26% said web scraping had the greatest impact on revenue of any external data collection method.
Web scraping came only just behind internal data collection (28%).
In contrast, only 12% believed manual data collection had the biggest influence, alongside third-party traditional data aggregators (12%).
Third-party alternative data aggregators came in fifth with just 11%.
Over a quarter (26%) of financial services organisations believe web scraping has had the greatest impact on revenue out of all external data collection methods. This is one of the key findings from the new Oxylabs white paper, Alternative Data Unlocks Key Decisions in the UK & US Finance Industries, demonstrating how web scraping is shaping the sector in terms of gathering data and using it to improve services and generate revenue.
In cooperation with Censuswide, Oxylabs surveyed 500 UK-based and 501 US-based senior data decision-makers from financial services companies, gathering insights on attitudes towards data collection in the sector.
Every other data collection method was split almost evenly, but didn’t reach more than 13%. 12% of respondents believed manual data collection had the greatest impact on revenue, alongside third-party traditional data aggregators (12%), and then third-party alternative data aggregators (11%).
The findings highlight that web scraping has come to the forefront largely due to the rapidly growing amount of information available online. Harnessing this has allowed financial service organisations to gain insight into business performance and identify future investment opportunities, which has a major impact on future revenue.
Andrius Palionis, VP of Enterprise Sales at Oxylabs said: “Something we rarely consider is that web scraping provides otherwise unavailable datasets. Finance companies thrive off having insights others do not possess as it allows them to get one step ahead of competition. Web scraping lets them fill that niche ad infinitum as long as they can maintain the infrastructure and diversify sources of information.”
When looking at the survey results more closely, there exists a contrast between UK and US companies. In the UK, web scraping ranked as the leading collection method with the greatest impact on revenue (cited by 25%) while in the US it was in second place (27%), behind internal data (37%). Interestingly, UK companies were significantly more likely to choose “None of the above” (14%) compared to US companies (4%), indicating that some data collection methods might not be publicly known. Such a tendency could be explained through data collected from a wide variety of disparate sources without provisioning it from an aggregator, such as satellite imagery or weather reports.
Mr. Palionis continued: “Web scraping may not quite have come out on top among financial service organisations in the US, but the results still show it plays an integral role in driving revenue. The correct use of data allows an organisation to get a better understanding of its customers, create more enticing marketing campaigns and forecast demand with higher accuracy.
“The landscape of data is changing and so is the way it is being used. Web scraping has always played a pivotal role in helping organisations collect data that has been previously hard to reach but its role is now evolving. The final frontier is for financial organisations to fine-tune their data gathering processes to ensure revenue is maximised in the long term”, concluded Palionis.
