The data is getting worse.
A third of investors who buy data say its quality has deteriorated over the past couple of years, according to a survey by consultancy Neudata, and some pin the blame on artificial intelligence.
Data is the lifeblood of any investing business, and hedge funds have relied on so-called alternative data — information that doesn’t come from traditional sources like SEC filings or exchange trading data — to help them make bets for years.
The advent of AI has led to a surge in new vendors selling data, thanks to the ease in accumulating and sorting it, as well as changes to how longtime sellers operate.
“There’s so much data now, and there are not a lot of experienced data scientists. These new places are just running it through LLMs,” said one fundamental equity investor who purchases external datasets and has noticed a slip in quality.
“Hallucinations still happen,” he said.
Why AI is to blame
AI is affecting data quality in two ways: Providers are using AI to generate or map the data they’re selling, or they’re using AI as a justification to cut or reassign people focused on data quality, said Daniel Entrup, cofounder of AggKnowledge, a data product business that works with sellers and buyers.
Both approaches have issues. Data buyers say that those relying on AI to create a data product to sell to hedge funds struggle to communicate how a final report or dataset was created — a critical piece of information hedge funds need to ensure they’re complying with federal regulations on information availability and privacy. There’s also a chance that the model interfacing with the fund is just not very good.
“The data might be there, but the LLM might be rubbish,” said Daryl Smith, the head of research at Neudata.
Hedge funds, often with much larger tech budgets and more sophisticated teams, would prefer to run the data through their own systems than rely on a vendor’s AI model. Plus, the work done on the raw data is a part of many managers’ secret sauce.
“Funds trust AI with their workflow much more than they trust it with their alpha,” Smith said.
Meanwhile, Entrup has noticed an uptick in basic data hygiene issues in recent months as industry priorities have shifted to finding new use cases for AI. His business does data “revision” work that cleans up errors in datasets clients use, and he said this type of work has increased significantly.
“How much does quality matter? For a hedge fund, it matters a lot,” he said.
The frustrating thing for many buyers is that these AI initiatives are not what they would have tasked their data providers with.
“Show me the customer research where clients were asking for it,” Entrup said.
There’s been no business pickup for the data vendors that are using more AI in their processes. One data buyer at a medium-sized hedge fund said they’ll use a vendor’s LLM if it’s offered for free, but not if they have to pay. Smith notes his firm hasn’t found a notable increase in revenue for vendors that use AI tools.
And it only takes one hallucination to ruin a dataset.
“Once you have a failed signal, it’s polluted data,” said one hedge fund manager.
