Partner warns ‘huge danger’ of replacing reasoning skills

A Goldman Sachs partner leading one of the bank’s flagship artificial intelligence projects warned that AI’s spread across Wall Street risks hobbling the thinking capabilities of the next generation of financiers.

“There’s a huge danger here that in the era of AI, we outsource our reasoning to these models, and we have cognitive atrophy that stops us being able to reason from first principles ourselves,” said Chris Churchman, who leads Goldman’s digital platform for institutional clients called Marquee.

The comments came during the latest episode of the firm’s “Exchanges” podcast, according to a transcript provided exclusively to CNBC.

Just as people lost navigation and memorization skills with modern inventions, bankers risk losing analytical abilities if algorithms handle all the heavy lifting, Churchman said.

“Reasoning is still important,” he said. “You still need to reason about [problems] and structure it into an argument, and now we’re delegating reasoning.”

Wall Street’s push to enmesh AI into all of its trading and banking processes could be a kind of devil’s bargain: It will make the industry more profitable today while potentially eroding the talent it needs for tomorrow. With AI taking over more of the routine work that has traditionally taught young bankers and traders how to think and make decisions, firms risk sacrificing the culture that turns junior employees into seasoned Wall Street talent.

It could even reduce the need for junior bankers in the first place. Last year, CNBC reported that Wall Street firms were examining ways of using AI to lower the ratio of junior bankers to senior employees.

Banks need to find a balance between using AI and preserving Wall Street’s apprenticeship culture, said Churchman, who ran currency trading at UBS before joining Goldman in 2021.

“You learn by doing, and a lot of knowledge is tacit, it was never written down,” he said.

Goldman needs “to make sure we don’t lose that tacit and intuitive knowledge that some of our best people have today [and] to ensure the next generation have it too,” Churchman said.

For instance, junior traders learn by fielding client pricing requests under supervision of experienced risk takers, Churchman said.

“We can absolutely automate that,” he said, “but then do we get the senior traders that fully understand?”

Systems must be designed so that employees still call the shots in high-stakes, high-uncertainty decisions rather than becoming passive operators, Churchman said.

Even Goldman, one of the world’s top investment banks, hasn’t yet “figured out” how it will manage the transition the company has begun, said Churchman, who is also co-chair of the firm’s Global Banking and Markets AI working group.

Also in the podcast interview, Churchman shared lessons from implementing AI into Marquee, which is used by hedge funds and other institutional clients to access Goldman’s market data, research, risk analytics and trade execution services.

The Marquee AI platform is only available to Goldman employees for now, he said.

The toughest challenge, from a technical standpoint, is in ensuring that AI answers are 100% factual and can be audited, he said. While consumer AI chatbots warn users of possible mistakes, in high finance, the tolerance for errors is low.

Churchman said that in developing the firm’s AI platform for clients, the software made a startling admission.

“When we challenged it hard, at least it was honest,” Churchman said. “It was like, ‘Look, in the end, I’m better at sounding thorough than being thorough.'”