Click. Drag. Train.
Loading audio narration…
More companies are monitoring what you do on your work laptop or phone. Surveillance could also help show AI how you do your job.
Across industries, companies are investing heavily in AI agents that can automate tasks and make decisions. To build the best agents, employers often need high-quality data to model real work.
Now, on top of using software to ensure you’re working and following the rules, companies can use your clicks as how-to data for those agents.
“This is the evolution of workplace surveillance — from measuring work to learning how to replace it,” said Dan Schawbel, managing partner at the research firm Workplace Intelligence.
Watching how you work
Employer surveillance has grown because of the rise of remote work and because of a proliferation of tools that allow for monitoring, the US Government Accountability Office reported in September.
Employers such as AT&T have used technology to keep tabs on who shows up at the office, while JPMorgan is tracking software engineers’ AI use and monitoring them via dashboards, Business Insider reported Thursday.
However, increasingly, companies don’t just want the report you drafted or the code you (or, more likely, your AI) produced. They want to see how you made the proverbial sausage.
That’s because how you work — from emails to Slack messages — can be a goldmine for employers trying to understand the ways you make decisions and get things done.
For companies, the appeal is clear. Detailed, real-world examples of how workers work are among the most valuable inputs for building effective agents. It’s not like training a large language model, where scraping the internet might be enough. Worker data, sometimes called “digital exhaust,” is often highly specific and directly relevant to a company’s operations.
Meta is deploying an internal tool that tracks employee activity — including things like keystrokes and mouse movements — to help train AI systems, Business Insider reported on Tuesday.
The goal, the tech giant said in an internal memo, is to better understand how workers complete tasks so that AI agents can replicate or assist with those processes. Some Meta employees raised concerns about how closely the company would monitor their work.
A Meta spokesperson said in a statement on Wednesday that agents need real examples of how people use computers.
“There are safeguards in place to protect sensitive content, and the data is not used for any other purpose,” the spokesperson said.
Workplace Intelligence’s Schawbel said that for any company with loads of high-achieving workers, understanding more about how employees do their jobs brings enormous benefits.
“Companies aren’t just tracking productivity anymore. They’re capturing institutional knowledge in real time,” he said. “This is unbelievably valuable.”
An incomplete picture
While some companies might dream of training agents based on what workers do all day, it’s not easy to do.
Employers are generally sitting on far more worker data than they can readily use, said Emily Rose McRae, senior director analyst at the research firm Gartner.
Often, she said, companies store this information and don’t use it — or decide not to keep it — because doing so is costly and poses security risks.
While AI’s ability to make sense of this information can be “incredibly powerful,” she said, the data companies have doesn’t necessarily capture the full picture of what a worker does.
“Even the data we have may not be the data we need to be able to understand people’s jobs,” McRae said.
Training AI on workers’ usage data can be easier for certain roles, such as software development, where much of the work is done on computers, and there are clearly defined steps, she said.
“A lot of other work is a little bit more complicated than that,” McRae said.
She credited Meta for its decision to disclose its monitoring to workers. Too often, she said, employees aren’t aware of what companies are collecting.
“If you don’t tell people and they find it out, it feels like a betrayal,” McRae said.
‘Trust erosion’
Schawbel sees increased monitoring generally as a sign of a continued “trust erosion” between workers and employers. Companies can get away with even unpopular moves because of the power they tend to hold in the job market, he said.
That’s likely one reason — besides keeping their job — why workers fearful of being replaced by AI might still go along with this type of monitoring, Schawbel said.
“If training these AI agents that might take my job in two years buys me two years to hold this job, so be it,” he said.
Schawbel expects that, tech challenges aside, more companies will likely try to use the data they collect to train agents. In part, because they’re investing so heavily in AI.
For tech firms, which are under pressure from investors to show the fruits of their AI outlays, the benefit is twofold, he said: They get to use AI to improve their own operations by introducing agents, and companies can show potential customers for their AI products and services how to do it, too.
Schawbel said that even as companies consider automating some positions, leaders have long touted workers as being a company’s most valuable resource. Now, because workers can train AI, they can be a company’s “best technology asset,” too, he said.
Do you have a story to share about your experience with AI at work? Contact this reporter at [email protected].
