Eighty-five percent of consultants working alone solved a hard business problem. Colleagues given GPT-4 were 19 percentage points less likely to get it right. On a different set of tasks, the same experiment found the opposite: higher quality, more speed, more work finished. The tool did not change. The job did.
Those figures come from a 2023 field experiment at Boston Consulting Group, written up by Fabrizio Dell’Acqua and colleagues as Navigating the Jagged Technological Frontier. The scholarly version is a working paper on SSRN, not a finished journal article. Read it that way.
Inside the frontier, and just outside it
The experiment involved 758 junior, client-facing consultants. Everyone first did a similar task without AI, so the team had a baseline. People were then sent either into creative product work or into business problem solving, and from there into GPT-4 with about 30 minutes of training, GPT-4 with no training, or no GPT-4 at all.
This was not a volunteer afternoon. Participation showed up in biannual reviews. Finishing the work fed annual bonuses. The top 20 percent were flagged to managers. Ninety-nine percent of those given access actually used the model. The analysis controlled for more than 20 background factors, including education, geography, and prior experience with generative AI.
The useful idea in the paper is a jagged frontier. Current models are relatively good at novel, useful ideas of the kind already thick in their training data. They are weaker when the job is to weigh messy qualitative and quantitative evidence and land on a defensible call. Two tasks can feel equally hard in a Monday staff meeting and still sit on opposite sides of that line.
Inside the line, the gains were large. Across eighteen realistic consulting tasks, consultants using AI produced work more than 40 percent higher in quality than the control group, finished tasks 25.1 percent faster, and completed 12.2 percent more of them. Lower performers got that lift too. Relative to their own baseline scores, people below the average performance threshold improved about 43 percent. People above it improved about 17 percent. On this side of the frontier, the leveling story is real. Both groups got better, and the people who started behind gained more.
Outside it, the story flips. The business problem was built so GPT-4’s first pass would be wrong, while a consultant could still solve it. Human graders who did not know who had used the tool scored the creative work. The business task was simpler to mark: right or wrong. BCG’s account of the experiment says 85 percent of the control group found the correct answer. Dell’Acqua and colleagues report that consultants using AI were 19 percentage points less likely to produce a correct solution.
BCG’s account is that many of those users accepted the erroneous output once the rationale that followed sounded convincing. The working paper does not isolate that explanation as the sole cause. What it does establish is a split between over-reliance and people who kept their own judgment in the loop. A smooth wrong answer is easier to wave through than a clumsy one. That last line is a reading of the result, not an extra measurement.
What is left when the model leaves
People who wrote essays with a large language model later reported the least ownership of the prose, and struggled to quote their own sentences accurately. People who wrote with no external tool reported the most ownership.
That finding comes from a June 10, 2025 MIT Media Lab preprint, Your Brain on ChatGPT, led by Nataliya Kosmyna and Pattie Maes and posted as arXiv:2506.08872. It is an essay study, not a client floor, and it has not been through journal peer review.
Fifty-four people wrote across three sessions in one of three conditions: an LLM, a search engine, or no external tool. Eighteen came back for a fourth session and switched. LLM users had to write on their own. Brain-only users were handed the LLM. Those sessions were spread out. The abstract says LLM users underperformed over four months, but the public description does not show a continuous four-month trial. It shows a few writing sessions and a small switch group.
Using EEG, language analysis, teachers, and an AI judge, the team found brain-only writers had the strongest, most widely distributed connectivity. Search users sat in the middle. LLM users showed the weakest. The authors read that as effort dropping as the external tool took more of the load. In the switch, they read the group coming off the LLM as under-engaged, and the group moving from unaided writing into the LLM as showing better memory recall. Treat the brain readings as the authors’ interpretation of a small sample, not as a management metric. Ownership and recall are the findings a firm can actually picture.
The authors call the accumulation cognitive debt: thinking handed off, then not fully recovered.
Do not mash these studies into one statistic. BCG measured performance while the tool was available, with job incentives and several hundred consultants. MIT measured brains and memory on a writing task, then watched eighteen people try the work again without the model. Neither is a multi-year study of professional judgment. Together they still point at the same operating risk. The failure mode is over-reliance. They do not support a vague claim that AI trains early-career staff badly.
Questions worth asking before you automate the reps
- Does AI reliably turn weaker performers into strong ones?
On tasks inside the model’s range, lower performers in this experiment gained more than stronger ones, and both groups improved. On a problem the model could not solve correctly, AI use made consultants 19 percentage points less likely to be right than colleagues working alone.
- If 85 percent can solve the hard problem alone, why did the tool make the group worse?
BCG’s account is that many GPT-4 users accepted an erroneous first pass once the rationale sounded convincing, while others kept their own judgment. The paper does not isolate that rationale as the sole cause.
- Is cognitive debt just a metaphor?
In the MIT preprint it is tied to lower ownership and trouble quoting one’s own sentences, alongside weaker measured engagement in a small sample. It is an essay study (54 people, 18 in the switch) and a preprint, not a verdict on every profession.
- Should early-career staff be cut off from AI agents and copilots?
No. A ban is a blunt answer these studies do not support. Keep some unaided reps: draft before opening the tool, make people attack the model’s answer, and take the assistant away on a sample of work to see who can still defend it.
- What does this mean for AI automation on client work?
As an implication, not a result either study measured: a faster draft is not the same as a person who can hold the recommendation when the model is not in the meeting.
Firms like the leveling result because it sounds like free training. Sometimes the tool really does pull a weak draft up into something a client can read, and the quality gap on tasks inside the frontier (more than 40 percent) is not a rounding error. The quieter risk is that the reps that used to build taste, structure, and nerve get replaced by review. Review looks like work. It is also where a fluent error survives, because disagreeing with it feels like pedantry.
Count the drafts shipped and the quarter looks like a win. Occasionally close the tool, hand back a flawed draft, and ask for the fix in the person’s own words. That is how you find out whether AI for business trained a professional, or merely rented one for the quarter.
A file that does not belong in the staff meeting
Separate from the consulting and memory studies above, this NBER working paper is about diet, not models.