Harvard study of 718 firms finds coding agents add 30% more code but no more finished work

What did the Harvard study find?

The paper is “Artificial Intelligence in the Firm: Bottlenecks in Software Production” by Fiona Chen and James Stratton of Harvard University. It is Chen’s job-market paper; the first version is dated 7 January 2026 and the current version 4 August. It drew wide attention in the past few days after Ars Technica wrote it up on 9 October under the headline that AI coding agents generate more code, but not more software.

The finding is simple to state. When a firm brings in AI coding agents, its engineers produce a lot more code. The firm does not, on the evidence of its own issue tracker, ship measurably more finished work, and it does not employ fewer people. The extra code gets stuck at the next step, where colleagues have to review it.

That is a firm-level version of a pattern our guide on whether an AI pilot is saving time or moving work into review describes for single teams. The value of the paper is scale: hundreds of companies, measured from their own engineering systems rather than from surveys of how productive people feel.

Where does the data come from?

Jellyfish sells an analytics platform that pulls in a company’s GitHub activity, Jira issues, Google Calendar meetings and HR data, plus usage records from GitHub Copilot, Cursor and Claude Code. The authors use data from client firms that opted in to research: 725,938 workers across 718 firms. The average firm had 1,652 employees and 241 active engineers in January 2023. Just over half (52%) sell software licences, 30% run marketplaces or similar services, and 14% build software for their own operations, such as trading systems.

Adoption dates come from two signals. Assistant adoption is the first month a firm bought GitHub Copilot or Cursor seats. Agent adoption is the earliest of Claude Code API activity, an agent bot account on GitHub, or signatures such as “Co-authored-by: Claude Code” or “Created by Devin” in commits and pull requests. By those measures, the share of firms using any coding agent went from near zero in October 2024 to more than 95% by January 2026.

The method is a staggered difference-in-differences design. Because firms adopted at different times, often depending on how long security and legal approval took, early adopters can be compared with firms that had not adopted yet. The key assumption is that, without the tools, both groups would have followed similar paths. The authors check the result against several newer estimators and report no significant differences.

How much more code did coding agents produce?

A lot more. After agent adoption, lines of code added per worker-month rose 30%, commits rose 20% and pull requests rose 23% (1.22 more per worker-month on a baseline of 5.34). All three are statistically significant. The authors note that these are on the low end of earlier studies, partly because they measure firm adoption rather than individual use. A year after adoption, only about 20% of engineers at the median firm were using an agent.

Earlier autocomplete-style assistants did much less. Copilot or Cursor seats raised lines of code 12%, commits 9% and pull requests 5%, and only the commit effect was significant. Agents, which write and submit code from a prompt, are a different kind of change.

One senior engineer quoted in the paper describes the pull: with Claude Code, “I can write a draft in two seconds and it’s going to be right 90% of the time,” a speed-up he puts “on the order of 100x” for routine tasks. The averages are far smaller than that, but they are large for a firm-wide measure.

Why didn’t finished software go up?

The authors measure output with Jira: an issue counts as resolved when its code reaches production, and an epic when a whole feature is done. After agent adoption, resolved issues rose by 0.12 per worker-month on a baseline of 3.67, about 3%, and the change was not significant. Their confidence interval rules out a rise larger than 12%, well short of the 30% jump in code. Epics showed no significant effect either. A machine-learning estimate of issue size found no sign that teams had switched to bigger tasks.

The explanation is review. Engineers in the sample spend only 30% to 42% of their time writing code, depending on whether you trust a survey of 100 engineers or an estimate from the activity logs. After agents arrived, the time from pull request submission to merge rose by 3.45 days on a baseline of 7.03, or 49%. The share of pull requests with a formal request for changes went from 13% to about 25%. Comments per pull request rose 35%. Pull requests did not get bigger, so the extra scrutiny is not just more lines per request.

Firms responded by pulling more people into review: the share of workers reviewing at least one pull request in a month rose from 29% to about 33%. AI review tools have not taken over. Nearly 80% of firms had used one by March 2026, but AI wrote only 23.3% of review comments, and only 10.8% of pull requests got any AI comment. Tools such as the Sashiko and Boro kernel reviewers point to where that could change.

Did coding agents cut jobs?

Not in this data. Measured by LinkedIn profiles, total employment showed what the authors call a precise zero after agent adoption, ruling out a fall of more than 2.9%. Engineering headcount in Jellyfish was noisier but also flat, ruling out a fall of more than 13.8%. They found no significant change for junior or senior staff separately, despite other studies that have reported losses among entry-level workers.

The authors caution that this is a short-run, single-firm comparison. Hiring may respond slowly, and a firm that gains from AI could take business from rivals that then cut staff, which this design would not see.

What should engineering leaders take from it?

The authors’ first lesson is about measurement: lines of code and commits overstate the return on AI, so the place to look is the whole pipeline through to deployment. Their second is that the gain only arrives if firms invest in the next bottleneck. That means more review capacity, clearer standards for agent-written code, and tests that catch problems before a person has to. Our guide to where to put human review in an AI workflow covers the trade-offs.

The paper also gives a reason review slows down. In its model, AI raises the volume of draft code and can also change how many bugs that code carries, and a junior engineer quoted in the study describes new rules requiring senior approval for AI-generated code after outages. Companies rolling out agents widely, such as Barclays with Claude Code, have a reason to watch review queues as closely as adoption numbers. Review time is also a cost line, as our guide to what an AI bill really includes explains.

What are the study’s limits?

It is an observational study, not a randomised trial. If firms that adopt agents early also differ in management or growth, the estimates could be biased, although the authors argue that procurement delays are unrelated to growth. Adoption is detected only through business licences and visible signatures, so personal accounts and unsigned use are missed. Jellyfish clients who opt in to research may not represent all software companies.

Jira resolutions are a rough proxy for software output; they do not capture quality, revenue or speed to customers. The data ends in March 2026, before newer agents and AI review tools spread further. Jellyfish reviewed the paper only for proprietary information, and the research was funded in part by a Stripe Economics of AI Fellowship and Schmidt Sciences.

We read the current PDF and Ars Technica’s report on 12 October 2026. The charts in this article are AI Lookout’s, drawn from the paper’s published estimates.