AI Increases Code by 30%, But Not Completions | Survey of 700+ Companies
機械翻訳 / Machine-translated
A
AI Friends
2日前・16分で読める
機械翻訳 / Machine-translated
@aifriends
AI Friends（https://aifriends.jp）のクロスポスト公式アカウント。AIツールの紹介・使い方・できることを、中学生でもわかるやさしい日本語で届けます。
Have you heard stories about AI tripling development speed and wondered whether to adopt it yourself? When Harvard University researchers examined data from over 700 companies, they found that while code volume increased, the number of completed tasks did not. This article breaks down the reasons and offers tips for adoption.
On October 9, 2026, Ars Technica reported on research by Fiona Chen and James Stratton of Harvard University. The title: "AI coding agents are writing more code, but not more software."
The study examined data from over 700 software companies collected by engineering analytics firm Jellyfish. The dataset covered more than 700,000 employees and approximately 300 million work records (commits, pull requests, etc.) from 2021 through March 2026.
A coding agent is an AI that, when given instructions by a person, writes code on its own and proceeds through to revisions. The study compared how activity levels changed before and after each company adopted these tools.
After adoption, companies changed on average as follows:
So far, this looks like "AI improved productivity."
However, the resolution rate of issues managed in Jira (tasks such as bug fixes and feature additions) showed no statistically significant improvement. In other words, the volume of code written and the volume of work completed were not moving in tandem.
It is somewhat like a moving situation where boxes keep piling up in a room, but the number of fully unpacked rooms never increases.
What the researchers zeroed in on was the verification work that happens after code is written. A pull request must be read and approved by someone else before it is merged into the main codebase. This process is called review.
After adoption, the following changes emerged in reviews:
Imagine that thanks to AI, everyone on the team is submitting more change requests. Reviewers have the same amount of time, so the queue of pending reviews just keeps growing. And because the number of requested revisions also increases, requests and fixes go back and forth repeatedly.
As of March 2026, 80% of the measured companies were using AI-powered code review. AI was involved in 23.3% of review comments and 10.8% of pull requests. Yet the bottleneck in the flow has not been cleared.
This finding is not isolated. The National Bureau of Economic Research (NBER) working paper "Writing Code vs. Shipping Code" examined over 500,000 GitHub developers.
The cumulative effect on commit counts was 30% for autocomplete, 180% for interactive agents, and 240% for autonomous agents. Yet this shrinks to 80% for project counts and 30% for release counts.
The authors attribute this to the strong complementarity between AI tasks and human tasks — human work becomes the weakest link that determines the overall pace. The estimated elasticity of substitution between AI and human work is 0.23.
Furthermore, across four major app markets, the number of new apps increased substantially, while total usage did not.
Research measuring AI productivity has produced mixed results. Here are three key studies side by side:
In the METR study, developers predicted beforehand that they would be "24% faster," and even after finishing still felt they had been "about 20% faster." The actual result was the opposite.
However, METR itself notes that this finding is already dated. In February 2026, they released new data using tools from the second half of 2025. Because the target populations and time periods differ across studies, directly comparing numbers is risky.
This research examined overseas companies and did not directly study Japanese firms. Nevertheless, the structural pattern of review becoming a bottleneck may well apply to Japanese teams.
Consider, for example, a small development team of three people that adopts AI. Even if the number of change requests per person doubles, if only one experienced person can perform reviews, work concentrates on that individual. New feature releases might actually slow down.
The same applies to companies that outsource development. Even if the volume of delivered code increases, if the people responsible for acceptance testing (confirming deliverables) are insufficient, completion will not come any faster.
When measuring the effects of adoption, it is more important to track the number of completed tasks and the number of days to release than lines of code.
This study has its limitations. Because it relies on observational data, it cannot be definitively stated that agents caused all of the observed changes.
The data ends in March 2026, so new tools and shifts in usage patterns that emerged afterward cannot be measured. The 49% figure also represents elapsed time in review, not actual hours worked by humans.
The study also does not claim that "AI is not producing new software." What was found is that there was no statistically significant improvement in the number of tracked issues completed.
On the other hand, an Appfigures survey covered by MacStories shows that the number of apps published in Q1 2026 was up 60% compared to the same period the previous year. However, author John Voorhees also notes that this type of figure should be viewed with caution.
Not at all. Lines of code and commits have increased. What could not be confirmed was a corresponding increase in completed tasks — that is the finding here.
80% of measured companies were already using AI review, yet the study shows the bottleneck persists. Simply delegating to AI may not be enough.
This research did not directly examine Japanese companies, so no definitive answer can be given. The most reliable approach is to measure your own organization's task completion rates and release frequency.
That remains unknown. The data only goes through March 2026 and does not include newer tools released since then.
Start by measuring not the volume of code, but the number of completed tasks and time spent waiting for review within your own team.
This article is a cross-post from AI Friends.
