WHEN AI BUILDS ITSELF
What Anthropic Published, and Why It’s Alarming
The paper comes from the Anthropic Institute, written by Marina Favreau and Jack Clark. Here is what’s in it.
80% of new code is now written by Claude. More than 80% of the code merged into Anthropic’s own codebase is machine-written, not human-written.
8x engineering throughput. Anthropic’s engineers are shipping eight times as much code per quarter as they were a year ago.
Task horizons exploded. Claude Opus 4.6 can now complete tasks that take a skilled human 12 hours. A year ago the ceiling was four minutes. On the METR autonomy benchmark, the time horizon AI can work unsupervised is doubling every four months, up from every seven.
A 2027 projection. If the trend holds, Anthropic states that by the end of 2027 Claude will handle week-long tasks on its own.
Research taste is the last human job. The paper says the only remaining frontier for their human engineers is “research taste,” the judgment of which experiments are worth running, and they expect to automate even that within a year.
Why it’s alarming is the conclusion they drew from their own data. A company about to IPO at a trillion-dollar valuation, with 640% user growth, used the paper to call for a temporary global pause on frontier AI. Their words: “We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up.” A lab with every incentive to stay quiet through its IPO instead published evidence that AI is now improving AI, and asked the world to build a brake.
The technical report also included two employee quotes that landed hard. One engineer: “It’s been five months since I last wrote any code myself.” Another: “On days where everything works well, I can’t help but think nothing I do matters.”
“I’m calling it here. This is early start of recursive self-improvement.” — Peter H. Diamandis
What the Mates said:
Alex does not expect a hard takeoff. He thinks we pass through human and superhuman performance smoothly, the way we passed the Turing test “with a whimper, not a bang.” He rejects the idea that the last 20% is asymptotic: autonomy horizons are heading toward effectively infinite.
Salim read the timing as a tell. If progress were going to stall, Anthropic could have waited. They published because internally they have a clear sense of when the threshold gets crossed.
Dave made the key technical point: recursive self-improvement does not require an Einstein-level AI. It requires faster inference and better chips. A 100x performance gain at the bottom of the stack, already in the pipeline, can push the system over the line.
There’s a governance story attached. In the last 24 to 48 hours, reporting indicates the White House has converged on a version of Senator Sanders’ proposal: a government equity stake in the frontier labs as the basis for a universal basic dividend. Both parties, the same idea, at the same moment the labs hit self-improvement. Anthropic’s own paper invokes the post-Cuban-Missile-Crisis nuclear treaty as the precedent: not a unilateral stop, but a coordinated option to slow down.
“We drove straight through the Turing test with a whimper, not a bang. I suspect we go straight through recursive self-improvement the same way.” — Alex Wissner-Gross
No comments:
Post a Comment