44:52Matthew Berman · Essay
It's starting...
A 45-minute walk through Anthropic's internal data showing AI crossed from coding assistant to primary engineer — and a frank read on what that means for humans.
June 5thThe argument in one line.
METR's independent assessment says Anthropic's model progress has roughly doubled in speed, most of it caused by AI doing AI research, with no identified bottleneck stopping further compounding.
Read if. Skip if.
READ IF YOU ARE…
- You follow frontier AI news and want the actual claims of the METR report on Anthropic explained in plain language.
- You build on top of Claude or other frontier models and want to understand how fast the underlying capability curve may be moving.
- You want to know what evidence exists for recursive self-improvement beyond lab marketing.
SKIP IF…
- You want a neutral technical audit of the methodology; this is a commentary that mostly reads the report aloud with reactions.
- You want to know what to do about it; the video describes the trend but gives no practical guidance.
The full version, fast.
METR, an independent evaluator, spent a few months inside Anthropic and concluded that by October 2026 Anthropic is improving model capabilities on well-specified tasks at about twice its historical rate, possibly 1.3x to 3x. It attributes roughly 80 percent of that acceleration to AI itself rather than more compute or hiring, citing employee interviews, lines of code, and an inflection that starts with Opus 4.5. METR found no clear bottleneck over the next 12 months, though Anthropic limited access and redacted parts of the analysis.
Free for members
Chat with this breakdown — free.
Sign in and you get 23 free chat messages on us — ask for the hook, quote a framework, find the exact transcript moment, generate a markdown action plan. Bring your own key when you want unlimited.
Create a free account →Where the time goes.
00:00 – 00:52
01 · Cold open and context
The claim that recursive self-improvement has begun and why third-party evaluators were given access.
00:52 – 03:10
02 · METR's headline finding
Anthropic is improving about 2x faster than before, mostly attributed to AI, with caveats on Anthropic's control over publication.
03:10 – 05:20
03 · The inflection at Opus 4.5
The acceleration chart and why it lines up with late 2025.
05:20 – 06:50
04 · Sponsor: JetBrains Air
Sponsor read for JetBrains Air and IntelliJ IDEA.
06:50 – 09:40
05 · Compounding loops and compute efficiency
How better models, better compute use, and amplified researchers feed each other, plus no identified bottlenecks.
09:40 – 13:00
06 · The AECI curve and ruling out compute
Opus 4.5 and Mythos on the capability curve, and METR's evidence that compute scale was not the cause.
13:00 – 16:40
07 · Employee interviews and evidence
Anonymous researcher quotes on speedups, being bottlenecked on ideas, and signs of autonomous research.
16:40 – 18:10
08 · How the METR engagement worked
Two and a half staff, three questions, and Anthropic's control over access and scope.
18:10 – 20:08
09 · Appendix highlights and what it means
Unreleased internal models, saturated benchmarks, and the worry about no longer understanding AI.
Lines worth screenshotting.
- METR estimates Anthropic is improving model capabilities on well-specified tasks at about twice the rate it did between March 2024 and October 2025.
- The acceleration is attributed to AI itself, about 80 percent by METR's guess, not to extra compute or more researchers.
- The inflection in Anthropic's internal capability measure lines up with the release of Opus 4.5 in late 2025.
- METR says it received convincing evidence the acceleration was not driven by a jump in training compute, though Anthropic redacted the details.
- Anthropic employees interviewed said they are bottlenecked on ideas and compute, no longer on code.
- Anthropic's compute efficiency went from 0.67x in Q4 2025 to break-even in Q1 2026 and 2x in Q2 2026, according to the S-1 figures Berman cites.
- METR did not identify any clear bottleneck in algorithms, training compute, experiment compute, or data over the next 12 months.
- Anthropic revoked METR's access to compute allocation data midway through the engagement.
- Anthropic can unilaterally approve or reject METR's data requests and decide what gets published, so the findings are not a fully independent audit.
- The report suggests a permanent slope change in progress is more plausible than a one-time jump at the Mythos Preview release.
- Anthropic has had to make its internal benchmarks harder because new models are saturating them.
- Researchers say AI is not yet making fully autonomous contributions to the most important parts of model training, but shows early signs of it.
What METR says about AI speeding up AI
WHAT TO LEARN
An independent evaluator concludes AI is already the main driver of faster progress at Anthropic, and it found no obvious bottleneck, though Anthropic controlled what was shared.
- Look at who controlled the data: Anthropic could approve, reject, and redact, so treat the findings as strong signal with real limits, not a full audit.
- Acceleration claims need a counterfactual. METR compared actual capability growth to a trend line and tried to rule out compute and hiring as causes.
- Self-reported uplift, lines of code, and headcount are weak alone but become informative when several independent lines of evidence point the same way.
- Compounding comes from several loops at once: better models, more efficient compute use, amplified researchers, and revenue funding more experiments.
- The remaining human edge cited by researchers is ideas and research direction, but early signs suggest models are beginning to contribute there too.
- If benchmarks saturate and internal models are ahead of public ones, public releases understate where the frontier actually is.
Terms worth knowing.
- Recursive self-improvement
- A loop where an AI system helps build better AI systems, which then help build better ones faster still.
- METR
- An independent evaluation organization that tests frontier AI models and labs for dangerous capabilities and research acceleration.
- AECI
- Anthropic's internal measure of model capability, used in the report to chart progress over time and the excess growth attributed to AI.
- Compute efficiency
- How much useful research or model quality a lab gets out of each unit of its limited data center capacity.
- Intelligence explosion
- The idea that AI-driven research compounds on itself so progress speeds up sharply, described in the Situational Awareness paper.
Things they pointed at.
19:50linkWeight Space Programming paper
09:20linkSituational Awareness paper
10:40linkGoogle AlphaEvolve
Lines you could clip.
00:00
“Recursive self-improvement has already begun, and this comes with proof from a third-party evaluator.”
19:20
“There is literally no blocker between what is happening now and what artificial super intelligence or recursive self-improvement looks like.”
Word for word.
Read-along
Don't just watch it. Burn it in.
See every word as it's spoken — crank it to 2× and still catch all of it. The same dual-channel trick behind Amazon's Kindle + Audible.
Recursive self -improvement has already begun, and this comes with proof from a third -party evaluator that had near unfettered access to internal anthropic systems. researchers, and basically everything else. So this all really started a few months ago when leaders of the top AI labs, including Anthropic and OpenAI, all called for pacing of the frontier because they were nervous about recursive self -improvement and then eventually super intelligence.
And at that time, they all said, hey, we will allow third -party evaluators to basically have employee -like access internally. unfettered access to all of our systems, our latest models, our employees, our chat history, everything. And so the company Meter, which is a well -regarded independent evaluation company.
worked with Anthropic, very hands on and got nearly full access internally. And so the first results are here. And I have to be honest, it is making me a little bit nervous.
And so we're going to go over this report together. I'm going to show you some of their findings and what this truly means for the future of AI over really the next. one to two years.
This is the report, Independent Assessment of AI R &D Acceleration at Anthropic in October 2026. So what they are looking for are signs that AI is improving itself, signs that AI is accelerating AI research and development. And they found it.
Now, before we get into the results, I want to point out one thing. I mentioned that Anthropic and OpenAI had said, hey, we're going to give employee -like access to third -party evaluators. It turns out this was not part of that program.
And so Anthropic was able to basically approve or deny access to anything. And not only that, even after access was given and Meter was able to make some predictions and determinations, Anthropic was able to determine what they published and what they didn't.
But even with that, these findings are incredible. And so I am going to show you how they were able to prove that AI research and development is absolutely accelerating strictly because of AI. So basically, AI is able to improve itself.
It's able to accelerate the speed at which Anthropic is researching and developing their latest models. And yes, you probably already guessed this. The models are getting better and they're getting better more quickly than ever.
And it is almost entirely attributable to AI help. All right, let's get into this. Most meter staff who worked on this project now believe that as of October 2026, Anthropic is likely improving the capabilities of their models on well -specified tasks at around twice the rate they did between March 2024 and October 2025.
And so when they're talking about well -specified tasks, they're... generally talking about the AI R &D process. They're talking about the systems around that.
They're talking about writing code to help them with these systems. And they go on to say they would not be surprised if the current rate of growth was closer to 1 .3x or 3x the historical rate. And now here's the important part.
It's not because Anthropic is hiring more people, getting better AI researchers, having more compute. It's... None of that.
They were really able to isolate the cause of the acceleration to be AI progress itself. And they even pointed out in late 2025, that's when the latest version of Opus came out. And that is when models became incredibly good at writing code, like industry changing good.
at writing code it happened all around the end of last year and they say yes the acceleration at that point was likely due to non -ai factors and those are more compute more hiring breakthroughs discovered by human researchers but fast forward to october 2026 we guess that the majority perhaps about 80 of the current acceleration is due to ai and they're going to not only prove it they're also going to explain how it's happening.
All right, so they put together this fake chart and yes, it's fake, but it basically could be real. What we're seeing with this blue line is the improvement in model capability. And if It followed the historical trends.
We would continue on this dotted blue line. Okay, so it's pretty linear. Then they measured AECI, which is basically Anthropix internal measure of the capabilities of their models without AI.
And we see this gray line. but when we have that same measure excess growth due to ai we have this orange line and we can just see the inflection point at the end of last year and again it lines up perfectly with what all developers experienced at the same time their entire job changed in october of last year because of opus 4 .5 and it was just absolutely mind -blowingly good at writing code and not just writing code but really like building entire features and entire pieces of architecture.
And it has only gotten better since then. And if you're writing as much code as Anthropic is writing today, which is obvious now based on meters research, you need better tools to do it. And that's why I want to tell you about today's sponsor.
So if you watch this channel at all, you know, I am all in on AI coding agents. And the really hard part is the code that's almost right. And so that's why today's sponsor JetBrains built JetBrains Air and IntelliJ Idea.
You can run multiple agents at once without having to re -explain your code base every time. You see which sessions are running, done, or waiting on you. Comment on a line in the diff and it goes straight back to the agents to fix.
And because IntelliJ IDEA already understands every usage, dependency, and type in your code, your agents use fewer tokens because they are taking fewer wrong turns. And then the debugger, test, and profiler are all one click away. So you can make sure nothing breaks you merge.
You have the final say on what ships. Try JetBrains Air and IntelliJ IDEA with the link down below. Let them know I sent you.
Here's where it gets concerning and this is why I am nervous. A key concern is that acceleration from the use of AI could start to compound and they're already seeing that because everything compounds on itself. We have the models getting better, which means they're able to discover how to make themselves better.
more quickly. They're also figuring out how to utilize their limited compute capacity better than ever. And the human researcher output is being amplified because of all of this.
And all of it amplifies each other. That is the intelligence explosion that was described in the situational awareness paper. And this was actually revealed in their S1, the document leading up to their IPO, which is imminent.
They showed that their utilization of compute, the data centers. the limited amount of compute they have is actually increasing tremendously. So these models are getting more efficient.
And of course, when they're getting more efficient, they can do more research. You can throw more tokens at a problem. You also earn more money, right?
You have more revenue, which they've also shown. And it's all this crazy compounding loop. So look at Q4 in 2025.
The compute efficiency was 0 .67x. Then... Q1 2026, break even.
Q2 2026, it is 2x. So again, all of these breakthroughs are primarily being accelerated by artificial intelligence being able to figure out how to make itself better. And don't forget Google's Alpha Evolve paper, which came out a while ago, in which they basically pointed AI at improving Google's overall hardware architecture, which allowed them a...
1 % improvement in efficiency. And if 1 % does not sound like a lot, imagine 1 % at Google scale. That is billions and billions and billions of dollars of savings that AI was able to discover for itself.
It's able to now run itself more efficiently. This is what we're seeing with this meter paper. And meter even goes as far to say they don't see any other bottlenecks between where we are today and an absolute explosion.
in artificial intelligence recursive self -improvement listen to this we did not identify any clear bottlenecks limited inputs that would prevent further growth or steep diminishing returns to further scaling that would get in the way of such compounding acceleration that is a wild statement they're basically saying we're off to the races this is it there's nothing between us and the singularity.
Next, because of this, we believe there is a significant risk that further capabilities progress could lead to even faster acceleration over the coming year. Year. singular.
They go on to say Anthropic appears to have made as much progress on benchmarks in the past year as they did between October 2023 and October 2025. But again, I think we're all feeling this. The models are getting so much better.
The model releases are getting more frequent. We're all seeing it and they have shown it now. and again they've isolated it to being because of ai accelerated development i mean here's the perfect chart to prove it this is a e c i again this is anthropic's internal model capability evaluator and what we see is a really you know more or less linear growth line all the way through the end of 2025 and then something happens and so here we have opus 4 .5 which was the absolute inflection point and then It just accelerates.
Mythos comes out at the early part of this year, and it just continues to accelerate. Again, if it followed the trajectory historically, we would be down here. But no, no, we're up here.
And if you project out into the future, this curve is actually getting steeper. That is the acceleration that they're describing, the steepness of the curve. The acceleration itself is accelerating.
And again, they really had to take their time and make sure that this acceleration wasn't caused by anything else. The first thing that they point out is that we received convincing evidence that the acceleration was not due to an acceleration in the scale of training compute, meaning Even though Anthropic has been able to acquire much more compute, they have evidence, although they go on to say that further details on this evidence were redacted by Anthropic, but they have the evidence that shows.
The acceleration was not due to some explosion in compute capacity because we all know Anthropic doesn't have that. They, at the beginning of this year, were struggling to serve their models. I mean, they're much better now.
They were able to acquire more compute. They're also, as we talked about, able to squeeze a lot more efficiency out of their models, again, because of AI R &D. So this is just all so wild.
And not only that, they interviewed employees at Anthropic and they asked them like, hey, are you? more productive? And if so, can you estimate how productive you are?
Now, yes, granted, this is like a very casual interview and it's not scientific, but it's a good signal to have. So let's read what it says. We have many lines of evidence relevant to AI's contribution to progress in different parts of the model training pipeline, including interviews, self -reported uplift, headcount, lines of code, experiment usage.
and qualitative reflections from leadership. So not only those interviews, but they were actually able to look directly at the model training pipeline, what was happening, how much efficiency they were gaining, how many additional researchers joined the team, how many lines of code were being written, and all of this gave them enough signal to determine, yes, this is all because of AI.
So let me actually give you some of the quotes from those interviews. Now, they did not. put names on this which of course to protect the privacy of those researchers so here are a few quotes could definitely endorse a greater than 2x speed up my inside view is by far the largest effect is model acceleration the ai acceleration is larger than all the other effects now only bottlenecked on ideas and compute, no longer bottlenecked on code.
Now for this one, I want to pause for a second. Now only bottlenecked on ideas and compute, but meter showed that the compute wasn't a bottleneck. So this is obviously a little bit of.
contradiction here, but that's okay. However, bottlenecked on ideas is interesting because this has been long talked about where even though AI is incredibly good at doing research, at writing code, at verifying experiments, the one thing that it really hasn't been great at is discovering a research direction that has a lot of promise.
That is possibly still the realm of humans, at least. for the short term. But I'm going to come back to that because meter shows that there are some hints that these models are actually able to generate.
meaningful research direction hypotheses, I would definitely endorse greater than two years of progress in one year. And I don't think four is crazy. So again, they're saying two to four X improvement.
And so there's just a bunch of quotes just like this. And here they talk about AI being able to make meaningful contributions to new research directions. Third, we understand that AI is not yet able to make highly significant and fully autonomous contributions to the most important parts of the.
the model training process. However, researchers described significant signs of life implying such contributions could occur soon. And then you start to think about some of the math research and proofs that have come out of open AI recently.
That is more or less net new knowledge to humanity. And so I do think these models given enough compute can actually just test everything. Like maybe they have some intuition on research direction, but with enough compute, they don't actually need to have much intuition.
Like the intuition will help save money, save compute. But ultimately, if you could just go out and test everything like that's. the same thing, why not?
And again, they're doubling down on basically not seeing any other bottlenecks between now and RSI. We did not see evidence for significant bottlenecks in algorithmic progress, training compute, experiment compute, or data that will take effect in the next 12 months. Anthropic redacted our analysis of a number of specific bottlenecks.
All right, so how did this program actually work? How did Meter get access and what did that access look like? Well, first, they only sent two and a half full -time equivalent employees into Anthropix.
And this engagement started in mid -August, right around the time that the whole Pace AI thing started happening. And they wanted to answer three specific questions. One, has the growth rate of Anthropix internal capabilities accelerated?
What share of any acceleration is attributable to AI? And are there important bottlenecks that could slow growth or limit acceleration in the near term? and on all three of those they got the answer that maybe all of us intuitively knew but now we have some proof yeah there we're accelerating and it is because of ai so meter had the ability to request access to any information we believed relevant to understanding ai r d acceleration however anthropic could unilaterally approve or reject our requests in practice we often negotiated with anthropic staff and compromised towards data that was informative not extremely sensitive and easy to share.
And again, this meter program with Anthropic is not the same thing that Anthropic said they were going to do with giving employee like access. to their systems so we're still waiting on that and they specifically called it out here note that neither meter nor anthropic intended for this project to involve full employee -like access and specifically this project is not the one described in anthropic's october 15th update on independent evaluators now one part of this program meter was originally going to get access to understanding the compute allocation that anthropic decides that means how much is going towards r d how much is going towards inference serving Claude to all of us and how much is going into whatever else they allocate compute towards.
However, Anthropic decided mid -program to revoke that access. And every time I hear Anthropic is revoking access or something like that, I think about their IPO because. Maybe they were just nervous, hey, if this information comes out, it might delay our IPO.
We've already put out our S1, kind of lining up the dominoes to reaching the IPO. So it is very interesting to see and to think about what some of those internal conversations may have been like. So the contracted scope of this project originally included a verification of Anthropix compute allocation, especially the allocation between capabilities and safety.
However, Anthropix changed the scope of this project midway to exclude compute verification and instead just focus on AI R &D acceleration. This compute was canceled before meter came to any substance. conclusions on compute usage at Anthropic.
Anthropic stated this was due to wanting to keep the engagement focused on risk assessment for the time being and that it would revisit compute allocation verification later on. Interpret that. how you will all right so here are a few things from the appendices that really stood out to me so one new internal models these are the models that we don't know about yet they are using them to accelerate their own development before they give it to anybody else those new internal models make a permanent slope change look more plausible than a one -time jump at mythos preview mythos preview was a massive improvement But what they are saying is that that massive improvement was not specifically because there's something special about mythos.
No, no. It is simply because of that compounding effect of the acceleration of AI R &D. And they even were able to see it with internal unreleased models.
Look at this. This is model three, model four. Mythos five was back here.
They have models that we don't even know about yet. Next, Anthropic has actually had to make their benchmarks more difficult because their new models are basically saturating them. They're too good.
They're too smart. And they just have to keep making those benchmarks more difficult. And researchers inside Anthropic are saying for the first time, these models are becoming actually useful for autonomous experimentation.
And so what does all of this tell us? I think it tells us what we already felt, which is, yeah, AI is Now, what makes me nervous is actually seeing a third party independent, well -regarded evaluator actually with access to anthropic internals saying, yes, this is happening and there's nothing in the way.
There is literally no blocker between what is happening now and what artificial super intelligence or recursive self -improvement looks like. And where I start to get especially nervous. is when we stop being able to understand AI completely.
And we already started to see some signs of that with the research paper that just came out this week, Weight Space Programming, where the frontier AI models are actually able to create new models by writing the weights directly. And I made an entire video about that. Go watch that right now.
The Hook
The bait, then the rug-pull.
Recursive self-improvement has already begun, says Berman, and the proof is a METR report written after the evaluator got deep access inside Anthropic. He reads it section by section and tells you why it makes him nervous.
How they asked for the click.
VERBAL ASK
19:50next-video
“Go watch that right now.”
A short pointer to the Weight Space Programming video at the end.
MENTIONED ON CAMERA
Visual structure at a glance.
