Modern Creator
Why Modern Creator?
Theo - t3․gg · YouTube

Oh No, the New Grok Model Is Actually Good

A day spent hammering Grok 4.5 inside Cursor turns a skeptical hook into a genuine benchmark scare for the rest of the frontier field.

Posted
2 days ago
Duration
Format
Review
educational
Views
129K
4.4K likes
Big Idea

The argument in one line.

Grok 4.5 is a genuine top-four coding model that beats Anthropic's Opus 4.8 and every Google model on real benchmarks at roughly a fifth of the price, marking the fastest competitive turnaround of any AI lab the reviewer has seen.

Who This Is For

Read if. Skip if.

READ IF YOU ARE…
  • You use Cursor, an AI coding agent, or a multi-model workflow and choose models partly on cost per task, not just raw capability.
  • You track frontier-model launches and want a skeptical, hands-on read instead of the vendor's own benchmark blog post.
  • You're deciding whether to add Grok to a coding stack that currently runs on Claude or GPT models.
SKIP IF…
  • You don't do AI-assisted coding — this is a developer-tooling benchmark breakdown, not general AI news.
  • You've already written xAI off and don't want a review that complicates that.
TL;DR

The full version, fast.

xAI's Grok 4.5 launched with bold claims of near-frontier coding performance at a fraction of the cost, and a full day of hands-on testing mostly backs it up: it scores fourth on the Artificial Analysis Intelligence Index, beats every Google model on coding benchmarks, and burns roughly a fifth to a third the tokens of Anthropic's Opus and Fable lines to hit a comparable score. Pricing is $2 per million input tokens and $6 per million output under a 200K-token window (doubling above that, capped at 500K), undercutting Fable 5 by 5-10x. In real use it handled multi-PR code review, follow-up corrections, and a screenshot-based bug fix cleanly without losing context, and it built a working 3D aquarium game from a single prompt — a result no prior model has matched. Its clear weak spot is orchestration: unlike Fable and GPT-5.5, it doesn't break work into coordinated sub-agent tasks, so it reads as the best version of the previous model generation rather than a true peer to the newest one.

Free for members

Chat with this breakdown — free.

Sign in and you get 23 free chat messages on us — ask for the hook, quote a framework, find the exact transcript moment, generate a markdown action plan. Bring your own key when you want unlimited.

Create a free account →
Chapters

Where the time goes.

00:00 – 01:06

01 · The claim: matching Fable 5 for a fraction of the price

Theo opens skeptical of xAI's launch claims, then shows the Artificial Analysis Coding Agent Index chart where Grok 4.5 sits neck-and-neck with GPT-5.5, just below Fable 5, and ahead of Opus 4.8.

01:06 – 03:09

02 · Sponsor: the button that doesn't notify you

A real personal frustration story about LG's broken 'Notify Me' button leads into Firecrawl's new page-monitoring feature as the sponsor read.

03:09 – 05:44

03 · xAI's official pitch: scale, training, one-shot builds

xAI's announcement covers DeepSWE placement (third, behind Fable and GPT-5.5), a jump to 1.5 trillion parameters signaling a full new base model, and one-shot app builds like a 3D solar system simulator.

05:44 – 07:11

04 · Cursor's numbers and the subsidy war

Cursor's own blog post adds Terminal-Bench 2.1 results and details the joint training deal, plus the subsidy economics that let labs offer $14,000 of inference for a $200 subscription.

07:11 – 09:23

05 · CursorBench: the chart Grok wins, then the asterisk

Grok 4.5 looks unbeatable on the CursorBench cost/score chart until xAI's own disclosure reveals a Cursor codebase snapshot leaked into training data, tainting the result.

09:23 – 10:52

06 · Pricing: $2 in, $6 out, until 200K tokens

Grok 4.5's headline pricing undercuts Fable 5 by 5-10x, but the price doubles above 200K tokens of context and caps at 500K — a structure Theo ties to xAI's GPU-reselling economics.

10:52 – 13:56

07 · Intelligence Index and startling token efficiency

Grok 4.5 ranks fourth on the Artificial Analysis Intelligence Index and uses a fraction of the tokens Opus and Fable need per coding task, driving its actual cost per task well below rivals.

13:56 – 15:53

08 · Where it doesn't win

On SkateBench, Grok 4.5 scores the lowest of any frontier lab while running relatively expensive, a reminder that no model wins every benchmark.

15:53 – 20:15

09 · Real test: auditing a launch checklist inside Cursor

Theo runs Grok 4.5 through a messy real-world workflow auditing his own product, Lakebed — multiple PRs, follow-up corrections, and a screenshot-based fix — and it holds context throughout.

20:15 – 22:02

10 · One prompt, a working 3D aquarium

Grok 4.5 turns Theo's 2D game FISHSLOP into a full 3D environment with modeled creatures from a single prompt, outperforming every other model he's tried for 3D generation.

22:02 – 24:43

11 · Verdict: the best of last-gen, not yet this-gen

Theo closes by arguing Grok 4.5 is a huge leap for xAI but still lacks the sub-agent orchestration of the newest model generation — the best PS2 game ever, with the PS3 already out.

Atomic Insights

Lines worth screenshotting.

  • Grok 4.5 scored fourth on the Artificial Analysis Intelligence Index, landing just behind GPT-5.5 and just ahead of Anthropic's Sonnet 5.
  • On coding-specific benchmarks, Grok 4.5 needed only 2 million tokens to reach a score that took Anthropic's Opus line 9.2 million tokens and its Fable line 7.2 million.
  • Grok 4.5 costs $2 per million input tokens and $6 per million output, versus roughly $10 in and $50 out for Fable 5 — a 5 to 10x price gap.
  • Crossing 200,000 tokens of context doubles Grok 4.5's price to $4 in and $12 out, and the model is capped at 500,000 tokens total.
  • xAI disclosed that an earlier snapshot of Cursor's real codebase was unintentionally included in Grok 4.5's training data, inflating its CursorBench score, and removed that data from future models.
  • Grok 4.5 averaged 31 cents per task across a benchmark suite where GLM 5.2 cost 37 cents and Fable 5 cost $2.75 for the same work.
  • Cursor's own subsidy math shows a user can pay OpenAI $200 directly and unlock up to $14,000 of inference through a partner-discounted plan — the kind of subsidy this Grok deal finally gives Cursor to offer.
  • Grok 4.5 was trained at 1.5 trillion parameters, roughly three times the ~500 billion parameter count of the previous Grok generation, indicating a full new base model rather than an incremental update.
  • Grok 4.5 averaged 2,100 tokens per response on reasoning-heavy benchmarks, one of the heaviest reasoning footprints tested, yet still landed cheaper per task than several lighter competitors.
  • In a full day of real coding work — auditing a launch checklist, splitting follow-up PRs, fixing a bug from a pasted screenshot — Grok 4.5 tracked instructions across a long, messy conversation without losing the thread.
  • Grok 4.5 built a full 3D aquarium game with modeled creatures and environment geometry from a single prompt, a result the reviewer says beat every other model he's tested for 3D generation, including Fable.
  • Grok 4.5's clearest gap isn't raw intelligence — it's orchestration: it doesn't spin up and coordinate sub-agents on large work the way the newest generation of models (Fable, GPT-5.5) does.
  • Terminal-Bench 2.1 placed Grok 4.5 at 83.3%, essentially tied with GPT-5.5's 83.4% and just under Fable's 84.3%.
  • GLM 5.2 costs roughly $1.10 per million input tokens and $4.40 output on standard providers, yet Grok 4.5 beat it on the Intelligence Index despite GLM's heavier token usage.
  • The reviewer had predicted in April that xAI could stage a comeback within six months to a year — Grok 4.5 arrived faster than that best-case estimate.
Takeaway

Grok 4.5 nails cost and coding benchmarks but hasn't caught up on orchestration.

WHAT TO LEARN

A model can out-benchmark and out-price the frontier on raw coding tasks while still trailing the newest generation on the one skill — coordinating sub-agents across long, messy work — that actually decides whether a workflow scales.

01The claim: matching Fable 5 for a fraction of the price
  • A single benchmark you already trust is a faster reality check on a launch's bold claims than reading the vendor's own marketing post.
  • A model can be 'neck and neck' with the top of the field on one index while still ranking behind it on price and token use — check both numbers, not just the score.
03xAI's official pitch: scale, training, one-shot builds
  • A parameter count jump this large (roughly 500 billion to 1.5 trillion) is a real signal of a full new base model, not a fine-tune of the previous generation.
  • One-shot build demos are a decent proxy for how much scaffolding a model needs before you'd trust it with real work.
04Cursor's numbers and the subsidy war
  • When a subscription price looks too good to be true, check the subsidy math: paying a lab directly can cost far more than the same usage unlocked through a partner's discounted plan.
  • A joint-training deal with a company that has real usage data can move a model's benchmark rank faster than more raw compute alone.
05CursorBench: the chart Grok wins, then the asterisk
  • A benchmark chart that looks too good relative to the rest of the field is worth scrolling past the headline number for the fine print.
  • A lab disclosing that a benchmark was contaminated by training data, instead of quietly leaving it up, is a meaningfully different trust signal than staying silent.
06Pricing: $2 in, $6 out, until 200K tokens
  • Compare full pricing structures, not just headline per-token rates — a price that doubles above a context threshold changes the real cost of long sessions.
  • Token efficiency can matter more than the sticker price per token when comparing total cost across models.
07Intelligence Index and startling token efficiency
  • A model landing fourth on an overall intelligence index can still be the most cost-efficient option if its token usage per task is dramatically lower than higher-ranked competitors.
  • Cost-per-task comparisons reveal price gaps that per-million-token pricing alone hides.
08Where it doesn't win
  • No single model wins every benchmark — check the specific one that matches your workload rather than trusting one overall score.
  • A model that reasons longer per response isn't automatically worse value if its per-task cost still comes out lower.
09Real test: auditing a launch checklist inside Cursor
  • The everyday test that matters isn't a benchmark score — it's whether a model can track a long, messy back-and-forth without losing the thread.
  • A model's willingness to say a task is only partially done, and list exactly what's still open, is more useful than one that claims false completion.
10One prompt, a working 3D aquarium
  • A model's advertised specialty doesn't predict its performance in an adjacent domain — it's worth testing capabilities outside the marketed use case.
  • Being clearly ahead of every other model tested on a task doesn't mean the output is production-ready; 'better than anything else available' and 'good enough to ship' are different bars.
11Verdict: the best of last-gen, not yet this-gen
  • The trait currently separating 'this generation' models from 'last generation' ones is orchestration — breaking large work into sub-agent tasks — not raw benchmark intelligence.
  • A lab can make the single biggest generational leap of any competitor and still be trailing the frontier by one full capability tier.
Glossary

Terms worth knowing.

Artificial Analysis Coding Agent Index
A composite benchmark from Artificial Analysis that averages a model's performance across coding-specific tests like DeepSWE and Terminal-Bench into one comparable score.
CursorBench
Cursor's internal benchmark suite built from real, ambiguous, multi-file tasks pulled from actual user sessions inside the Cursor editor.
DeepSWE
A software-engineering benchmark that measures how a model performs on realistic, agentic coding tasks rather than isolated code snippets.
Token efficiency
How many tokens a model needs to consume to complete a task. A model that reaches the same score with fewer tokens costs less to run even at a higher per-token price.
Mixture of experts
A model architecture that routes each request to a subset of specialized internal sub-networks instead of using the full model every time, which can lower compute cost per query.
Context window
The maximum amount of text, measured in tokens, a model can consider at once within a single conversation or task.
Cached tokens
Previously processed input tokens a provider can reuse instead of reprocessing from scratch, typically billed at a steep discount versus fresh input tokens.
Resources

Things they pointed at.

01:06productLG UltraGear 52G930B / 32GX870B monitors
02:10toolFirecrawl ↗
05:44linkxAI Grok 4.5 launch blog
11:14toolArtificial Analysis Coding Agent Index / Intelligence Index ↗
15:53productLakebed (the reviewer's own cloud product, used as the real-world test case)
20:15productFISHSLOP (the reviewer's own 2D game, rebuilt in 3D during the demo)
Quotables

Lines you could clip.

08:17
“Grok 4.5 High is looking insane by this chart, even crushing out GPT.”
the exact beat right before the benchmark twist lands→ TikTok hook↗ Tweet quote
13:56
“They went from in the thousands on the human baseline to 1,543, leapfrogging across multiple labs' entire last decade.”
a concrete, quotable stat about the size of xAI's jump→ newsletter pull-quote↗ Tweet quote
23:23
“They just put out the best PS two game ever, but the PS three's been out for two months.”
a tight analogy that reframes the whole review in one line→ IG reel cold open↗ Tweet quote
The Script

Word for word.

Read-along

Don't just watch it. Burn it in.

See every word as it's spoken — crank it to 2× and still catch all of it. The same dual-channel trick behind Amazon's Kindle + Audible.

metaphoranalogy
00:00A new model just dropped, and its creators are making some very bold claims. Specifically, they're saying that for dev work, it should compare to models like Fable five at a fraction of the cost. The model is Grok five from SpaceX AI, now partnered with Cursor, And I was a little skeptical after hearing this, but it turns out I was actually testing it.
00:18Over the last twenty four hours, Cursor gave me early access to a new model that I thought was gonna be a new composer, and I was pretty impressed with it. I learned today that that model was actually Grok 4.5, and now that I'm seeing the benchmarks, yeah, it's a pretty damn good model.
00:32This is from the artificial analysis code index, which combines a handful of benches that I actually like and trust. And according to this bench, Grok four five is neck in neck with GPT five five and just barely below Fable, while also beating out Opus four eight.
00:47While I think these numbers might be a little bold based on my experience using it, they're not that far off. Grok 4.5 has been genuinely impressive and it has a few things that are truly novel to it that I never would have guessed. And at its current price, it's a steal, especially with the 50% discount they're currently offering for people using it through tools like Cursor.
01:06I wanna break down all of the good, the bad, and the ugly with this model, but first, a quick word from today's sponsor. Normally, ads are showcasing all the cool things AI can do. I'm gonna do something a little different here.
01:16I wanna show you something AI failed hard at with me. LG, notoriously wonderful at naming, announced a monitor I've been really excited about since January that still hasn't come out yet.
01:26This monitor has a bunch of cool tech that I'm excited about. That's not what I'm here to talk about though. I'm here to talk about my attempts to get it.
01:32When you click the notify me button, it doesn't notify you. It launches a broken JavaScript thing, which in other browsers lets you sign up for notifications. And it doesn't even have the monitor I want in the options.
01:46So I have no way of knowing when this monitor comes out. So I did what any nerd would do. I asked my Hermes agent to monitor it and let me know when it comes out.
01:54And it did every single day because it was waiting for content on the page to change and the content was changing but never changing in the ways that mattered and my AI was not smart enough to realize that the reads it was getting were not actual changed content on the page. All I wanted to know is when this monitor came out for sale.
02:13Today's sponsor is Firecrawl and they make it way easier for your agents to scrape the web. That in and of itself would have been really useful for this type of thing. If it translates the page to markdown, it's more likely to notice when things have changed.
02:23It can get the page in markdown, it can get a screenshot of the page, or it can get the plain text. Super useful. But what it can also do is a new feature they just added called monitoring, where you schedule a recurring check to detect changes on a page.
02:37This makes this exact task trivial, and not only is it easy, it falls entirely under the free tier. And even if it didn't fall under the free tier, I could have just forked it and ran it myself because they're open source.
02:48Fire crawl has pretty much everything your agents need to scrape the web. From search, to URL specific readouts, to interactions, to monitoring, to an MCP server to make it way easier to connect, that doesn't even need an API key so it's trivial to set up.
03:00I can see why these guys are hiring right now. They're clearly doing really well. All of this is so useful.
03:05And if you wanna see it yourself, check them out now at swaydiv.linkfirecrawl. As I mentioned before, I've been using this model for a ton of real work as well as some silly demos like making three d games and whatnot. I've been going at it nonstop for the last twenty four ish hours and I have a lot of thoughts.
03:20But first, I wanna start with the official reporting. Introducing Grok 4.5. It's SpaceX AI's smartest model built for coding, agentic tasks, and knowledge work.
03:28Let's talk about what we care about, real world engineering excellence. Grok four five was trained on datasets spanning knowledge in coding, science, engineering, and math. With both intelligent and efficient reasoning, Grok 4.5 excels at real engineering tasks and it exceeds comparable leading models at many of these tasks.
03:45So DeepSWE, which is the software bench that I've been repping pretty hard, I find to be a very reasonable benchmark measuring how work actually happens with agents. Obviously, Fable is still number one.
03:56DVT 5.5 is still number two. We don't have numbers for 5.6 yet. Excited to see those.
04:00But for now, third place is Grok. Massively beating out any Google model. I don't think any are even referenced here.
04:06Opus four eight is falling a decent bit behind, and then four seven is way lower. This is a new, like, frontier tier that we are seeing happen, and Grok four five is on the line between last gen and this gen in a lot of ways. 4.5 was trained across tens of thousands of GB 300 GPUs, which is the newest technology from NVIDIA.
04:25Training and stability techniques designed for large scale runs beyond raw token volume, we invested heavily in data filtering and curation, deduplication, quality scoring, and domain focus selection, so the data mixture stayed high coverage and high signal. That is interesting.
04:39This model does seem to be a full new base like pre training rather than an adjustment on previous Grok models. The biggest indicator of that is that they mentioned it's a 1.5 trill per m model, where previous ones were only 500 bill per m. So clearly a whole new model here.
04:54Their RL training covered hundreds of thousands of tasks centered on multi step software engineering and other technical work with automated and model based grading. Our stack is built for highly asynchronous training so rollouts can run for many hours while learning continues across tens of thousands of GPUs. The result is more intelligent and efficient reasoning on real engineering and agentic tasks.
05:15They give some examples of one shotted tasks that they have the model make, and it's impressive. It does three d particularly well, which I will be sure to talk about in a bit. It's also quite fast, both because it uses not too many tokens, and because it is running faster in general on the infra they're serving it on, at roughly 80 per second.
05:34Cursor, who is now part of SpaceX AI, has their own article about this drop, and it has a couple more interesting details I wanted to jump on. First off, they have more benchmarks, including Terminal Bench 2.1, where it performed just barely below GPT five five and pretty close to Fable with an 83.3 versus eighty three point four and eighty four point three respectively.
05:53SWE bench, which I don't care about, it's a bad benchmark, so I'll skip it. And then deep SWE, which we mentioned before, it's doing very well on. SWE bench pro, which again, bad benchmark, I don't care.
06:02Cursor subscription plans for individuals and teams include significant usage of the model with double usage for the first week. This is the other exciting thing. One of the problems Cursor has as a business is that they struggle to compete with the labs just doing this crazy subsidization, because you can pay OpenAI $200, and then get up to $14,000 of inference, not even counting resets.
06:22That's hard for them to compete on. They've won on Enterprise still because the API rates are what the enterprises often have to pay. They don't get that crazy subsidization from the labs.
06:31But winning on individuals has been tough, and this is a huge win for them there because they finally have models they can kind of subsidize. Not that they have to though because the price is really good. We haven't got there yet, but we will in just a bit.
06:42As they mentioned, Grok four point is a mixture of experts model they trained jointly with SpaceX AI. So this is a model that was still a Grok model, still SpaceX AI focused, but they came in and trained it jointly, bringing data as well as their own processes. Training included trillions of tokens of cursor data, which capture a wide range of user interactions with code bases and software tools.
07:02This dataset lets the model learn both from existing software as well as developer agent interactions, capturing how developers work and how agents interact with their environments. I've seen a lot of good examples here, which we will definitely showcase. When they trained Composer 2.5 to be a coding specialist, Grok four point five kept the training data intentionally mixed and more broad.
07:22This involved drawing on high quality stem tasks, research papers, and other knowledge work so that the model gained proficiency across a wide range of domains. They made a bunch of difficult RL problems for the model because even a lot of the traditionally hard problems are now trivial for models. And in RL, they wanna have really, really difficult stuff.
07:40And I think that's why it's benching so well. It can get, it can just go on those types of big, bold, hard tasks. You might have noticed one bench missing from here though.
07:49Cursor bench. That is certainly not because it performed poorly. As we see here, the top right being the cheapest and the best.
07:57It performed comparable to Fable five High for a significantly lower price, where Fable five High cost $8.77 per task, and Grok 4.5 costs a dollar and 51¢ for a slightly higher score at that tier.
08:11Obviously, the best is still Fable on Max, even though it's double the price of Fable on High. But yeah, Grok 4.5 High is looking insane by this chart, even crushing out GPT.
08:24But how? Like, can't possibly be that good, right? Well, you scroll a little, you see why they did not include this information.
08:31Grok 4.5 has an advantage on CursorBench. An earlier snapshot of the cursor code base was unintentionally included in training.
08:39The exact score impact is unclear. Data has been removed from future models. For a rundown of third party benchmark scores, see the Grok 4.5 launch blog.
08:48Yep. They accidentally put Cursor's actual code in the training data, and since CursorBench is based on real problems that they have in Cursor and working on Cursor, this bench is now kind of tainted, at the very least, in the Grok world.
09:03It's a shame, because I like this bench, but mistakes happen. I'm happy they were transparent about it, and that they're not advertising Grok 4.5 via this benchmark publicly, which other labs may have done similar things too.
09:16They're being straightforward and transparent with it. I appreciate them for that. Now I wanna talk about the price.
09:22Grok 4.5 is $2 per million tokens in and $6 per million tokens out.
09:29That makes it comically cheaper than a lot of competing models. For example, Fable is $10 per million tokens in and 50 per million tokens out, between five and almost 10 x the cost. And that ignores the fact that Fable's relatively token hungry, and this model seems to be less so.
09:46Hard to know for sure until we've really put it through its paces, but based on all the benches I've seen and all the work I've done with it, it's relatively efficient. It is also worth noting that this price only applies under 200,000 tokens of context.
09:59If you go over, the prices double to $4 per million tokens in and 12 per million tokens out. Still way cheaper than any other model at this tier, but it only goes at the 500 k tokens.
10:11It's kind of weird to have a model that can go over 200 k and charges more, but is still under a mil. Specifically, like, 200 k to 500 k is not that much more context, and to build twice as much for it, especially to build twice as much on output, feels a little much to me.
10:28I feel like they're reaching a little here. My guess is the reason they did that is they wanted to get the input and output token costs for the base tier 200 k version as cheap as possible, and the GPUs that this is running on are also being resold to companies like Anthropic and Google with massive markups. They have to make sure that it's priced in a way where they're not losing too much money that could have been made from reselling GPUs, but at the same time, is priced cheaply enough to actually compete with those labs.
10:57SpaceX AI has put themselves in a weird spot here, but I think they navigated okay according to everything I've been seeing and all the use I've been having so far. Let's go over the artificial analysis numbers, and then I will dive into my experience. SpaceX AI's Grok four five scored a 54, which places it fourth in the artificial analysis intelligence index.
11:14Kinda wild to see a different color in the top again. It's been a very long time since I saw purple all the way up here. It is right behind GPT 5.5 and just ahead of Sonnet five based on the intelligence index, crushing GLM 5.2, which is even more interesting when you realize how expensive GLM 5.2 is to run.
11:32GLM five two's base price on a lot of providers is a dollar 10 in roughly, and $4.40 out roughly. There are some providers that offer quite a bit cheaper now, like, Noveda has a temporary 60% off, DeepInvra has it at like $3 ish per mil out.
11:49When you remember how much more token hungry GLM five two is, you realize it's kind of been crushed by Grok four five. Here we could see the actual costs incurred per task average across the entire suite, and Kimi k two six was about 35¢ per task, GLM five two is about 37¢ per task, and then Grok four five which scored way higher than those other models was only 31¢ per task.
12:13And for reference, Fable five was $2.75 for the same work. Yeah.
12:19It's an efficient model. And if you look at the intelligence versus cost chart, you see it is very well positioned. It is just on the edge of the green box, you know, the the good spot that almost nothing is in.
12:30Both Grok 4.5 and Gemini 3.1 Pro score very well here, and Grok 4.5 is meaningfully more intelligent while also being in this price range. This gets much crazier with the coding focused benches though, because Grok came out swinging here, crushing every Google model by a large margin. And if we look at the token usage per task for the code work, you'll see Opus and Fable both massive token hogs at 7.2 mil for Fable, and 9.2 mil for Opus.
12:58Buy five on x high is still in the 6 mil range or so. Buy five on medium was only 3.5 mil tokens, but all the way at the end here, Grok build with Grok 4.5, only 2 mil tokens to get that high of a score.
13:10That is an insane level of token efficiency. I never would have guessed that this model would be so efficient, but it really is, and the result is that it feels way faster to use, and the bill ends up being cheaper than you might have expected. It kinda makes this a great go to default code model that you bring other things in to clean up after if you use it, and it doesn't do quite what you want.
13:30Artificial analysis also calls out that it does very well on agentic tasks, things that are multi step, where it has to call tools and synthesize information. One of the most cost efficient models to run for near frontier intelligence. Yep, it is insanely cheap.
13:42The token efficiency combined with the low price is what makes it so compelling. As a coding agent, Grok 4.5 and Grok build is on par with 5.5 and offers efficiency benefits. Still insane, because 5.5 was such an efficient model, getting more efficient is just unbelievable.
13:56And when you see how big of a jump this is for them, it's kind of insane. They went from in the thousands on the human baseline to 1,543, leapfrogging across multiple labs entire like last decade.
14:09It's crazy how far they have jumped on all of these things. And benches like Tau two, they are now number one in the world in. Just crazy, crazy leap.
14:18And it really shows the benefit that Cursor brings xAI in being part of the business. It seems like Cursor's combination of like data and RL process has been incredibly beneficial to x so far.
14:30That does not mean it scores well in everything though. For example, in SkateBench, it ended up being quite expensive because it kept thinking and reasoning trying to figure out the tricks, and it still only got a 76%, which is the lowest score from a Frontier Lab on the max reasoning settings, while also being relatively expensive at 1.3¢ per run.
14:49Not as bad as something like Sonnet five, which costs way, way, way, way more than it should have at 15¢ per run. Literally 10 x the cost for a lower score. When you compare it to something like Gemini three one Pro preview, it ended up being a little over half the price, but a meaningfully lower score.
15:07Yeah. I was hoping it would be a little cheaper here, but it does seem very determined to get answers as it averaged at 2,100 tokens per response, making it one of the most heavy reasoning models I have used here where it really thought before giving an answer.
15:21All of this said, benchmarks are benchmarks, and a lot of them don't measure how it feels to use the model in the real world. For example, a lot of the Google models score great and when I use them for code, they just don't feel particularly good to use. So how has Grok 4.5 been in my usage?
15:38I'll be frank, I'm impressed. I'm working hard on my new cloud product, Lakebed, and I'm really close to shipping. I wanted to spend a lot of time the last few days doing a big pass, auditing it, finding any potential issues, whether it's security, maintenance problems, things that aren't great to have in an open source project, that type of stuff.
15:55I had to do an audit and it did a pretty good job, but found most of the things that Fable and GPT five six found. However, it did a great job when I asked it to start fixing the things.
16:06And when I noticed how well it was doing, I started to push it a little hard. I opened with working on hardening Lakebed for its first public release. I have the following PR up, which addresses the majority of the remaining issues with the link to the PR.
16:17Here's the report for the majority of those issues with a post plan link. Have we resolved them? Are there other things worth solving before launch?
16:24An issue with the Cloud environments, I didn't even set up yet, and I was doing this all remote. I think I was actually doing it from my phone if I recall. So I said all of that, it went through the PR, it said that PR 92 does not close the report, it closes two hard stops cleanly and one only partially.
16:38Most of the launch gates are still open. It said explicitly what is resolved, gave some caveats, and then gave me a list of things that were still open and should not be treated as done.
16:48All of this is great. This is a really good way to process and synthesize and to give me this info. It feels very opus y, is how I would put it.
16:57So I end up having questions because there was a lot of text, so I went through it all. I called out a couple different pieces. I grabbed this section and I only cared about issues three and five because I had questions about what it meant by these things, and then after it had a different section with a lot more stuff.
17:13These types of things could be confusing for models because there was two lists that had a number three and a number five in them. So I give a tiny prefix of which list I'm referring to. I thought this might trip it up, because that's just a lot of context to get through, and I'll be real, a lot of like the open weight and frankly not frontier stuff tends to struggle once you get it to this point.
17:33And it did a great job. It addressed all of my concerns very well and very directly. It called out the split release thing and described what it meant, called out what it thinks I should put on SecurityMD, and then it went through all of these different issues that I had questions about and helped me prioritize them.
17:48I gave it a little bit more feedback on my thoughts there, but specifically here is where it gets fun. I called out I don't care about the token URL issue, but issue two, which was bound public work.
17:58I said I like their ideas for it. Make a separate PR that addresses all of the concerns it raised. Then I asked it to do an investigation for this part, and then I asked it to hold on to other things.
18:09It made two PRs because there was two issues it was told to address, and it addressed both of them and did a great job. It also answered my questions. Remember, this this is a lot of different pieces I asked for here.
18:21I said to do a PR for splitting up the actions earlier and a separate PR here for the public work bindings that it was mentioning. And it was able to, in just one run, make both of those PRs and also address my other questions and give me a to do list on all the things I have to go do before we go live.
18:39I noticed there were some comments on the PR, so I asked it to look. I just said both PRs have review comments to address, and it addressed them all. I then used Cursor's built in babysit skill to monitor both of the PRs and continue addressing things that come up on them, and it succeeded.
18:53I then asked another agent about these PRs, and they said a couple different things should be fixed. So I literally just screenshotted it, both to test its visual capabilities, but also out of laziness, pasted the screenshot and said, you make these changes to 93? And it did.
19:07This is great. This is the type of back and forth and complex multi target tasks and work that you could make most models do if you break them up really carefully and cleanly and try to not bloat the context.
19:21Even a model like Five Five, in my experience, would get confused at some point during this, and get too fixated on something from three messages ago instead of doing what I'm asking now. I had no ergonomic issues with Four Five here at all. It was actually very pleasant to work with.
19:36I kept making my responses like worse and worse almost intentionally just to see where it would stumble, and it didn't.
19:43Did it find everything Fable found? No. Was its code as thorough as 56?
19:49No. Was it able to go back and forth with me on really big heavy tasks and be pleasant to use while also being very fast and cheap? Absolutely, yeah.
19:59In a lot of ways, I see this model as a good alternative to something like OPUS four eight, and it kind of shits all over something like, I don't know, GLM five two. I have no interest in that model anymore, other than the fact that it's open weight, which is genuinely really cool and I appreciate them for that. But Grok four five is a weirdly good default code model.
20:18So I wanted to push further, and I did. I did a game I built previously, my little fish slop thing, and I asked it to make the game in three d. It did, and it has some issues, like this layout is broken because of the model.
20:30I didn't choose for it to be panned hard to the side like this. Wait till you see what happens when we open the game. It successfully made a full three d environment, modeled all of the different creatures itself, as well as all of the geometry of the things on the bottom of the tank.
20:46And it did a way better job with that than any other model I've used, even models like Fable and five six, which I have given access to three d modeling tools as well, this crushed all of them.
20:58Obviously, the models are far from great, especially the creature models and the model for the submarine. And it also got the control super wrong, where a goes right and d goes left for some reason.
21:11They, yeah, they screwed these things up. And it also just doesn't control very cleanly.
21:17I gave it a follow-up telling it to fix it, and it fixed the pointer and the clicking, but it didn't fix a lot of the rest. One of the models I was really impressed with in here though was the enemy model for the aliens that invade the tank. It did a very good job with that.
21:31But yeah, I did not expect this to have three-dimensional taste. And I'll say outright, a lot of these, while far from like, I would ship this confidently tier, are so far ahead anything I've gotten from any of the other models that I can confident say this is the first model to be almost decent at three d modeling in game engines like Three.
21:53Js. Take it as you will, this is not bad. I I am more impressed than I expected to be regarding a Grok model's three d capabilities.
22:02I do have one last thought I wanna discuss here, which is this idea of model generations. I do really feel like Fable's a new generation of model, and I have a lot of thoughts on where g b d five six falls here. Video probably coming tomorrow, depending on a lot.
22:17We'll see. The reason I bring this up is because Fable and Five Six, in many ways, feel very different. And as silly as it is, even something like Sonnet Five has a bit of that feeling too.
22:28The difference is in the model's ability orchestrate. They can step up a level and prompt sub agents and orchestrate big work into lots of smaller chunks to go longer, do more, and complete more difficult tasks.
22:41This is why I got so addicted both to Fable and the g p t five six. I don't necessarily see that capability in Grok 4.5. My attempts to break up sub agents with it were admittedly limited, but I was not super impressed.
22:54It didn't seem to have the same nuance in how it would break work up and delegate it, and it would often get stuck as a result of a certain process it ran hanging and then not knowing how to clean up after. Some of this is harness specific, some of this is model specific, some of this is just being behind. But would say in many ways, what they built with Croc 4.5 is less a comparable model to this new generation with things like Fable and GPD five six.
23:19More it's really, really impressive what they did with the previous technology. If you were to think of this in gaming, for example, they just put out the best PS two game ever, but the PS three's been out for two months. And as such, I'm very excited to see if they can level up to a PS three generation game, and I think they can.
23:37They have all of the pieces, and the amount that they just jumped in such a short window is truly a sight to behold. I don't think any lab has had a jump like this ever other than it may be arguably like deep seek. Going from forgotten in the conversation and just reselling GPUs to beating out practically everyone else in the space at the tier you're competing in for cheaper and more efficient models, it's impressive.
24:03And we shouldn't be sleeping on SpaceX anymore. I posted in April that I legitimately believed x a I could have a crazy comeback. I thought it would take six months to a year.
24:12I didn't think it would take two and a half. I am blown away. These guys are cooking again.
24:17I have no idea where this will all end up, but I'm thankful somebody other than Google is actually competing now. This is the first real player that Anthropic and OpenAI have had to be scared of in quite a while, and I hope they are. I hope the result of Grok four five is faster, cheaper, and smarter models for everyone because that's what we want in the end.
24:34So congratulations to SpaceX AI for catching up. I didn't think you had it in you, but clearly you do. I can't wait to see what comes out next.
24:41And until next time, peace
The Hook

The bait, then the rug-pull.

The title reads like a joke — dry sarcasm from a reviewer bracing for another over-hyped launch post. The opening minute plays it that way too, right up until the actual benchmark chart shows up, and the joke stops being a joke.

CTA Breakdown

How they asked for the click.

VERBAL ASK
01:06link
“check them out now at soydev.link/firecrawl”

Opens with a real personal failure story — a broken LG 'Notify Me' button — before pivoting to Firecrawl's monitoring feature as the actual fix, which makes the sponsor read feel earned rather than bolted on.

MENTIONED ON CAMERA
02:10toolFirecrawl ↗
11:14toolArtificial Analysis Coding Agent Index / Intelligence Index ↗
FROM THE DESCRIPTION
PRIMARY CTAWhere the creator wants you to go next.
OTHER LINKSAlso linked in the description.
Storyboard

Visual structure at a glance.

cold open
hookcold open00:01
sponsor: LG's broken notify button
ctasponsor: LG's broken notify button01:45
xAI's one-shot build examples
valuexAI's one-shot build examples05:24
CursorBench chart
valueCursorBench chart07:52
the contamination admission
valuethe contamination admission08:48
pricing at a glance
valuepricing at a glance09:25
Artificial Analysis Intelligence Index
valueArtificial Analysis Intelligence Index11:14
auditing Lakebed's launch PRs in Cursor
valueauditing Lakebed's launch PRs in Cursor19:00
FISHSLOP in 3D
valueFISHSLOP in 3D20:32
closing verdict
ctaclosing verdict24:35
Frame Gallery

Visual moments.

cold open00:01Frame at 00:28 from Oh No, the New Grok Model Is Actually Good00:28Frame at 00:47 from Oh No, the New Grok Model Is Actually Good00:47Frame at 01:12 from Oh No, the New Grok Model Is Actually Good01:12Frame at 01:18 from Oh No, the New Grok Model Is Actually Good01:18sponsor: LG's broken notify button01:38
cold open00:01Frame at 00:28 from Oh No, the New Grok Model Is Actually Good00:28Frame at 00:47 from Oh No, the New Grok Model Is Actually Good00:47Frame at 01:12 from Oh No, the New Grok Model Is Actually Good01:12Frame at 01:18 from Oh No, the New Grok Model Is Actually Good01:18sponsor: LG's broken notify button01:38Frame at 02:00 from Oh No, the New Grok Model Is Actually Good02:00Frame at 02:13 from Oh No, the New Grok Model Is Actually Good02:13Frame at 02:30 from Oh No, the New Grok Model Is Actually Good02:30Frame at 02:56 from Oh No, the New Grok Model Is Actually Good02:56Frame at 03:07 from Oh No, the New Grok Model Is Actually Good03:07Frame at 03:33 from Oh No, the New Grok Model Is Actually Good03:33Frame at 03:51 from Oh No, the New Grok Model Is Actually Good03:51Frame at 04:10 from Oh No, the New Grok Model Is Actually Good04:10Frame at 04:28 from Oh No, the New Grok Model Is Actually Good04:28Frame at 04:47 from Oh No, the New Grok Model Is Actually Good04:47Frame at 05:05 from Oh No, the New Grok Model Is Actually Good05:05xAI's one-shot build examples05:24Frame at 05:42 from Oh No, the New Grok Model Is Actually Good05:42Frame at 06:01 from Oh No, the New Grok Model Is Actually Good06:01Frame at 06:20 from Oh No, the New Grok Model Is Actually Good06:20Frame at 06:38 from Oh No, the New Grok Model Is Actually Good06:38Frame at 06:57 from Oh No, the New Grok Model Is Actually Good06:57Frame at 07:15 from Oh No, the New Grok Model Is Actually Good07:15Frame at 07:34 from Oh No, the New Grok Model Is Actually Good07:34CursorBench chart07:52Frame at 08:11 from Oh No, the New Grok Model Is Actually Good08:11Frame at 08:29 from Oh No, the New Grok Model Is Actually Good08:29the contamination admission08:48Frame at 09:06 from Oh No, the New Grok Model Is Actually Good09:06pricing at a glance09:25Frame at 09:43 from Oh No, the New Grok Model Is Actually Good09:43Frame at 10:02 from Oh No, the New Grok Model Is Actually Good10:02Frame at 10:21 from Oh No, the New Grok Model Is Actually Good10:21Frame at 10:39 from Oh No, the New Grok Model Is Actually Good10:39Frame at 10:58 from Oh No, the New Grok Model Is Actually Good10:58Frame at 11:14 from Oh No, the New Grok Model Is Actually Good11:14Artificial Analysis Intelligence Index11:28Frame at 11:49 from Oh No, the New Grok Model Is Actually Good11:49Frame at 12:12 from Oh No, the New Grok Model Is Actually Good12:12Frame at 12:39 from Oh No, the New Grok Model Is Actually Good12:39Frame at 12:49 from Oh No, the New Grok Model Is Actually Good12:49Frame at 13:07 from Oh No, the New Grok Model Is Actually Good13:07Frame at 13:30 from Oh No, the New Grok Model Is Actually Good13:30Frame at 13:44 from Oh No, the New Grok Model Is Actually Good13:44Frame at 13:56 from Oh No, the New Grok Model Is Actually Good13:56Frame at 14:16 from Oh No, the New Grok Model Is Actually Good14:16Frame at 14:40 from Oh No, the New Grok Model Is Actually Good14:40Frame at 14:59 from Oh No, the New Grok Model Is Actually Good14:59Frame at 15:17 from Oh No, the New Grok Model Is Actually Good15:17Frame at 15:36 from Oh No, the New Grok Model Is Actually Good15:36Frame at 15:54 from Oh No, the New Grok Model Is Actually Good15:54Frame at 16:13 from Oh No, the New Grok Model Is Actually Good16:13Frame at 16:31 from Oh No, the New Grok Model Is Actually Good16:31Frame at 16:50 from Oh No, the New Grok Model Is Actually Good16:50Frame at 17:08 from Oh No, the New Grok Model Is Actually Good17:08Frame at 17:27 from Oh No, the New Grok Model Is Actually Good17:27Frame at 17:45 from Oh No, the New Grok Model Is Actually Good17:45Frame at 18:04 from Oh No, the New Grok Model Is Actually Good18:04Frame at 18:22 from Oh No, the New Grok Model Is Actually Good18:22Frame at 18:41 from Oh No, the New Grok Model Is Actually Good18:41auditing Lakebed's launch PRs in Cursor19:00Frame at 19:18 from Oh No, the New Grok Model Is Actually Good19:18Frame at 19:37 from Oh No, the New Grok Model Is Actually Good19:37Frame at 19:55 from Oh No, the New Grok Model Is Actually Good19:55Frame at 20:14 from Oh No, the New Grok Model Is Actually Good20:14FISHSLOP in 3D20:32Frame at 20:51 from Oh No, the New Grok Model Is Actually Good20:51Frame at 21:09 from Oh No, the New Grok Model Is Actually Good21:09Frame at 21:28 from Oh No, the New Grok Model Is Actually Good21:28Frame at 21:46 from Oh No, the New Grok Model Is Actually Good21:46Frame at 22:02 from Oh No, the New Grok Model Is Actually Good22:02Frame at 22:23 from Oh No, the New Grok Model Is Actually Good22:23Frame at 22:42 from Oh No, the New Grok Model Is Actually Good22:42Frame at 23:01 from Oh No, the New Grok Model Is Actually Good23:01Frame at 23:19 from Oh No, the New Grok Model Is Actually Good23:19Frame at 23:38 from Oh No, the New Grok Model Is Actually Good23:38Frame at 23:56 from Oh No, the New Grok Model Is Actually Good23:56Frame at 24:15 from Oh No, the New Grok Model Is Actually Good24:15closing verdict24:35
Watch next

More from this channel + related breakdowns.

43:15
Theo - t3․gg · Tutorial

A proper guide to Fable 5

How Theo turned a returned, unmetered Claude release into a five-and-a-half-hour unattended agent run that cleared a month of stalled pull requests for about $150.

July 6th
23:24
Theo - t3․gg · Talking Head

You were lied to about Fable

A 23-minute rebuttal of three viral claims about Anthropic's returning Fable model — that it's nerfed, that its subscription pricing is a bait-and-switch, and that it's too expensive to run.

July 4th
Chat about this