36:00Theo - t3․gg · Review
GPT-5.6: The Review
Theo spends 36 minutes putting real numbers behind the GPT-5.6 hype — Sol, Terra, and Luna, benchmarked against Claude Fable, one blog chart at a time.
July 12thThe argument in one line.
Kimi K3, a 2.8-trillion-parameter open-weight model from Moonshot, has closed the gap with the best closed frontier models on coding, UI generation, and 3D work at roughly Sonnet-level pricing — but it ships with real usability rough edges and no built-in refusal behavior once its weights go public.
Read if. Skip if.
READ IF YOU ARE…
- You use AI coding agents daily (Claude Code, Codex, OpenCode) and are deciding whether a new model is worth switching to or adding to your rotation.
- You track AI economics — token pricing, subscription tiers, and how open-weight releases pressure closed-lab pricing.
- You're curious about frontier-adjacent open-weight models for coding, front-end UI generation, or long-running agentic workflows.
- You follow AI security and safety implications of increasingly capable models releasing with no usage restrictions.
SKIP IF…
- You're looking for a beginner tutorial on prompting or using AI tools for the first time.
- You want a model you can run on your own hardware — Kimi K3 needs supercomputer-scale infrastructure even at reduced precision.
- You need a fully polished, low-friction model today — Moonshot's own release notes admit real UX limitations.
The full version, fast.
Kimi K3 is Moonshot's new 2.8-trillion-parameter open-weight model, and after a full day of stress-testing, Theo finds it genuinely frontier-competitive — winning or tying frontier labs on coding, browser-agent, and front-end benchmarks, while pricing in at roughly Sonnet levels ($3 in / $15 out per million tokens). It handled a 3+ hour codebase port, generated a working 3D game and GBA emulator, redesigned a marketing site, and ran a 32-agent security audit that closed models often refuse. The catch: Moonshot's own release notes admit it's excessively proactive and has UX rough edges, it uses roughly twice the tokens of GPT-5.6 Sol to do the same work, and because it ships as open weights on July 27 with no built-in refusal behavior, its offensive security capability is now available to anyone.
Free for members
Chat with this breakdown — free.
Sign in and you get 23 free chat messages on us — ask for the hook, quote a framework, find the exact transcript moment, generate a markdown action plan. Bring your own key when you want unlimited.
Create a free account →Where the time goes.
00:00 – 01:17
01 · Cold Open: A Real Frontier Contender
Theo states his thesis up front: open-weight models had stalled behind Fable/GPT-5.6 Sol, but Kimi K3's benchmarks suggest that gap just closed. He previews the video (usage tips, security implications) and mentions his terminal froze from overuse.
01:17 – 03:13
02 · Sponsor Break: Depot
Mid-roll read for Depot — a CI engine and Docker build cache that speeds up GitHub Actions and Docker builds, usable from coding agents via CLI.
03:13 – 05:58
03 · Moonshot's Pitch: 2.8 Trillion Parameters
Theo reads Moonshot's own announcement: Kimi K3 is a 2.8T-parameter model with native vision and a 1M-token context window. He does the math on model size (2.8TB at FP8, 1.4TB at FP4) and bluntly debunks any claim that this could run as a 'local model.'
05:58 – 09:05
04 · Benchmark Gauntlet: SWE Benches to BrowseComp
Walks through DeepSWE, FrontierSWE, terminal bench, Program Bench, SWE Marathon, and general agent evals (GDPval, Jobbench, Spreadsheetbench). Kimi K3 lands close behind or ahead of GPT-5.6 Sol and Fable across most. BrowseComp stands out: Kimi K3 beats the field on both score and cost per task.
09:05 – 11:41
05 · Access, Pricing, and the July 27 Weights Release
Kimi K3 is live today via kimi.com, Kimi Work, Kimi Code, and the Kimi API — all Chinese-hosted, since the weights themselves don't release until July 27. Pricing: $0.30 cache hit, $3/mil in, $15/mil out — close to standard Sonnet pricing.
11:41 – 13:15
06 · Architecture: Kimi Delta Attention + Sparse MoE
Covers the two architectural bets behind K3 — Kimi Delta Attention and Attention Residuals — plus a sparse Mixture-of-Experts setup (16 of 896 experts active) that keeps a model this large computationally viable, yielding a claimed 2.5x gain in scaling efficiency.
13:15 – 16:36
07 · Long-Horizon Coding Test + Front-End Arena Score
Theo's real-world stress test: porting his old ping.gg codebase, a job that ran 3+ hours and 122 tasks off a single prompt before hitting a context-window ceiling (caused by a subscription limit, not the model itself). He also flags Kimi K3's 76% win rate on the Frontend Code Arena leaderboard.
16:36 – 18:27
08 · 3D Game Demos: GBA Emulator + Fish Slop Submarine
Kimi K3 builds a playable 3D Game Boy Advance emulator and, live on stream, ports Theo's 'Fish Slop' project into a working 3D submarine game complete with textures and sound effects — a capability Theo says every other model he's tried has badly fumbled.
18:27 – 19:53
09 · Chip Design, Knowledge Work, and Video Editing Claims
Briefer callouts: Moonshot claims strong chip-design and ASIC research assistance, leading knowledge-work benchmarks (GDPval, DeckBench, FinanceBench) with polished infographic/dashboard generation, and even automated video-editing capability Theo hasn't tested himself.
19:53 – 22:36
10 · Third-Party Validation: Artificial Analysis Scores It
Independent validation from Artificial Analysis: Kimi K3 ranks as the third-smartest model measured (intelligence index 57), just behind Fable and GPT-5.6 Sol, with a standout GDPval jump over the prior open-weight leader GLM 5.2.
22:36 – 25:43
11 · The Cost Reality Check + Moonshot's Own Disclosed Limitations
The $15/mil output price isn't actually cheap once you factor in that Kimi K3 burns roughly 2x the tokens of GPT-5.6 Sol on the same task. Moonshot's own docs admit a UX gap versus Fable/Sol, call out excessive proactivity, and flag sensitivity to thinking-history continuity across sessions.
25:43 – 29:21
12 · Front-End Showcase: Redesigning t3.gg and Liquid Glass
Theo has Kimi K3 generate five marketing-page redesigns for t3.gg (pill nav, glow/bento styles, an accidental 'cringe terminal' theme), then a genuinely-improved true-black sidebar redesign for his own product. A community example recreates macOS 27's Liquid Glass UI from scratch as a working web app.
29:21 – 33:00
13 · A Real PR, Then the Lakebed Security Audit Begins
Using OpenCode as the harness (Kimi isn't natively in Claude Code or Codex), Kimi K3 self-corrects a browser-access mistake, ships a real before/after PR, then Theo escalates to a much bigger test: a deep security audit of his production app, Lakebed.
33:00 – 36:39
14 · Security Audit Results + Self-Organizing Sub-Agent Workflows
Kimi K3 runs roughly 32 agents across discovery, verification, and synthesis phases to audit Lakebed for real vulnerabilities — a task Theo says Fable and Sol often refuse outright. He also notices it spontaneously restructures a to-do list into parallel workflows and lets sub-agents check off their own tasks, something he's never seen another model do.
36:39 – 41:35
15 · How to Use It Today: Subscriptions, API, and the Final Verdict
Practical breakdown of kimi.com subscription tiers ($20/$40/$100/$200) versus the platform.kimi.ai API, plus Theo's own cost tally (~$63 for a full day of hard testing). Closes with a prediction that this release should worry Anthropic's upcoming Opus pricing.
Lines worth screenshotting.
- Kimi K3 is a 2.8 trillion parameter open-weight model, nearly triple the size of the previous largest open model (Kimi K2, at 1 trillion params).
- At FP4 precision, Kimi K3's weights alone are roughly 1.4 terabytes — too large to run on any consumer hardware, despite hype calling open models 'local models.'
- Kimi K3 pricing is $3 per million input tokens and $15 per million output tokens, roughly matching Anthropic's standard (non-discounted) Sonnet pricing.
- Kimi K3 uses about twice as many output tokens as GPT-5.6 Sol to solve the same tasks, which cancels out most of its raw per-token price advantage.
- Moonshot's own release notes disclose that Kimi K3 is 'excessively proactive' and prone to making unrequested decisions during long agent runs — a limitation the company is warning users about in its own documentation.
- On Artificial Analysis's hallucination benchmark (AI Omniscience), Kimi K3 scored as one of the most honest open-weight models tested, while GPT-5.6 Sol scored lower because it more often guesses instead of admitting uncertainty.
- A community developer used a swarm of Kimi K3 agents to recreate macOS 27's Liquid Glass interface as a working web app, burning 60% of a monthly subscription allotment in the process.
- Kimi K3 completed a security audit across roughly 32 parallel agents (discovery, verification, and synthesis phases) on a real production codebase — a task category that competing closed models often refuse outright.
- Because Kimi K3 ships as open weights on July 27, the same offensive-security capability that makes it useful for defense work is now available to attackers with no built-in refusal behavior to stop them.
- Hosting Kimi K3 at usable speed requires a 'super node' of 64 or more high-end accelerators — an estimated $2.6 million in hardware just to run inference.
- Kimi K3 spontaneously broke a single coding task into multiple parallel workflows and let its own sub-agents check items off a shared to-do list, a coordination pattern the reviewer had not seen from any other model, including Fable.
- A full day of aggressive testing — including a 3+ hour continuous coding run — cost roughly $63 on Kimi K3, versus the reviewer's typical $300-400/day on Claude and up to $1,000/day on OpenAI models for comparable work.
- Historically, open-weight Kimi models have only ended up 11-15% cheaper on third-party hosting than Moonshot's own pricing, suggesting Kimi K3 won't get dramatically cheaper once its weights are public.
- Kimi K3 has native vision built in at launch, while rival open-weight model GLM 5.2 still lacks vision capability entirely.
- Kimi K3's reasoning is fully visible (not hidden server-side like Anthropic or OpenAI), which let the reviewer catch it wasting time reasoning about whether a coding rule applied inside a Markdown planning file.
Open weight caught up. Trust didn't.
OPEN WEIGHT SHIFT
A 2.8-trillion-parameter open-weight model now matches frontier coding and UI performance at roughly Sonnet-level pricing, which is genuinely useful for builders and genuinely alarming for AI safety once the weights ship publicly.
01Cold Open: A Real Frontier Contender
- A single benchmark chart isn't proof a model is good in practice — Theo only trusted Kimi K3 after a full day of real coding, UI, and agent work, not after reading Moonshot's own numbers.
- When a lab claims 'frontier-level' performance, check whether that's against the newest closed models or last generation's — Kimi K3 still trails GPT-5.6 Sol and Fable, it's just much closer than any open model before it.
03Moonshot's Pitch: 2.8 Trillion Parameters
- A model's raw parameter count tells you almost nothing about whether you can run it yourself — do the math on memory footprint (2.8TB at FP8, 1.4TB at FP4) before believing 'open weight' means 'runs on your machine.'
- Treat 'this can run locally' claims from a lab or hype cycle with suspicion until you've checked the actual hardware requirement — a 1.4-terabyte model needs a data center, not a desktop.
04Benchmark Gauntlet: SWE Benches to BrowseComp
- Benchmark scores that don't account for token usage overstate cost efficiency — a model can look cheaper per-token and still cost more per-task if it uses twice the tokens to get there.
- A model beating the field on both capability score and cost-per-task on the same chart (like Kimi K3 on BrowseComp) is a rarer and more meaningful signal than winning on capability alone.
05Access, Pricing, and the July 27 Weights Release
- When comparing a new model's price to an incumbent, use the price you're actually being charged (including temporary discounts), not the vendor's list price.
- A model that isn't hosted in your region yet is a data-handling decision, not just a pricing one — Theo explicitly avoids sending sensitive data through Chinese-hosted infrastructure until the weights are self-hostable.
06Architecture: Kimi Delta Attention + Sparse MoE
- A claimed 'X times more efficient' architecture number is only meaningful in context — 2.5x scaling efficiency matters because it's what let a 2.8T model train in a reasonable timeframe at all, not because bigger multipliers are inherently good.
- Sparse activation (using a small fraction of total experts per request) is what makes an enormous model computationally usable — total parameter count and 'active' parameter count are two different numbers worth knowing separately.
07Long-Horizon Coding Test + Front-End Arena Score
- Before trusting an AI system with a multi-hour autonomous task, know its context-window ceiling under your specific setup — Theo's failure came from a subscription limit, not a model limit, and he didn't find out until 3 hours in.
- A model completing 122 tasks off a single prompt with no follow-up steering is a stronger coherence signal than any single benchmark score — real endurance shows up in unattended runs, not scripted evals.
083D Game Demos: GBA Emulator + Fish Slop Submarine
- 3D and visual-reasoning capability is still a reliable way to separate genuinely capable models from ones that are just good at text — most models fumble basic 3D placement, which makes a working textured 3D game a meaningful signal.
- Vision-in-the-loop iteration (a model taking its own screenshots and adjusting from them) is a distinct capability from static code generation — it's worth evaluating separately when judging a model for UI or visual work.
09Chip Design, Knowledge Work, and Video Editing Claims
- When a model is unusually good at a narrow, unglamorous skill (chip design, GPU kernel optimization), that's often a sign of what a lab optimized for internally, and worth investigating even if it's not your use case.
- Claims a reviewer hasn't personally verified (like automated video editing) should be flagged as such rather than repeated as fact — untested claims are still useful signal, just lower-confidence.
10Third-Party Validation: Artificial Analysis Scores It
- Independent third-party benchmarks are worth cross-checking against a vendor's self-reported numbers, especially on dimensions vendors don't advertise themselves, like honesty about not knowing an answer.
- A model ranking 'third smartest' behind two others is still major news if the prior third-place holder (Opus, in this case) just got displaced — relative rank shifts matter as much as absolute scores.
11The Cost Reality Check + Moonshot's Own Disclosed Limitations
- Read a vendor's own limitations page, not just their benchmark page — Moonshot's public admission that K3 is 'excessively proactive' is more useful for deciding whether to trust it unsupervised than any leaderboard score.
- A cheaper per-token price doesn't guarantee a cheaper bill — always multiply by expected token usage per task before deciding a new model will actually save money.
12Front-End Showcase: Redesigning t3.gg and Liquid Glass
- Real UI improvement is easiest to judge on a redesign task you already have strong opinions about — the clearest win came from a sidebar Theo had already been iterating on himself, not a cold-start demo.
- Generating multiple design variants from one prompt (rather than committing to a single direction) is a useful way to evaluate a model's UI taste before picking a direction to refine.
13A Real PR, Then the Lakebed Security Audit Begins
- A model that reports its own mistakes as it makes them (and cleans them up automatically) is meaningfully safer to leave unsupervised than one that fails silently — that behavior is worth testing deliberately, not just capability.
- Testing a model on a task you'd normally get refused for (like a real security audit) is the clearest way to find the actual edge of a model's guardrails, rather than assuming based on marketing.
14Security Audit Results + Self-Organizing Sub-Agent Workflows
- A model's willingness to do a task a competitor refuses isn't automatically a feature — it's a tradeoff between usefulness to you today and misuse risk once the same capability is available to anyone.
- The most impressive agentic behavior often isn't a benchmark score at all — it's watching a model reorganize its own to-do list into parallel workflows and delegate task tracking to sub-agents without being told to.
15How to Use It Today: Subscriptions, API, and the Final Verdict
- Before committing budget to a new model, run your own worst-case cost estimate from a hard day of real usage rather than trusting sticker price — the actual day-one cost only made sense in contrast to usual spend on other models.
- A cheap-looking model from a new entrant can still pressure incumbent pricing even if you never switch to it yourself — competitive pressure is a real reason to track a release even when you don't plan to adopt it.
Terms worth knowing.
- Open-weight model
- An AI model whose trained parameters (weights) are published for anyone to download and run, as opposed to a closed model only accessible through a company's paid API.
- Mixture of Experts (MoE)
- An architecture where a model has many specialized sub-networks ('experts') but only activates a small subset for any given input, cutting compute cost relative to the model's total size.
- Context window
- The maximum amount of text (measured in tokens) a model can consider at once, including the conversation history, files, and its own working output.
- Long-horizon coding
- Coding tasks that require a model to work autonomously for extended stretches — hours rather than minutes — while staying coherent across hundreds of steps.
- FP8 / FP4
- Numeric precision formats used to store and run a model's weights. Lower precision (FP4) shrinks a model's memory footprint at some cost to accuracy.
- BrowseComp
- A benchmark that measures how well an AI agent can browse the web and complete research-style tasks, scored against the dollar cost of completing each task.
- Token caching / cache hit
- A pricing discount applied when a model reuses previously-processed input tokens (like a long system prompt) instead of reprocessing them from scratch.
- Diarization
- The process of automatically identifying and labeling which speaker is talking at each point in an audio recording.
- Frontier model
- Industry shorthand for the most capable AI models available at a given time, typically from top labs like Anthropic, OpenAI, or Google.
- AI Omniscience (hallucination benchmark)
- Artificial Analysis's test of whether a model admits uncertainty versus fabricating an answer — models score positively for saying 'I don't know' and negatively for confidently lying.
Things they pointed at.
20:40toolArtificial Analysis
13:50toolArena AI (Frontend Code Arena)
03:46toolHugging Face
29:40toolOpenCode
33:00productLakebed
13:15productping.gg
17:37productFish Slop
27:30linkmacos27.kimi.page (Max Weinbach demo)
37:41linkplatform.kimi.ai (Kimi API)
Lines you could clip.
04:00
“Anyone who's telling you that this is the future of local models has no idea what they're talking about and they should be ignored forever.”
04:20
“This model requires supercomputers to be used.”
16:25
“Man, Anthropic has to be terrified of this release more than anybody.”
28:40
“I think we finally have a model that's good at solving real world UI tasks without having to pay Anthropic massive amounts of money.”
33:30
“All of that effort that Anthropic and OpenAI have been putting in to use their models for defense work and blocking them from doing offensive work is very helpful because that is no longer going to be enough to protect us now that we have an open weight model this capable.”
40:10
“I'm doing like a thousand dollars a day with OpenAI models right now just on weird side projects, and I do at least 3 to $400 a day with Fable for my day to day work.”
Word for word.
Read-along
Don't just watch it. Burn it in.
See every word as it's spoken — crank it to 2× and still catch all of it. The same dual-channel trick behind Amazon's Kindle + Audible.
metaphor
00:00Kimi k three is here and it is a huge leap for open weight models. I'm gonna be honest and tell you guys that I just haven't been that hyped about open weight stuff recently because it hasn't been close to where we're at with models like Fable and GPT five six. The open weight frontier caught up to where we were before with models like Opus 4.8, kind of, but nothing has come close to surpassing it, especially for day to day use on complex coding work, especially once you start orchestrating really long runs where agents spin up tons of sub agents for complex tasks.
00:28Think this might have changed because Kimi k three is genuinely on the line for Frontier, if not surpassing where we're already at for various different things. The benchmarks are showing some pretty absurd numbers with Kimi k three beating out GPT five six Soul in various tasks, as well as Fable and others, and the very least, it's neck and neck throughout pretty much every bench I've seen.
00:47The benchers only tell one part of the story though, so I spent the whole day building as much as I could with Kimi k three, just to push it to its limits and see what it's capable of. And I'm gonna be real, I'm blown away. There are definitely some rough edges and I'll do my best to show you guys how to work around them.
01:03My terminal froze because I was doing so much though, so I'm gonna have to fix that first. This video is gonna have a lot of fun in it, from how to maximize your usage of the model, to addressing the confusion around the different ways to use the model, because there are quite a bit, talking about how the world is seeing this and what the impact might be, both on how we do dev work, as well as the economy, but also, possibly most importantly, the security implications of a release like this.
01:26Because this model is going to be open weight, and when you have a model this capable with no restrictions, there are some real concerns we're gonna have to address. I'm gonna go fix my terminal, and while I'm doing that, I hope you don't mind a quick break for today's sponsor.
01:39If you use GitHub Actions or Docker, trust me, you're gonna wanna watch this one, because Depot's today's sponsor, and they made both way, way better. Depot's fully compatible with GitHub Actions, but they also built their own alternative CI engine that is way faster, and it can also be called from your coding agents using a CLI, which allows your agents to get feedback much faster than they would if they had to run all that stuff locally or wait for your PR to build it for you.
02:02If you do use them for your normal actions, you'll still see crazy speed ups, though, up to 10 times faster. Docker's where they shine even more, though, making your Docker builds 40 times faster for real world use cases, and not just in the cloud, on your machine too. The Depot CLI is a drop in replacement for the Docker CLI that caches all of the layers on their CDN, which means all of your employees that are building the same images can all have way faster builds fetching from that cache instead of having to do the whole thing on their machine.
02:28And somehow, this all just got even faster with Depot Metal. As a friend of the Depot team, I am blown away at how far they went with Depot Metal. They went as deep as they possibly could on AWS, specking out machines directly that they own and control, managing the storage themselves as well.
02:45And the results show why they made these changes. It's already 30% faster for existing Depot CI or Sandbox runs. But more importantly, they managed to move from their roughly ten second VM spin up time to sub second spin ups, which is crazy for CI jobs.
02:59There's a reason companies like Posthog and PlanetScale do all of their builds on Depot, and you can figure it out yourself at soydev.link/depot. In case you thought I was joking, I'm not. I actually have to force quit CMUX right now in order to do what I was working on.
03:13Let's start with what the official Moonshot team had to say about this release. Today, we're introducing Kimi k three, our most capable model. Kimi k three is a 2,800,000,000,000 parameter model built on our Kimi delta attention and attention residuals with native vision capabilities and a 1,000,000 token context window.
03:28These are two really nice big changes. Things like GLM 5.2 don't have vision at all, which is one of the most annoying parts of using them. And the 1,000,000 token context window is also super useful, even if it's not available in the subscriptions, which we'll talk about in a bit.
03:43It's very nice to have when you do need it, and this model is huge, so it'll be beneficial for a lot of different things. It's crazy how just a year ago the Kimi models had no vision, had short context, it didn't even have reasoning, and they've somehow caught up to the frontier in that time.
03:58Say they just raised around $2,000,000,000 and have raised almost 4,000,000,000 total. Makes sense that they're aiming for the stars, like this is a moonshot in the most literal sense, especially at the size of 2,800,000,000,000 parameters.
04:10You're curious how big this is, the rough math for FP eight is that a trillion parameters is roughly a terabyte. So 2,800,000,000,000 is 2.8 terabytes of data.
04:19At FP four, it gets cut roughly in half, so it's only 1.4 terabytes. Oh man, that's so much more reasonable. To be very, very clear, anyone who's telling you that this is the future of local models has no idea what they're talking about and they should be ignored forever because a 1.4 terabyte model is not fitting in memory on any computer owned by anyone watching this.
04:39And if I'm wrong about that, please contact me. I would love to borrow your machines for some fun work. Seriously though, this is not running on anything any of us have in our homes unless you happen to live in like the Colossus data center.
04:50This model requires supercomputers to be used. We don't know how big the closed weight models from Frontier Labs are, but we've seen estimates between one and three trillion params for a model like Opus, usually in the one to 2,000,000,000,000 range.
05:04So having a roughly three trill model that's open weight is insane. This is a massive leap in the size of models that are available for us to use, and I genuinely feel bad for our friends over at Hugging Face having to host this and deal with people downloading two plus terabytes of data to use it.
05:20They won't have to for a bit though, because their planned release date for the weights is July 27. So the weights aren't available yet, which sadly means we have to use their APIs, which they're a Chinese company, so take that as you will.
05:33They're gonna have access to your code. Some people freak out about this, and I can understand why. Let's talk about the performance a bit.
05:39They say that it still trails the most powerful proprietary models like Fable five and Five Six Soul, but it also demonstrates Frontier level performance across their evaluation suite, consistently outperforming other tested models. I mentioned these in the intro, but we'll go through the benches real quick here. Deep SWE, which is my current favorite software dev bench, chose Five Six Soul slightly ahead of Fable five, seventy three to 70, and then Kimi K three behind them at roughly the same rate at a 67.5, which puts them ahead of GPT 5.5, Opus four point and GLM 5.2.
06:08Where things start to get more interesting is when we go down a bit to Frontier SWE. I'm coming around to this bench because it seems more focused on how likely the code is to merge, not just how well it solves the problem, And Fable five definitely writes the most mergeable code, in my opinion, in my real world projects.
06:26And now that I've read more of the bench, I understand why it scores like this. I still think it's a little higher than it should be in the gaps, bigger than I would expect in real world usage, but Kimi K three sliding in between Fable and Soul with a 10 lead on Soul is kind of nuts. It suggests this model is more tasteful, is the best I can put it.
06:46Like it writes code that better, it just looks and feels better rather than just finding a solution to the problem. They have their own internal bench where it scored just ahead of Opus four eight and behind K three, but it's worth noting their internal bench puts Soul behind five five, so I don't know how trustworthy it is.
07:03And then over here, we get the terminal bench where it beats out Opus and Fable five, and is just barely behind Soul. Program bench, which I haven't really seen much of, but it is world class there, just barely beating out Soul, which beats out Fable five by a little more. And SWE Marathon, a new bench that I know a lot are fond of.
07:20Somehow Opus four eight was the lead before, even against Soul and Fable five, but KimiK3 has now come out in front. General agent evals are also quite interesting. Things like GDP val, Jobbench, Spreadsheetbench, which is now led by KimiK3, to whoever is really into spreadsheets and open weight models, it must be a phenomenal day for you.
07:40Congratulations. Browse comp scoring so high is one of the most interesting to me though, because I'm a huge fan of browser use and computer use now after not caring about it for like two plus years. Because the models got way better at it, it's much more interesting.
07:53Having an open weight model that can do this is genuinely fascinating, because that means, hypothetically speaking, if I could afford the hardware to run it on, I could have a fully offline runner that can control my computer and do real work without having to send my data to Anthropic or OpenAI.
08:11Generally, if you wanna do actual computer use work right now, you're just expecting to send all of those screenshots of your machine to one of those labs. Not great. You hear, solution, there.
08:22It's expensive, but in the future, if the costs come down and the opportunity to use models of this capability becomes more available, that is huge. I mentioned before that it has vision capabilities, and it seems to be pretty solid there too. But what's really cool about the browser stuff, as I was hinting at before, is not just that it is industry leading by its score on these benches, it's also comically cheaper than the Frontier is.
08:44It's funny because I was just glazing this chart in the g p d five six video about how much better Soul was compared to everything else on it, but I don't know if that's fair anymore because Kimi k three max gets even cheaper than Soul Max, roughly the same price as Soul High, but a meaningfully better score. That said, an open weight model that is priced as cheap as possible, competing neck and neck with Five Six Soul on cost, shows just how far OpenAI has gone in reducing costs to the best of their ability.
09:13$5.06 Sol is still the fewest tokens to solve these types of problems and benchmarks. But since the tokens are so much more expensive, Kimi k three ekes out a win there. We'll talk about cost more in a little bit, but I'll just give the numbers now so you have them.
09:2630¢ for a cash hit, $3 per million tokens in, and $15 per million tokens out. For reference, the standard API price for Sonnet models is $3 per 15 per mil out, but Anthropic is temporarily offering a discount of 2 per mil in and 10 per mil out right now. So this is roughly a Sonnet level priced model, but it doesn't have a lot of Sonnet's problems, which we'll definitely talk about in a bit.
09:48Kimi k three is available today on kimi.com, Kimi Work, Kimi Code, and the Kimi API. I would ignore the two in the middle here. I will talk about the kimi.com and Kimmy API stuff in a bit when I talk about how to use the model.
09:59At launch, it will use max thinking effort by default with low and high effort modes to be introduced in subsequent updates. This is one of the most interesting things I saw about this release is it doesn't offer reasoning controls at all yet. You just use it and it gets used on Macs.
10:13They're currently working closely with inference partners and open source maintainers to align technical details and ensure reliable rollout across the ecosystem. This is also exciting to see. I know the Kimi guys have been pretty good about making it clear which providers are and aren't hosting their models correctly with benchmarks to verify the likelihood of any given provider actually hosting the model properly, which is great because this model is not trivial to host because it's an interesting implementation.
10:37We'll talk briefly about that in a bit. They also haven't put out their technical report yet, which will have a lot more of those details. Moonshot has historically had the biggest open weight models with the Kimi line.
10:47They put out Kimi k two as a trillion per amp model all the way back in 07/11/2025. And they stayed at that size for all of their releases from that point forward until now, where Kimi k three is a huge jump of 2,800,000,000,000 per ams, putting them ahead of everyone else, even huge models like DeepSeek v four, which was 1,600,000,000,000 on the pro version.
11:08I feel bad for Thinking Machines. They were so hyped to put out Inkling, and it's only a trail per ams and didn't bench great, and now with this coming out the day after, oof, I feel bad for those investors even more so. K three is built on their Kimi Delta Attention and Attention Residuals models, two architectural updates designed to improve how information flows across sequence length in model depth.
11:28They've also scaled up the mixture of expert sparsity, effectively activating 16 of 896 experts while paired with a stable latent MOE framework.
11:37Again, not gonna be trivial to host this, because they invented a lot of their own solutions to make a model this big and capable without the compute cost being absurd. The memory cost is still very high because you're gonna need a lot of this in memory for it to make any sense at all. But the sparse nature of how these experts are traversed should hopefully keep costs from being too absurd once the model's in memory.
11:59When you combine all of those updates with the refined training and data recipes that they produced, they end up with a 2.5 x improvement in overall scaling efficiency, which is a pretty big jump. These guys have always been pretty good about sharing their advancements and the cool things they do, so I'm extra excited for that technical report to come out.
12:15But now let's talk about what using it looks like, on the coding side especially. Kimi k three has strong long horizon coding performance, meaning it can run long tasks with minimal human oversight. One of my favorite test tasks for these much bigger, more capable models is to take my old code base for ping.
12:30Gg, which is a Zoom app for content creators doing live collaborations, and see if the model is capable of porting it. It's been running for like three plus hours, and it was doing great until like ten minutes ago where it hit a limit on the context window size.
12:44This is a mistake that's partially my fault because I didn't realize that the subscriptions that you can use for Kimi code don't actually give you the full 1,000,000 token context window, and with my setup using it with Claude code, it was hitting a much smaller limit. So I'm gonna really quickly change that max size and get that compacted so it can keep going because I want to see if that run can complete.
13:06Sadly, I won't be able to compact using Kimi k three because again, it won't respond to the API request. I'll be able to get it compacted with Fable and keep the run going in just a moment though.
13:16The fact that it was able to get through a 122 tasks with a single like paragraph and a half prompt and no additional insight or effort from me is unbelievable though. I've never seen an open weight model come close to staying coherent for even half this length, much less like actual real world massive migration work.
13:35It's impressive. And since it also has visual reasoning, it's way more capable of UI type work too. It leverages screenshots and visuals to optimize game dev, front end, and CAD.
13:45And those visual capabilities are nuts when it comes to front end code. We'll talk more about this later, I'm sure, but just know in advance that Kimi K three is really, really good at front end, at least according to Arena AI. From my experience using it, it has its quirks, but it is very impressive, and I cannot wait to show you guys just how cool some of the UI it creates is.
14:04On the very opposite end, we have the kernel optimization capabilities, where apparently they use it to optimize GPU kernels, and it did a pretty damn good job. After being active for around fifteen hours, it saw slightly better improvements than even Fable did, and quite a bit better than five five and five six Soul somehow, which is nuts.
14:23They have four types of kernel optimizations that they benched here, and somehow k three came out near frontier or above frontier in all of them. It's also really interesting to see how big the gap is between soul and fable in a lot of these as well. This might be a good bench for them to actually like put out in the future, not just because they're leading it, but because it's fascinating to see how big the gaps are.
14:44This is also scary for companies like Anthropic who have went out of their way to hide the model's capability of helping with ML work, like kernel optimization for hosting models. They went out of their way to keep the model from sharing those things, and it's one of the restrictions they have on both Fable and Mythos. So it's fascinating to see Kimmy bragging about how good their model is at this when they plan to release the weights, because that means this capability is now in the hands of everyone to an extent.
15:10Man, Anthropic has to be terrified of this release more than anybody. In the late stages of Kimi k three development, an early version of k three handled the majority of the team's kernel optimization work.
15:20That's pretty nuts. They also tested if it could build a complete GPU programming stack and compiler from scratch. It developed Mini Triton, a compact Triton like compiler with its own tile level IR layer over MLIR.
15:32I'm sure much smarter people will know what this all means and be really pumped about it. Yeah, it seems like they did a really good job training the model to do this type of stuff. Here's where I'll be much more useful.
15:42Game dev and digital creation. K three combines strong three d reasoning, coding, and vision capabilities to turn concepts, images, and videos into fully playable interactive experiences. It achieves a true vision in the loop by seamlessly iterating between code and live screenshots, instantly seeing and refining outputs.
15:59I actually watched it do this live when I was having it do some changes to t three code, where it would spin it up, get a screenshot, look at it, think about it, and then change what it was doing. Apparently, it created everything here, including the rider and horse models. So it, like, actually understands three d.
16:16I'm gonna have to play more. I want to see this in action. I just spun up Pi to go do a three d port of Fish Slop.
16:24We'll see how it goes. Hopefully not well, because I don't wanna have to record more after I'm leaving. I'm actually supposed to be at an event right now, but had to film this because it's such a cool model.
16:33Yeah, the fact that it's able to do this type of three d work and modeling is crazy. Every model I've tried so far is just so rough at three d stuff that I've been impressed by like circles being placed properly sometimes. If this model can actually do three d, I'm gonna have to that's gonna change things.
16:52Oh, shit. Oh, shit. Theo from the future here, it got pretty far in fish slop three d, so I wanted to check this out quick.
16:59Holy shit. The three d game is like actually working. Sorry about the audio.
17:05Can I mute that easily? No, I can't. I have no idea what it sounds like.
17:09I'm not putting on my headphones to see. I I I'm Oh, here he is.
17:22Holy shit. The fish textures. They're not perfect at all, but this is the best fish model I've seen any lab create.
17:29And the submarine is phenomenal too For a model generated for a three d web game like this. It's got its issues for sure, but like, it's still working on it.
17:38It just said like the game works. Let me do more. But the fact that it's already this far?
17:45Unbelievable.
17:49Okay. Holy shit.
17:55Yeah. God, even has sound effects and things. This is so much better than I would've expected.
18:01It's still working. Like, I'm checking this before it's done, and it keeps pulling in visualizations, like, ran it.
18:06And I saw the little picture in the Pie history where it actually, like, pulled it up. It was like, it's working. Let's continue.
18:11Let's fix all these things. It's a good fucking model. Apparently, it's also good at chip design.
18:17This is interesting. It might have real, like, beyond what the Frontier currently allows capabilities that none of the other Frontier labs are focusing on. This is actually arguably one of the problems with this fixation on code that both OpenAI and Anthropic have as they try to win an enterprise.
18:33They're not finding new capabilities the same way that they used to. Here it seems like this new Frontier open weight model is actually able to explore things that the labs here have just not explored. And hopefully, as long as the three d setup is as cool as it seems, that's gonna be huge.
18:49It's also incredible at knowledge work according to them. Benchmarks like Online e x p Bench, Deck Bench, and Finance Bench, it is industry leading in. A lot of that's probably because of how good it is at visualization stuff.
19:00They had to do a bunch of research work, and it did a pretty good job even at generating the actual reports, which is pretty damn cool. Got infographic style presentations.
19:08It's still got that, like, AI generated vibe that a lot of those have.
19:14They show some dashboards. It does really like this bento box style UI layout, but it makes them in a very pretty way.
19:23And the animation taste is actually quite good, too. I've been impressed with it. Apparently, it's also good at video editing, which is crazy.
19:29I will not have any time to try this anytime soon, but if my team ends up liking it, I'll be sure to share that in the future. I'm not that into AI based video editing, because video editing is actually quite fun and not too tedious if you are any good at it at all. And they had to edit a lot of their videos for the launch of the model, having it go through lots of clips, handling clip selection, motion matched cuts, frame accurate, beat synchronization, audio processing, and multiple rounds of revision.
19:52That's pretty cool. They have some cool info here about how they were able to get a model of this size to be trainable in a reasonable timeframe and compute wise in in a stable fashion, because the bigger it gets, the harder it gets. They found some fun tricks like using f p four weights for the actual stored weights and params, but then using f p eight for the activations once the model's actually running so that it has better short term memory and it doesn't have to waste a ton of RAM in order to preserve the actual weights.
20:20At least that's my rough understanding. Smarter people, feel free to correct me in the comments. Remember that supercomputer thing I said?
20:26Since inference efficiency likewise benefits from larger high bandwidth communication domains, we recommend deploying k three on super node configurations with 64 or more accelerators. 64 h one hundreds is a bit rough. That's 2,600,000 to host the model.
20:43Yeah. Very local friendly. So that's what we have from them.
20:47Let's take a quick look at what others have said. I mentioned Arena AI gave it an absurd score for front end stuff. We'll show some front ends in a bit.
20:54We're gonna start with artificial analysis because the model, according to them, is the third smartest ever. So pretty much out of nowhere, we had two drops in a row that took the frontier away from OpenAI and Fable just going back to back forever.
21:11And suddenly, we have Grok four five and Kimi k three coming up for those third place spots right behind Soul and Fable. Normally, Opus would be up there too pushing these back, and it's not anymore because both x a I and Kimi have gotten their shit so together that they are leapfrogging. And they are so far ahead of what Google is cooking, it's hilarious.
21:33I honestly think Google, if they had any reasonable way to buy a Chinese company like Moonshot, they probably have to at this point, because they are just so behind in comparison.
21:43Okay, slight correction. Grok four five is behind both Terra and Opus, as well as five five, so it's not really up at this frontier level, but Kimi k three is.
21:53It is just behind Soul and Fable according to artificial analysis's benches. But things get much more interesting as we dig in more. I'll read what they said on Twitter first.
22:02Kimi k three scores 57 on the artificial analysis intelligence index. Its intelligence is comparable to Opus four eight and five five, but remains slightly behind Fable five and five point six SOL. Moonshot has expressed plans to release the 2,800,000,000,000 parameter models weights, which would make it the leading open weight model.
22:17It's got really strong agentic performance, as I mentioned before. It's meaningfully better than models like GLM five two, going from sixteen sixty eight on g d p val from 1514, which was the open weight frontier before, and even ahead of models like Opus four eight.
22:33The second highest score that I've ever seen on artificial analysis's knowledge bench called Briefcase. It will absolutely be the leading open weight model, not a surprise there. The cost per task is similar to Five Six Soul, but it's half the price of Opus and higher than all of the open weight peers.
22:48Not just because the model pricing is more expensive, but also remember, the amount of tokens it uses is important too. For reference, in output tokens per task, Kimi k three is a decent bit lower than Fable five, which is the most token efficient model AmphoraBic's put out in a while. But compared to things like Five Six Tera or even Soul all the way back here, 15 k tokens per task, 23 is more, but also not much more when you consider the price difference, and it's so much less than other models, especially other noisy open weight ones, things like DeepSeek v four Pro or even worse, v four Flash, which has 45 k tokens for the same tasks, it's pretty token efficient.
23:26And that's awesome to see because historically, only OpenAI has really focused on token efficiency. Now we have both Grok and Kimi surrounding them in their little cheap token section here, finally driving the industry towards more efficient reasoning.
23:3821% more efficient than k two six was, even though the model is bigger and the tokens are more expensive. Has native multimodal capabilities, as I mentioned before, very, very good there.
23:48The AI Omnition score is also very interesting. If you're not familiar with ThisBench, it's artificial analysis attempt to measure hallucinations, specifically if the model is going to tell you when it doesn't know something versus will it make up something instead.
24:01And Kimi K three is one of the best open weight models at not hallucinating. In fact, I think it is the best by quite a bit. Even five two is a much worse score here.
24:10And this test is fun because you can get into the positive when you say I don't know the answer, and you go negative when you lie. So even some very good models, like Five Six Soul, end up scoring a lot lower than they should here because they're a little too quick to lie when they don't know, and the extra points they get for the things they do know get canceled out a bit by that.
24:28And Five Six Luna gets hit real hard with this, getting into the negative as a result. It's one the most honest open weight models we've ever seen, and that's genuinely exciting because open weight models have not been good at this historically. But if your goal is to use this model to save a bunch of money, you probably shouldn't be too excited yet because that $15 out cost is not cheap.
24:47And since it uses twice as many tokens as something like Sol, cancels out the 50% discount that you're getting, and you also lose a lot of the niceties that you get from Modern Frontier Labs. Moonshot themselves even said that the model still has a noticeable gap in user experience compared with Fable five and five six Sol.
25:04So if Sol is as cheap, if not cheaper for your use case, it might end up being better just because it's nicer to work with. And I have noticed that with Kimmy myself. I noticed things like when a workflow finishes, it responds to the workflow finishing instead of giving me the context I need.
25:18Those are things you can work around, but you're gonna have to do that with this model because it's not as RL'd on the expectations we have as users, and they just don't have as much data because they're not getting feedback from people the same way. Because the vast majority of users of the Kimi models are using them on other providers, so they don't get any of the telemetry they would need to improve these things.
25:37I'm sure it will get better over time, but not surprised that there's a real usability gap here. They also call out that it's too proactive, , as well as the sensitivity to thinking history that it needs its thinking.
25:50Thankfully, it's an open weight model, so we get the thinking data too. It's not like Anthropic or OpenAI where the thinking is hidden on some server. All reasonable, really cool callouts.
25:57I love this level of transparency from a lab that's releasing something this important. Okay. Enough of the research side.
26:03Let's talk about the actual front end capabilities. Because I know a lot of you guys are excited about this. I did one of my recent favorite demos, which is to have things redesign the t three code marketing site.
26:12This is the design I came to after working with the Claude Design product, my own bringing in a lot of back and forth. I got it here. There are little things I would change.
26:19Obviously, I wanna make the cursor icon a different color so it fits better, but you get the idea. It's not bad. I love my little carousel here with all the nice things people said about t three code.
26:28So let's look at what it did. I told it to do five different designs on different URLs to keep them varied, and here's what it gave me so far. This is the first one.
26:37It's doing pills as certain OpenAI models really like to do, so that was interesting to see. I did this with OpenCode as the harness, you were curious. So that's number one.
26:46Not bad, but not something I would ship. Here's two. It looks really nice, except for the fact that this type of design has been copied by so many models for so many things that it's not as cool anymore.
26:59I also can't help but notice that this top bar doesn't have enough content, so it's cycling, but it's half empty now as a result. Has a lot of nice little animations when you hover over things, which is cool.
27:11Still not perfect though. But like, considering this is an open weight model, unbelievably cool.
27:19Which over here, and now we get the cringe terminal one. I was so shocked that this snuck in that I actually asked if it had pulled in my UI skill or something, because I thought I had removed it. And it hadn't It's almost like the Claude design skill got baked in though, which is For the fourth design, it channeled more of what I already had, but with a surprisingly not too cringe glow in the corners, also fixing this icon to be the right color.
27:43Still don't necessarily love how it structured things here too bubbly, but you could steer this somewhere good for sure. I do think the gradients are pretty cool.
27:53And then we have the bento box version, so I knew it would do one of these. I don't think it fits for the type of product that t three code is, but it's not a bad design at all. So from just this one pass, I would say that for doing a marketing page, it's slightly better than what I get out of OpenAI models, but slightly behind what I would expect from Claude.
28:12But marketing pages are far from the best way to measure the UI capabilities of a model. So I gave a slightly harder task. It was actually something I was already working on.
28:20I've been overhauling the sidebar in t three code. I wanna find a better way to do it, something that's a little more flexible based on how it's being used.
28:29So I already had this build where I redid the sidebar. And this build has a much better experience there, but I've noticed that the color isn't great. This is dark and this isn't, and if anything, that should probably be flipped.
28:40And I also am hating the gray more and more since I had to remove the noise because of performance things that'll be in an upcoming video. Keep an eye out for that. Fable screwed up the performance of the app and I had to remove a bunch of shit to fix it.
28:49So I asked it to do a darker redesign with true black instead of grays as often. And this is what it made. I will be frank.
28:58This is better than what we had. It made a better design than what we were already doing with an open weight model that is a third the price of the Frontier models from the other company that makes ones good at UI.
29:12I think we finally have a model that's good at solving real world UI tasks without having to pay Anthropic massive amounts of money. Very exciting. Another cool UI example that Mac shared here is a recreation of Mac OS 20 seven's Liquid Glass styles in a real web app that you can load that Kimi built all of from scratch.
29:32And they're also hosting it on kimi.page, their little like web hosting thing. And this is nuts.
29:37To have something that looks and works this well that like a open weight model generated for not too expensive is kind of crazy.
29:46It has a lot of little things that are broken in it, but like, this is not bad at all for something that a model generated. Seriously, this is dope.
29:54What was much more exciting to me is how well it did this work. I did this work through OpenCode's bindings in t three code because as I've mentioned many times, t three code is not a harness. You have to bring something like ClaudeCodeCodecs or OpenCode in order to use t three code.
30:08So since Kimi isn't available in ClaudeCode or Codecs, wink, I'll show something soon, I used OpenCode. And this was a fun bit of work. I told it to rethink the core UI for t three code.
30:20It's currently too gray. In dark mode, I want a cooler black and white layout with hard blacks inspired by ChatGPT. I gave it a screenshot of the ChatGPT UI so it could have that as a reference.
30:30And then an hour and five minutes later with one steer where I told it that it was opening things incorrectly, because it kept trying to run the dev command to see. When it did that, it opened my browser and burned the one time token that we use in t three code to verify who is connecting so it couldn't access it the ways it wanted to.
30:48But once I told it that and it stopped letting it open the browser, then it was able to access it, check its work, and keep refining until it got somewhere that it liked. I also apologize for opening new tabs constantly because I kept forgetting the no browser command.
31:02Sorry about that. Everything I started is killed and the temp homes have been removed. Also, parallel sessions dev server, codec m p p ports got caught in a broad cleanup kill.
31:09I restarted those with their original flags. Nice. So it made mistakes.
31:13It told me about them immediately and then cleaned them up. Other models will make similar mistakes, but they don't necessarily tell you when they do, so that was very nice to see. I asked it to do a more machine task, not like building a project, but fixing a config, because I wanted to run this in isolated environment so it didn't affect my existing t three code install, just so I could open it in the browser and play with it a bit as I did here.
31:34But in order to do that, I needed content. So I told it to figure out how to get things from my official t three history, which was breaking this build because I have all my sidebar changes. And it managed to pull over the history, modify it, and get it in a state where I could actually work again.
31:49Thought that was really cool that it could do that type of thing. I then asked it in the same thread to file a PR. Follow revoke conventions for PR naming, minimal description, include before and after photos, use the file upload skill.
32:00If you don't have the skill, look at the global codex and Claude skills, you can find it there. I added this because I didn't have the skill in OpenCode and I wanted to go find it, and it did. And it got it working.
32:09It again struggled a bit with getting the browser going, but once it realized it could do the no browser command and also open in Puppeteer, it pulled it together and it made this PR where it shows the before and the after. And you can very clearly see how much better the after is here. I'm I'm impressed.
32:27It did a great job with this work. I didn't hold back in any way, I didn't give it an easier thing to see if it could do it. I worked with this the way I work with other models, and it did all of the work very well in one thread without having to do any custom anything.
32:42Since it did so well on those tasks, I decided to push its limits with these bigger overhauls. I did have the issue with the token window and compaction with the previous thread as I mentioned. I'm working on fixing that now, but that did not stop me from having success with some other really big jobs.
32:56This was one of those jobs. I asked to do a deep audit for security issues on my cloud product that I'm building called Lakebed. This is one of those tasks that I get a lot of refusals for when I use Fable and Soul for it.
33:07So I was curious to see if it would refuse, as well as what it would find, because I have a concern with a really powerful open weight model like this coming out. And it did it. It spun up a bunch of agents to go do discovery on specific things.
33:19It then followed up with a verification pass for all of those things that it found doing 25 verification agents. And then it synthesized that all at the end, ran out of token space, I compacted, continued, and it succeeded.
33:31It gave me useful feedback on things I can do to secure further. This is good on my end because I can use this to secure my systems, but it's gonna be really bad for us in the future when attackers start using it for similar things. All of that effort that Anthropic and OpenAI have been putting in to use their models for defense work and blocking them from doing offensive work is very helpful because that is no longer going to be enough to protect us now that we have an open weight model this capable.
33:59And if you're curious how Moonshot is thinking about security and safety, the word security appears inside of some of the demos on their page, but not in the actual contents of it. And the word safety does not appear on the page at all.
34:11As I mentioned before, they have not put out a system card yet. They plan to put out the technical reporting in the future, but we have no info about safety and security and what the impacts on those will be from this model yet.
34:22We don't even know how they thought about safety and security during training. So that is a thing that is worth being concerned about. And if I talk about just how concerned I am, this video will be much longer, so go check out my other videos about security and my concerns in that general space.
34:35Since the reasoning is visible in this model, I did take the opportunity to read said reasoning a little bit, and I don't know if the other labs do this too because most of them don't share the reasoning, but it was interesting to see the dumb things the model would get stuck on. For example, in my global agents m d, I'm very strict with the models that I don't want them to write any all over the place.
34:54I want them to actually use TypeScript for its types. So when it was working on a plan in Markdown, it read the detail that we need to ensure no any per user preference.
35:05The plan is Markdown, not code, so okay. Like it took a second to think about that.
35:11Time was spent reasoning about the fact that it shouldn't use any, the TypeScript syntax inside of a markdown file.
35:19When you combine that type of bad reasoning with the model's 20 TPS out, which is less than half of what we see from other labs, the result is that it feels slow. It's spending time doing things it shouldn't.
35:31Others might as well, but we don't see that data. At the very least, would guess OpenAI doesn't because their models are so efficient with reasoning. But that is time I spent and money I spent on a thing that doesn't actually affect the quality of the outputs.
35:42Here's another weird experience I had. I mentioned this one earlier where I had it doing a big workflow to write a plan up for me, and at the end, it didn't tell me what to do. It ended with no new report here, just an idle status ping.
35:57The plan in plan slash OverhaulMD remains current with all three audits incorporated. It was done. It had completed the work, but it was responding to the sub agents and the ping, not to me, the user who was setting this off asking it to do the work.
36:12Just a weird thing that I had to notice and be like, oh, it's not going anymore. I need to do a follow-up. The reason I tend to use these models in Claude code is because I really like workflows.
36:21I've talked a lot about this in previous videos. Go watch any of my videos about Cloud Code versus Codex or why I used five six in Cloud Code, probably has more detail there, as well as how I did this. That method worked perfectly for Kimi, by the way.
36:32It was very easy to just do slash model, Kimi k three, and it just worked. But what really blew me away was that Kimmy k three can use sub agents and workflows and all of these things correctly, but it did it differently.
36:44Normally, when I tell a model to use a workflow, it will use one workflow to break down a big task. Kimi made a bunch of workflows for various phases of the work it was going to do with the port of Ping to a modern stack. It broke up all these different phases, created workflows for them, and started running them.
36:59But the most interesting thing is that it allowed the sub agents to complete tasks on the to do list a level higher. So it had a to do list, it turned the to do list into workflows, started running some of them, and I noticed the to do is just like randomly getting crossed off throughout, and that was because it gave those sub agents the ability to check them off.
37:17Never seen that before, even Fable doesn't do that inside of Cloud Code from my experience. So that was really cool and fascinating to me. Probably time to talk about how you can use the model today.
37:25Since the weights aren't available yet, the only way to use it is through the Kimi platform. To be fair, this has been the case with Anthropic and OpenAI historically. They do usually set up their models on Bedrock with AWS as well as GCP and Azure depending on which lab and which models and which provider.
37:41So there are some other options, but at the very least, you always have a lot of options that are based out of The US. That is not the case for Moonshot. Moonshot is a Chinese company, which means either of these solutions are going to go through Chinese servers, some amount.
37:55I would not use either of these methods for real important sensitive data at all yet. Wait till the waits are out, you'll be able to do that then. But for now, these are your two options.
38:04You have the subscription on kimi.com or the API, which is platform.kimi.ai. These are both hard to find, so I ended up accidentally spending a $100 on the API because I forgot I had a $40 subscription on Kimi.
38:16They've an interesting breakdown of subscriptions. They have a $20, a $40, a $100, and a $200. And I ended up hitting the $40 pretty fast with k three, at least the five hour window I hit in about thirty minutes of my early testing.
38:28So I bumped up to the $200 tier. And despite my borderline abusive usage, I have not even gotten to 50% of my five hour or 10% of my weekly since doing that upgrade.
38:39It seems to be quite generous, which makes sense because the model only costs $15 per mil token out, So I did a bunch of calculations on how much my usage has cost. For the $40 sub that I burned and upgraded, the $10 or so of API usage I did before realizing this, and all of my usage in that sub so far, my estimates from using CC usage and having my whole fleet analyzed through SOL is around $63 of usage.
39:05That is not bad at all considering how much work I've been getting done with this model. I'm impressed. For context, I'm doing like a thousand dollars a day with OpenAI models right now just on weird side projects I have it burning and looping on, and I do at least 3 to $400 a day with Fable for my day to day work.
39:22I've done a similar amount of work with this model and ended up being way cheaper. So that's good to see, hopeful. Not similar to the amount of work I do with Soul to be clear.
39:31I do way too much with that. But the cost to actual work done compared to Fable, it's a really good ratio.
39:37And it does also instinctively so far seem slightly cheaper than $5.06 on lower reasoning levels for my day to day work. What this means for you is that the API is actually a pretty reasonable option because the costs aren't too bad, But I would start with a subscription at least like the $40 tier to play with it if you're curious enough to do said play.
39:57And if you do sign up for either of these, you can bring the API key from them into OpenCode, therefore making it work in something like, I don't know, t three code. It's also worth noting that the subscriptions work inside of the CLI proxy that I use for routing my Claude code to both Claude code and Codex and now also to Kimi, that's been pretty nice too.
40:14If you'd prefer to wait until the waits are released so you can use this on American servers, I totally understand. But if your reason for waiting is cost, don't. I ran a bunch of numbers in previous Kimi models when you compare the price they released them at to the price that was available on other hosts and providers.
40:30They were cheaper, but it was like 11 to 15% at best. These models are just massive, so they're not cheap to host.
40:39And I would not expect this model to suddenly become $5 on other providers, unless they quantize the hell out of it and make it way dumber. That doesn't mean the open weight nature of the model isn't great for the market as a whole.
40:50It's going to make pricing way more competitive. And I would be genuinely surprised if this doesn't force Anthropic to rethink some of the Opus release that they have coming in the not too distant future. If they still price it the same they did before and the numbers end up worse than Kimi's, Anthropic's gonna be in a rough place.
41:06Think I've said everything I have to about this model. It's unbelievable at three d. It's really damn good at visual stuff, especially coding and UI.
41:13And it's just overall a really good model. It no longer feels like it's good for open weight. It now feels frontier class in many different ways.
41:21And while talking to it and working with it, is it quite as pleasant as the best models available? It's close enough that I am blown away. And I am so excited for the future of models like this and where they're going to go.
41:30Let me know how y'all feel about the Kimi K three launch, and until next time, peace nerds.
The Hook
The bait, then the rug-pull.
Theo opens by admitting he'd stopped caring about open-weight models — none had come close to Fable or GPT-5.6 Sol on real coding work. Kimi K3, he says, might change that: benchmarks put it neck-and-neck with the frontier, so he spent a full day building with it to find out if the numbers hold up.
Named ideas worth stealing.
09:05list
Kimi K3 Access Methods
- kimi.com (subscription)
- Kimi Work
- Kimi Code
- Kimi API (platform.kimi.ai)
Four ways to use Kimi K3 before its weights release; Theo says ignore Kimi Work and Kimi Code and just use the kimi.com subscription or the raw API.
Steal forAny evaluation of a new model's launch access tiers before deciding where to spend eval budget.
37:41list
Kimi Subscription Tiers
- $20/mo
- $40/mo
- $100/mo
- $200/mo
Theo blew through the $40 tier in about 30 minutes of aggressive testing and upgraded to $200, after which he stayed under 50% of his 5-hour cap and 10% of his weekly cap.
Steal forBudgeting a first real evaluation window for a new frontier-adjacent model.
How they asked for the click.
VERBAL ASK
01:30product
“check them out at soydev.link/depot”
Mid-roll sponsor read for Depot, framed as a natural break while Theo fixes his frozen terminal — low-friction, in-voice, doesn't disrupt the review's momentum.
MENTIONED ON CAMERA
FROM THE DESCRIPTION
PRIMARY CTAWhere the creator wants you to go next.
OTHER LINKSAlso linked in the description.
Visual structure at a glance.
