Big Idea

The argument in one line.

Video understanding for an AI model that only sees static images can be made fast and cheap by selecting frames intelligently instead of sampling every frame or decoding the whole file.

Who This Is For

Read if. Skip if.

READ IF YOU ARE…

You use Claude Code or Claude and want to hand it a video link instead of manually screenshotting and summarizing it yourself.
You build or maintain tools that process video and are hunting for ways to cut compute cost without losing accuracy.
You're curious how a two-hour interview can be summarized by an AI model that technically can't watch video at all.

SKIP IF…

You have no interest in AI coding tools or Claude Code specifically — this is a plugin walkthrough, not a general video-editing tutorial.

TL;DR

The full version, fast.

Claude has no native video model, so this open-source skill fakes video understanding by feeding it a flipbook of frames alongside a timestamped transcript. Version 1 sampled a fixed number of frames evenly across a video, which meant a two-hour video got roughly one frame every 72 seconds — mostly transcript with a few random screenshots. Version 2 fixes this with smart scene selection (only capturing frames where the picture actually changes), deictic gesture detection (grabbing the exact frame when someone says "look here"), and a new efficient mode that pulls keyframes already baked into the video file instead of decoding and comparing every frame, which is where the claimed 40x speed gain comes from. A frame-deduplication pass also strips near-identical frames before Claude ever sees them, cutting token costs on static talking-head footage. The video ends with a two-minute install/upgrade walkthrough and guidance on choosing between four detail levels: transcript-only, efficient, balanced, and an uncapped "token burner" mode for when accuracy matters more than cost.

Free for members

Chat with this breakdown — free.

Sign in and you get 23 free chat messages on us — ask for the hook, quote a framework, find the exact transcript moment, generate a markdown action plan. Bring your own key when you want unlimited.

Create a free account →

Chapters

Where the time goes.

00:00 – 02:14

01 · Claude just watched a 2-hour video in seconds

Introduces the /watch skill: paste any video link and Claude gets frames + transcript together, demonstrated demolishing a 2-hour interview in seconds. Notes 1,600+ supported sites and ~3,000 GitHub stars since launch, then announces v2.

02:14 – 02:54

02 · Smart scene selection

Explains why fixed-interval sampling breaks down on long videos (one frame per 72 seconds on a 2-hour video) and introduces scene-change detection as the fix.

02:54 – 03:15

03 · Deictic gesture targeting

Shows a case scene-selection alone would miss — a speaker saying "as you can see" mid-scene — and how deictic gesture detection grabs that exact frame.

03:15 – 03:53

04 · Token burner mode

Describes the uncapped mode that extracts every scene change up to two frames per second for maximum detail regardless of cost.

03:53 – 05:15

05 · Efficient mode

Explains the 40x speed source: pulling keyframes already baked into the video container instead of decoding and comparing every frame.

05:15 – 05:38

06 · Frame deduplication (cheaper tokens)

Addresses viewer feedback about token cost: a thumbnail-comparison pass discards near-duplicate frames before Claude ever sees them.

05:38 – 06:33

07 · Install + upgrade in two minutes

Walks through installing via Claude Code marketplace commands, enabling auto-update, or installing through skills.sh for other platforms.

06:33 – 08:31

08 · Picking your mode

Runs through the four detail-level modes — transcript, efficient (recommended default), balanced, and token burner — and how to set/override them, plus the Grok/OpenAI fallback key for transcription.

Atomic Insights

Lines worth screenshotting.

Claude has no built-in video model — this skill fakes video understanding by feeding it frames plus a timestamped transcript together.
Sampling frames evenly across a long video breaks down fast: a two-hour video gets one frame every 72 seconds, and a ten-hour video gets one every six minutes.
The fix isn't more frames, it's the right frames — scanning for genuine scene changes rather than sampling on a fixed interval.
Deictic gesture detection catches phrases like "look here" or "as you can see" in the transcript and grabs the exact frame being referenced, even mid-scene.
A token-burner mode removes the frame cap entirely, extracting every scene change up to two frames per second for when cost doesn't matter and total accuracy does.
Most of a video's frames are keyframes already baked into the compressed file roughly once a second; every frame in between only stores what changed since the last one.
Reading pre-baked keyframes directly, instead of decoding and diffing every frame, is what produces the 40x speed improvement over the prior version.
On a talking-head video, most frames are visually identical, so a duplicate-detection pass (shrink to thumbnail, compare, discard near-matches) avoids paying tokens for the same shot repeated dozens of times.
The tool works across more than 1,600 sites — YouTube, Zoom recordings, Looms, TikToks, and local files — not just YouTube.
The skill picked up nearly 3,000 GitHub stars in its first month, alongside hundreds of thousands of videos processed across platforms.

Takeaway

Pick the right frames, not more frames, to make AI video analysis fast.

WHAT TO LEARN

Since AI models see static images rather than video, the entire cost and speed problem comes down to which frames you hand over — sampling evenly wastes tokens on redundant shots while missing the moments that matter.

01Claude just watched a 2-hour video in seconds

An AI model with no native video understanding can still analyze video convincingly by combining frames with a timestamped transcript.
Free, fast public iteration (open-source, auto-updating) can take a tool from launch to nearly 3,000 stars and hundreds of thousands of processed videos in about a month.

02Smart scene selection

Fixed-interval frame sampling fails predictably on long content — the math (one frame per 72 seconds on a 2-hour video) makes the problem obvious once stated.
Detecting real scene changes instead of sampling on a timer guarantees every captured frame is a moment worth analyzing.

03Deictic gesture targeting

Scene-change detection alone misses references that happen mid-scene, like a speaker verbally pointing at something already on screen.
Listening for phrases like "look here" or "as you can see" in the transcript and capturing that exact timestamp closes that gap.

04Token burner mode

Removing the frame cap entirely (up to two frames per second) trades cost for completeness when accuracy matters more than price.
Offering an explicit "no limits" mode signals to users they can dial into it deliberately rather than being stuck with a fixed default.

05Efficient mode

Compressed video files already store periodic full keyframes for their own internal reasons — reading those directly skips the expensive decode-and-diff step entirely.
Speed gains this large (40x) tend to come from skipping unnecessary work altogether, not from marginal optimization of the existing approach.

06Frame deduplication (cheaper tokens)

On talking-head or static-shot footage, most extracted frames are visually near-identical, so paying full token price per frame wastes money on repeats.
A cheap thumbnail-comparison pass before the expensive step is a simple, general pattern for trimming redundant work in any pipeline.

07Install + upgrade in two minutes

Auto-update removes the burden of manually tracking new releases, which matters more the faster a tool is shipping.
Supporting multiple install paths (plugin marketplace, direct skill install, generic skills.sh) lowers friction across different user setups.

08Picking your mode

A single sliding scale from cheapest/fastest to most thorough/expensive is easier for users to reason about than a binary quality toggle.
Recommending a sensible default (efficient mode, for everyday use) while leaving power users an escape hatch (token burner) serves both audiences without extra complexity.

Glossary

Terms worth knowing.

Keyframe: A fully-stored image baked into a compressed video roughly once per second; frames between keyframes only record what changed, making keyframes much cheaper to extract than rebuilding every frame.
Deictic gesture: A verbal reference to something being pointed at or shown, like "look here" or "as you can see" — used as a cue to capture the exact frame being referenced.
Scene selection: Scanning a video for moments where the picture genuinely changes (new slide, cut, or visual) rather than sampling frames at fixed time intervals.
Token burner mode: An uncapped extraction mode that pulls every detected scene change as a frame, up to two frames per second, prioritizing completeness over cost.

Resources

Things they pointed at.

00:00toolwatch skill (GitHub, Claude Code plugin marketplace)

08:31linkskills.sh

Quotables

Lines you could clip.

00:00

“Most people don't know that Claude can now watch any video for you.”

clean cold-open hook stating the core claim→ TikTok hook↗ Tweet quote

07:01

“Instead of decoding all 24 frames a second, [Claude] just pulls the keyframes that the video already has baked in, and that's where the 40x comes from.”

the single sentence that explains the entire technical unlock→ newsletter pull-quote↗ Tweet quote

The Script

Word for word.

Read-along

Don't just watch it. Burn it in.

See every word as it's spoken — crank it to 2× and still catch all of it. The same dual-channel trick behind Amazon's Kindle + Audible.

metaphoranalogy

Most people don't know that Claude can now watch any video for you. But if you try doing this yourself without this skill, it just gives up and tells you it can't do it. The skill is called watch, and all you have to do is paste the link to literally any video on the Internet, and Claude can now watch it.

It demolishes this two hour interview with Boris Czerny in just five seconds. The skill gives Claude the video's frames and transcript together, so it doesn't just read a transcript, it actually sees what's going on on screen. And it works for anything, YouTube videos, looms, TikToks, or local files.

I'm not kidding. It supports over 1,600 sites at this point.

And since I first released it on GitHub last month, it's gained nearly 3,000 stars along with hundreds of thousands of videos across platforms. Today, I'm launching version two. It's up to 40 times faster, cheaper to run, and now handles ultra long videos with smart frame selection that captures the moments that matter.

All controlled by a single dial, you can slide from free four second skim to a full frame by frame deep read. In this video, I'll show you exactly how it works and how you can get it set up in Claude or Claude code in just a few seconds. So let's get into it.

Now before I tell you how I managed to make this skill 40 times faster than version one, I wanna show you one of the improvements I am most excited about and that's smart selection. Claude doesn't have a video understanding model out of the box, but it is multimodal, which means it can understand static images. So this skill gives Claude a flipbook of frames paired with a timestamped transcript, so it can understand exactly what is happening on screen and when.

But there was a problem with my original approach in version one. Before, the skill would grab a 100 frames equally across the entire video, which worked great on shorter videos. But on a two hour video like the one we just did, that's only one frame every seventy two seconds.

And on a ten hour video, that's one frame every six minutes. So Claude was basically just reading the transcript with a few random screenshots. Now to solve this, we can't just give Claude every single frame in a video either because that's going to be massively expensive and slow.

And at its core, the problem really isn't about the amount of frames either. It's about selecting the right ones. So that's why I've developed smart scene selection.

The skill now scans for points where the picture genuinely changes throughout the video. Think new slides, scene changes or visuals. So every single frame Claude looks at is a real moment worth analyzing, but it actually goes even further than that.

Now let's go back to that video with Cherny from earlier. Here he's presenting his Claude code workflow and literally says you can see while he's sharing his screen.

The old version of the skill would completely miss that because it didn't give Claude the ability to go and look at that part of the video specifically. And scene selection doesn't pick up on this either because it's not at the beginning of a new scene. So that's where dyadic detection comes in.

So now Claude reads the transcript first and actually scans for something called the dyadic gestures. Essentially, anytime someone says, look here, as you can see, watch that what happens, and it goes and grabs a frame at that exact second and includes it in the analysis.

But sometimes smart selection isn't enough and you just need more power. So I added a new optional mode I call token burner, and it lets Claude totally off the leash.

When in token burner mode, there is no hard cap on frames and Claude has the ability to scan and extract as many frames as smart selection can detect. That means every scene change in the entire video becomes a frame, up to two frames per second. This is for when you do not care about token cost and you just want Claude to see absolutely everything.

It's the heaviest setting and it gets expensive, so use with caution. And there's one more thing in here that's become my favorite feature in the entire skill.

Now let's get into how I made this skill 40 times faster than version one. That's because this version ships a new mode called efficient. It's for when you want Claude to actually see the video, but you don't need every last frame, just the vibe.

So if you're watching your usage limits, this mode is for you. Here's how it works. To keep file size down, a video doesn't store every frame in full.

Roughly once a second, it saves one complete snapshot. That's called a key frame. And every frame in between only records what changed since that last snapshot, which is far cheaper than storing a whole new picture 24 times a second.

Efficient mode takes advantage of that. Instead of decoding all 24 frames a second, Call just pulls the keyframes that the video already has baked in, and that's where the 40 x comes from. The normal mode has to rebuild every single frame and compare it to the one before it, just to spot where the scenes change.

Pulling key frames skips all of that. They're already sitting there ready to grab, so there's no rebuilding or comparing, and that's how you can get the 40 x speed gains. Now in the last video, I think over half the comments were about how many tokens this skill uses.

Let's actually talk about that for a second because I've shipped a feature that makes it way cheaper. As a few people pointed out, in most videos, a ton of the frames are basically identical. Take a talking head video for instance, where most of the time there's nothing on screen worth capturing.

The old version paid full price for every one of those frames, even though they were all the same shot. So now before Claude sees anything, the skill does a quick cleanup pass.

It shrinks every frame down to a tiny thumbnail and compares each one to the last. And if two frames are basically the same picture, it throws the duplicate away. So you only pay for frames that are actually different.

It only kills the frames that are genuinely doing nothing, so you keep all of the signal and stop paying for the pointless frames. Okay. Now let's get this running on your own setup and it takes about two minutes.

If you're on Claude code and you're installing for the first time, head to the link in the description below. Copy these two commands into Claude code. The first one adds the marketplace and the second one installs the watch skill from marketplace.

One thing to note though, make sure you've got auto update turned on. I'm shipping new features to this one pretty regularly, and auto update means every new release just flows to you automatically, so you don't have to think about it. And if you've already installed the first version, you don't need to reinstall anything.

Just go turn on auto update, or jump into the marketplace menu in Claude code and hit update to pull this new version. If you're already using regular Claude, not Claude code, you can just add the plugin marketplace straight from the customize menu.

And for anything else, you can install it through skills.sh. I put the full install instructions for every platform in the readme, so whatever you're running, there's a path for you there. Now when you first run it, the first thing it's going to ask you is which mode you want as your default.

So let me run you through all of the options now so you know which one to pick. The first mode is transcript. In this mode, there's no frames at all.

This is the fastest option because Claude pulls the transcript only and reads it in a few seconds. Next, there's efficient. This grabs the key frames so Claude actually sees the video, but stays quick and cheap.

This is the one I set as your default for everyday use. Gets you the vibe of almost any video without burning much usage at all. Balanced.

This is proper scene aware mode. There's more frames and they land on real moments. Use this when the visuals actually matter and you want Claude to get it right.

And if you're not sure, this is a safe pick. Token burner is the last one where there's no limit and every scene change becomes a frame. This is expensive, so save it for the long important videos where you want Claude to genuinely see everything.

You can set a default now and override it on any video later, so don't overthink it. I'd start with efficient or balanced.

Once you've done that, it'll ask for a Grok or OpenAI API key. That's only used for transcription as a backup when there's no captions available. The skill will always pull free captions off the video first and the key is there for any videos that don't have it like a Loom or a raw local file.

I'd go with Grok because it's free and a lot faster than OpenAI. Paste it once and you're set. That's watch skill version two.

I use this skill every day, and it's plugged straight into my AI operating system. So every video I would have watched manually, Claude watches for me and actually writes clean notes and files it away totally automatically. I've got it keeping track of every creator in the AI space so I always know what's going on and who's posting what.

I broke down that entire system in the original video and that's where I also go deep on the pricing and exactly what it costs to run, plus a couple of other features that we didn't have time to cover here. So if you want to see how this actually fits into a system like that, that's the next video to watch and it's linked

The Hook

The bait, then the rug-pull.

Claude can't watch video on its own — but a free Claude Code skill fakes it convincingly, and the creator just shipped a version that does it 40 times faster.

Frameworks

Named ideas worth stealing.

06:33list

Four detail-level modes

Transcript (fastest, no frames)
Efficient (keyframes only, default)
Balanced (scene-aware, more frames)
Token burner (uncapped, every scene change)

A single dial for how deeply Claude analyzes a video, trading cost/speed against visual completeness.

Steal forany tool that lets users trade cost against thoroughness on a sliding scale rather than a binary toggle

CTA Breakdown

How they asked for the click.

VERBAL ASK

05:38product

“head to the link in the description below... copy these two commands into Claude code”

Soft, tool-native CTA embedded mid-tutorial (install steps) rather than a hard pitch; description also links a free-download opt-in and a paid AI-strategy call.

FROM THE DESCRIPTION

PRIMARY CTAWhere the creator wants you to go next.

Storyboard