Give Claude vision: watch any video or carousel

What you're building

Right now Claude can read a transcript. That tells you what was said, and it misses everything that isn't spoken: the on-screen text, the pacing, the fonts, the graphics, where the cut lands.

By the end of this you'll be able to hand Claude the actual frames, and ask it questions like "what's the on-screen text in the first three seconds" or "describe the visual effect in this reel" and get a real answer.

You provide a link. This performs a download plus a frame extraction. You receive a folder of stills Claude can read like a flipbook.

The two things to install

1. ffmpeg, which does the actual frame cutting. On a Mac, run brew install ffmpeg. On Windows, download it from ffmpeg.org. It's free and it's the only tool here that never costs anything.
2. An Apify account for Instagram, at apify.com. You need this because Instagram doesn't let you download a reel directly. Free tier gives you 5 dollars of credit a month, which is roughly a hundred reels. Grab your API token from Settings, then Integrations.

Pull the video off Instagram

Paste this into your terminal, swapping in your token and the reel URL. It asks Apify for the post and saves the response.

curl -s -X POST "https://api.apify.com/v2/acts/xMc5Ga1oCONPmWJIa/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"username":["https://www.instagram.com/reel/SHORTCODE/"],"includeTranscript":true}' \
  -o post.json

Then pull the direct video link out of that response and download the file:

python3 -c "import json; print(json.load(open('post.json'))[0]['videoUrl'])" > url.txt
curl -L "$(cat url.txt)" -o video.mp4

Cut it into frames

This is the step that does the watching. One still every two seconds, scaled down so the files stay small:

ffmpeg -i video.mp4 -vf "fps=0.5,scale=360:-1" frame_%02d.jpg

fps=0.5 means one frame every two seconds. For a fast-cutting reel where things change quickly, use fps=1.5 instead and you'll get roughly one still per beat.

The contact sheet trick

Uploading forty separate images is slow. Tile them into a single grid instead, and Claude reads the whole thing in one go:

ffmpeg -i video.mp4 -vf "fps=0.5,scale=320:-1,tile=4x4" sheet_%02d.jpg

That gives you 16 frames per image. One sheet usually covers a whole reel.

Hand it to Claude

Drag the frames or the contact sheet into Claude, then paste one of these depending on what you want back.

To understand what the video does

These are sequential frames from a short-form video, in order, roughly two seconds apart. Walk me through what happens: the on-screen text at each stage, where the cuts land, the visual effect being used, and how the whole thing is structured. Tell me what makes it hold attention.

To reverse-engineer the effect so you can recreate it

These are sequential frames from a video effect, in order. Work out exactly how it was made, then write me a short prompt I could paste into an image-to-video tool to recreate it. Name the specific words in that prompt that are doing the work.

To pull the hook structure out

These frames are the first six seconds of a reel. Tell me the exact on-screen text, when each line appears, and what the hook is doing to stop the scroll. Then give me three hooks built on the same mechanism for a different topic.

This works on carousels too

A carousel comes back from the same command as a list of child posts. Each slide is either an image or a video. Save every slide with this:

python3 -c "
import json, urllib.request
d = json.load(open('post.json'))[0]
for i, c in enumerate(d.get('childPosts') or []):
    u = c.get('videoUrl') or c.get('displayUrl')
    urllib.request.urlretrieve(u, f'slide_{i:02d}.jpg')
    print('saved', i, c.get('type'))
"

Then drag the slides into Claude and ask it to read the whole deck. This is how you break down someone's carousel structure in about thirty seconds: cover, the promise, how the middle slides carry it, and where the call to action sits.

YouTube and TikTok

Swap Apify for yt-dlp, which is free and needs no account. Install it with brew install yt-dlp, then:

yt-dlp -f mp4 -o video.mp4 "PASTE_THE_URL_HERE"

Everything after that is identical. Same ffmpeg command, same prompts.

What this actually costs

1. ffmpeg and yt-dlp are free, permanently, with no account.
2. Apify has a free tier of 5 dollars of credit a month that doesn't roll over. Scraping alone runs about 2.60 dollars per thousand posts, so the free tier covers far more than you'll use. Transcripts are what actually eat the credit, so leave includeTranscript off when you only want frames.
3. It's a terminal workflow. If you've never opened Terminal, do the first one on a short reel so the whole thing takes two minutes rather than twenty.
4. Public posts only. Private accounts won't return a video URL.

The links

🎞️ ffmpeg: ffmpeg.org/download.html
📥 Apify: apify.com
🔎 The Instagram reel scraper: apify.com/apify/instagram-reel-scraper
📺 yt-dlp: github.com/yt-dlp/yt-dlp
🤖 Claude: claude.ai