Measure a reference

Authoring from memory is the biggest self-inflicted error source: guessed hex, guessed positions, guessed timings all compound. Measuring is cheaper than guessing. When you have reference footage (a video or image you’re trying to match), pull the exact numbers out of it and feed them straight into the RenderRequest.

1. Extract a frame at a timestamp

# one frame at t = 3.5s → frame.png (fast seek before -i)
ffmpeg -ss 3.5 -i reference.mp4 -frames:v 1 -update 1 frame.png

# a strip every 0.5s to find the exact moment something happens
ffmpeg -i reference.mp4 -vf fps=2 frames/%04d.png

2. Sample a color at a pixel

Pick the pixel off the extracted frame, then read it as hex. The frame’s pixel space is the canvas space, so (x, y) maps directly (see step 3).

# ImageMagick: hex at pixel (640, 360)
magick frame.png -format '%[hex:p{640,360}]' info:      # -> 0B1B3A

# or Python (Pillow)
python -c "from PIL import Image; print('#%02x%02x%02x' % Image.open('frame.png').getpixel((640,360))[:3])"

Sample a few nearby pixels and take the mode — antialiasing and compression make any single pixel noisy. Feed measured colors into a brand kit — POST /v1/brand-kits extracts one from a URL, and PATCH /v1/brand-kits/{brand_kit_id} overrides individual fields — so the whole scene stays on-palette.

3. Convert a pixel position to a `RenderRequest` value

Positions are center-based percentages of the canvas (see Author a video). Measure the pixel center of the element in the reference frame, then:

x_percent = round(100 * pixel_x / frame_width)   # → "x": "42%"
y_percent = round(100 * pixel_y / frame_height)  # → "y": "58%"

For sizes, measure the element’s pixel width/height and divide by the frame dimensions the same way ("width": "38%"), or use "140px" if the reference and output resolutions match. Font size is pixels at the canvas width — the renderer normalizes font_size by width only, so when the reference frame’s width differs from your canvas, scale a measured glyph height by the width ratio, never the height ratio.

4. Measure timings

Step through the strip from step 1 to find the exact frame an event happens on, then time = frame_index / fps. Do this for every entrance, cut, and audio hit you need to match, and use the measured seconds directly as the element’s time (or a motion’s time) in the RenderRequest.

# find the frame index of a cut, then divide by the source fps
ffprobe -v error -select_streams v:0 -show_entries stream=r_frame_rate -of csv=p=0 reference.mp4

5. See many moments at once

POST /v1/preview with contact_sheet: {"detail": "balanced"} (optionally scoped by window) returns a grid of frames sampled at composition-aware moments — element entrances, transitions, caption starts — each cell tagged with its at and a reason (first_frame, element_start, transition, caption, uniform, last_frame). The sampling plan comes back even on dry_run, before any render. Use it to locate the moments worth comparing, then at for the pixel-exact single frame.

The loop

Measure → author with the measured numbers → contact sheet to find the moments that are off → preview at that timestamp (POST /v1/preview with at) → compare the preview frame to the reference frame → correct the few numbers that are off → repeat. Preview the same timestamp you measured, so the two frames are directly comparable. This tight measure-compare loop is what turns an approximate reconstruction into a match.

MCP Server

Agent Skills

Measure a reference

1. Extract a frame at a timestamp

2. Sample a color at a pixel

3. Convert a pixel position to a `RenderRequest` value

4. Measure timings

5. See many moments at once

The loop

​1. Extract a frame at a timestamp

​2. Sample a color at a pixel

​3. Convert a pixel position to a RenderRequest value

​4. Measure timings

​5. See many moments at once

​The loop

1. Extract a frame at a timestamp

2. Sample a color at a pixel

3. Convert a pixel position to a `RenderRequest` value

4. Measure timings

5. See many moments at once

The loop