Skip to main content

Expressiveness Contract — API exposure plan (Task #4)

Lift render-node’s AGENT_EXPRESSIVENESS_CONTRACT.md (§1–§6, on render-node main = production) into the public API so agents can author everything the engine renders. Production runs the C++ engine only — the contract’s “Rust path: warn-skip” deltas do not gate exposure, so capabilities advertise the full contract flat (no engine-aware lower bound).

The pipeline (every feature moves these together)

Verification

  • Unit: schema accepts; translator emits the contract’s exact field names/values.
  • Golden payloads: render-node:tests/mainTests/ReferenceVideoExitTests.cpp holds the exact engine JSON for the 4 target scenes — assert translator output matches it.
  • Pixel parity (follow-up): drive render-node’s headless runner + probe frames (needs Task #3’s runner; not wired yet — noted so it isn’t mistaken for done).

Phase status

Phase 1 — §1 common transform/looks

  • Blend modes — enum 10→17 (separable + W3C non-separable); BLEND_MODE_MAP → camelCase; un-rejected; emitted on video/text/sticker/progress/audioViz.
  • Perspective tiltrotation:{x,y,z} object on video/sticker (overrides scalar); un-rejected.
  • maskShape (circle/diamond/hexagon/star/heart/triangle) — analytic SDF mask on all visual elements
  • backdropBlur — frosted-glass panel (radius px) on all visual elements; shared _common_visual helper
  • [~] nested shadow/border already emitted; per-corner cornerRadius object deferred (uniform works)
  • full filters grade — +temperature/tint/vibrance/highlights/shadows on video+sticker

Phase 2 — new element kinds → milestone: 4 exit-test scenes authorable

  • gradients[] (§5e — golden-payload verified) · [x] groups[] + stack layout (§5) · [x] shapes[] SVG path/stroke/morph/trim (§5b — golden-verified)

Phase 3 — §4 text + §3 animations

  • per-word style (W3 — golden-verified, completes the 4-scene milestone) · [x] range selector (W2) · [x] value counter · textWrap/[x]glow/RTL
  • audio generators · [x] customAnimations registries · loop/calculateTimings (present) · [x] keyframed effect intensity (§2)

Phase 4 — §5c/5d expressive ✅

  • speed ramps (playbackRateKeyframes) · motion blur (params.motionBlur) · track mattes (matte) · animated masks (mask+maskKeyframes)

Phase 5 — §6 canvas + §2 tail ✅

  • background.gradient · aurora + generic effect props · W3 transitions (whipPan/cinematicZoom/burnEdge/shatter/domainWarp) · watermark
  • chroma_key: already exposed via chroma_settings (HSV); the newer keyR/keyG/keyB YCbCr path deferred (verify before touching working code)

Phase 6 — consolidation

  • capabilities completeness audit (verified: 7 element kinds, 17/17 blend modes, aurora, W3 transitions, gradient/shape fields — all advertised, CI-enforced by test_capabilities)
  • SDK version bump 0.1.0 → 0.2.0
  • Remaining (infra-blocked / decisions): elements reference docs; deprecate flat legacy fields (breaking — needs SemVer decision #2); render-parity CI gate needs the headless runner (Task #3) — interim coverage is golden-payload assertions against ReferenceVideoExitTests.cpp

Phase 7 — the shape fill union

  • shape.fill union + CSS absorptionfill takes a hex colour, "none", a Gradient object, or a CSS linear-gradient(...)/radial-gradient(...) string parsed API-side (api/schemas/css_gradient.py). The CSS string never reaches storage or the wire: it is normalized to the Gradient object during validation, so a malformed one is a 422 with an RFC 6901 pointer rather than a translator code. Same union on gradient.gradient and request-level background_gradient, because an agent that learns it on one tries it on the others within a turn.
  • Wire form: §4f (shapes[].fill as an object) is on render-node main but not on the deployed image, where a non-string fill reads as the "#ffffff" struct default — a white rectangle behind a 200. So the translator fans a gradient-filled rectangular shape into the gradients[] alias, behind engine_shape_fill_gradient (default off). Superseded 2026-08-28 — see Phase 8. render-node main is what the farm runs; the fan-out and the flag are deleted and the gradient rides on shapes[].fill directly.
  • The gradient element is permanently supported … The steering is “prefer the shape fill for new work”, never “stop using this”. Reversed 2026-08-28 by owner decision — see Phase 8. The element is removed from the API.

Phase 8 — removing the gradient element (2026-08-28 → 2026-08-31)

Owner decision, taken with the breaking-change consequences stated in writing. Three removals, no compatibility shims — but rolled out in two deploys, not one (see the sequencing note at the end of this section; a single deploy is not orderable against the migration it needs).
  • The pre-§4f fan-out. engine_shape_fill_gradient, Translator._reject_gradient_fanout and Translator._shape_as_gradient are gone. A shape’s gradient fill goes straight to shapes[].fill as an object. Every rejection the fan-out raised (gradient fill + non-rectangular path / stroke / morph / trim / blend_mode / mask* / matte / backdrop_blur) disappears with it — the combinations now translate.
  • The gradient element. GradientElement, ElementType.GRADIENT, Translator._gradient and the emitted params.gradients[] array are gone. {"type": "gradient"} is a 422 from the discriminated union; the type no longer appears in GET /v1/capabilities. gradients[] stays a parsed part of the renderer contract (CONTRACT.md §16.2) — the API simply never emits it, and omits the key rather than sending it empty.
  • Stored-document compatibility. StrictModel._drop_unknown_keys_on_stored_docs, STORED_DOC_CONTEXT and is_stored_doc are gone: a persisted document is now validated exactly as strictly as a fresh POST. Because of that, alembic 0013 migrated projects.render_request_json, project_versions.render_request_json and jobs.input_json forward first — rewriting every stored gradient element into the shape form and dropping keys retired earlier.
  • The 422 explains itself. api/schemas/retired.py maps ("type", "gradient") to the replacement tag, the prose and the literal JSON to paste, and api/schemas/errors.py swaps that in for the bare union_tag_invalid. Retiring the next tag is one entry in that table, not a new special case in the error path.
  • Retiring a field from here on means migrating storage in a change that ships before the one that drops the field. There is no longer a net between a stale stored key and a failed read.
Sequencing, as actually executed. 699d9b5 landed all three removals plus alembic 0013 in one commit, and the pre-deploy review showed that combination cannot be rolled out: a migration and a deploy are separate events, and whichever ran first left the other side reading a spelling it did not declare (migrate first → the running API meets a shapes[].fill OBJECT where it declares a STRING; deploy first → every stored type: "gradient" element is union_tag_invalid). Either way GET /v1/projects 500s for an affected workspace and in-flight INGESTING renders fail terminally. So:
  1. EXPAND (5b04599, deployed 2026-08-28). Reads both spellings — the element accepted and normalised into a shape, retired keys via the compat net — and writes only the new one. 0013 present, safe, idempotent, not required.
  2. MIGRATE (2026-08-29). 0013 applied everywhere. projects 11/8/0, project_versions 102/94/0, jobs 390/294/84 (scanned/rewritten/unreadable). The 84 are unrelated pre-existing debt — retired enum VALUES and three .mkv source_urls — bounded by the try/except guards in JobService.lint_stored and ::finalize_render_dispatch, which are NOT part of the compat net and stay.
  3. CONTRACT (2026-08-31, this state). The removals above, plus the ("type", "gradient") entry promoted from PENDING_RETIREMENTS into RETIRED_TAGS. api/services/doc_migration.py stays in full — it is how a restored backup or an archived payload moves forward.

Status: Phases 1–5 complete; Phase 6 audit + version bump done. ~22 commits, full suite 1091 pass.

Open API-design decisions

  1. Groups public shape — top-level groups[] on RenderRequest (matches engine) vs element-nesting sugar.
  2. Legacy field migration — deprecate-and-map the flat x_rotation/border_radius-px fields, or keep both (SemVer impact on SDK)?
See memory: exposure-catch-up-task, api-engine-capability-drift, render-node-main-is-production.