Designing for Character Consistency in Generative Video

I've been refining the craft around AI-generated animated videos on platforms like Runway, and the hardest part is holding a character consistent across shots. A single generation can look excellent; by shot ten the character has often drifted in scale, wardrobe, or facial structure, and the short stops holding together.

The pipeline that addresses it is built like any production system I ship, with one source of truth, rules every prompt follows, and a review gate before the expensive step. Those are engineering habits, and they carried into generative video without much translation. Here is the whole thing, start to finish, on one character from a short called First Day.

Start with one image

Everything begins with one approved image of the character. This is Bao, the lead. At this point he exists as a single render, from one angle, in one pose. Generate the next shot from this alone and the model has to re-invent everything it cannot see, which is where drift starts.

Turn it into a master sheet

I first treated drift as a prompting problem and spent time on richer descriptions, added constraints, and fixed seeds. Those changes reduced the symptom without removing it. The drift returned a few generations later, which told me the cause was upstream of the prompt.

The fix was to consolidate the character onto a single master sheet, one canvas holding the full turnaround and the expression range, referenced as one asset. Scale drifted whenever individual angle or expression sheets were passed as the primary reference in place of that consolidated sheet, because given an ambiguous reference the model returned an ambiguous result, so consistency turned out to be a property of the inputs.

Generate every shot from that sheet

Every shot references the master sheet first. These three frames are the same Bao at the bus stop, in the classroom, and at his desk next to Sage, generated across different scenes, camera angles, and lighting. The face, scale, and wardrobe hold because each generation instances the same source, and in the bus stop frame he faces away, so that frame shows scale and wardrobe only.

Then turn it into video

The shots assemble into the finished short. You can watch the result here: First Day. That cut still shows some drift, and regenerating it through the current pipeline is the next step.

Motion is where drift is most visible, because the eye tracks a face as it moves, so a moving shot is the real test of whether the earlier steps held.

Four layers, each with one responsibility

The walkthrough works because the production sits on four layers, each with a single responsibility.

Source of truth. The characters, locations, and fixed facts of the project, built once, versioned, and referenced by ID. Nothing here is generated mid-session.

Production standards. How to generate correctly, applied to every shot and every short: camera and continuity rules, the order references are passed, and the documented failure modes.

Shot plan. What to make and in what order, with the source-of-truth asset IDs linked inline.

Operator. The run instructions saved in the workflow. They name the two inputs that change per scene, the script and the location anchor, keep the master sheets fixed, and stop the run for review before animation.

The separation is what makes drift diagnosable. When it appears, the layer that produced it is identifiable at once: a character changing scale points to the source-of-truth layer, and the fix belongs there. That layer is stabilized before anything above it depends on it.

The pipeline as built

The pipeline is a Runway workflow with 369 saved versions, and in one run a scene description goes in and eight animated shots come out in 8 to 12 minutes, across four stages.

1. Claude Opus reads the scene and writes nine panel prompts as JSON, one character sheet panel and eight shots.
2. GPT Image 2 renders a keyframe for each panel. Every keyframe receives all eight approved master sheets and the scene's location anchor, and each keyframe feeds the next as a continuity reference.
3. Claude Opus writes a motion prompt for each approved keyframe, based on what is visible in the frame.
4. Seedance 2.0 animates each keyframe at 1080p with diegetic sound.

The storyboard stage returns a JSON contract, and a parse step reads each key by path.

A review gate before animation

Between the keyframes and the animation, a review gate stops the run. Every frame is checked for Bao's shirt, backpack, paws, and eyes, for wardrobe that stays the same frame to frame, for lighting direction, for sketch artifacts or background text, for species features, and for composition against the script beat. A frame that fails is fixed or regenerated before animation starts, because an animated clip cannot be undone and the credits spent on it cannot be recovered.

Where it still breaks

The architecture addresses character and spatial consistency and camera behavior, and on-screen text remains a limitation of current video models, handled as a post-production step. The exact reference specification is tuned per production and maintained as part of the build.

The production rules came from failures observed while running the pipeline: a reversed camera axis made two characters swap positions between shots, and a character faced the lens because its eye-line was never specified.

Why the methodology transfers

I built the shorts to prove out the methodology, and it carries directly into marketing, advertising, and promotional work, because the hard part is the same. A defined source of truth, production rules, and a review gate before the expensive step. It runs on Runway today, and the pipeline is saved as a JSON graph, and the shot plan is a JSON contract, so another platform would change how the workflow is represented while the workflow itself stays the same.