I've been refining the craft around AI-generated animated videos on platforms like Runway, and the hardest part is holding a character consistent across shots. A single generation can look excellent; by shot ten the character has often drifted in scale, wardrobe, or facial structure, and the short stops holding together.
The pipeline that fixed it is built like any production system I ship, with one source of truth, a gate before every generation runs, and a score on every output after. Those are engineering habits, and they carried into generative video without much translation. Here is the whole thing, start to finish, on one character from a short called First Day.
Start with one image
Everything begins with one approved image of the character. This is Bao, the lead. At this point he exists as a single render, from one angle, in one pose. Generate the next shot from this alone and the model has to re-invent everything it cannot see, which is where drift starts.
Turn it into a master sheet
I first treated drift as a prompting problem and spent time on richer descriptions, added constraints, and fixed seeds. Those changes reduced the symptom without removing it. The drift returned a few generations later, which told me the cause was upstream of the prompt.
The fix was to consolidate the character onto a single master sheet, one canvas holding the full turnaround and the expression range, referenced as one asset. Scale drifted whenever individual angle or expression sheets were passed as the primary reference in place of that consolidated sheet. Given an ambiguous reference the model returned an ambiguous result, so consistency turned out to be a property of the inputs rather than something a prompt could recover.
Generate every shot from that sheet
Every shot references the master sheet first. These three frames are the same Bao at the bus stop, in the classroom, and at his desk next to Sage, generated across different scenes, camera angles, and lighting. The face, scale, and wardrobe hold because each generation instances the same source instead of re-inventing the character from the prompt.
Then turn it into video
The shots assemble into the finished short. You can watch the result here: First Day.
Motion is where drift is most visible, because the eye tracks a face as it moves, so a moving shot is the real test of whether the earlier steps held.
Why it holds: the architecture underneath
The walkthrough works because the production sits on four layers, each with a single responsibility.
Source of truth. The characters, locations, and fixed facts of the project, built once, versioned, and referenced by ID. Nothing here is generated mid-session.
Production standards. How to generate correctly, applied to every shot and every short: camera and continuity rules, the order references are passed, and the documented failure modes.
Shot plan. What to make and in what order, with the source-of-truth asset IDs linked inline.
Operator. The single entry point, loaded first each session. It decides what else loads and what must be true before generation runs.
The separation is what makes drift diagnosable. When it appears, the layer that produced it is identifiable at once: a character changing scale points to the source-of-truth layer, and the fix belongs there. That layer is stabilized before anything above it depends on it.
A gate before generation, an evaluation after
The part that made the pipeline reliable is standard production engineering.
A pre-generation gate runs before any generation. It confirms a small set of conditions, and if any is unmet, generation does not run: the required reference exists, the camera axis is stated explicitly, and the prompt is structured environment-first.
An evaluation rubric runs after. Each generation is scored against defined criteria for character consistency, spatial accuracy, motion, and pacing. Output below the threshold does not enter the timeline; it is rejected, its root cause is recorded, and that failure is added to an anti-pattern registry so it is caught earlier next time.
For anyone who has built evaluations for an LLM, this is the same discipline applied to generative video: a check on input, a score on output, and a growing record of known failure modes. The creative quality improves because the surrounding system declines to ship a degraded result.
Where it still breaks
The architecture addresses character and spatial consistency and camera behavior. On-screen text remains a genuine limitation of current video models and is handled as a post-production step rather than solved at generation time. The exact reference specification and the scoring thresholds are tuned per production and maintained as part of the build.
Most of the anti-pattern registry came from failures observed while running the pipeline, not from assumptions about what might go wrong. A reversed camera axis made two characters swap positions between shots. A character faced the lens because its eye-line was never specified. Each of these produced a specific rule and a documented fix that the pipeline now applies before the same failure can recur.
Why the methodology transfers
I built the shorts to prove out the methodology, and it carries directly into marketing, advertising, and promotional work, because the hard part is the same. A defined source of truth, a gate before generation, and an evaluation after. None of it is tied to one platform or one model; the four layers hold wherever the generation runs.
The advantage in generative media has moved off the models themselves and onto the judgment around them: which model to reach for, which workflows hold up, and which tools reach an outcome without burning tokens or time. Treating the pipeline as instrumented infrastructure is what makes that judgment repeatable instead of a guess each time.
Frequently asked
- Why do AI-generated characters drift between shots?
- Drift comes from an ambiguous reference layer, not the prompt. I hold characters consistent by defining the character, location, and world as one source of truth that every generation references, so scale, wardrobe, and face stay locked across shots.
- How do you keep AI-generated characters consistent across a video?
- I treat the character, location, and world as static state that every shot references, gate each generation so it cannot run without valid references, and score every output against a rubric before it ships. Better prompts alone do not hold; the consistency lives in the architecture around them.
- How do you run quality control on AI-generated video?
- Every generation is scored against a defined rubric for character consistency, spatial accuracy, motion, and pacing, and anything below the bar is rejected and logged as a documented failure the pipeline catches the next time.
- Does this depend on a specific AI tool or model?
- No. The workflow is platform- and model-agnostic, so it holds as tools change and lets me pick the model that reaches an outcome without burning time or budget.
- Does this transfer to professional creative production?
- Yes. It is an end-to-end, documented workflow, so it hands off to a creative or production team and applies the same way to marketing, advertising, and promotional content.