I built a system that makes videos by itself. It goes looking for a story, checks whether it is actually true, writes the script, draws every frame, writes the music and posts the result. One of them is below.
What surprised me is that almost everything it does is turn work down. That is also the part I care about most, and the reason has more to do with what I watched as a kid than with any of the code.
Where the taste came from
I watched a lot of LEMMiNO growing up. The channel is known now for hour long documentaries, but back then it was mostly top ten facts videos and I worked through nearly all of them. I could not have said why they felt better than everything else on my screen. Watching them again later, I think it was restraint. Nothing was on screen that did not need to be there, and he would hold a shot long after most editors would have cut away, which made the quiet feel deliberate instead of empty.
Then I started making my own things. Lyric videos, typography moving to music, a few of which passed fifteen thousand views while I was still at school. That taught me something no coding tutorial had. A frame arriving four hundred milliseconds late is not a matter of taste, it is simply wrong, and no amount of good type will rescue it. The only online course I have ever paid for was about video editing, even though code was what I did every day.
3Blue1Brown came later. Grant Sanderson animates mathematics with a Python library he wrote himself, manim, and in that work the picture is not decoration for the argument, it is the argument. Watching a proof move is a different experience from reading one. That set the bar I have measured my own work against since.
Generating was never the hard part
Very little of making a video happens in the timeline. By the time anyone opens an editor, the decisions that matter have already been made: which fact deserves the attention, what to leave out, how long to sit on something before moving on.
Better people said this first. Walter Murch, who cut Apocalypse Now, ranks emotion above continuity in In the Blink of an Eye and weights it more heavily than every other factor combined, which leaves the mechanically correct cut near the bottom of the list. Edward Tufte argues the same thing about the printed page in The Visual Display of Quantitative Information: every mark should carry information and the rest should go. Tufte's ink and LEMMiNO's silence are one instinct in two media, and both of them are about subtraction.
So when people say that AI can make videos now, I think they are describing the last ten percent and calling it the whole job. The first ninety is deciding what is worth making, and that is the part I wanted to build.
What I built
Here is the output. Two and a half minutes, made end to end by the pipeline below. It found the story, checked it, wrote the script, had its own judge throw out the first version, then drew every frame and wrote the score.
Every stage is a written rule file rather than a prompt. One stage can send work backwards and another can drop it outright.
Every morning the first stage works through a few thousand items from a fixed set of places: aggregators, subreddits, launch boards, release feeds, package registries and the trending panels on the big platforms. Almost nothing survives. Whatever does has to trace back to something the agent fetched and can quote, an API response or a commit or a filing, not an article written about one. A good day gives me two stories. Plenty of days give me none, and none is the right answer rather than a failed morning. The restraint I liked as a viewer turns out to be most of the work as a builder.
If I had to keep one piece of this, it would be the judge. A second agent reads the finished script knowing nothing about where the story came from, so it has only what a viewer would have, and if it cannot say what the video is claiming, the script goes back. Everything after that gate is craft. The gate is what stands between the pipeline and confident nonsense.
The words are generated and I am not going to pretend otherwise. The pictures are held to a different standard. Anything carrying a fact, a number or a chart or a diagram, is drawn by code from the figures themselves, because an image model will happily produce a chart that looks right and is wrong. Generated images appear only as abstract backgrounds, tinted into the palette, holding no information. A picture is allowed to set a mood and never allowed to be the evidence.
The rules are where the taste lives
Underneath all of it sit eight rule files, about four and a half thousand lines. They are not a style guide. They are a record of things that already went wrong, each written as an instruction, a consequence and the evidence that the mistake was real.
Exempt any block whose content is a counting number. Averaging sub-samples while digits change prints two numbers on top of each other.
7,784with ghost glyphs was called the worst frame in the reel by two independent judges, twice, before the cause was found.
I had added motion blur to keep the animation from feeling mechanical and it quietly corrupted every number that counted up on screen. The rule that fixes it is one sentence long and cost two rounds of review and a re-render to find.
The rules I am proudest of do not ask anyone to be careful. They make the mistake unavailable. Narration is generated and measured before a single frame exists, so a cut landing inside a word stops being unlikely and becomes impossible. Toyota calls this poka-yoke, mistake proofing: shape the process so the wrong move is not on the table. My better rules work that way. My weaker ones are still reminders, and I can tell which is which by how often they get broken.
This is also why the corrections hold, which is the part I would defend hardest. Agent frameworks have memory now, so the honest comparison is not against a stateless prompt but against memory that accumulates on its own. A rule file is written by hand after a specific failure, then reviewed and committed, so it can be read, argued with and reverted. Accumulated memory is a side effect of running. A written rule is a decision someone made. I would rather own four and a half thousand lines I can defend line by line than a larger store I have never read.
Where it actually is
Most of the rule files are not tuned and I know roughly where each one is weak. Some videos come out flat. Some scenes still explain when they should show. The pacing is better than it was and still not reliable. When something is wrong I write the rule that stops it recurring, so the curve is real but slow, bounded by how much of my own attention I am willing to spend watching output.
For work that has to explain something and be right about it, this is the strongest tool I have. For a talking head, a product ad or generated footage the commercial tools beat it easily. HeyGen's Video Agent turns one prompt into a finished multi-scene video and publishes its production behaviour as agent skills; Runway, Veo and Kling generate footage I cannot produce at all. All of them are built around the prompt, which assumes the subject has already been chosen. Nothing stops someone adding the choosing step on top and somebody probably will. The difference is emphasis rather than capability. I started one step earlier, at whether anything deserves to be made at all.
Where it goes, and when it will be available
Explaining things is what this is good at today and not where I want it to stop. Live footage, cutting against music, the kind of edit that makes someone feel something rather than understand something: none of that is close yet.
Running it costs almost nothing. The voice runs locally, the frames are Pillow, the score is NumPy, the encoding is ffmpeg, so a finished video costs electricity and about half an hour of laptop time. Whatever this becomes will not be gated behind per-render pricing.
I am releasing the harness publicly in the next few days: the rule files, the build scripts and the validation gates. I would rather hand over the rules than the videos. Whether a machine can make a video is settled. What stops it making a bad one is still open, and that is the question I find interesting.
Built with
- Pillow for every frame
- NumPy for the score, synthesised sample by sample
- FFmpeg for encoding, and for joining the audio to the picture
- Kokoro for narration, running locally, no key, nothing leaving the machine
- Google Gemini for the abstract backgrounds, and as a second opinion judge on scripts
- Simple Icons for official brand marks, so a logo is never drawn from memory
- Music: Spy Groove by Kevin MacLeod (incompetech.com), CC BY 4.0
- manim by Grant Sanderson, not used here, but the reason I wanted to build any of this
- LEMMiNO and 3Blue1Brown for the standard
- Walter Murch, In the Blink of an Eye, and Edward Tufte, The Visual Display of Quantitative Information, for the reason I think editing is the last ten percent