Batch generation without babysitting, and waking up to usable output

A model that takes minutes per frame is unusable as an interactive tool and entirely reasonable as an overnight job. Nineteen frames is an hour; a hundred is a night.

What makes the difference is whether the run can proceed without you. A queue that stops on the first failure, or that produces files you cannot trace back to their prompts, turns an unattended night into a wasted one.

Three properties make a batch trustworthy: it continues past failures, it records what produced each output, and it tells you what happened.

Wrap each item so an error is recorded and the loop moves on. A batch that halts at item four leaves ninety-six items ungenerated and you find out in the morning.

for job in jobs:
    try:
        run(job)
        log(job, "ok")
    except Exception as e:
        log(job, f"failed: {e}")
        continue

The exception is a failure that will clearly affect everything — the model failing to load, the disk full. Detect those specifically and stop, because continuing produces a hundred identical errors.

Write a small file next to each output with the prompt, the seed, the model and the settings — or embed it in the filename where it is short enough.

out/
  shot-04.jpg
  shot-04.json     prompt, seed, model, steps

In the morning you will want to re-roll three of them with one change. Without the recipe, that means reconstructing what produced each — which is guesswork by then.

Name outputs by their slot rather than sequentially, so a regenerated frame overwrites the right one and the set stays coherent.

Build the runner so it accepts a target and regenerates only that one:

python3 batch.py                    # everything
python3 batch.py shot-04 shot-11    # just these

This is the single feature that makes a batch pipeline practical rather than theoretical. Out of a hundred frames, a handful always need another attempt, and the re-roll path is used far more than the full run.

Have it read the saved recipe by default so a re-roll is a genuine repeat, with the option to override one value.

At the end, write a summary: how many succeeded, how many failed, which ones, and how long it took.

done: 94 ok, 6 failed, 5h 12m
failed: shot-07, shot-19, shot-33, shot-40, shot-52, shot-88

That is what you want to see first thing, and it converts directly into the re-roll command.

Keep the detailed log separately for when a failure needs investigating, but do not make reading it the price of finding out whether the night worked.

Always run a short subset before committing to the full batch. A wrong prompt template, a wrong output path or a wrong resolution repeated a hundred times overnight is a night lost, and it is entirely preventable by checking three.

Also confirm there is disk space for the whole run and that the machine will not sleep. Both are boring and both have ended batches at forty percent.

Then start it and leave. The point of the whole arrangement is that the machine works while you do not.