All guides

Local AI line · stop 05 of 14 · 22 min · members

Running a small model locally, and designing around its two failure modes

A small model that punches above its size, and the two failure modes you have to design around.

Free with an account

Sign in to read.

Membership is free: an account opens all 86 script pages. The Lab, Studio Canvas and the paid guides need the $99 pass, paid once. Already signed in on this browser? The page opens by itself.

01

The appeal

A small model that punches above its parameter count.

Fast, modest in memory, and good enough for a great deal of real work.

Smaller models occupy a useful position: quick enough to iterate with, light enough to run alongside everything else you have open, and capable enough that the output is usable rather than merely indicative.

For a machine that cannot comfortably hold a large checkpoint, this is not a compromise — it is the option that lets local generation exist at all.

The trade is specific rather than general. Small models are not uniformly worse; they are worse at particular things, and knowing which lets you design work that avoids them.

02

Failure one

extremities.

Hands and feet degrade faster than anything else as model size drops.

Fingers merge, counts go wrong, joints bend incorrectly. This is the most visible weakness of smaller models and it does not improve with more attempts.

Design around it. Frame so hands are outside the shot, obscured, holding something, or small in frame. All of these are ordinary compositional choices rather than obvious workarounds.

Where a hand must be prominent, generate the frame without it and composite, or accept that this particular shot needs the larger model. Re-rolling is the one approach that reliably wastes time.

03

Failure two

Complex scenes with several interacting elements.

Capacity shows up as an inability to keep multiple things coherent at once.

A single subject on a plain ground is well within reach. Three people interacting in a detailed environment is where a small model shows its size — spatial relationships become incoherent, objects merge, and the scene reads as assembled rather than photographed.

The practical rule is one main subject and a simple environment. That is also, incidentally, a strong compositional constraint, and work made under it frequently looks more deliberate than work with more in the frame.

For genuinely complex scenes, build them in layers: generate elements separately and compose. This is better practice regardless of model size.

04

Prompting it

Shorter prompts, concrete language.

Smaller models have less capacity to reconcile competing instructions.

Long prompts stacked with style references perform worse here than on larger models. The capacity to hold many constraints simultaneously is exactly what is reduced.

Write shorter and more concrete. Name the subject, the material, the light and the framing, and leave out the adjectives. Three specific instructions land better than ten vague ones.

Test whether it prefers natural sentences or comma-separated terms — small models often have a clear preference, and matching it produces a noticeable improvement for no cost.

05

Where it is strong

Speed, iteration and volume on simple subjects.

Play to this rather than pushing it towards its limits.

Single-subject portraits, product shots on plain grounds, textures, backgrounds, and any work where you need many variations quickly — all of these sit comfortably inside its capability.

It is also an excellent exploration tool ahead of a larger model, for the same reason a fast model is: resolving composition cheaply and spending the expensive generation on the answer.

The mistake is treating it as a lesser version of a big model and being disappointed by the same prompts. Treated as a different tool with a different envelope, it earns its place.

06

Deciding

Map its envelope once, on your own subjects.

An afternoon that tells you what to send where.

Run your usual subjects through it and note which come back reliably and which do not. That list is worth more than any general assessment, because it is specific to what you actually make.

Keep it as a routing rule: these subjects go to the small model, these to the large one. Then the choice is automatic and you stop losing time to generations that were never going to work.

Re-check after any significant update. Small models improve quickly, and an envelope mapped a year ago is probably too conservative.