Transferring motion from a reference clip, and where the technique falls apart

Motion transfer reads the movement in a driving clip and applies it to a different subject. In principle this separates two things that are normally fused: what moves, and what is moving.

It is genuinely useful when the movement is the hard part — a camera move you cannot describe, a specific rhythm, a gesture with particular timing. Describing those in words rarely gets you there; showing them can.

It also fails in ways that are predictable once you understand what is actually being transferred, which is less than the name suggests.

Camera movement transfers most reliably — a push, a pan, a rise. It is a global transformation of the whole frame and there is nothing subject-specific about it.

Large body movement transfers reasonably: walking, turning, sitting, a broad gesture. The pose sequence is legible and maps onto a different figure.

Rhythm and timing transfer well too, which is why the technique suits work cut to music. The beat structure of the movement survives even where the detail does not.

Fine hand movement, facial expression and anything involving fingers degrade badly. These depend on precise anatomy, and mapping between two different bodies loses exactly that precision.

Fast movement also suffers. The model has fewer frames to work from per unit of change, and it fills the gap by inventing.

Interaction with objects is the hardest case: the driving clip's subject was holding something the new subject is not, and the transferred motion describes a grip that has nothing to hold.

A driving clip framed as a wide shot of a standing figure transfers well onto another wide shot of a standing figure. The same motion applied to a close-up does not, because the mapping between frames is doing too much work.

Match framing, subject scale within the frame, and rough proportions. Where those align, the transfer looks natural; where they differ substantially, the result has a characteristic sliding quality.

Clean driving footage matters too. Motion blur and low contrast make the movement harder to read, and errors in reading it become errors in the output.

A specific camera movement — a particular speed, a slight drift, an imperfect handheld quality — is very hard to specify in language and easy to demonstrate.

Shooting ten seconds on a phone to use as a driving clip is often faster and more accurate than iterating on prompt wording, and it gives you exactly the move you had in mind.

For subject movement, be more sceptical. Describing an action in the prompt is frequently as good and much less constrained than mapping someone else's performance onto your subject.

Check hands, feet and face specifically, at full size. These are where the mapping breaks first, and they are easy to miss while watching the overall movement, which is usually convincing.

Check the final second as well. As with any generated video, error accumulates, and a transfer that was clean at the start may have drifted by the end.

Where the broad movement is right and the detail is wrong, consider whether the shot can be reframed to exclude the detail. That is usually faster than another attempt, and it works.