Awaken — build 3D games in your browser

Buy a character pack and an animation pack from the same store and they will not fit together. The animations are authored on the store's own rig; your character has its own skeleton, its own proportions, and its own idea of which way each bone points. Making one drive the other is retargeting, and it is the single hardest piece of maths in this engine.

Awaken's answer: one clip library stored in rig space, re-baked onto any character on import. This post is about why the obvious approach fails, what replaced it, and a bug I spent days proving did not exist.

The obvious approach, and why it breaks

The textbook retarget bridges two rigs through a shared reference pose. Put both in a T-pose, and for each bone compute a fixed correction:

A[b] = charTpose[b] · clipTpose[b]⁻¹

then apply that correction to every animated frame. It is one matrix per bone, computed once. It is also what I shipped first.

It works when both rigs were exported by the same person on the same day. Across two independently-exported rigs it falls apart, because nothing forces two skeletons to agree on bone roll. A bone's direction is determined by where its child sits. Its roll, the rotation about its own length, is a free parameter the exporter picks. Measured across the rigs I test on:

Disagreement	Measured
One arm's authored roll	flipped about 180°
Spine bones	57° to 106° apart, per bone
Head and clavicle axes, across families	wrong entirely

The T-pose bridge does not correct that mismatch. It bakes it in, because the correction is derived from the same authored axes that disagree. The result is an animation that plays with the elbows in the wrong plane and the spine subtly screwed, and it looks like the maths is broken rather than like the input is.

Retarget from positions, not from axes

The fix is to stop asking the bone which way it thinks it is oriented.

For each bone, build an orthonormal frame out of pure geometry:

X is the bone's direction at the canonical T-pose, derived from body axes rather than from its bind direction, which differs between an A-pose and a T-pose export.
Y is a body reference made perpendicular to X: up for the arms, forward for the legs and the axial column.
Z is X cross Y.

Every input to that frame is a bone position. Roll never touches a position. So two rigs of the same rough shape produce the same frame even when their authored rolls differ by 180 degrees, and expressing the motion in that frame transfers it exactly.

flowchart TD
  A["Clip bone motion
(source rig space)"] --> B["Express in the source's
geometric frame"]
  B --> C["Frame-to-frame:
same body geometry"]
  C --> D["Re-express in the target's
geometric frame"]
  D --> E["Character bone rotation"]
  R["Authored bone roll"] -.->|"never consulted"| C

Twenty bones go through the frame map: both clavicles, the arms and wrists, the legs and feet, and the axial column from hips through spine, chest, upper chest, neck and head.

Toes are deliberately absent. Their direction in a T-pose is ill-defined, and a frame built on an ill-defined direction is worse than no frame. They follow the frame-mapped foot at bind, which reads correctly.

The wrist only works because the T-pose canonicalisation puts both rigs' hands palm-down first. Without that step the hands are two different conventions and the frame is comparing incomparable things.

Which way is forward?

The frame needs a forward axis, and a skeleton does not carry one. You infer it from the body.

My first version used the toes: feet point forward, so the hip-to-toe vector is forward. That is true for a character standing still, and false for a great many clips. Anything authored from a stride rest pose (one foot planted ahead of the other) has toes that disagree with the body, and those clips baked mirrored backwards: the character ran away from the direction it was facing.

Forward is now a vote between two cues: the foot-to-toe direction, and the knee's displacement forward of the hip-to-ankle midline. A bent knee points forward almost regardless of stance. When the cues cancel out completely the function returns null and the caller falls back rather than picking a confident wrong answer.

That last part matters more than the cue itself. A geometric inference that cannot admit failure will hand you a normalised vector built from noise.

The part I gave up on

Hands and fingers are still left at bind pose.

I made four attempts at automatic roll correction for them. Each fixed one rig family and broke another, because a hand has no long axis to build a frame from and no unambiguous body reference. The finger retarget transfers a clip's absolute curl and spread where a clip carries it, but a general roll fix I would trust across families does not exist in this codebase.

That is written down in the docs as a limitation rather than left for someone to discover. A character holding a sword still holds it; a character doing something expressive with its hands does not.

The bug that was not there

For a while I was certain the retarget was producing twisted, backwards characters. The evidence was a character in the viewport, plainly wrong.

I spent days on the maths. Then I stopped and built six headless measurements instead: bone-direction error against the source clip, per-axis rotation deltas, root displacement, limb-plane normals, and a silhouette comparison rendered offscreen and diffed, all run on a real FK character with real clips from another family.

Every measurement came back clean. The retarget was correct, and had been correct the whole time. The character looked wrong for a reason outside the retarget entirely.

I did not enjoy that, and it is the most useful thing in this post. "It looks wrong" identifies a symptom, and a symptom in a chain of six transforms tells you nothing about which transform is at fault. The instinct to fix the most mathematically intimidating link is exactly backwards: it is the link you are least able to verify by eye, so it is the one where a "fix" is most likely to be a random walk that happens to make this one character look better.

Measure the link before you edit it. If you cannot measure it, that is the thing to build first.

Making it fast enough to actually use

An animation pack is not one clip, it is hundreds. Importing 660 of them took long enough that I assumed the FBX parser was the problem.

Profiling said otherwise: roughly 90% of the time was per-frame retarget sampling, not parsing. Every frame of every clip was re-walking structure that never changes between frames.

Two fixes, in order:

Hoist the invariants. The FBX node structure and each bone's constant matrices are the same on frame 1 and frame 900. Computing them once rather than per frame was about 1.6x.
Move it off the main thread. Clip baking is embarrassingly parallel, one clip per task, so it went to a worker pool. The editor stays responsive and the import scales with cores.

WebAssembly was the tempting third option and I did not take it. The work is already numeric and monomorphic, which is the case where a JIT does well; the win would have been small and the cost of a second toolchain is not.

What I would tell someone starting this

Retargeting is not one algorithm, it is a stack of conventions that other people chose and did not write down. The maths is the easy half.

The decision that mattered most was building the correction from positions. Every input derived from an authored axis is an input two rigs are free to disagree about. Positions are the part of a skeleton everybody agrees on, and everything that survived contact with real content is built out of them.