Awaken
← All posts
#10importersunityreverse-engineeringhashing

The ids Unity computes and never writes down

Post #3 was a tour of four asset formats at byte level. This one is about a single number in one of them, because that number cost me more than the rest of the Unity importer put together.

A Unity scene records a per-instance override like this:

m_IsActive: 0
target: {fileID: -3329563214062771084, guid: 8f2c…, type: 3}

That is a real override from a Synty ship prefab. It means hide one specific sub-object of that model. Which sub-object? The number is a 64-bit local fileID of something inside an FBX. Nothing in the package says which.

The table Unity stopped shipping

Older Unity wrote a lookup table into the model's .meta file, fileIDToRecycleName: fileID to name, right there, free. Post #3 leans on it to resolve material slots.

Unity 2019 introduced fileIdsGeneration: 2 and replaced that field with internalIDToNameTable. It writes it empty. Not sometimes: as a rule. Unity can regenerate every id on demand at import time, so it has no reason to persist the map, and the importer that reads the package afterwards is not Unity's problem.

The consequence in a stock pack is total: every override that names a sub-object is unattributable. A ship prefab that hides one rolled-up sail hid nothing, so it drew both sails at once. A creature whose tentacles are posed by bone overrides stood with its tentacles straight out. The data was right there in the YAML, addressed by a number I could not decode.

The only fix I had was harvesting the ids out of a Unity install: run Unity once, let it import the pack, read the ids back out of its Library/Artifacts database, and ship an enriched copy of the pack. It works. It also means a browser-based importer whose whole pitch is no install needs you to install Unity first. That is not a fix, it is an admission.

Reading the function out of the binary

Unity's macOS binary ships symbols. Following AssetImporter::GenerateHashBasedFileID into AssetImporter::MakeHashKey, the key it builds is a string:

"Type:" + <serialized type name> + "->" + <identifier> + <collision index, decimal>

and the id is that string through UNITY_XXH64, seeded 0, reinterpreted as a signed 64-bit integer:

id = BigInt.asIntN(64, xxh64(utf8(key), 0n))

The collision index starts at 0 and only increments when the resulting id is already taken inside the same imported asset. Across an 88-model harvest it was 0 for all 2,544 objects, so in practice it is a constant that exists to be correct rather than to be used.

xxHash64 itself is Yann Collet's reference algorithm, which is public and easy to port. The whole thing looked like an afternoon.

The identifier is a path, not a name

It was not an afternoon, because the identifier is the part that is easy to get wrong, and every wrong guess produces ids that are confidently, uniformly incorrect.

The identifier is not the object's name. It is its position in the imported prefab, built by AssetImporter::RegisterGameObjectHierarchy as a /-joined path starting at a fixed root, //RootNode/root, with one segment per node. A GameObject contributes its name. A component contributes its type name, not its name:

GameObject "Hips" under "root"   →  //RootNode/root/root/Hips
its Transform                    →  //RootNode/root/root/Hips/Transform
its MeshRenderer                 →  //RootNode/root/root/Hips/MeshRenderer
the Mesh asset named "Hips"      →  Hips          (registered by name, not by path)

Note the doubled root. The model prefab has its own root GameObject, and the FBX's top-level node sits under it. Except when it does not: a model whose FBX has a single top-level node has that node merged into the prefab root, while a model with several top-level nodes gets a synthetic root named after the file.

That single rule is worth the whole post. Scored against Unity's own tables:

Rule Ids reproduced
Merge a single top-level node into the root 2604 / 2604
Do not merge 1000 / 2604

Same hash, same key format, same everything else. Get the merge wrong and you reproduce 38% of the ids, which is worse than reproducing none, because 38% looks like a bug in the remaining 62% rather than a wrong premise.

The finished function reproduces 2,544 of 2,544 harvested records and every row of an enriched pack, with no fitted parameters. A stock pack now resolves activation masks, bone poses, mount points and material attribution with no Unity anywhere in the loop. The harvest script survives only as a fallback for what computation cannot reach.

flowchart TD
  A["Override names
fileID -3329563214062771084"] --> B["Walk the FBX node tree"]
  B --> C["Build the identifier path
//RootNode/root/… + type"]
  C --> D["key = Type:T->identifier + index"]
  D --> E["xxHash64(key, seed 0)
as signed 64-bit"]
  E --> F{"Matches the
override's id?"}
  F -->|yes| G["That is the sub-object"]

Three defects it flushed out

Being able to compute the ids meant I could compare my import against Unity's own tables row by row, instead of against my own eye. Three real bugs surfaced immediately, and none of them would have been visible any other way.

The merged root's Transform must not be named. Every placement in a scene writes that transform. Naming it made the importer read each placement as a bone pose: 1,001 bogus mesh bakes, the placement applied twice, and a ship with a visibly broken bow. After the fix, 15 poses remain, and all 15 are genuinely articulated.

Rest-world transforms ignored a node's own RotationPivot. The mast attach points carry their offset only there, so all three read as the origin, and every mast mounted at the same spot.

Activation is layered, not subtractive. A ship preset overrides the mast prefab, and its m_IsActive list contains re-enables as well as disables. The importer only ever subtracted, so an ON was invisible and every ship inherited the mast's default. Activation is now resolved per sub-object through the chain: model default, then each enclosing prefab, then the scene placement, outermost winning. Demo_Island_01 now varies the way Unity does. It was 34 sails down and 0 tied; it is now 14 down and 20 tied, which is what the scene author built.

Why this one was worth the time

Most importer work is a grind of formats. This was the opposite: one number, a wrong assumption about how a hierarchy is named, and a factor-of-two difference in how much of a pack comes across correctly.

It also moved a whole class of content from "needs Unity installed" to "works on a stock package". That is the difference between an importer you can put in a browser tab and one you cannot, which is the entire premise of this engine.

The identifier rule is the part I would tell anyone attempting the same thing. The hash is public. The key format is a short string. The way a model prefab's root is assembled from the FBX's top-level nodes is the undocumented bit, and it is where all the difficulty lives.

← All posts