Mixamo characters

Mixamo and Fuse characters brought in, their materials fixed, and drawn through anim.

A Mixamo or Adobe Fuse character reaches a game through the anim rig file: the rig names the body under bodies, and the bake plays every clip of the rig onto it in Blender, packs it with gltfpack and writes a glb to the loader model entry of the same name. A character that is a new kind with rules of its own, and not only a new body for a kind that exists, is designed first, as Designing a kind says.

#Setting up

The bake reads BLENDER, PATCHWORK_LIBRARY and GLTFPACK, the native gltfpack 1.3 or later, from the shell, else from .env at the repository root, copied from .env.example. yarn health checks each and says the fix for any that fails.

#Inspecting the FBX

1. Open the new body and the body it replaces in Blender. Its bones are mixamorig:<Name>, the same names as the clips'; a rig with other names takes no clip.
2. Its height is a person's, about 1.6 to 1.9 m. A body of other proportions still plays every clip, since only the hips' position is keyed.
3. Images named …_diffuse, …_gloss, …_specular and …_normal mark a Fuse or Mixamo spec/gloss body, whose materials the bake converts.

#Swapping or adding a body

1. Put the character in the library, PATCHWORK_LIBRARY, else ~/patchwork-library, as characters/<name>/<name>.fbx. The library holds farmer_woman, farmer_man, farmhand, mechworker, soldier, soldier_woman, xbot (Mixamo's X Bot, meshes Alpha_*) and ybot (Y Bot, Beta_*).
2. The rig file's bodies is { <model entry>: "<fbx under the library>" }. To swap a body, point its existing key at the new FBX; the kind, the catalogue entry, the glb path and every scene keep their names.
3. To add one, add a key, a model entry of that name in world.loader.assets, and a world.anim.kinds.<kind> naming it.
4. Bake every body in one run, from the repository root:

   /Applications/Blender.app/Contents/MacOS/Blender -b --factory-startup --python plugins/anim/anim.export.py -- \
       <rig.json> <scratch>/table.json <body>=<out.glb> [<body>=<out.glb> ...]

   The rig's table is measured on the first body named and serves all of them. Each body's bone ends are measured too, into table.ends.json beside the table.
5. Each body's export prints PACK <out.glb> raw=<bytes> packed=<bytes> meshes=<names>. A glb whose asset.generator is not gltfpack was not made by the bake.
6. Write the measured table and ends back into the rig file, keeping the rig's layout, so only the numbers and the bodies lines change.

#A new kind that walks and is animated

Start from the kind in the game that already walks on walker and is drawn by anim: its rule, its walker, anim, hitscan3d and ragdoll3d entries in the scene, and its rig file. Then decide each of these in the scene:

1. Its rig. A body that moves as the existing kind moves is one more key in that rig's bodies. A body that moves differently, unarmed, a worker or a civilian, gets a rig file of its own beside its glb, with its own clips, roles (still alone for a body that never leaves the ground) and stances, and only the clips it plays: every clip in a rig is baked onto every body of it.
2. Its clips. ~/troupe-collection/packs/ holds the packs, such as Male Locomotion Pack 30FPS for an unarmed idle and walk, and packs/mixamo-motions/ single clips such as Talking At Watercooler.fbx. A walk is a gear (loop, gear: 0). An idle or a talk played in place is a loop played as an act, or a gesture on the upper channel over the still role.
3. What it is solid to. world.walker.kinds.<kind>.bodies lists the kinds it walks round, and another kind walks round it only if that kind's bodies lists it. A bystander that must not block the player lists only its own kind.
4. Whether it can be shot. A kind is a body to hitscan3d only with a world.hitscan3d.kinds entry. Scenery has none, no hp and no hit act.
5. Where it stands and walks to. Rows in actors, never a list on world or in its definition: its own, and for the places it walks between, a kind of their own with one actors entry each. The place a row has taken is a field of that row, and a row chooses only among places no other row holds.

#Materials

Blender reads a spec/gloss material as fully metallic with the gloss map as roughness, which draws cloth and skin as polished metal. The bake fixes every material whose roughness is fed by an image named gloss: metallic 0, roughness from that image inverted, no specular map, and alpha from the base colour image clipped at one half, or none when it is opaque. It logs METAL_ROUGH <material> alpha=clip|opaque for each.

1. Check the bake printed a METAL_ROUGH line for each material of each Fuse body.
2. In the glb, every material has "metallicFactor": 0 and a …_gloss_rough image, and "alphaMode": "MASK" where hair, lashes or brows are.
3. Render the baked glbs in their idle clip, three to a picture. Blender's glTF importer refuses a packed glb, so each is unpacked first, its meshopt buffers decoded with three.js's decoder. Cloth is matte, skin is skin and hair is cut out. A metallic sheen means a material's roughness is fed by an image not named gloss.

#Props on a bone

1. world.anim.kinds.<kind>.props is a list of { model, bone, at, turn }: at in metres and turn in degrees, a three.js 'XYZ' Euler, in the bone's frame without scale.
2. Two Mixamo bodies roll their hand bones differently, often by 10 to 25 degrees, so refit each prop for the new body. Pose both bodies on the first frame of the clip the prop is held in and find the at and turn that keep the prop where it was. Three.js's 'XYZ' is Blender's 'ZYX'.
3. Write the new numbers into every scene that holds the kind's anim table.

#Checking it

1. Look at the body in a gym scene: the game's own scene with the bodies standing from the first step, render3d's camera (prefabs.render3d.camera at and look, in metres) a few metres in front at eye height, and no player, vehicle or HUD. A body in the scene's actors with grounded true stands on the ground on its first step.
2. Run the tests beside the kinds that draw the body, the scenes whose props were refit, and the gym's own <gym>.sim.test.ts.