Xbox capsule shading

Source: the original Xbox default.xbe from the playtester’s disc. Read with the public NV2A descriptions (xemu’s vertex-program and register-combiner sources, read only; the XDK/nxdk D3DPIXELSHADERDEF layout).

Xbox vs PC data. The disc’s Map/, Effect/, Player/, com/ etc. are the PC Assets/Data/ files with the same names and sizes. com/ (459 files) is byte-identical; every stNN.dat (stage layout and light script, fog included) is identical. The .alr and .e archives keep their sizes but differ in a few small runs (st03.alr: about 45 KB of 10.7 MB in 94 runs; a0.e: about 0.3 MB of 8.3 MB): the same contents; in the .e packs the differences are the uncompressed textures, stored row by row on the PC (“Effect packs and the Xbox / PC effect comparison”). Xbox-only: BGM/ and the root .xbe files; PC-only: the .cso shaders, Effect/eftNNNN.* (an older revision of the effects that neither build plays: both load the packed aN.e), MenuD/, Extras/.

Shader table. The game’s shaders are compiled into .data: per shader, an Xbox vertex program (one header dword: 0x2078 | instruction count << 16; then 4-dword NV2A instructions, the last with bit 0 of dword 3 set), one pad dword, a 240-byte D3DPIXELSHADERDEF, then a 16-byte record (name pointer, a format word, a pointer, a small count). The name record follows its shader (checked: the shader before the vkCAPM2A record is the two-texture additive one the PC calls vkCAPM2A, the one before vkCAP01 has the PC vkCAP01’s three-layer form). vkCAP02’s program sits at 0x2d3f80 (28 instructions), its combiner setup at 0x2d4148, its record at 0x2d4238; vkCAP02M2’s record at 0x2d4810.

Vertex program (vkCAP02). The same terms as the PC vertex shader: position by the matrix in c20–c23; the direction to the eye from c15 (object-space eye), normalised; the half vector normalise(eye direction − L) with L = c4 (the direction the light travels); light, hemisphere axis (c10) and half vector projected on (−T, −B, +N) for the light and axis and (T, B, −N) for the half vector, each packed ×c0.w + c0.w (0.5): the light into the diffuse colour, the hemisphere axis into the specular colour, the half vector into texture coordinate 3 (passed through unclamped, texture mode “pass-through”). Texture coordinates 0–2 are the vertex UV × c12.z plus the scroll offsets c90/c91. The vertex alpha goes to the diffuse alpha. The fog output is the clip-space w (the view depth). T, B, N are inputs 6, 7 and 1.

Combiners (vkCAP02, 8 stages, one constant pair per stage). Inputs t0 (colour), t1, t2 (bump maps, 2-D), t3 (half vector), v0 (light vector, vertex alpha), v1 (hemisphere axis):

  1. n = expand(t1) + expand(t2) — stored in a combiner register, so each channel is clamped to [−1, 1] (the PC keeps the full [−2, 2] sum).
  2. n·n, and n × constant0.a (0.5).
  3. n’ = n + (1 − sat(n·n)) · n/2: the “unsigned invert” input mapping clamps n·n to [0, 1] first, so the factor is never below 1 and the step never turns the normal inward (on the PC 1 + (1 − n·n)/2 goes negative for n·n > 3). Result clamped to [−1, 1].
  4. d = L·n’ and s = S·n’ (both expanded from the colours, dot products).
  5. diffuse = sat(d) × light colour (constant 3); h = n’·H; k = s + constant7.a (0.5).
  6. hemisphere = sat(k)·sky (constant 6) + (1 − sat(k))·ground (constant 7) (PC: 1 − k unclamped); sat(h)² in an alpha register.
  7. lit = (sat(diffuse) + sat(hemisphere)) × vertex alpha, clamped; sat(h)⁴.
  8. lit × t0; specular colour (constant 2) × vertex alpha; sat(h)⁸. Final combiner: out = f·(lit·t0) + (1 − f)·fog colour + sat(h)⁸·(specular·alpha), with f the fog factor; alpha = constant 0’s blue (the PC’s c0.z). So the Xbox core is the PC formula with every intermediate held in [−1, 1] or [0, 1], plus fog. The constant values themselves are uploaded at run time (not in the definition); we take the PC’s (0.5 and the light records). Evidence: in vkCAP02’s definition both per-stage constant arrays are zero; the stage-constant mappings send stage 1’s constant 0 to register 0, stage 4’s pair to 3/7, stage 5’s to 6/7, stage 7’s constant 0 to 2, and the final combiner’s constant 0 to register 0 (the rest unused). The values in those registers are set by the game’s draw code.

The other capsule layers (same scan of the table). vkCAP01, vkCAPM2MM2 and vkCAPM2M compute the PC formulas; every intermediate stays within [0, 1], so the register clamps change nothing. Two differ only by a clamp: vkCAP04M4 sums t0·t1·(1 − t2.a) + t2·t2.a + t3 into a register (clamped to 1) before multiplying by the vertex colour (the PC multiplies the unclamped sum), and the additive shells vkCAPM2A / vkCAPM2AM2 add t0·vc + t1·vc into a register (clamped to 1) before the vc.a factor (PC: (t0 + t1)·vc·vc.a, clamped at the end). The Xbox look uses the clamped forms (XboxLook.CORE04_XBOX, SHELL_XBOX). None of these layers is fogged.

The glow quad (CapsuleLook.GLOW_SHADER, the PC’s vkcolanim) is vkCOLANIM in the Xbox table (definition at 0x2d13f0; the disc also has vkCOLANIM2/3/4/A, not used by capsules). Its combiner multiplies t1 by t0, then by the colour constant, and outputs that, alpha from the constant; no fog. Every factor is in [0, 1], so the register clamps change nothing: the same as the PC’s sat(t0·t1·colour). The Xbox look keeps the PC glow shader.

Effect on the cores. With the sum clamped and no inward flip the bump normals stay facing out: the cores are brighter and the lit side follows the light (see REPORT of xboxlook for the 24-step orbit luma, PC vs Xbox math).