St03 sky brightness against the Xbox: the sky model’s vertex-colour domes

Read from default.xbe (Xbox) and PDUWP.exe (PC) on the data machine, the stage files of both builds, and probe renders of our build against the xemu intro pan. Proof of the layout: converter/svcdump.py (struct based, synthetic test tests/test_svcdump.py).

The answer. The game’s st03 sky is not “clouds over a dark clear colour”. Its sky model (stage model 2) has three layers and pdconv converts one: it has no branch for vertex format 0x0B (position + colour, no texture), so the two svc layers are dropped (“3 index buffers -> 1 visual”). The dropped outer layer is an opaque blue-grey backdrop dome, (76, 80, 102) / 255 = (0.30, 0.31, 0.40) over the whole upper sky. The clouds are added on top of it. With that dome drawn, our sky box reads the xemu frames to within 0.01..0.03 (table below) (H: data, code, renders). The same kind of layer is dropped on 16 of the 19 versus stages.

Vertex format 0x0B and the sv* shaders

St03’s sky model (identical bytes in the Xbox and the PC st03.alr)

Index buffer Shader, flags, blend Shape Vertex colour Winding
0 svc, 0x10001 (opaque) outer dome, radius 12 398, y −3 620..3 620, 329 vertices, 576 triangles by height: (18, 19, 24) for y ≤ −2 794, (34, 36, 46) for −2 384..−1 385, (56, 59, 74) for −843..263, (76, 80, 102) for y ≥ 801 all 576 face the centre
1 sbtscloud3, 0x43d, source alpha / one cloud dome, radius 11 742, 736 triangles (format 0x03) all face the centre
2 svc, 0x10011 (blend on), one / one inner dome, radius 6 098, y −2 010..2 030, 306 vertices, 608 triangles white below y 513; (76, 76, 56)..(253, 255, 189) above all 608 face away from the centre

All 19 versus stages: layers of stage model 2

No other stage model has format 0x0B primitives. “Dropped” = not converted before skycol2 (11-15); since then pdconv converts every layer listed here (pdconv --stats lists them with flags, blend word and triangles; st02 / st13 / st22 convert byte for byte as before).

Stage Dropped svc / svcal layers (vertex colour range, R G B) Other layers
st01 backdrop r 12 699: (25, 0, 0)..(255, 255, 255) 8 cloud layers
st08 backdrop r 12 699: (0, 0, 0)..(255, 255, 127); svcal one / one, r 2 083, 8 triangles: (0, 0, 0)..(165, 144, 0) 8 cloud layers
st02 none sc dome, clouds, sc inner
st12 backdrop r 12 415: (6, 6, 6)..(38, 38, 38) clouds
st22 none sc dome, 2 cloud layers
st03 backdrop (above); inner dome (culled) clouds
st13 none sc dome, clouds, sc inner
st23 backdrop r 12 390: (30, 35, 35)..(46, 58, 54); svcal one / one, r 3 258, 304 triangles: (0, 0, 0)..(248, 236, 112) clouds
st04, st24 backdrop r 13 512: uniform (120, 104, 130) clouds
st14 backdrop r 13 512, flags 0xd: uniform (176, 176, 176) clouds
st05 backdrop r 12 398: (46, 41, 36)..(255, 229, 202) clouds, scal inner
st15 backdrop r 12 487, flags 0x9: (0, 0, 0)..(255, 255, 255) clouds, scal inner
st25 backdrop r 12 487: (10, 14, 19)..(255, 255, 255) clouds, scal inner
st06 backdrop r 12 398: (0, 0, 0)..(124, 157, 142) clouds, sc inner
st16 backdrop r 12 398: (0, 0, 0)..(254, 207, 188) clouds, sc inner
st26 backdrop r 12 398: all black clouds, sc inner
st07 backdrop r 12 398: (21, 13, 13)..(255, 240, 196); second svc, r 7 004, y 982..1 705, triangle list (primitive type 5): (29, 16, 11)..(115, 64, 44) clouds, sc inner
st17 backdrop r 12 398: (21, 27, 48)..(43, 54, 92); second svc, r 9 821, y 1 304..2 383: (24, 31, 65)..(33, 41, 87), mixed winding (164 toward / 102 away) clouds, sc inner

The backdrops are opaque (flags 0x10001; st14 0xd and st15 0x9 have other low bits, blend bit clear) and gradients by height on most stages, so each of these skies is today missing its base colour and horizon gradient, not only st03. St26’s is black (no change).

The Xbox sky draw, fog and light-set selection (default.xbe, H)

The Xbox code has the PC’s structure, function for function:

What Xbox PC
Sky object draw 0xb1580 0x1401c87f0
Light set builder 0x896e0 0x140199670
Fog apply for a set 0x89440 0x140199360
Set selection (calls the fog apply) 0x89a40 (as “Sky and far scenery draw”)
Set records, 0x58 bytes each 0x2c4980
Light records, 0x70 bytes each (index set · 8 + 0x40 + k) 0x2e65d0
Current set 0x396b10

Check against xemu (probe renders, nothing committed)

Our build at 461794f with two probe flags: set 2’s point light on (as “Sky colour”), and the st03 svc backdrop drawn from svcdump.py --json (unlit, opaque, back faces culled, depth writes off, before the clouds, unfogged). Intro shot 2 at tick 150, Xbox look, glow on, 1280 × 720; sky box = x 0.02..0.68, y 0.20..0.50 of the picture; xemu = three frames of the intro pan (x1125_5/15/25).

Picture Mean RGB Dark half Bright half Clipped
xemu frame 15 (0.48, 0.52, 0.60) (0.28, 0.32, 0.40) (0.68, 0.72, 0.80) 14 %
xemu frame 25 (0.39, 0.43, 0.52) (0.27, 0.31, 0.39) (0.52, 0.56, 0.66) 0 %
xemu frame 5 (camera nearer the white core) (0.74, 0.78, 0.83) (0.52, 0.58, 0.67) (0.95, 0.97, 0.99) 44 %
ours + backdrop, 8 scroll phases (0.40..0.61, 0.43..0.65, 0.52..0.72) (0.27..0.34, 0.31..0.38, 0.39..0.47) (0.52..0.89, …) 1..35 %
ours + backdrop, phase 25 000 (0.40, 0.43, 0.52) (0.27, 0.31, 0.39) (0.52, 0.56, 0.66) 5 %
ours + backdrop, phase 12 500 (0.50, 0.54, 0.63) (0.29, 0.34, 0.42) (0.70, 0.74, 0.83) 16 %
ours without backdrop (point light on), same tick as the next row (0.38, 0.41, 0.46) (0.16, 0.19, 0.23) (0.59, 0.63, 0.68) 10 %
ours + backdrop, same tick (3 000) (0.66, 0.69, 0.76) (0.41, 0.45, 0.54) (0.92, 0.94, 0.98) 40 %
ours + backdrop, no glow, tick 3 000 (0.51, 0.55, 0.64) (0.30, 0.34, 0.42) (0.73, 0.77, 0.86) 16 %
ours + backdrop, PC look, tick 3 000 (0.68, 0.70, 0.76) (0.44, 0.46, 0.54) (0.92, 0.94, 0.98) 39 %
backdrop alone (clouds hidden, no glow) exactly (76, 80, 102) / 255 over the upper sky 0

Implementation notes for the rebuild (items 1-4 built in skycol2 11-15: pdconv writes the layers as glTF primitives with COLOR_0 and a material whose extras carry pd_shader, pd_flags (the u32 at entry + 0x0C) and pd_blend (the u16 at + 0x14), single-sided, untextured; Godot PdData.vcol_sky_material, PdData.frame_clear_color. Item 5 was built in skycol, as the material’s cloud_p_low since skycol2)

  1. pdconv: convert format 0x0B primitives of stage model 2 (position + COLOR_0 from the D3DCOLOR, no UV, no normal), with the material’s shader name, flags and blend word, in index buffer order. Today they are dropped on 16 stages. Use the structs in svcdump.py / Esper-RE; strips and (st07) lists.
  2. Godot: draw sv / svc / svcal unlit as the vertex colour only (no texture, no material colour, no light), depth writes off, back faces culled by the stored winding (this hides st03’s inner dome; st17’s second layer has mixed winding and loses 102 faces, as in the game), blend from the material: flag bit 4 clear = opaque, set = the blend word (one / one on st03 / st08 / st23). Order inside the sky model = index buffer order (backdrop before clouds).
  3. Fog: clouds keep set 2’s depth fog in the Xbox look (now decoded). Leave svc / svcal unfogged in both looks until the c16 upload is decoded (XboxLook.FOGGED lists svc, svcal today: take the sky model’s out).
  4. Clear the frame to black, not the 0x2001 shadow colour.
  5. Xbox look: clamp the packed point vector to [0, 1] (already task 3 of “Sky colour”; now H).
  6. Expect at st03 intro shot 2, Xbox look, with the point light: sky box mean (0.40..0.61, 0.43..0.65, 0.52..0.72) by phase, dark half about (0.28, 0.32, 0.40), cloud gaps exactly (76, 80, 102) / 255 where no cloud covers them. Re-shoot every stage of the table above (st01, st04 / st24, st14, st05, st12 and st23 gain the most).
  7. Open: what SVC uploads to c16 (the backdrop’s fog coordinate on the Xbox); the exact cloud phase of the footage; the second set-2 selection site at Xbox 0x10d94a.