IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo module

mla_stamp

Debug-only per-warp clock stamps for sparse MLA decode (-D MLA_STAMP=1).

Lane 0 of a warp writes one clock64 word to buf[((cta * STAMP_WARPS) + warp) * STAMP_SLOTS + slot] for CTAs with a linear index below STAMP_MAX_CTAS. The buffer address is parked in the last 16 bytes of dynamic smem (the CTA record below it), which the launch reserves only when stamping is on, so role functions need no extra argument. Off, every call elaborates to nothing.

Warp 8 (the Q load) carries no role stamps, so its row holds the CTA-level stamps thread 0 writes at the CTA's exit: entry at ST_START, exit at ST_END, the walk's key count at ST_NTILES, the block and grid indices at ST_BLOCK/ST_GRID, and the prologue and epilogue sync points as tile-0 events (CTA_*). They are on whenever MLA_STAMP is, whatever the role mask.

comptime values​

CTA_ALLOC​

comptime CTA_ALLOC = 5

The TMEM-alloc warp, after its alloc.

CTA_INDEX_START​

comptime CTA_INDEX_START = 7

The index warp, at its role's start.

CTA_INITS​

comptime CTA_INITS = 4

The barrier-init thread, after its inits, fence and prefetches.

CTA_INITS_UNFENCED​

comptime CTA_INITS_UNFENCED = 8

The barrier-init thread, after its inits, before their fence.

CTA_LATE_DEALLOC​

comptime CTA_LATE_DEALLOC = 10

Written by warp 9 itself after thread 0's exit: after the TMEM dealloc.

CTA_LATE_INBOUND​

comptime CTA_LATE_INBOUND = 9

Written by warp 9 itself after thread 0's exit: after the pair's tail waits.

CTA_LATE_RELEASED​

comptime CTA_LATE_RELEASED = 11

Written by warp 9 itself after thread 0's exit: after the allocation-permit release, before the TMEM dealloc.

CTA_OFFSETS​

comptime CTA_OFFSETS = 3

Thread 0, once the offsets are known and the early exits are passed.

CTA_ROLES_DONE​

comptime CTA_ROLES_DONE = 2

After the CTA barrier every role reaches when its walk is done.

CTA_SETUP​

comptime CTA_SETUP = 0

After the CTA barrier that publishes the barrier inits and TMEM base.

CTA_SYNC_IN​

comptime CTA_SYNC_IN = 1

After the pair's entry cluster sync.

CTA_ZEROED​

comptime CTA_ZEROED = 6

Thread 0, after its share of the NoPE-tail zeroing.

MLA_STAMP​

comptime MLA_STAMP = (get_defined_int[StringSpan("MLA_STAMP"), Int(0)]() != Int(0))

ST_BLOCK​

comptime ST_BLOCK = 6

CTA row only: block_idx x | y << 16 | z << 40; ST_GRID packs grid_dim the same way.

ST_END​

comptime ST_END = 1

ST_END_NS​

comptime ST_END_NS = 5

ST_EVENTS​

comptime ST_EVENTS = 4

ST_GRID​

comptime ST_GRID = 7

ST_MAX_TILES​

comptime ST_MAX_TILES = (Int((add get_defined_int[StringSpan("MLA_STAMP_SLOTS"), Int(256)](), -8)) // Int(4))

ST_NTILES​

comptime ST_NTILES = 3

ST_SMID​

comptime ST_SMID = 2

ST_START​

comptime ST_START = 0

ST_START_NS​

comptime ST_START_NS = 4

ST_TILE0​

comptime ST_TILE0 = 8

Event e of tile t lands at ST_TILE0 + t * ST_EVENTS + e.

STAMP_CTA_WARP​

comptime STAMP_CTA_WARP = 8

STAMP_MAX_CTAS​

comptime STAMP_MAX_CTAS = get_defined_int[StringSpan("MLA_STAMP_MAX_CTAS"), Int(1024)]()

STAMP_ROLES​

comptime STAMP_ROLES = get_defined_int[StringSpan("MLA_STAMP_ROLES"), Int(127)]()

Bit r enables role r: 0 softmax, 1 correction, 2 gather, 3 QK, 4 PV, 5 index, 6 re-swizzle.

STAMP_SLOTS​

comptime STAMP_SLOTS = get_defined_int[StringSpan("MLA_STAMP_SLOTS"), Int(256)]()

STAMP_SMEM_BYTES​

comptime STAMP_SMEM_BYTES = Int(128) if MLA_STAMP else Int(0)

STAMP_SMX​

comptime STAMP_SMX = (get_defined_int[StringSpan("MLA_STAMP_SMX"), Int(0)]() != Int(0))

Softmax (role 0) stamps that sum in shared memory (-D MLA_STAMP_SMX=1).

One elected lane of each softmax warp adds the 32-bit clock of tile event e to word e of its row (red.shared.add), so each tile costs four fire-and-forget adds and no global store; the per-tile mean of interval e -> e + 1 is (word e + 1 - word e) / tiles. The exit copies the row to the warp's stamp slots ST_TILE0 + e, with ST_START left 0 (see stamp_finish). The per-tile global stores, and their 64-bit base held across the loop, perturb the softmax too much to time it. The rows take the CTA record's event words (the Layout G carve has no room for more), so the CTA events are off under this knob.

STAMP_WARPS​

comptime STAMP_WARPS = 16

Functions​

Was this page helpful?