For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo module
mamba2_ssd_state_mi355
The chunked Mamba-2 scan's chunk state on MI355 (gfx950).
S[slot, h, p, n] = sum_t exp2(cs_last - cs_t) dt_t x[t, h, p] B[t, g, n]
over each chunk slot's tokens, on the chunked scan's varlen contract
(mamba2_ssd_chunked): x and B rows of the packed conv output at the
slot's first token, fp32 dt and log2 cumsum [H, slots, CHUNK], and S fp32
[slots, H, P, N] for the state passing. A slot's dt is 0 past its
sequence's end and its cs is flat there, so cs_last is cs[CHUNK - 1].
A persistent workgroup walks work items of the used slots, a slot and two
heads of one group. Two load waves write each head's weights w_t into LDS
and stream 32-token key blocks (B's rows and both heads' x rows; rows past
the slot's end are zeros) through a 4-deep LDS ring. Eight MFMA waves each
take one head and two 16-wide N tiles over all of P, accumulating
S^T[n, p] += B^T (w x) with 16x16x32 bf16 MFMAs whose operands are read
token-major with ds_read_tr16_b64; w scales x's fragments in fp32 before
they round to bf16. A lane stores 16 contiguous bytes of S.
This is the Moka component Kernels/test/gpu/Moka/amd/moka_amd/kernels/ ssd_varlen.mojo (ssd_state_amd_plan), statement for statement, with no
Moka in it.
comptime values
NWORDS
comptime NWORDS = 12
OFF_END
comptime OFF_END = 77824
OFF_KEYS
comptime OFF_KEYS = 0
OFF_W
comptime OFF_W = 73728
SB_PITCH
comptime SB_PITCH = 288
SKEY_BYTES
comptime SKEY_BYTES = 18432
STATE_LDS
comptime STATE_LDS = 77872
SX_OFF
comptime SX_OFF = 9216
SX_PITCH
comptime SX_PITCH = OX_PITCH
W_BYTES
comptime W_BYTES = 2048
W_DEPTH
comptime W_DEPTH = 2
W_KEYS
comptime W_KEYS = 0
W_W
comptime W_W = 8
Functions
-
ssd_state_mi355: S for every used chunk slot and head intostates([slots, H, P, N] fp32). x and B start at their first element, rowsx_ldandbc_ldapart. -
ssd_state_mi355_kernel: Persistent: workgroup c takes items c, c + grid, ... Waves 0-1 load, 2-9 compute.