IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo module

mamba2_ssd_state_mi355

The chunked Mamba-2 scan's chunk state on MI355 (gfx950).

S[slot, h, p, n] = sum_t exp2(cs_last - cs_t) dt_t x[t, h, p] B[t, g, n]

over each chunk slot's tokens, on the chunked scan's varlen contract (mamba2_ssd_chunked): x and B rows of the packed conv output at the slot's first token, fp32 dt and log2 cumsum [H, slots, CHUNK], and S fp32 [slots, H, P, N] for the state passing. A slot's dt is 0 past its sequence's end and its cs is flat there, so cs_last is cs[CHUNK - 1].

A persistent workgroup walks work items of the used slots, a slot and two heads of one group. Two load waves write each head's weights w_t into LDS and stream 32-token key blocks (B's rows and both heads' x rows; rows past the slot's end are zeros) through a 4-deep LDS ring. Eight MFMA waves each take one head and two 16-wide N tiles over all of P, accumulating S^T[n, p] += B^T (w x) with 16x16x32 bf16 MFMAs whose operands are read token-major with ds_read_tr16_b64; w scales x's fragments in fp32 before they round to bf16. A lane stores 16 contiguous bytes of S.

This is the Moka component Kernels/test/gpu/Moka/amd/moka_amd/kernels/ ssd_varlen.mojo (ssd_state_amd_plan), statement for statement, with no Moka in it.

comptime values​

NWORDS​

comptime NWORDS = 12

OFF_END​

comptime OFF_END = 77824

OFF_KEYS​

comptime OFF_KEYS = 0

OFF_W​

comptime OFF_W = 73728

SB_PITCH​

comptime SB_PITCH = 288

SKEY_BYTES​

comptime SKEY_BYTES = 18432

STATE_LDS​

comptime STATE_LDS = 77872

SX_OFF​

comptime SX_OFF = 9216

SX_PITCH​

comptime SX_PITCH = OX_PITCH

W_BYTES​

comptime W_BYTES = 2048

W_DEPTH​

comptime W_DEPTH = 2

W_KEYS​

comptime W_KEYS = 0

W_W​

comptime W_W = 8

Functions​

  • ​ssd_state_mi355: S for every used chunk slot and head into states ([slots, H, P, N] fp32). x and B start at their first element, rows x_ld and bc_ld apart.
  • ​ssd_state_mi355_kernel: Persistent: workgroup c takes items c, c + grid, ... Waves 0-1 load, 2-9 compute.

Was this page helpful?