IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

ssd_out_sm100

def ssd_out_sm100(ctx: DeviceContext, total_len: Int, batch: Int, slots: Int, nheads: Int, ngroups: Int, x_addr: Int, b_addr: Int, c_addr: Int, x_ld: Int, bc_ld: Int, d_addr: Int, dt_addr: Int, cs_addr: Int, h_addr: Int, y_addr: Int, info_addr: Int)

Computes y for every used chunk slot and head.

Args:

  • ​ctx (DeviceContext): The device context.
  • ​total_len (Int): Tokens of the packed batch.
  • ​batch (Int): Sequences.
  • ​slots (Int): Chunk slots of the table (chunked_slots).
  • ​nheads (Int): Heads.
  • ​ngroups (Int): B and C groups.
  • ​x_addr (Int): The first element of x, [T, H, P] bf16 with rows x_ld apart.
  • ​b_addr (Int): B's first element, [T, G, N] bf16 with rows bc_ld apart.
  • ​c_addr (Int): C's first element, as B.
  • ​x_ld (Int): Row stride of x in elements (a multiple of 8).
  • ​bc_ld (Int): Row stride of B and C in elements (a multiple of 8).
  • ​d_addr (Int): D, bf16 [H], or 0 for none.
  • ​dt_addr (Int): The softplus dt, fp32 [H, slots, CHUNK].
  • ​cs_addr (Int): The log2 decay cumsum, fp32 [H, slots, CHUNK].
  • ​h_addr (Int): The state entering each slot, bf16 [slots, H, P, N].
  • ​y_addr (Int): The output, bf16 [T, H, P], contiguous.
  • ​info_addr (Int): The chunk table, int32 [3 slots + 2 batch].

Was this page helpful?