For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
MLAKPoolRingClose
struct MLAKPoolRingClose
Registers the attention.mla.kpool.ring_close graph op with the graph compiler.
Implemented traits
Methods
execute
static def execute[*, head_dim: Int, kpool: Int](pooled: T, closed_pool: T, tail: T, k: T, gate: T, ape: T, input_row_offsets: T, cache_lengths: T, slot_idx: T, ctx: DeviceContext)
Closes each request's pending tail-ring pool, ragged and unconditional -- see kpool_ring_close_kernel's docstring for the pool-splitting semantics this implements.
Parameters:
Args:
- pooled (
T): Output[batch, head_dim]pooled keys, meaningful only whereclosed_poolis non-negative. - closed_pool (
T): Output[batch]pool id closed this call, or -1. - tail (
T): Persistent slot-indexed ring[max_slots, 2, kpool, head_dim]. Read only. - k (
T): This call's layer-normed keys[total_tokens, head_dim]. - gate (
T): This call's gate scores[total_tokens, head_dim]. - ape (
T): Within-pool position embedding[kpool, head_dim], float32. - input_row_offsets (
T): Token row offsets per request[batch + 1]. - cache_lengths (
T): Cached-prefix length per request[batch]. - slot_idx (
T): Ring slot per request[batch]. - ctx (
DeviceContext): Device context for GPU execution.