For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
MLAKPoolCompress
struct MLAKPoolCompress
Registers the attention.mla.kpool.compress graph op with the graph compiler.
Implemented traits
Methods
execute
static def execute[*, head_dim: Int, kpool: Int](pooled: T, k: T, gate: T, ape: T, input_row_offsets: T, pool_row_offsets: T, cache_lengths: T, ctx: DeviceContext)
Compresses each complete k-pool into one candidate key.
Parameters:
Args:
- pooled (
T): Output[total_pools, head_dim]pooled keys. - k (
T): Layer-normed indexer keys[total_tokens, head_dim]. - gate (
T): Per-token gate scores[total_tokens, head_dim]. - ape (
T): Within-pool position embedding[kpool, head_dim], float32. - input_row_offsets (
T): Token row offsets[batch + 1]. - pool_row_offsets (
T): Pool row offsets[batch + 1]. - cache_lengths (
T): Cached-prefix length per request[batch]. A pool covers absolute positions, so this is what places the call's tokens on the pool grid. - ctx (
DeviceContext): Device context for GPU execution.