IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

Struct_mha_ragged_paged_scalar_args

struct Struct_mha_ragged_paged_scalar_args

Registers the attention.mha.ragged.paged graph op with the graph compiler.

mixed_step_split is the KV cache's split record (kv_cache.mixed_step_split). On NVIDIA, a valid record runs the decode rows and the prefill rows of one replica batch as two views, each on its own route, into the one output. Zeros, an invalid record, a strided operand, mask operands or another target keep the single launch.

Implemented traitsโ€‹

AnyType, Deinitable, Movable

Methodsโ€‹

executeโ€‹

static def execute[MaskIns: TypeList[MaskIns.values], //, mask_str: StaticString, local_window_size: Int = Int(-1), v_head_dim: Int = Int(-1)](output: T, q: T, input_row_offsets: T, kv_blocks: T, page_stride: T, cache_lengths: T, kv_lookup_table: T, max_prompt_length: T, max_cache_length: T, layer_idx: UInt32, scale: Float32, mha_decode_dispatch_metadata: T, mixed_step_split: T, mask_operands: TensorTuple[*MaskIns.values], context: DeviceContext)

Was this page helpful?