For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo module
mla_decode_dispatch
comptime values
logger
comptime logger = Logger(stdout, prefix=String(""), source_location=False)
MAX_FOLD_Q
comptime MAX_FOLD_Q = 8
MIN_FOLD_Q
comptime MIN_FOLD_Q = 2
MLA_SPARSE_KV_FORM_NVFP4
comptime MLA_SPARSE_KV_FORM_NVFP4 = 2
MLA_SPARSE_KV_FORM_PLAIN
comptime MLA_SPARSE_KV_FORM_PLAIN = 0
MLA_SPARSE_KV_FORM_SCALED_FP8
comptime MLA_SPARSE_KV_FORM_SCALED_FP8 = 1
MLA_WS_POISON
comptime MLA_WS_POISON = (get_defined_int[StringSpan("FA4_WS_POISON"), Int(0)]() != Int(0))
Structs
-
MLADispatchScalarArgs: Pre-computed MLA decode args for the legacy (non-capturable) path.
Functions
-
compute_mla_dispatch_key_runtime: Returns(batch_size, q_max_seq_len, num_partitions, kernel_variant), the capture key of a decode dispatch. -
compute_mla_dispatch_scalars: Pure computation of the packed MLA dispatch metadata. -
compute_mla_dispatch_scalars_runtime: Binds a runtime Q-head count to a comptime dispatch instantiation. -
launch_mla_sm100_decode_enqueue_kernel: -
launch_mla_sm100_decode_fp8_per_token_scale_rope_aware: Launch the FP8 per-token-scale rope-aware MLA decode kernel with split content/rope TMAs. -
launch_mla_sm100_decode_native_fp8: Launch the native FP8 MLA decode kernel with FP8 Q TMA. -
launch_mla_sm100_decode_native_fp8_layout_g: Launch the Layout G native FP8 MLA decode kernel (BM=32, 5-stage). -
launch_mla_sm100_decode_sparse_qkv_fp8: Launches the unified sparse MLA decode kernel. -
mla_decode_sm100_dispatch: -
mla_decode_sm100_sink_split_k: -
sparse_decode_partitions_for_rows: Returns the shape-pure split-K count a sparse FP8 verify ofrows_per_requestrows per request shares with the one-row decode step.