For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo module
dispatch
Provides the SM100 (Blackwell) flash-attention host-side dispatch entry point that selects a 1Q or 2Q FA4 kernel configuration, builds the Q/K/V/O TMA descriptors and tile scheduler, and enqueues the kernel onto the device.
comptime values
FA4_FORCE_2Q
comptime FA4_FORCE_2Q = (get_defined_int[StringSpan("FA4_FORCE_2Q"), Int(0)]() != Int(0))
FA4_WS_POISON
comptime FA4_WS_POISON = (get_defined_int[StringSpan("FA4_WS_POISON"), Int(0)]() != Int(0))
logger
comptime logger = Logger(stdout, prefix=String(""), source_location=False)
SK_WS_KEYS_PER_PARTITION
comptime SK_WS_KEYS_PER_PARTITION = UInt32(384)
SPLIT_KV_1Q_MAX_Q_ROWS
comptime SPLIT_KV_1Q_MAX_Q_ROWS = 128
SPLIT_KV_KEYS_PER_PARTITION
comptime SPLIT_KV_KEYS_PER_PARTITION = 512
WS_G_FUSED
comptime WS_G_FUSED = 0
WS_G_FUSED_1Q
comptime WS_G_FUSED_1Q = 3
WS_G_NONE
comptime WS_G_NONE = 2
WS_G_UNFUSED
comptime WS_G_UNFUSED = 1
WS_RAW_GRID_CLAMP
comptime WS_RAW_GRID_CLAMP = UInt32(4)
WS_SWEEP_MAX_RAW_GRID
comptime WS_SWEEP_MAX_RAW_GRID = UInt32(24)
Functions
-
mha_sm100_dispatch: Dispatches the SM100 FA4 flash-attention kernel for a prefill or decode workload. -
mha_split_kv_decode_num_partitions: Returns the split-K partition count for decode and speculative verify over a split KV cache (key wider than value), the count an engine puts in the dispatch metadata. -
ws_p_ceiling: Capture-invariant partition-count ceiling for the split-K crossover: follows one GPC wave's SM-fill, holds the plateau throughWS_SWEEP_MAX_RAW_GRID, then decays toward one partition.