For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
scale_and_mask_helper
def scale_and_mask_helper[p_type: DType, p_layout: TensorLayout, mask_t: MHAMask, group: Int, num_n_mmas: Int, WN: Int, MMA_N: Int, simd_width: Int](p_reg_tile: TileTensor[p_type, p_layout, Engine=p_reg_tile.Engine, address_space=AddressSpace.LOCAL, linear_idx_type=p_reg_tile.linear_idx_type], scale_log2e: Float32, num_keys: Int, bound: Int, lane: Int, warp: Int, mask: mask_t, kv_tile_start_row: Int) where ((p_layout.rank == Int(2)) & (Int(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(p_layout.__shape_types), [idx: __mlir_type.index] p_layout.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(p_layout.__shape_types), [idx: __mlir_type.index] p_layout.__shape_types[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(p_layout.__shape_types), [idx: __mlir_type.index] p_layout.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(p_layout.__shape_types), [idx: __mlir_type.index] p_layout.__shape_types[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(p_layout.__shape_types), [idx: __mlir_type.index] p_layout.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(p_layout.__shape_types), [idx: __mlir_type.index] p_layout.__shape_types[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(p_layout.__shape_types), [idx: __mlir_type.index] p_layout.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(p_layout.__shape_types), [idx: __mlir_type.index] p_layout.__shape_types[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType)))) == Int(2)))
Apply softmax scaling and masking to one P = Q·Kᵀ MMA result tile in registers.
Multiplies each element by scale_log2e and then applies mask to
out-of-bounds and masked positions. Only threads with lane < 4 * group
carry meaningful data; other threads return immediately. Designed for the
decode inner loop where P is a 1-D column-vector across the key dimension.