IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

scale_and_mask_helper

def scale_and_mask_helper[p_type: DType, p_layout: TensorLayout, mask_t: MHAMask, group: Int, num_n_mmas: Int, WN: Int, MMA_N: Int, simd_width: Int](p_reg_tile: TileTensor[p_type, p_layout, Engine=p_reg_tile.Engine, address_space=AddressSpace.LOCAL, linear_idx_type=p_reg_tile.linear_idx_type], scale_log2e: Float32, num_keys: Int, bound: Int, lane: Int, warp: Int, mask: mask_t, kv_tile_start_row: Int) where ((p_layout.rank == Int(2)) & (Int(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(p_layout.__shape_types), [idx: __mlir_type.index] p_layout.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(p_layout.__shape_types), [idx: __mlir_type.index] p_layout.__shape_types[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(p_layout.__shape_types), [idx: __mlir_type.index] p_layout.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(p_layout.__shape_types), [idx: __mlir_type.index] p_layout.__shape_types[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(p_layout.__shape_types), [idx: __mlir_type.index] p_layout.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(p_layout.__shape_types), [idx: __mlir_type.index] p_layout.__shape_types[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(p_layout.__shape_types), [idx: __mlir_type.index] p_layout.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(p_layout.__shape_types), [idx: __mlir_type.index] p_layout.__shape_types[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType)))) == Int(2)))

Apply softmax scaling and masking to one P = Q·Kᵀ MMA result tile in registers.

Multiplies each element by scale_log2e and then applies mask to out-of-bounds and masked positions. Only threads with lane < 4 * group carry meaningful data; other threads return immediately. Designed for the decode inner loop where P is a 1-D column-vector across the key dimension.

Was this page helpful?