IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

mixed_gemv_store_epilogue

def mixed_gemv_store_epilogue[static_N: Int, c_layout: TensorLayout, a_layout: TensorLayout, b_layout: TensorLayout, c_engine: TensorEngine, a_engine: TensorEngine, b_engine: TensorEngine, EpilogueFnType: ElementwiseEpilogueFn](c: TileTensor[.float32, c_layout, MutAnyOrigin, Engine=c_engine], a: TileTensor[.bfloat16, a_layout, ImmutAnyOrigin, Engine=a_engine], b: TileTensor[.float32, b_layout, ImmutAnyOrigin, Engine=b_engine], m: Int, n: Int, k: Int, epilogue_fn: EpilogueFnType, ctx: DeviceContext)

Launches the mixed GEMV with its output stored through epilogue_fn.

The bf16 activation is widened to fp32 in registers and dotted against the unchanged fp32 weight in one launch, as router_gate_mixed_gemv does, and every output element goes through epilogue_fn instead of a plain store. That is what lets a caller whose projection carries elementwise work -- a scale chain the graph would otherwise stage as a launch of its own -- keep one launch for the whole projection. gemv_split_k_epilogue_fn is the same kernel gemv_split_k is, so an epilogue that stores the untouched value reproduces the plain launch bit for bit, and the fp32 form of the same kernel over a pre-cast activation is the reference both are compared to.

The B200 bands are router_gate_mixed_gemv's: tile_m and tile_n move with the row count, and neither reorders the K reduction. Only the SM100 launch is served here -- the [896, 7168] cold-weight table and the gfx950 [128, 6144] sweep are expressed through gemv_split_k parameters (small_m_static_k, small_m_k_chunk, expert_major_grid, act_wide_k) that this kernel variant does not take, and a non-SM100 NVIDIA target's fp32 matmuls take a dispatch of its own -- so a caller on another target stages its own chain instead.

Parameters:

  • ​static_N (Int): Static output width (weight rows); selects the check_bounds_n guard at compile time.
  • ​c_layout (TensorLayout): Layout of the fp32 output tensor. Only its shape is read: the epilogue owns every store.
  • ​a_layout (TensorLayout): Layout of the bf16 activation tensor.
  • ​b_layout (TensorLayout): Layout of the fp32 weight tensor.
  • ​c_engine (TensorEngine): Engine of the fp32 output tensor.
  • ​a_engine (TensorEngine): Engine of the bf16 activation tensor.
  • ​b_engine (TensorEngine): Engine of the fp32 weight tensor.
  • ​EpilogueFnType (ElementwiseEpilogueFn): Type of epilogue_fn (inferred).

Args:

Was this page helpful?