IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

mhc_post_kernel

def mhc_post_kernel[dtype: DType, OutLayoutType: TensorLayout, StreamsLayoutType: TensorLayout, YLayoutType: TensorLayout, PostLayoutType: TensorLayout, CombLayoutType: TensorLayout, hc_mult: Int, hidden_size: Int, block_size: Int, non_temporal: Bool](output: TileTensor[dtype, OutLayoutType, MutAnyOrigin], streams: TileTensor[dtype, StreamsLayoutType, ImmutAnyOrigin], y: TileTensor[dtype, YLayoutType, ImmutAnyOrigin], post: TileTensor[.float32, PostLayoutType, ImmutAnyOrigin], comb: TileTensor[.float32, CombLayoutType, ImmutAnyOrigin])

Writes one token's slice of the mHC streams back.

Block (row, kb) owns columns [kb * chunk, (kb + 1) * chunk) of token row, where chunk = block_size * width. Each thread loads y and all hc_mult streams for its width columns and writes all hc_mult outputs, so no data crosses threads and no LDS is needed. The post and comb coefficients depend only on row, so the compiler keeps them in scalar registers.

Parameters:

  • ​dtype (DType): Element type of the streams, y and the output.
  • ​OutLayoutType (TensorLayout): Layout of output.
  • ​StreamsLayoutType (TensorLayout): Layout of streams.
  • ​YLayoutType (TensorLayout): Layout of y.
  • ​PostLayoutType (TensorLayout): Layout of post.
  • ​CombLayoutType (TensorLayout): Layout of comb.
  • ​hc_mult (Int): Number of parallel residual streams.
  • ​hidden_size (Int): Width of one stream.
  • ​block_size (Int): Threads per block.
  • ​non_temporal (Bool): Whether to mark the loads and stores non-temporal.

Args:

Was this page helpful?