For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
mhc_post_kernel
def mhc_post_kernel[dtype: DType, OutLayoutType: TensorLayout, StreamsLayoutType: TensorLayout, YLayoutType: TensorLayout, PostLayoutType: TensorLayout, CombLayoutType: TensorLayout, hc_mult: Int, hidden_size: Int, block_size: Int, non_temporal: Bool](output: TileTensor[dtype, OutLayoutType, MutAnyOrigin], streams: TileTensor[dtype, StreamsLayoutType, ImmutAnyOrigin], y: TileTensor[dtype, YLayoutType, ImmutAnyOrigin], post: TileTensor[.float32, PostLayoutType, ImmutAnyOrigin], comb: TileTensor[.float32, CombLayoutType, ImmutAnyOrigin])
Writes one token's slice of the mHC streams back.
Block (row, kb) owns columns [kb * chunk, (kb + 1) * chunk) of token
row, where chunk = block_size * width. Each thread loads y and all
hc_mult streams for its width columns and writes all hc_mult
outputs, so no data crosses threads and no LDS is needed. The post and
comb coefficients depend only on row, so the compiler keeps them in
scalar registers.
Parameters:
- dtype (
DType): Element type of the streams,yand the output. - OutLayoutType (
TensorLayout): Layout ofoutput. - StreamsLayoutType (
TensorLayout): Layout ofstreams. - YLayoutType (
TensorLayout): Layout ofy. - PostLayoutType (
TensorLayout): Layout ofpost. - CombLayoutType (
TensorLayout): Layout ofcomb. - hc_mult (
Int): Number of parallel residual streams. - hidden_size (
Int): Width of one stream. - block_size (
Int): Threads per block. - non_temporal (
Bool): Whether to mark the loads and stores non-temporal.
Args:
- output (
TileTensor[dtype, OutLayoutType, MutAnyOrigin]): Updated streams. Shape:[num_rows, hc_mult, hidden_size]. - streams (
TileTensor[dtype, StreamsLayoutType, ImmutAnyOrigin]): Incoming streams. Shape:[num_rows, hc_mult, hidden_size]. - y (
TileTensor[dtype, YLayoutType, ImmutAnyOrigin]): Sublayer output. Shape:[num_rows, hidden_size]. - post (
TileTensor[.float32, PostLayoutType, ImmutAnyOrigin]): Weight ofyin each output stream. Shape:[num_rows, hc_mult]. - comb (
TileTensor[.float32, CombLayoutType, ImmutAnyOrigin]): Stream mixer;comb[t, j, k]weighs input streamjin output streamk. Shape:[num_rows, hc_mult, hc_mult].