For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
dispatch_mask
def dispatch_mask[mask_type: String, local_window_size: Int = Int(-1)](callback_fn: T)
Instantiate an MHAMask that reads no operands by name and invoke a callback with it.
Parameters:
- mask_type (
String): Name of the mask (e.g."causal","null","sliding_window_causal"). Must match one of theMaskNameconstants. - local_window_size (
Int): Sliding-window or chunk size for masks that require it. Must be-1for masks that ignore it and positive for masks that require it.
Args:
- callback_fn (
T): Parametric callback invoked with the resolved mask.
Raises:
Compile-time assertion error if mask_type is unrecognised, if it
names a mask that reads operands, or if local_window_size is
inconsistent with the selected mask.
def dispatch_mask[MaskIns: TypeList[MaskIns.values], //, mask_type: String, local_window_size: Int = Int(-1)](mask_operands: TensorTuple[*MaskIns.values], callback_fn: T)
Build an MHAMask by name from its graph operands and invoke a callback with it.
Resolves the mask string to one of the built-in MHAMask implementations
at compile time, builds it with make from mask_operands, and calls
callback_fn with it. This lets callers write mask-agnostic kernels
while still specialising the generated code per mask type, and lets one
kernel take every mask whatever operands it reads.
Parameters:
- MaskIns (
TypeList[values]): The mask operand types. - mask_type (
String): Name of the mask (e.g."causal","null","sliding_window_causal","segment"). Must match one of theMaskNameconstants. - local_window_size (
Int): Sliding-window or chunk size for masks that require it. Must be-1for masks that ignore it and positive for masks that require it.
Args:
- mask_operands (
TensorTuple[*MaskIns.values]): The mask's graph operands: none for the masks that read none, the ids and row offsets for"segment", the mask for"additive", the bias, cache lengths and row offsets for"relative_logits". - callback_fn (
T): Parametric callback invoked with the resolved mask.
Raises:
Compile-time assertion error if mask_type is unrecognised, if the
operands don't match the mask, or if local_window_size is
inconsistent with the selected mask.