IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

dispatch_mask

def dispatch_mask[mask_type: String, local_window_size: Int = Int(-1)](callback_fn: T)

Instantiate an MHAMask that reads no operands by name and invoke a callback with it.

Parameters:

  • ​mask_type (String): Name of the mask (e.g. "causal", "null", "sliding_window_causal"). Must match one of the MaskName constants.
  • ​local_window_size (Int): Sliding-window or chunk size for masks that require it. Must be -1 for masks that ignore it and positive for masks that require it.

Args:

  • ​callback_fn (T): Parametric callback invoked with the resolved mask.

Raises:

Compile-time assertion error if mask_type is unrecognised, if it names a mask that reads operands, or if local_window_size is inconsistent with the selected mask.

def dispatch_mask[MaskIns: TypeList[MaskIns.values], //, mask_type: String, local_window_size: Int = Int(-1)](mask_operands: TensorTuple[*MaskIns.values], callback_fn: T)

Build an MHAMask by name from its graph operands and invoke a callback with it.

Resolves the mask string to one of the built-in MHAMask implementations at compile time, builds it with make from mask_operands, and calls callback_fn with it. This lets callers write mask-agnostic kernels while still specialising the generated code per mask type, and lets one kernel take every mask whatever operands it reads.

Parameters:

  • ​MaskIns (TypeList[values]): The mask operand types.
  • ​mask_type (String): Name of the mask (e.g. "causal", "null", "sliding_window_causal", "segment"). Must match one of the MaskName constants.
  • ​local_window_size (Int): Sliding-window or chunk size for masks that require it. Must be -1 for masks that ignore it and positive for masks that require it.

Args:

  • ​mask_operands (TensorTuple[*MaskIns.values]): The mask's graph operands: none for the masks that read none, the ids and row offsets for "segment", the mask for "additive", the bias, cache lengths and row offsets for "relative_logits".
  • ​callback_fn (T): Parametric callback invoked with the resolved mask.

Raises:

Compile-time assertion error if mask_type is unrecognised, if the operands don't match the mask, or if local_window_size is inconsistent with the selected mask.

Was this page helpful?