IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo module

operand_masks

Attention masks built from graph operands.

Each mask reads its operands only through load, so the graph compiler may fuse an operand's producer into it. The view parameters default to NoView; make returns the mask instantiated with the operands' real views.

Structs​

  • ​AdditiveMask: Adds bias[b, q - p, k] (rank 3) or bias[b, h, q - p, k] (rank 4) to the score, where p is the bias's key length minus its query length. Positions outside the bias are masked.
  • ​RelativeLogitsMask: Causal (or sliding-window causal when window_size > 0) visibility, plus an additive bias by relative distance.
  • ​SegmentMask: Segment mask over ragged sequences: a query sees a key iff either token's segment id is -1 or the two ids are equal.

Functions​

  • ​aligned_load: Loads width elements of view at idx, as one aligned vector when it can on GPU.

Was this page helpful?