For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo package
nn
Provides neural network operators for deep learning models.
Packages
-
attention: Multi-head attention (MHA) and multi-head latent attention (MLA) kernels. -
conv: Convolution kernels (1D, 2D, 3D, transposed). -
sampling: Token-sampling kernels: penalties, top-k/top-p masking and sampling.
Modules
-
activations: The module contains implementations of activation functions. -
arange: Implements thearangeoperation that fills a tensor with evenly-spaced values over a range. -
arg_nonzero: Implementsarg_nonzero, which returns the indices of all nonzero elements in a tensor. -
argmaxmin: Provides a CPU implementation of the argmax reduction. -
argmaxmin_gpu: Provides a GPU argmax of logits plus a bias over the inner-most dimension. -
argsort: Provides CPU and GPU implementations of argsort that return indices permuting a tensor into sorted order. -
bicubic: This module provides CPU and GPU implementations for bicubic interpolation. -
broadcast: Implements numpy-style tensor broadcasting for CPU and GPU targets. -
chunked_qkv_rope_store: Fuses QKV unpacking or head-major projections, partial NeoX RoPE, and paged cache writes. -
concat: Implements tensor concatenation along a specified axis for CPU and GPU targets. -
cumsum: Implements the ONNX CumSum operator, computing prefix sums along a specified tensor axis. -
fold: Implements the fold operation. -
fused_qk_rope: Provides fused query/key rotary position embedding (RoPE) kernels with integrated KV-cache writes. -
gated_residual_rms_norm: A residual boundary with a learned gate, in one persistent launch. -
gather_scatter: Implements gather and scatter operations for CPU and GPU, including indexed reductions. -
gemv_partial_norm: Fused matvec (M=1) + partial RMS norm on B200. -
head_rms_norm_rope: Muse Spark's per-head attention norms, one launch each. -
ihc: Identity hyper-connection (iHC) site kernels. -
index_fp8: Implements tensor indexing and gather kernels with FP8 quantization support on Blackwell and AMD CDNA4 GPUs. -
index_tensor: Implements numpy-style advanced tensor indexing (getitem and setitem) for CPU and GPU. -
indexer_candidates: Prepare sparse indexer candidates and restore request-local token positions. -
indexer_k_store: Normalize, rotate and store projected index keys in a paged quantized cache. -
indexer_qk_quantize: One-launch epilogues of a DSA lightning indexer with SSMax. -
irfft: Inverse real FFT kernel using cuFFT. -
kv_cache: -
kv_cache_gather: Gathers rows of a paged KV cache by an explicit per-row slot list. -
kv_cache_ragged: Implements KV-cache kernels for ragged (variable-length) sequences used in continuous batching. -
learnable_2d_interp_pos_emb: Learnable 2D interpolated position embedding (Kimi K2.5 MoonViT3d). -
mhc: Manifold-Constrained Hyper-Connections (mHC) gates and stream write-back. -
moe: Implements Mixture-of-Experts (MoE) routing, token dispatch, and expert computation kernels. -
moe_indices_packed: The MoE index build of packed routing rows, in one CTA. -
mtp_eh_norm: Input projection for a multi-token-prediction draft layer. -
nms: Implements non-maximum suppression (NMS) for object detection bounding-box filtering. -
normalization: -
pad: Implements tensor padding with constant or edge values for CPU and GPU. -
pad_gpu: Implements GPU-specific tensor padding kernels with constant, reflect, and repeat (edge-fill) strategies. -
paged_store_utils: Slot, RoPE and quantization helpers for paged quantized cache stores. -
pool: Provides average- and max-pooling kernels with configurable padding, dilation, and stride for CPU and GPU. -
rand_normal: Generates tensors filled with values drawn from a normal (Gaussian) distribution for CPU and GPU. -
rand_uniform: Generates tensors filled with values drawn from a uniform distribution for CPU and GPU. -
randn: Provides a thin wrapper aroundrandnto fill aTileTensorwith standard-normal random values. -
repeat_interleave: Implementsrepeat_interleave, which repeats each tensor element a specified number of times along an axis. -
reshape: Implements the reshape operation that reinterprets a contiguous tensor's data under a new shape. -
resize: Implements tensor resize (upsample/downsample) with nearest, linear, and cubic interpolation. -
roi_align: Implements RoI Align, which extracts fixed-size feature maps from regions of interest using bilinear interpolation. -
roi_align_gpu: Implements the GPU RoI Align kernel. -
rope: -
rope_split_store: Fused rope + split + KV store kernel. -
routed_rows_finalize: The MoE combine with the routed-rows norm folded in, in one launch. -
shapes: Provides output-shape computation utilities for sliding-window operations such as pooling and convolution. -
shard_and_stack: Implements shard-and-stack: shards a tensor across devices and stacks the shards into a higher-rank output. -
short_conv_ring: Depthwise causal short convolution over a ring of past inputs. -
slice: Implements the ONNX Slice operator, selecting sub-tensors along specified axes with start, stop, and step. -
softmax: Provides numerically stable softmax kernels for CPU and GPU, including fused and online variants. -
spatial_merge: Implements spatial merge, which compresses vision token grids by merging spatial blocks before attention. -
split: Implements the tensor split operation, dividing a tensor into chunks along a specified axis. -
tile: Implements the ONNX Tile operator, which replicates a tensor a specified number of times along each axis. -
topk: Provides top-K selection kernels using warp- and block-level reductions for CPU and GPU. -
topk_bitonic: Block-wide bitonic sort top-k for the MLA/MSA indexer. -
topk_filtered: The filtered top-k select for long indexer rows that fill the GPU. -
topk_per_row: Top-k with a pick count per row, for the DeepSeek-V4 indexer. -
toppminp: Provides CPU implementations of top-p (nucleus) and min-p sampling for autoregressive token generation. -
toppminp_gpu: Provides GPU implementations of top-p (nucleus) and min-p sampling for autoregressive token generation. -
tpool_patch_merger: Implements the temporal-pool patch-merger kernel that reduces vision token sequences via average pooling.