IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo package

nn

Provides neural network operators for deep learning models.

Packages​

  • ​attention: Multi-head attention (MHA) and multi-head latent attention (MLA) kernels.
  • ​conv: Convolution kernels (1D, 2D, 3D, transposed).
  • ​sampling: Token-sampling kernels: penalties, top-k/top-p masking and sampling.

Modules​

  • ​activations: The module contains implementations of activation functions.
  • ​arange: Implements the arange operation that fills a tensor with evenly-spaced values over a range.
  • ​arg_nonzero: Implements arg_nonzero, which returns the indices of all nonzero elements in a tensor.
  • ​argmaxmin: Provides a CPU implementation of the argmax reduction.
  • ​argmaxmin_gpu: Provides a GPU argmax of logits plus a bias over the inner-most dimension.
  • ​argsort: Provides CPU and GPU implementations of argsort that return indices permuting a tensor into sorted order.
  • ​bicubic: This module provides CPU and GPU implementations for bicubic interpolation.
  • ​broadcast: Implements numpy-style tensor broadcasting for CPU and GPU targets.
  • ​chunked_qkv_rope_store: Fuses QKV unpacking or head-major projections, partial NeoX RoPE, and paged cache writes.
  • ​concat: Implements tensor concatenation along a specified axis for CPU and GPU targets.
  • ​cumsum: Implements the ONNX CumSum operator, computing prefix sums along a specified tensor axis.
  • ​fold: Implements the fold operation.
  • ​fused_qk_rope: Provides fused query/key rotary position embedding (RoPE) kernels with integrated KV-cache writes.
  • ​gated_residual_rms_norm: A residual boundary with a learned gate, in one persistent launch.
  • ​gather_scatter: Implements gather and scatter operations for CPU and GPU, including indexed reductions.
  • ​gemv_partial_norm: Fused matvec (M=1) + partial RMS norm on B200.
  • ​head_rms_norm_rope: Muse Spark's per-head attention norms, one launch each.
  • ​ihc: Identity hyper-connection (iHC) site kernels.
  • ​index_fp8: Implements tensor indexing and gather kernels with FP8 quantization support on Blackwell and AMD CDNA4 GPUs.
  • ​index_tensor: Implements numpy-style advanced tensor indexing (getitem and setitem) for CPU and GPU.
  • ​indexer_candidates: Prepare sparse indexer candidates and restore request-local token positions.
  • ​indexer_k_store: Normalize, rotate and store projected index keys in a paged quantized cache.
  • ​indexer_qk_quantize: One-launch epilogues of a DSA lightning indexer with SSMax.
  • ​irfft: Inverse real FFT kernel using cuFFT.
  • ​kv_cache:
  • ​kv_cache_gather: Gathers rows of a paged KV cache by an explicit per-row slot list.
  • ​kv_cache_ragged: Implements KV-cache kernels for ragged (variable-length) sequences used in continuous batching.
  • ​learnable_2d_interp_pos_emb: Learnable 2D interpolated position embedding (Kimi K2.5 MoonViT3d).
  • ​mhc: Manifold-Constrained Hyper-Connections (mHC) gates and stream write-back.
  • ​moe: Implements Mixture-of-Experts (MoE) routing, token dispatch, and expert computation kernels.
  • ​moe_indices_packed: The MoE index build of packed routing rows, in one CTA.
  • ​mtp_eh_norm: Input projection for a multi-token-prediction draft layer.
  • ​nms: Implements non-maximum suppression (NMS) for object detection bounding-box filtering.
  • ​normalization:
  • ​pad: Implements tensor padding with constant or edge values for CPU and GPU.
  • ​pad_gpu: Implements GPU-specific tensor padding kernels with constant, reflect, and repeat (edge-fill) strategies.
  • ​paged_store_utils: Slot, RoPE and quantization helpers for paged quantized cache stores.
  • ​pool: Provides average- and max-pooling kernels with configurable padding, dilation, and stride for CPU and GPU.
  • ​rand_normal: Generates tensors filled with values drawn from a normal (Gaussian) distribution for CPU and GPU.
  • ​rand_uniform: Generates tensors filled with values drawn from a uniform distribution for CPU and GPU.
  • ​randn: Provides a thin wrapper around randn to fill a TileTensor with standard-normal random values.
  • ​repeat_interleave: Implements repeat_interleave, which repeats each tensor element a specified number of times along an axis.
  • ​reshape: Implements the reshape operation that reinterprets a contiguous tensor's data under a new shape.
  • ​resize: Implements tensor resize (upsample/downsample) with nearest, linear, and cubic interpolation.
  • ​roi_align: Implements RoI Align, which extracts fixed-size feature maps from regions of interest using bilinear interpolation.
  • ​roi_align_gpu: Implements the GPU RoI Align kernel.
  • ​rope:
  • ​rope_split_store: Fused rope + split + KV store kernel.
  • ​routed_rows_finalize: The MoE combine with the routed-rows norm folded in, in one launch.
  • ​shapes: Provides output-shape computation utilities for sliding-window operations such as pooling and convolution.
  • ​shard_and_stack: Implements shard-and-stack: shards a tensor across devices and stacks the shards into a higher-rank output.
  • ​short_conv_ring: Depthwise causal short convolution over a ring of past inputs.
  • ​slice: Implements the ONNX Slice operator, selecting sub-tensors along specified axes with start, stop, and step.
  • ​softmax: Provides numerically stable softmax kernels for CPU and GPU, including fused and online variants.
  • ​spatial_merge: Implements spatial merge, which compresses vision token grids by merging spatial blocks before attention.
  • ​split: Implements the tensor split operation, dividing a tensor into chunks along a specified axis.
  • ​tile: Implements the ONNX Tile operator, which replicates a tensor a specified number of times along each axis.
  • ​topk: Provides top-K selection kernels using warp- and block-level reductions for CPU and GPU.
  • ​topk_bitonic: Block-wide bitonic sort top-k for the MLA/MSA indexer.
  • ​topk_filtered: The filtered top-k select for long indexer rows that fill the GPU.
  • ​topk_per_row: Top-k with a pick count per row, for the DeepSeek-V4 indexer.
  • ​toppminp: Provides CPU implementations of top-p (nucleus) and min-p sampling for autoregressive token generation.
  • ​toppminp_gpu: Provides GPU implementations of top-p (nucleus) and min-p sampling for autoregressive token generation.
  • ​tpool_patch_merger: Implements the temporal-pool patch-merger kernel that reduces vision token sequences via average pooling.

Was this page helpful?