For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo module

logprobs

Registers log-probability computation graph ops used by sampling pipelines.

`comptime` values

comptime logit_dtype = DType.float32

Element type of logit tensors used by log-probability ops.

comptime offset_dtype = DType.uint32

Element type of row offset tensors used by log-probability ops.

comptime token_dtype = DType.uint32

Element type of token id tensors used by log-probability ops.

FixedHeightMinHeap: Maintains a fixed-capacity min-heap of key/value pairs used to track top-k logits.
LogProbabilitiesRagged: Registers the compute_log_probabilities_ragged graph op computing per-token log probabilities over ragged batches.

compute_log_probabilities_1tok: Computes log probabilities for a single token position from its logits row.
compute_log_probabilities_ragged_shape: Computes the output shapes for the ragged log-probabilities op.