For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
compute_log_probabilities_1tok
def compute_log_probabilities_1tok[levels: Int](output_token_index: Int, lp_logits: TileTensor[.float32, Engine=lp_logits.Engine, address_space=lp_logits.address_space, linear_idx_type=lp_logits.linear_idx_type], lp_tokens: TileTensor[.uint32, Engine=lp_tokens.Engine, address_space=lp_tokens.address_space, linear_idx_type=lp_tokens.linear_idx_type], logits: TileTensor[.float32, Engine=logits.Engine, address_space=logits.address_space, linear_idx_type=logits.linear_idx_type], tokens: TileTensor[.uint32, Engine=tokens.Engine, address_space=tokens.address_space, linear_idx_type=tokens.linear_idx_type], sampled_tokens: TileTensor[.uint32, Engine=sampled_tokens.Engine, address_space=sampled_tokens.address_space, linear_idx_type=sampled_tokens.linear_idx_type], logit_row_offsets: TileTensor[.uint32, Engine=logit_row_offsets.Engine, address_space=logit_row_offsets.address_space, linear_idx_type=logit_row_offsets.linear_idx_type], token_row_offsets: TileTensor[.uint32, Engine=token_row_offsets.Engine, address_space=token_row_offsets.address_space, linear_idx_type=token_row_offsets.linear_idx_type], lp_output_offsets: TileTensor[.uint32, Engine=lp_output_offsets.Engine, address_space=lp_output_offsets.address_space, linear_idx_type=lp_output_offsets.linear_idx_type])
Computes log probabilities for a single token position from its logits row.
Parameters:
- levels (
Int): Number of heap levels controlling the top-k width; the output stores2**levelsentries per token. Whenlevels <= 0, only the sampled token's log probability is emitted.
Args:
- output_token_index (
Int): Global row index intolp_logitsandlp_tokensfor this output position. - lp_logits (
TileTensor[.float32, Engine=lp_logits.Engine, address_space=lp_logits.address_space, linear_idx_type=lp_logits.linear_idx_type]): Output tensor of shape[num_output_tokens, 2**levels]receiving the log probabilities. - lp_tokens (
TileTensor[.uint32, Engine=lp_tokens.Engine, address_space=lp_tokens.address_space, linear_idx_type=lp_tokens.linear_idx_type]): Output tensor of shape[num_output_tokens, 2**levels]receiving the token ids paired withlp_logits. - logits (
TileTensor[.float32, Engine=logits.Engine, address_space=logits.address_space, linear_idx_type=logits.linear_idx_type]): Input logits ragged by batch, shape[total_rows, vocab_size]. - tokens (
TileTensor[.uint32, Engine=tokens.Engine, address_space=tokens.address_space, linear_idx_type=tokens.linear_idx_type]): Previously generated tokens across all batches, ragged. - sampled_tokens (
TileTensor[.uint32, Engine=sampled_tokens.Engine, address_space=sampled_tokens.address_space, linear_idx_type=sampled_tokens.linear_idx_type]): Most recently sampled token for each batch, used when this position is the last token in its sequence. - logit_row_offsets (
TileTensor[.uint32, Engine=logit_row_offsets.Engine, address_space=logit_row_offsets.address_space, linear_idx_type=logit_row_offsets.linear_idx_type]): Per-batch start offsets into the first axis oflogits. - token_row_offsets (
TileTensor[.uint32, Engine=token_row_offsets.Engine, address_space=token_row_offsets.address_space, linear_idx_type=token_row_offsets.linear_idx_type]): Per-batch start offsets intotokens. - lp_output_offsets (
TileTensor[.uint32, Engine=lp_output_offsets.Engine, address_space=lp_output_offsets.address_space, linear_idx_type=lp_output_offsets.linear_idx_type]): Per-batch start offsets into the output row axis, mapping each output index to its batch.