For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
argmax_with_bias_gpu
def argmax_with_bias_gpu[dtype: DType, bias_dtype: DType, output_type: DType](ctx: DeviceContext, logits: TileTensor[dtype, Engine=logits.Engine, address_space=logits.address_space, linear_idx_type=logits.linear_idx_type], bias: TileTensor[bias_dtype, Engine=bias.Engine, address_space=bias.address_space, linear_idx_type=bias.linear_idx_type], position: Int, output: TileTensor[output_type, Engine=output.Engine, address_space=output.address_space, linear_idx_type=output.linear_idx_type])
Takes the argmax of one position's logits plus a per-row bias.
For each row r, writes the index of the largest
float32(logits[r, position, i]) + float32(bias[r, i]). Both inputs
are upcast to float32 in registers and added there, which is exact for
16-bit inputs, so the result is the same as materializing the float32 sum
and taking its argmax: the lowest index wins a tie, NaN never wins,
and a row with nothing above -inf reports 0. No input is copied or
relaid out: the kernel reads logits[:, position, :] in place.
Parameters:
- dtype (
DType): Element type of the logits. - bias_dtype (
DType): Element type of the bias. - output_type (
DType): Element type of the output indices.
Args:
- ctx (
DeviceContext): Device context for launching the kernels. - logits (
TileTensor[dtype, Engine=logits.Engine, address_space=logits.address_space, linear_idx_type=logits.linear_idx_type]):[rows, positions, n]logits, contiguous. - bias (
TileTensor[bias_dtype, Engine=bias.Engine, address_space=bias.address_space, linear_idx_type=bias.linear_idx_type]):[rows, n]bias, contiguous. - position (
Int): Which position oflogitsto read, in[0, positions). - output (
TileTensor[output_type, Engine=output.Engine, address_space=output.address_space, linear_idx_type=output.linear_idx_type]):[rows]output indices.