IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

argmax_with_bias_gpu

def argmax_with_bias_gpu[dtype: DType, bias_dtype: DType, output_type: DType](ctx: DeviceContext, logits: TileTensor[dtype, Engine=logits.Engine, address_space=logits.address_space, linear_idx_type=logits.linear_idx_type], bias: TileTensor[bias_dtype, Engine=bias.Engine, address_space=bias.address_space, linear_idx_type=bias.linear_idx_type], position: Int, output: TileTensor[output_type, Engine=output.Engine, address_space=output.address_space, linear_idx_type=output.linear_idx_type])

Takes the argmax of one position's logits plus a per-row bias.

For each row r, writes the index of the largest float32(logits[r, position, i]) + float32(bias[r, i]). Both inputs are upcast to float32 in registers and added there, which is exact for 16-bit inputs, so the result is the same as materializing the float32 sum and taking its argmax: the lowest index wins a tie, NaN never wins, and a row with nothing above -inf reports 0. No input is copied or relaid out: the kernel reads logits[:, position, :] in place.

Parameters:

  • ​dtype (DType): Element type of the logits.
  • ​bias_dtype (DType): Element type of the bias.
  • ​output_type (DType): Element type of the output indices.

Args:

Was this page helpful?