For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
Struct_gumbel_argmax_from_residual
struct Struct_gumbel_argmax_from_residual
Registers the sampler.gumbel_argmax_from_residual graph op.
Draws one token per row proportionally to max(p - q, 0), the rejection
sampler's residual, by Gumbel-max over its ln. Both the subtract and the
clamp happen in registers inside the sampling pass, so the residual is
never written to or read from HBM: the alternative materializes the
proposal mass and the residual, two [rows, vocab] round-trips.
fallback carries one drafted token id per row; a non-negative entry
marks a row with no proposal distribution, and that row subtracts a
one-hot at the token instead of q. The value is bit-identical to
subtracting a materialized one-hot row, and it is what the caller
already has on hand, so passing the flag costs nothing.
Arity is fixed, so this is a separate registration rather than a mode on
sampler.gumbel_argmax_from_probs.
Implemented traitsโ
Methodsโ
executeโ
static def execute[_trace_name: StaticString](out_tokens: T, seed: T, probs: T, sub: T, fallback: T, ctx: DeviceContext)