For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
Struct_sampler_apply_packed_bitmask_with_penalties
struct Struct_sampler_apply_packed_bitmask_with_penalties
Registers sampler.apply_packed_bitmask_with_penalties: the packed grammar mask plus presence/frequency penalties in one output, so the penalties ride the mask's full pass over the logits instead of a copy.
Implemented traits
Methods
execute
static def execute[dtype: DType, penalty_type: DType, //, target: StringSpan[ImmStaticOrigin], _trace_name: StringSpan[ImmStaticOrigin]](output: ManagedTensorSlice[IOSpec[_, _].Output, static_spec=output.static_spec], logits: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=logits.static_spec], packed: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=packed.static_spec], fill_value: Scalar[dtype], compressed_frequency_data: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=compressed_frequency_data.static_spec], frequency_offsets: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=frequency_offsets.static_spec], frequency_penalty: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=frequency_penalty.static_spec], presence_penalty: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=presence_penalty.static_spec], ctx: DeviceContextBase)