For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

LogitPenalties

LogitPenalties

class max.nn.legacy.sampling.LogitPenalties(data, offsets, frequency, presence)

source

Bases: object

Presence/frequency penalties, one CSR row per logit row.

Logit row i owns data[offsets[i]:offsets[i+1]], [token, count] pairs counted over the tokens generated before that row’s position. A negative token is padding.

Parameters:

• data (TensorValue)
• offsets (TensorValue)
• frequency (TensorValue)
• presence (TensorValue)

apply()

apply(logits)

source

Returns 2-D logits with the penalties subtracted.

Costs a copy of the logits; prefer apply_with_bitmask() wherever a grammar mask is applied anyway.

Parameters:

logits (TensorValue)

Return type:

TensorValue

apply_with_bitmask()

apply_with_bitmask(logits, packed, fill_val)

source

Masks logits and subtracts penalties in the mask’s own pass.

Masking matches apply_packed_bitmask, and penalties are subtracted from the tokens the mask keeps.

Parameters:

• logits (TensorValue)
• packed (TensorValue)
• fill_val (float)

Return type:

TensorValue

data

data: TensorValue

source

frequency

frequency: TensorValue

source

from_inputs()

classmethod from_inputs(values)

source

Binds the four values input_types() declared.

Parameters:

values (Sequence[TensorValue])

Return type:

LogitPenalties

input_types()

static input_types(device)

source

Declares the four inputs, in the order from_inputs() reads.

Parameters:

device (DeviceRef)

Return type:

list[TensorType]

offsets

offsets: TensorValue

source

presence

presence: TensorValue

source