For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

SamplerInputs

SamplerInputs

class max.pipelines.sampling.SamplerInputs(top_k, max_k, temperature, top_p, min_top_p, min_p, seed)

source

Bases: object

Container for sampler inputs.

Parameters:

• top_k (Buffer)
• max_k (Buffer)
• temperature (Buffer)
• top_p (Buffer)
• min_top_p (Buffer)
• min_p (Buffer)
• seed (Buffer)

as_list()

as_list()

source

Returns the sampler inputs as a list of buffers.

Return type:

list[Buffer]

create()

classmethod create(batch, device)

source

Create sampling parameter tensors from context batch.

Parameters:

• batch (list[TextContext]) – List of context objects containing sampling parameters.
• device (Device) – Device to place the tensors on.

Returns:

SamplerInputs containing the sampling parameter tensors.

Return type:

Self

max_k

max_k: Buffer

source

min_p

min_p: Buffer

source

min_top_p

min_top_p: Buffer

source

seed

seed: Buffer

source

temperature

temperature: Buffer

source

top_k

top_k: Buffer

source

top_p

top_p: Buffer

source