For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

ReturnHiddenStates

ReturnHiddenStates

class max.nn.ReturnHiddenStates(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)

source

Bases: str, Enum

Which hidden states a transformer model returns per sequence.

NONE returns none, LAST returns the final token’s hidden states, and ALL returns every token’s. See each member’s description for the other modes.

ALL

ALL = 'all'

source

Return every token’s hidden states.

ALL_NORMALIZED

ALL_NORMALIZED = 'all_normalized'

source

Return every token’s hidden states after the model’s final norm.

LAST

LAST = 'last'

source

Return the final token’s hidden states.

LAST_PER_DEVICE

LAST_PER_DEVICE = 'last_per_device'

source

Return the final token’s hidden states as per-device tensors, one per device, without collapsing identical replicas.

NONE

NONE = 'none'

source

Return no hidden states.

SELECTED_LAYERS

SELECTED_LAYERS = 'selected_layers'

source

Return hidden states captured at specific layer indices.