For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python function

modulev3_ragged_kv_symbolic_input_tree

modulev3_ragged_kv_symbolic_input_tree()

max.pipelines.lib.interfaces.modulev3_ragged_kv_symbolic_input_tree(*, kv_params, device_refs)

source

Returns the named inputs of a ModuleV3 ragged KV model.

ModuleV3 forward expects (tokens, return_n_logits, input_row_offsets, *kv), a different order than ragged_kv_symbolic_input_tree().

Parameters:

• kv_params (KVCacheParamInterface) – The KV cache parameters that supply the kv subtree.
• device_refs (list[DeviceRef]) – The devices the model runs on.

Returns:

The tree of input types, keyed batch and kv.

Return type:

OrderedDict[str, object]