For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python function

vision_merge_inputs

vision_merge_inputs()

max.pipelines.lib.interfaces.vision_merge_inputs(device_refs, dtype, hidden_size, *, rows_dim)

source

Returns the per-device vision embeddings and where they merge.

Parameters:

• device_refs (Sequence[DeviceRef]) – The devices that each take a copy of the embeddings.
• dtype (DType) – The dtype of the embeddings.
• hidden_size (int) – The width of each embedding row.
• rows_dim (str) – The name of the embeddings’ row dimension, which each architecture spells its own way.

Returns:

The embedding types under image_embeddings and the positions in the token stream that their rows replace under image_token_indices, one of each per device.

Return type:

VisionMerge