For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python class
GraphPipelineModelWithKVCache
GraphPipelineModelWithKVCache
class max.pipelines.lib.GraphPipelineModelWithKVCache(pipeline_config, session, devices, kv_cache_config, weights, adapter, return_logits, *, memory_plan, return_hidden_states=ReturnHiddenStates.NONE, max_batch_size=1)
Bases: PipelineModelWithKVCache[BaseContextType, _ModelConfigT, Model]
Graph-API pipeline model with shared compile-and-load template.
Subclasses override _build_graph_for_compile() (and optionally
_create_model_config(), _init_distributed_runtime()) rather than
duplicating weight loading, timing, and EP batch-processor wiring.
ModuleV3 models and multi-graph VLMs should inherit
MultiGraphPipelineModelWithKVCache instead.
-
Parameters:
-
- pipeline_config (PipelineConfig)
- session (InferenceSession)
- devices (list[Device])
- kv_cache_config (KVCacheConfig)
- weights (Weights)
- adapter (WeightsAdapter | None)
- return_logits (ReturnLogits)
- memory_plan (MemoryPlan)
- return_hidden_states (ReturnHiddenStates)
- max_batch_size (int)
load_model()
load_model(session)
Build, compile, and load the model graph into session.
-
Parameters:
-
session (InferenceSession)
-
Return type: