For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python class
MemoryEstimator
MemoryEstimator
class max.pipelines.MemoryEstimator
Bases: object
Plans device memory for a pipeline and estimates what it will use.
plan()
classmethod plan(pipeline_config, arch, draft_arch=None)
Runs memory planning and returns the finished plan.
Called by the registry’s retrieve_factory after the config is
constructed. Gathers the sizes and the draft-model bound that
plan_from_sizes() needs and runs it. Nothing is written back
to pipeline_config, which keeps carrying the
construction-resolved values unchanged.
-
Parameters:
-
- pipeline_config (PipelineConfig)
- arch (SupportedArchitecture)
- draft_arch (SupportedArchitecture | None)
-
Return type:
plan_from_sizes()
classmethod plan_from_sizes(pipeline_config, model_config, arch_config, devices, model_weights_size, activation_memory_size, signal_buffer_size=0, arch=None, max_batch_size=None, draft_max_seq_len=None)
Plans memory from precomputed weight, activation, and buffer sizes.
Estimates the footprint, reserves the KV and vision budgets, and
bounds max_length and max_batch_size to what fits. Callers
that have only a PipelineConfig should use
plan(), which derives the sizes and calls this.
draft_max_seq_len is the draft model’s own sequence-length limit
under speculative decoding. It bounds max_length because the draft
shares the target’s KV cache, and it is passed in because the
estimator never sees the draft architecture.
-
Returns:
-
The finished
MemoryPlan. Nothing is written back topipeline_config. -
Parameters:
-
- pipeline_config (PipelineConfig)
- model_config (MAXModelConfig)
- arch_config (ArchConfig)
- devices (list[Device])
- model_weights_size (int)
- activation_memory_size (int)
- signal_buffer_size (int)
- arch (SupportedArchitecture | None)
- max_batch_size (int | None)
- draft_max_seq_len (int | None)
-
Return type: