For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python module
max.pipelines.lib.decoder_scratch
Workspace buffers a registered decoder’s kernels write each step.
A decoder whose kernels need workspace sized for the largest batch, such as
a sparse attention indexer’s score table, declares it with a
scratch_types classmethod that returns one
BufferType per buffer. The framework then:
- allocates each buffer once, at its declared size;
- adds them to the graph’s inputs as the
scratchfield, after the per-request state; - passes the same buffers every step.
Unlike per-request state, nothing in them outlives a step, so a request owns no part of them. Their shapes are static, so an engine that serves the graph allocates them at the declared size too.
Workspace
DecoderScratch | A decoder's workspace buffers and the graph inputs they fill. |
|---|
Construction
build_decoder_scratch | Allocates the workspace a registered decoder declares, if any. |
|---|