For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

ProfilingConfig

ProfilingConfig

class max.pipelines.ProfilingConfig(*, config_file=None, section_name=None, gpu_profiling='off', kernel_trace_max_passes=64, kernel_trace_max_batch_links=128, kernel_trace_max_spans=20000, kernel_trace_max_capture_bytes=67108864)

source

Bases: ConfigFileModel

Configuration for the GPU (NVTX/Nsight) profiler and kernel tracing.

max serve has no command-line flags for the kernel_trace_* limits; set them in the profiling section of a --config-file.

Parameters:

• config_file (str | None)
• section_name (str | None)
• gpu_profiling (Literal['off', 'on', 'detailed'])
• kernel_trace_max_passes (int)
• kernel_trace_max_batch_links (int)
• kernel_trace_max_spans (int)
• kernel_trace_max_capture_bytes (int)

gpu_profiling

gpu_profiling: GPUProfilingMode

source

Whether to enable GPU profiling of the model.

kernel_trace_max_batch_links

kernel_trace_max_batch_links: int

source

The most request links that one max.batch span carries.

When tracing exports spans, max serve emits a max.batch span for each forward pass at MAX_SERVE_KERNEL_TRACE_LEVEL=batch or higher, and for each pass that runs a request traced with the x-max-trace-level header, linked to the spans of the requests it ran. The default is the OpenTelemetry SDK’s per-span link limit, and a higher value raises that limit too.

kernel_trace_max_capture_bytes

kernel_trace_max_capture_bytes: int

source

The largest per-request kernel capture file, in bytes, that the replay reads.

max serve replays a kernel capture armed by a request’s x-max-trace-level header into spans once it stops. A larger capture file stays on disk but exports no spans, and the server warns.

kernel_trace_max_passes

kernel_trace_max_passes: int

source

The most forward passes that one per-request kernel capture records.

When tracing exports spans and MAX_SERVE_KERNEL_TRACE_HEADERS is on, max serve captures the GPU kernels of the forward passes that run a request whose x-max-trace-level header asks for them. A capture stops after this many passes, traced or not, and its requests are traced no further.

kernel_trace_max_spans

kernel_trace_max_spans: int

source

The most spans that replaying one per-request kernel capture exports.

Once a kernel capture armed by a request’s x-max-trace-level header stops, max serve replays its GPU activity into spans. Every traced pass’s max.batch.gpu span comes first, and spans past this many are dropped, with a warning.

model_config

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': True, 'strict': False}

source

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_post_init()

model_post_init(context, /)

source

This function is meant to behave like a BaseModel method to initialise private attributes.

It takes context as an argument since that’s what pydantic-core passes when calling it.

Parameters:

• self (BaseModel) – The BaseModel instance.
• context (Any) – The context.

Return type:

None