For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Environment variables

This page documents all environment variables you can use to configure MAX behavior. These variables control server settings, logging, telemetry, performance, and integrations.

How to set environment variables

You can set environment variables in several ways:

# Export in your shell
export MAX_SERVE_HOST="0.0.0.0"

# Pass to Docker container
docker run --env "MAX_SERVE_HOST=0.0.0.0" modular/max-nvidia-full:latest ...

# Use a .env file in your working directory
echo "MAX_SERVE_HOST=0.0.0.0" >> .env

Configuration precedence

When you configure the same setting in multiple places, the following precedence applies (highest to lowest):

1. CLI flags or direct Python initialization: For example, --port 8080 or Settings(MAX_SERVE_PORT=8080). CLI flags pass directly to the Settings constructor, so they have the same precedence as direct Python initialization.
2. Environment variables: export MAX_SERVE_HOST="0.0.0.0"
3. .env file values: Values defined in a .env file in your working directory

Environment variables must use the exact names documented below.

Serving

These variables configure the MAX model serving behavior.

For more information on serving a model with MAX, explore the text to text and image and video to text guides.

Variable	Description	Values	Default
MAX_SERVE_HOST	Hostname for the MAX server	String	0.0.0.0
MAX_SERVE_PORT	Port for serving MAX	Integer	8000
MAX_SERVE_METRICS_ENDPOINT_PORT	Port for the Prometheus metrics endpoint	Integer	8001
MAX_SERVE_ALLOWED_IMAGE_ROOTS	Allowed root directories for file:// URI access, as a JSON array (for example, '["/srv/images"]')	JSON array string	Empty
MAX_SERVE_MAX_LOCAL_IMAGE_BYTES	Maximum size in bytes for local image files	Integer	20971520 (20 MiB)
MAX_SERVE_MAX_REQUEST_BYTES	Maximum size in bytes of an accepted HTTP request body. MAX rejects a larger request with HTTP 413, based on the declared Content-Length or on the bytes counted as the body streams in. Raise it for requests that inline large base64 media, or set to 0 to disable the limit.	Integer (≥ 0)	104857600 (100 MiB)
MAX_SERVE_MAX_BYTES	Server-level maximum size in bytes for media resolved from http(s):// or data: URIs. MAX applies it on top of any per-model cap, and the smaller of the two wins. Set to 0 to disable the server-level cap.	Integer	0
MAX_SERVE_MEDIA_KIND	Default media kind used in size-limit error messages when a resolver caller doesn't specify one	image, video	image
MAX_SERVE_MEDIA_URL_ALLOWED_HOSTS	Allowlist of otherwise-blocked internal hosts that MAX can fetch media from while SSRF protection stays on. Each entry is an exact hostname (case-insensitive) or an IP address or CIDR range, such as '["minio.internal", "10.0.0.0/8"]'	JSON array string	Empty
MAX_SERVE_MEDIA_URL_SSRF_PROTECTION_ENABLED	Protects against server-side request forgery (SSRF) when MAX fetches media from a client-supplied http(s):// URL. MAX checks the host at each redirect and rejects any host that resolves to a private, loopback, or otherwise non-public address. Turn this off only as a last resort, to restore the earlier unchecked behavior. To allow specific internal hosts while the protection stays on, use MAX_SERVE_MEDIA_URL_ALLOWED_HOSTS.	true, false	true
MAX_SERVE_API_TYPES	Configures which API types MAX serve exposes. Accepts a JSON array of API type strings (for example, '["responses"]'). Use this to enable the Responses API for tasks like image generation.	JSON array string	None
MAX_SERVE_GRACEFUL_SHUTDOWN_TIMEOUT_S	Seconds to wait for in-flight requests to finish after SIGTERM before canceling them and exiting	Integer	5
MAX_SERVE_MAX_QUEUE_SIZE	Cap on the request queue to the model worker. Once full, the server rejects new requests with HTTP 429 instead of enqueuing them. Pair with MAX_SERVE_MAX_PENDING_REQUESTS for effective backpressure	Integer	None (unbounded)
MAX_SERVE_MAX_PENDING_REQUESTS	Cap on the scheduler's pending prefill queue depth. When set, the model worker stops pulling new requests from the request queue once it holds this many not-yet-running requests	Integer (≥ 1)	None (unbounded)
MAX_SERVE_STREAM_MIN_CHUNK_TOKENS	Minimum number of tokens per streamed server-sent events (SSE) chunk. Larger values coalesce streaming output into bigger chunks without affecting time to first token	Integer	1
MODULAR_DKV_TENANT_ID	Tenant identity for the dKV external KV cache tier. Required when using the dKV connector	String	None
MODULAR_DKV_DISABLE_G0_TOUCH	Disable recency refresh (touch) of an external KV cache tier when the on-GPU cache serves a request's prefix	0, 1	0
MAX_SERVE_BATCH_PRIORITY	Batch scheduling strategy that controls how replicas prioritize prefill (context encoding) versus decode (token generation) requests	prefill_first, decode_first, balanced, per_replica	per_replica

Logging

These variables control logging behavior and verbosity.

You can read more about logs when using the MAX container.

Variable	Description	Values	Default
MAX_SERVE_LOGS_CONSOLE_LEVEL	Console log verbosity level	CRITICAL, ERROR, WARNING, INFO, DEBUG	INFO
MODULAR_STRUCTURED_LOGGING	Enable JSON-formatted structured logging for deployed services	0, 1	1
MAX_SERVE_LOGS_FILE_PATH	Path to write log files	File path	None
MAX_SERVE_LOG_PREFIX	Prefix to prepend to all log messages	String	None

Telemetry and metrics

These variables control telemetry collection and metrics reporting.

For more information, read about telemetry.

Variable	Description	Values	Default
MAX_SERVE_DISABLE_TELEMETRY	Disable remote telemetry collection	0, 1	0
MODULAR_USER_ID	User identifier for telemetry (for example, your company name)	String	None
MAX_SERVE_DEPLOYMENT_ID	Deployment identifier for telemetry (for example, your application name)	String	None
MAX_SERVE_OTLP_METRICS_ENDPOINT	Push a self-calibrating exponential-histogram shadow (<metric>.exponential) of every histogram metric to an OTLP endpoint, alongside the Prometheus /metrics histograms	Endpoint URL	None
MAX_SERVE_EPLB_PROFILE	Enable expert-parallel load balancing (EPLB) statistics profiling in max serve	0, 1	0

Debugging

The MODULAR_DEBUG environment variable enables one or more MAX debugging options. Set it to a comma-separated list of option names, using name=value for options that take a value:

export MODULAR_DEBUG=nan-check,assert-level=all

This table includes all options you can add to MODULAR_DEBUG:

Option	Description	Type	Accepted values	Default
sensible	Enable a curated default debugging set	Boolean	N/A	false
nan-check	Insert NaN/Inf checks on a sampled subset of floating-point kernel outputs (see nan-check-stride)	Boolean	N/A	false
nan-check-stride	When nan-check is enabled, check one of every N floating-point kernel outputs. Set to 1 for full coverage	Integer	Positive integer (≥1)	20
uninitialized-read-check	Detect reads of uninitialized memory	Boolean	N/A	false
device-sync-mode	Force synchronous GPU execution for debugging	Boolean	N/A	false
stack-trace-on-error	Show Mojo stack traces on runtime errors	Boolean	N/A	false
stack-trace-on-crash	Show Mojo stack traces on crashes	Boolean	N/A	false
source-tracebacks	Include Python source locations in error messages	Boolean	N/A	false
op-log-level	Log level for op-level execution tracing	String	trace, debug, info, warning, error, critical	off
assert-level	Assertion level for the Mojo standard library	String	none, warn, safe, all	none
print-style	Output format for tensor debug printing	String	compact, full, binary, binary_max_checkpoint	compact
ir-output-dir	Directory where MAX dumps intermediate compiler IR	Path	Filesystem path	unset

The following variables control memory-allocation debugging:

Variable	Description	Values	Default
MODULAR_DEBUG_DEVICE_ALLOCATOR	Comma-separated allocator debugging modes. poison-all fills every memory-manager allocation with a NaN-pattern byte; the uninitialized-read-check debug option automatically enables uninitialized-poison	Comma-separated modes	None
MODULAR_DEVICE_CONTEXT_MEMORY_MANAGER_POISON_PATTERN	Byte pattern written by the poison-all allocator mode	Byte (0–255)	Built-in NaN pattern

The following variable overrides the GPU compiler MAX uses:

Variable	Description	Values	Default
MODULAR_NVPTX_COMPILER_PATH	Path to a custom NVIDIA ptxas binary, useful as an escape hatch when the installed NVIDIA driver is too old for the bundled compiler	Path	Unset

Profiling

For GPU profiling details, see GPU profiling with Nsight Systems.

The following variable controls runtime profiling and tracing.

Variable	Description	Values	Default
MODULAR_ENABLE_PROFILING	Enable runtime profiling and tracing	off, on, detailed	off

Performance and caching

The following variables configure caching and memory behavior.

Variable	Description	Values	Default
MODULAR_MAX_CACHE_DIR	Directory to save MAX model cache for reuse	Path	$MODULAR_CACHE_DIR/.max_cache
MODULAR_CACHE_DIR	Configure cache directory for all MAX filesystems	Path	See note below
MODULAR_MAX_SHM_WATERMARK	Percentage of /dev/shm to allocate for shared memory. Set to 0.0 to disable shared memory.	Float (0.0–1.0)	0.9
MAX_EAGER_OP_PRECOMPILE	Controls how the eager interpreter compiles its built-in op targets. The default compiles each target lazily on its first dispatch, which avoids a long cold-cache compile when a program touches only a few targets. Set to 1 to precompile the full (device, dtype) matrix at import so steady-state dispatch never recompiles. MAX reads the value when the sweep runs rather than at import.	0, 1	0
MAX_EAGER_EXECUTOR	Select the eager execution backend	composite, jit, interpreter, compile	composite
MAX_EAGER_ALLOW_LAZY_COMPILE	Allow max serve to compile eager interpreter models on demand instead of requiring a warm cache from max warm-interpreter-cache	0, 1	0
MODULAR_DEVICE_CONTEXT_MEMORY_MANAGER_VMM	Toggle the VMM defragmenting allocator, which avoids external-fragmentation OOMs. Enabled by default on NVIDIA GPUs; opt in on AMD MI300-series GPUs with 1	0, 1	1 (NVIDIA), 0 (AMD)
MODULAR_ENABLE_APPLE_NAIVE_FA_DECODE	Set to 0 to opt out of the split-K decode attention kernel on Apple GPUs (paged KV cache MHA and GQA decode)	0, 1	1
MODULAR_MAX_RELEASE_FREE_HOST_MEMORY	Return freed host memory to the operating system after model loading completes	Any non-empty value	Unset (disabled)

The default search paths take the following precedence:

1. When the MODULAR_HOME environment variable is set, MAX uses $MODULAR_HOME/cache.
2. If $HOME/.modular exists, MAX uses $HOME/.modular.
3. Follow the XDG Base Directory Specification. When XDG_CACHE_HOME is set, MAX uses $XDG_CACHE_HOME/modular; otherwise it defaults to $HOME/.cache/modular.

Weight loading

The following variables control how MAX handles model weights when loading from a checkpoint.

Variable	Description	Values	Default
MODULAR_AUTO_CAST_WEIGHTS	Auto-cast loaded weights between float32 and bfloat16 when checkpoint and module dtypes mismatch (shape must match). Other dtype mismatches still raise.	true, false	true
APPLE_FLUX2_INT8_W8A8	On Apple M5 GPUs, controls int8 W8A8 quantization for FLUX.2 checkpoints: FLUX.2-klein bf16 checkpoints default to int8 W8A8 (set 0 to opt out); NVFP4 checkpoints opt into an int8 W8A8 requant at load with 1.	0, 1	1 (FLUX.2-klein bf16)
MODULAR_MAX_RELEASE_HOST_WEIGHTS	Set to 1 to release the host copies of model weights once the model is loaded on GPU, reducing host memory usage. GPU deployments only	1	Unset (disabled)

Hugging Face

Configure your Hugging Face integration with the following environment variable:

Variable	Description	Values	Default
HF_TOKEN	Hugging Face authentication token for accessing gated models	String	None

Related resources

• MAX container: Deploy MAX with Docker
• max serve CLI: Command-line options for serving