This repository contains the NVFP4 quantized checkpoint of Mistral Large 4 - Le Chonk, our largest and most capable model to date: a 1 trillion-parameter natively multimodal Mixture-of-Experts model with 49 billion active parameters per token (52 billion including embeddings and output layers), trained from scratch on 3,800 NVIDIA Grace Blackwell GPUs in Mistral's own datacenters in Europe.
You should observe similar performance to the FP8 checkpoint, with a substantially smaller memory footprint and faster inference.
[!Note] This card covers evaluation, installation and serving of the NVFP4 checkpoint. For the architecture, capabilities, recommended settings, usage examples see the main model page: Mistral-Large-4-1T-A52B.
Same model as Mistral Large 4: 1M context window, multimodal input (text and image, text output), reasoning effort configurable per request (low, medium, high), native function calling and strict JSON schema output, and an embedded DSpark speculative decoding draft head.
We release this model under the Mistral Ecosystem Open-Weight License: a modified version of the MIT license, for both commercial and non-commercial use — operating a Model-as-a-Service business requires a separate commercial license with Mistral.
Across all domains including multimodal reasoning (GDP.pdf, MathVision), long-context (Graphwalks 512k), coding agents (DeepSWE, SWE-bench Pro, Terminal-Bench 4), business automation (AutomationBench) and cyber (CyberGym) the NvFP4 checkpoint matches FP8 behaviour.
'low' / 'medium' / 'high' — default is 'high'.reasoning_effort="high" for complex prompts and agentic usage, 'low' for fast chat replies.temperature=1.0top_p=0.95We thank all the ecosystem folks who helped us throughout this release to integrate Le Chonk into their frameworks.
The model can be deployed with:
[!Note] For optimal performance, we recommend using the Mistral AI API if local serving is subpar.
Use the official vLLM Docker image, which includes support for Mistral Large 4:
docker pull vllm/vllm-openai:mistral-large-4
See also the vLLM recipe for Mistral Large 4, which covers both the FP8 and NVFP4 checkpoints, for ready-made serving configurations.
docker run --runtime nvidia --gpus all \
-v ~/.cache/huggingface:/root/.cache/huggingface \
-p 8000:8000 \
--ipc=host \
vllm/vllm-openai:mistral-large-4 \
--model mistralai/Mistral-Large-4-1T-A52B-NVFP4 \
--tensor-parallel-size 8 \
--kv-cache-dtype fp8_ds_mla \
--tool-call-parser mistral \
--reasoning-parser mistral \
--enable-auto-tool-choice \
--gpu-memory-utilization 0.85
Key parameter notes:
--kv-cache-dtype fp8_ds_mla: FP8 KV cache for the MLA layers, needed for this checkpoint.--tool-call-parser mistral and --reasoning-parser mistral: required for tool usage and to separate reasoning from content in responses.--max-model-len and/or --max-num-seqs (concurrency). See the vLLM guide on conserving memory.The checkpoint ships the same DSpark draft head as the FP8 model. To enable it, add the following argument to the serving command above:
--speculative-config '{"method": "dspark", "num_speculative_tokens": 3}'
num_speculative_tokens is set to 3, the depth of the draft block. See the main model page for the measured speedups.
For request examples (instruction following, strict JSON schema, vision and tool calls), see Ping the Server on the main model page and use mistralai/Mistral-Large-4-1T-A52B-NVFP4 as the model id.
Serve Mistral Large 4 NVFP4 with the SGLang library for production-ready inference. See also SGLang's cookbook, which covers both the FP8 and NVFP4 checkpoints.
Use the official SGLang Docker image, which includes support for Mistral Large 4:
docker pull lmsysorg/sglang:mistral-large-4
sglang serve \
--model-path mistralai/Mistral-Large-4-1T-A52B-NVFP4 \
--tp-size 4 \
--mem-fraction-static 0.85 \
--speculative-algorithm DSPARK \
--max-running-requests 64 \
--reasoning-parser auto \
--tool-call-parser auto \
--host 0.0.0.0 \
--port 30000
[!Note] Speculative decoding (DSpark) is on and this recipe pins
--max-running-requeststo 64. Adjust it to match your target concurrency — if you remove the flag, SGLang falls back to 48.
Key parameter notes:
--speculative-algorithm DSPARK: enables the embedded DSpark draft head, which proposes blocks of 3 tokens. Remove this flag to serve without speculation.--tp-size 4: example tensor-parallel size for a Blackwell node; adjust to your topology.--reasoning-parser auto and --tool-call-parser auto: let SGLang pick the reasoning and tool-call parsers for this model.This model is licensed under the Mistral Ecosystem Open-Weight License.
You must not use this model in a manner that infringes, misappropriates, or otherwise violates any third party’s rights, including intellectual property rights.