Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning

Nvidia has released Nemotron 3 Ultra, a 550-billion-parameter language model that uses a hybrid Mamba-Transformer architecture and is openly available on HuggingFace, according to a paper submitted to arXiv on June 12, 2026 [1][2]. The model, described as Nvidia's most capable to date, has 55 billion active parameters through a Mixture-of-Experts design and was pre-trained on 20 trillion text tokens before its context length was extended to 1 million tokens [1][2]. Post-training involved Supervised Fine Tuning, Reinforcement Learning, and Multi-teacher On-Policy Distillation [2]. The architecture incorporates several technologies, including LatentMoE, Multi Token Prediction, NVFP4 pre-training, multi-environment RLVR, and reasoning budget control [2]. Nvidia claims Nemotron 3 Ultra achieves up to approximately 6x higher inference throughput compared to state-of-the-art publicly available large language models while maintaining on-par accuracy [1][2]. The combination of throughput, accuracy, and the 1-million-token context window is designed to support long-running autonomous agentic tasks [2]. The company has open-sourced the base, post-trained, and quantized checkpoints, along with the training data and recipe, on HuggingFace [1][2]. The Nemotron family has evolved through several generations. Nvidia launched the dense Nemotron-4 models in 2024, followed by the Llama Nemotron reasoning models announced at the 2025 Consumer Electronics Show, and the hybrid Nemotron 3 family in late 2025 [3]. In March 2026, Nvidia formed the Nemotron Coalition, a group of AI labs collaborating on future open models [3]. Nvidia's push into open-source AI models comes as the company has cemented its dominance in AI hardware. As of early 2025, Nvidia controlled more than 80 percent of the market for GPUs used in training and deploying AI models and provided chips for over 75 percent of the world's TOP500 supercomputers [4]. The company's market capitalization surpassed $5 trillion in 2025, driven by demand for AI data center hardware [4]. The release also arrives amid intensifying competition from open-weight models developed by Chinese firms. DeepSeek, founded in 2023, launched its R1 model in January 2025 with training costs it claimed were a fraction of those for comparable U.S. models [9]. DeepSeek's use of Mixture-of-Experts layers and training on export-restricted chips sent what observers described as shock waves through the industry, contributing to a single-day loss of $600 billion in Nvidia's market value [9]. Alibaba Cloud's Qwen family has also distributed models under open-source licenses such as Apache 2.0 [11].