MiniOpt: Reasoning to Model and Solve General Optimization Problems with Limited Resources

Researchers have introduced MiniOpt, a reinforcement learning framework that trains compact language models to solve optimization problems through a “reasoning-to-model-and-solve” paradigm, achieving the highest average solving accuracy among models with fewer than 10 billion parameters [1]. The framework decomposes optimization reasoning into two stages: structured optimization modeling and executable solver generation [1]. A custom reward function called OptReward, which uses a hierarchical score structure, jointly evaluates the formulation and the solution without requiring expert demonstrations [1]. The authors also developed an optimization-oriented policy optimization strategy that improves exploration efficiency and stabilizes reinforcement learning for compact models [1]. Existing methods for applying large language models to optimization tasks typically depend on large-scale supervised datasets, costly reasoning annotations, and expensive intermediate step verification, which create substantial training overhead [2]. MiniOpt addresses these constraints by adopting a two-stage reinforcement learning pipeline. In the first stage, the model quickly learns the model-and-solve paradigm; in the second stage, it acquires strong optimization generalization ability [4]. OptReward verifies the completeness of problem modeling and avoids the need for content validation, further reducing the cost of verifying the model’s responses [9]. The MiniOpt series includes MiniOpt-3B and MiniOpt-7B variants [3]. Extensive experiments show that MiniOpt-3B exhibits strong optimization generalization across various optimization types, problem scenarios, and task domains [1]. For models with fewer than 10B parameters, the MiniOpt series achieves the highest average solving accuracy [1]. For models with more than 10B parameters, MiniOpt still shows competitive performance [1]. On the hard OptMATH-Bench, MiniOpt-3B achieved superior solving accuracy while consuming only 37.64% of the average output tokens required by DeepSeek-R1 [9]. The researchers note that MiniOpt requires less data volume and less detailed annotation during training, resulting in significantly lower computational costs for both training and inference [3]. Among the investigated model scales, MiniOpt-3B provided the best balance between optimization performance and computational efficiency [3]. The code is publicly available on GitHub [1]. The work was submitted to ICLR 2026 but was subsequently withdrawn [4].