Plan Then Action:High-Level Planning Guidance Reinforcement Learning for LLM Reasoning

A new two-stage framework, PTA-GRPO, has been proposed to improve both high-level planning and fine-grained Chain-of-Thought reasoning in large language models, according to a paper submitted in 2025 and revised in 2026 [1]. Large language models exhibit strong reasoning when using Chain-of-Thought prompting, but their token-by-token generation favors local decisions and lacks global planning, often producing redundant or inaccurate reasoning [1]. Existing remedies, including tree-based search and reinforcement learning, carry high computational costs and still struggle to yield reliable reasoning trajectories [1]. To address these gaps, researchers introduced Plan-Then-Action Enhanced Reasoning with Group Relative Policy Optimization, or PTA-GRPO [1]. The framework operates in two stages: first, a model summarizes Chain-of-Thought reasoning into compact high-level guidance, which is then used for supervised fine-tuning [1]. Second, a guidance-aware reinforcement learning method jointly optimizes the final output and the quality of that guidance [1]. The approach was evaluated on ten reasoning benchmarks spanning mathematics and natural sciences, using five diverse base models that cover multiple data modalities [1]. The results showed consistent improvements across models and tasks, which the authors describe as demonstrating strong effectiveness and generalization [1]. The work arrives amid a broader push to make AI systems more agentic. AI agents are a class of intelligent systems that can pursue goals, use tools, and act with varying autonomy, typically within human-defined objectives and constraints [3]. Artificial intelligence research has long pursued capabilities such as learning, reasoning, knowledge representation, and planning [4]. Since the 2020s, generative AI has become widely available, and an AI boom has coincided with advances in systems that can create and modify media [4]. In India, the AI market is projected to reach $8 billion by 2025, growing at a 40% compound annual growth rate from 2020 to 2025, driven by startups and government policies [5]. India accounted for the largest share of ChatGPT’s mobile app users and had the third-largest user base for DeepSeek in 2025 [5]. The PTA-GRPO framework was submitted to arXiv on 2 October 2025 by Zhihao Dou and revised on 25 May 2026 [1].