Self-Improving Language Models with Bidirectional Evolutionary Search

A new search framework called Bidirectional Evolutionary Search (BES) aims to overcome limitations in how language models improve themselves, according to a paper submitted to arXiv on 27 May 2026. The method couples forward candidate evolution with backward goal decomposition to escape narrow exploration paths that constrain widely used techniques like best-of-N sampling and tree search [1][2]. The framework addresses two fundamental weaknesses in current search-based self-improvement methods. First, these methods rely on sparse verification signals that provide limited guidance. Second, they construct candidates primarily through autoregressive expansion, which restricts exploration to regions where the model already assigns substantial probability mass [2]. BES introduces evolution operators in its forward search that recombine partial trajectories, generating candidates that a single model rollout would be unlikely to produce. The backward search then recursively decomposes the original task into checkable subgoals, creating dense intermediate feedback that steers the forward process [2]. The researchers provide theoretical backing for their approach. They show that candidates from expansion-only search remain confined to a narrow entropy shell, while evolutionary operators can break out of it. They further demonstrate that backward search can exponentially reduce the number of samples required to find a correct answer [2]. Search has become a central technique for improving language model performance both during post-training sample generation and at inference time. The dominant architecture for large language models since 2017 has been the transformer, which relies on self-attention mechanisms rather than the recurrent connections found in earlier neural network designs such as LSTMs and GRUs [3][5]. These models have driven what some observers call an ongoing AI spring, following periods of reduced research activity in the 1970s and 1980s known as the AI winter [5]. In experiments, BES delivered consistent gains on challenging post-training tasks where mainstream post-training algorithms failed to produce improvements. On three open problem-solving benchmarks evaluated at inference time, the framework outperformed existing open-source systems in both average and best-case performance [2]. Code and trained models have been released on GitHub under the Embodied-Minds-Lab organization [2].