Breaking the Filter Bubble: A Semantic Pareto-DQN Framework for Multi-Objective Recommendation

A new framework proposes using multi-objective reinforcement learning to break the filter bubble in recommender systems, treating user engagement, information diversity, and provider fairness as separate goals rather than a single target [1][2]. The work, submitted to arXiv on 23 Jun 2026, introduces a semantic multi-objective Markov decision process that deploys a Pareto-DQN agent [1][2]. Standard single-objective models, including traditional Deep Q-Networks, are ill-equipped to navigate the trade-offs between platform retention and societal values like information diversity and provider fairness, the authors state [2]. The architecture integrates high-fidelity semantic embeddings and treats engagement, diversity, and fairness as distinct, non-aggregable reward signals, avoiding the pitfalls of static reward scalarization [2]. Empirical evaluations on the MovieLens small dataset show that hypervolume-based action selection disrupts the feedback loops responsible for semantic collapse [2]. By sustaining high state-trajectory variance, the Pareto-DQN maps the Pareto frontier, achieving gains in auxiliary societal objectives with only marginal impacts on engagement [2]. The paper provides a path toward intrinsically aligned, responsible recommender systems [2]. The preprint appears on arXiv, an open-access repository of electronic preprints and postprints that is not peer reviewed [6]. Begun on August 14, 1991, arXiv passed the half-million-article milestone in 2008 and reached two million articles by the end of 2021; as of November 2024, the submission rate was about 24,000 articles per month [6]. The paper’s abstract page includes experimental tools developed under arXivLabs, a framework that allows community collaborators to build and share new features directly on the site [4][5]. arXivLabs was launched as a formalized framework to enable innovative collaborations with individuals and organizations [4]. “Members of our community want to contribute tools that enhance the arXiv experience, and we value that kind of community engagement,” said Eleonora Presani, arXiv Executive Director, at the time of the launch [4]. The Labs framework sets guidelines for collaborations, ensuring that partners share arXiv’s values of openness, community, excellence, and user data privacy [4]. Third-party collaborators have access only to minimal and anonymized data, and any other use not included in a written consent from arXiv is strictly prohibited [4]. Current Labs integrations visible on the paper’s page include the Bibliographic Explorer, which displays citation relationships, and the CORE Recommender, which surfaces relevant open-access papers from a global network of repositories [5]. arXiv is temporarily pausing new Labs proposals while its development team focuses on modernizing and moving arXiv’s systems to the cloud [3].