The Effect of Training Task Diversity on In-Context Learning through the Lens of Low-Dimensional Subspaces

Researchers have made new breakthroughs in understanding in-context learning and the persuasive capabilities of Large Language Models (LLMs). A recent study explains how training task diversity improves in-context learning with linear attention^[1].

A new analytical model presented in a paper submitted on June 5, 2026, to arXiv.org^[1] sheds light on the mechanisms behind in-context learning (ICL). The model demonstrates that training task diversity improves both the generalization and optimization trajectory of ICL with linear attention. The researchers modeled training task vectors as a mixture of low-rank Gaussians and showed that task diversity shortens the ICL plateau and enables out-of-distribution generalization. Meanwhile, another study on LLMs explored their persuasive potential through the lens of Jürgen Habermas' Theory of Communicative Action^[2]. The researchers found that LLMs can generate high-quality arguments and engage in nuanced communicative actions, conveying illocutionary intent such as conveying knowledge or building trust. Crowd-sourced workers even found LLM-generated counter-arguments more agreeable than human-written ones^[2].