The Holistic Storage of Verb+Up Phrases in Text-based and Audio-based Language Models

A new study from researchers on arXiv finds that both text-based large language models and an audio-based automatic speech recognition model store certain verb-plus-particle phrases as single units, a pattern driven by how often and how predictably those phrases appear in language [1]. The work, posted to the preprint server arXiv, examines a core tension in language: the need to retrieve memorized chunks while also applying abstract rules to create novel expressions [1]. The authors note that while abstract knowledge in language models has been studied, the holistic storage of multi-word units has received far less attention [1]. To address this, they probed the internal representations of text-based LLMs and an ASR model, specifically testing whether phrasal verbs formed with "up" — such as "pick up" or "give up" — develop distinct, unified representations [1]. The results showed that all models exhibited evidence of holistic storage, and that this tendency was shaped by the frequency and predictability of the phrase, supporting usage-based theories of language [1][2]. The linguistic category under investigation, the phrasal verb, combines a lexical verb with an adverbial particle to form a unit whose meaning is often not predictable from its parts [3]. A key syntactic feature of English phrasal verbs is that the particle can sometimes be separated from the verb and placed after the object, as in "turn the light off," a flexibility not available to prepositional verbs such as "look at" [3]. Prior neurolinguistic research using magnetoencephalography has suggested that the human brain processes phrasal verbs as single lexical units [3]. The new computational findings extend this line of inquiry into artificial neural networks. This investigation into holistic storage sits within a broader effort to understand what linguistic properties are encoded in model representations. Researchers have previously shown that language models can differentiate between light-verb constructions, such as "make a decision," and full-verb uses, such as "make a cake," even in minimal sentence contexts [4]. Other work has explored joint modeling of speech and text, developing language models trained on both discrete speech units and written words to improve cross-modal understanding [5]. The mental lexicon itself, the component of the human language faculty that stores information about word composition, meanings, and pronunciations, is understood not as a static list but as a dynamic network where entries are interconnected and constantly developing [7]. The new arXiv study contributes to this picture by providing computational evidence that frequency and predictability drive the chunking of multi-word expressions, mirroring the trade-off between storage and computation long observed in human cognition [1][2].