PACUTE: Phonology-, Affix-, and Character-level Understanding of Tokens for Filipino

A new diagnostic benchmark called PACUTE reveals that large language models struggle with the complex morphology of Filipino, a language rich in infixation and reduplication, with open-weight models performing near chance on core decomposition tasks. The benchmark, introduced in a paper by Jann Railey Montalan and colleagues, comprises 4,600 tasks designed to probe how well models understand the internal structure of Filipino words [1]. Filipino morphology is characterized by productive infixation, where affixes are inserted inside root words, and reduplication, alongside lexical distinctions often marked only by diacritics that are frequently omitted in written text [2]. Standard subword tokenizers systematically misalign token boundaries with morpheme boundaries in such languages, obscuring the character-level structure that underlies word formation [2]. PACUTE organizes its evaluation into a hierarchical diagnostic framework of six compositional levels, localizing precisely where a model's morphological understanding breaks down [2]. The first submission of the paper was a 240 KB file [1].