Label Shift Aware Adaptation for Online Zero-shot Learning with Contrastive Language-Image Pre-Training (CLIP)

A new method called Label Shift Aware (LSA) aims to improve online zero-shot learning for vision-language models by correcting mismatches between training and test data distributions, according to a paper submitted to arXiv in June 2026 [1]. The approach targets a specific weakness in models like Contrastive Language-Image Pre-Training (CLIP). When deployed for online zero-shot classification, where unlabeled test samples arrive sequentially, most existing methods adapt representations on the fly but ignore the original training distribution [1]. This can degrade performance when the label distribution in the test data differs from the source domain [2]. LSA reformulates the task as a domain adaptation problem, adapting CLIP's predictions to the target distribution using only unlabeled test data and applying a label shift correction [2]. The paper reports that LSA consistently outperforms current state-of-the-art online zero-shot learning methods based on CLIP across multiple datasets [2]. The work was posted on arXiv, an open-access repository for electronic preprints in fields including computer science that has been operating since August 1991 [8]. The repository passed the two-million-article milestone by the end of 2021 and now receives roughly 24,000 submissions per month [8]. The paper's abstract page also links to community-developed tools through arXivLabs, a framework launched in 2020 that allows third-party collaborators to build experimental features on the site, such as citation explorers and code finders [6][7]. arXiv states that these collaborators have access only to minimal, anonymized user data and are prohibited from using it for any purpose beyond ensuring feature functionality [6]. The LSA method does not modify CLIP's feature extraction or model parameters during inference, keeping the process fixed while only adjusting the output predictions [2]. This distinguishes it from representation-adaptation approaches that update internal states as new samples arrive. By focusing on the label distribution mismatch, the researchers address a gap they argue has been overlooked in prior online zero-shot learning work [2].