Is My Vision-Language Data in Your AI? Membership Inference Test (MINT) Demo 2

Researchers have introduced a new framework, the Membership Inference Test (MINT) Demo 2, designed to determine whether specific data was used to train a machine learning model, achieving up to 90% accuracy in detecting training data across image and text modalities [1][2]. The MINT technique provides an experimental method for auditing the training processes of AI models, a capability increasingly relevant as large language models (LLMs) and other systems are deployed widely [1]. The framework proposes multiple architectures that can be applied depending on the level of information available about the model being audited [2]. In tests, the system was evaluated using a popular face recognition model, four state-of-the-art LLMs, and multiple large-scale public image and text databases [2]. The research team reports accuracy levels in the detection of training data of up to 90% [1][2]. To make the tool accessible, the authors built a comprehensive web platform that expands MINT’s capabilities to both image and text modalities [1]. The platform integrates a technological stack that includes MINT, aMINT, and gMINT, allowing users to audit a wide range of models [2]. The demonstrator is hosted on arXivLabs, a framework that enables collaborators to develop and share new features directly on the arXiv website [1]. arXivLabs operates under values of openness, community, excellence, and user data privacy [1]. The release arrives as the machine learning community increasingly uses interactive demos to improve research transparency. Since October 2021, Hugging Face Spaces has been used to build and share over 12,000 open-source machine learning demos crafted by the community [3]. A collaboration between Hugging Face and arXiv now embeds these demos directly alongside papers, allowing users to find and try them immediately from a paper’s abstract page [3][4]. These demos are built using open-source tools such as the Gradio and Streamlit Python libraries and leverage models and datasets available on the Hugging Face Hub [3][4]. The push for auditing tools comes amid heightened scrutiny of training data practices. LLMs are language models with many parameters, trained with self-supervised learning on vast amounts of text [7]. Companies such as DeepSeek, a Chinese AI firm, have drawn attention for their training methodologies, reportedly training their V3 model for US$6 million, far less than the US$100 million cost for OpenAI’s GPT-4 in 2023 [6]. DeepSeek’s models are described as “open-weight,” meaning the exact parameters are openly shared, but the training data is not openly licensed [6]. The MINT Demo 2 aims to provide a practical tool to foster compliance with emerging AI regulations by offering a direct method to test for the presence of specific data in a model’s training set [2].