$M^3 QuestionIng$: Multi-modal Multi-span Medical Question Answering

A research team has proposed a new framework, $M^3QAFrame$, designed to improve medical question-answering systems by generating answers that combine text with relevant images drawn from source documents [1]. The framework, detailed in a paper submitted to arXiv on 19 May 2026, addresses a gap in current medical question-answering (MedQA) systems, which typically extract answers from text alone [1]. Real-world medical documents, however, frequently contain both textual and visual content, and answers that incorporate images can improve comprehension [1]. $M^3QAFrame$ takes a context, a query, and associated images as input, then uses a transformer-based architecture to process text and image embeddings and determine relevance [1]. The output is an answer containing both textual spans and relevant images [1]. To support the framework, the authors curated a new dataset called $M^3 QuestionIng$, which includes queries, medical contexts, medical images, and extractive answers [1]. Each query-answer pair in the dataset is also labeled with user intent and query type to improve query and context comprehension [1]. The paper states that extensive experiments show the approach consistently outperforms existing methods across various evaluation metrics [1]. The work appears on arXiv, an open-access repository for electronic preprints that, as of November 2024, receives about 24,000 submissions per month and hosts over two million articles [7]. The paper’s abstract page includes a tab for arXivLabs, a framework launched in 2020 that allows community collaborators to develop and share experimental tools directly on the site [6]. arXivLabs projects, such as the Bibliographic Explorer and CORE Recommender, operate under guidelines that require partners to share arXiv’s values of openness, community, excellence, and user data privacy [6].