Evidence-Informed LLM Beliefs for Continual Scientific Discovery

Researchers have proposed two AI systems to aid scientific discovery, addressing limitations in current large language models (LLMs). The first modifies LLMs to use non-stationary surprisal, while the second, Co-Scientist, generates and refines hypotheses for experimental verification.

A research paper on arxiv.org^[1] argues that current LLM-based scientific discovery methods, such as AutoDiscovery, treat surprisal as a static quantity. The authors propose updating LLM beliefs with evidence from previous hypotheses to compute non-stationary surprisal. They found that embedding-based retrieval-augmented generation over prior discoveries best anticipates eventual posteriors, increasing accumulated non-stationary surprisal by 30.62% on average compared to the original search procedure^[1]. Meanwhile, another paper on arxiv.org^[2] introduces Co-Scientist, a multi-agent AI system designed to help scientists discover new knowledge. Co-Scientist generates novel research hypotheses for experimental verification using Gemini for structured scientific thinking and hypothesis generation. The system's design involves agents continuously generating, critiquing, and refining hypotheses, accelerated by scaling test-time compute^[2]. Co-Scientist was validated on three biomedical applications: drug repurposing, novel target discovery, and explaining mechanisms of anti-microbial resistance. It helped identify new drug repurposing candidates and synergistic combination therapies for acute myeloid leukemia^[2].