To our knowledge, Hulu-Med is the first medical foundation model to natively process medical text, 2D images, 3D volumes, and surgical video in a single architecture. In our expanded study, it outperformed both GPT-5.2 and Gemini 3 Pro on 48 of 60 applicable benchmarks. DentVLM brings bilingual dental image interpretation into hospital workflows.
Hulu-Med
The version of Hulu-Med evaluated in our expanded study combines a sparse mixture of experts with unified spatiotemporal encoding for 2D images, 3D volumes, and video. We trained it in stages on 20.3 million curated samples and compared it with 68 models across 63 benchmarks.
Medical reasoning across modalities
- Medical text. Answer questions about biomedical literature, work through clinical cases in multiple languages, and respond across turns in a medical conversation.
- 2D images. Work with radiographs, pathology images, and fundus photographs. Evaluated tasks include visual question answering and chest X-ray report generation.
- 3D volumes. Process scans such as CT and MRI slice by slice. On CT benchmarks, Hulu-Med answers anatomical questions and generates radiology reports.
- Surgical video. Describe what happens across frames and answer questions about instruments, actions, and procedural phases.
In our expanded evaluation, Hulu-Med matched or surpassed specialist models on the tasks compared, including Merlin for 3D CT, MUSK for pathology, PanDerm for dermatology, and DeepRare for rare disease diagnosis.
We conducted five reader studies with board-certified physicians. On NEJM Image Challenge cases spanning more than 15 specialties, Hulu-Med achieved 85.1% diagnostic accuracy, ahead of both the NEJM submission baseline and our invited physician cohort. GPT-5.2 scored 65.2% and Gemini 3 Pro 68.5% on the same cases.
Open-source impact
According to our release records, Hulu-Med’s open-source releases have reached 20+ countries and regions and recorded approximately 400,000 cumulative downloads. In its first month, Hulu-Med reached No. 1 on Hugging Face’s medical model trending list, a category with roughly 4,000 models. The project has earned 1,000+ GitHub stars, and its paper has been cited by 100+ publications.
The Baichuan-M4 technical report cites Hulu-Med in its description of the system’s multimodal medical perception tools.
Code, model weights, and data
The released checkpoints can run locally. The Apache 2.0 repository provides inference notebooks, runnable examples for text and visual inputs, and MedUniEval evaluation scripts. It also tracks data preparation availability. The initial preprint documents data sources and access terms.
Supporting sources · Citation record · Baichuan-M4 report
DentVLM
DentVLM brings oral disease assessment, recognition of prior treatment, and orthodontic evaluation into one model. We trained it on 2.46 million bilingual question–answer pairs built from 110,447 dental images.
From dental images to answers and explanations
DentVLM answers clinical questions about dental images in Chinese and English. Its responses can explain the basis for a finding and specify where it appears in the dental arch.
DentVLM covers panoramic radiographs, lateral cephalograms, and five intraoral photo views. In the study’s head-to-head evaluation, DentVLM led the models tested on 34 of 36 bilingual tasks.
DentVLM has been deployed in hospitals across China to support dental image interpretation. In a retrospective study of 32 readers, DentVLM assistance reduced diagnostic time by 15–37% across reader groups.
Code, model weights, and data
The Apache 2.0 repository includes training, inference, and evaluation scripts plus example data formats. The model weights are available under CC BY-NC 4.0. The paper’s data availability statement explains how to request access to the clinical datasets.
Supporting sources · Zhejiang University project announcement