---
sd_hide_title: true
---

# MedVision

```{image} _static/medvision-logo.svg
:alt: MedVision
:width: 420px
:align: center
:class: only-light
```
```{image} _static/medvision-logo-dark.svg
:alt: MedVision
:width: 420px
:align: center
:class: only-dark
```

<div align="center">

**Dataset and benchmark for _quantitative_ medical image analysis**

[Paper](https://arxiv.org/abs/2511.18676) ·
[Project page](https://medvision-vlm.github.io) ·
[Dataset](https://huggingface.co/datasets/YongchengYAO/MedVision) ·
[Models](https://huggingface.co/collections/YongchengYAO/medvision-v0) ·
[Demo](https://huggingface.co/spaces/YongchengYAO/MedVision-V0-demo) ·
[PyPI](https://pypi.org/project/medvision-bm/) ·
[Code](https://github.com/YongchengYAO/MedVision)

</div>

---

MedVision measures whether vision–language models (VLMs) can read *quantities* off a
medical image — the size of a tumour in millimetres, the coordinates of a bounding
box, the angle between two anatomical landmarks — rather than only naming what they
see. It ships a large annotated dataset (22 source datasets, ~30.8M annotated
samples across CT, MRI, PET, ultrasound and X-ray), a reproducible benchmark harness,
and post-training recipes (SFT, RFT/RL, chain-of-thought, LoRA).

The `medvision_bm` package is the codebase behind the benchmark. This site documents
how to **run the benchmark**, **fine-tune models**, and **call the core API**.

## The three tasks

::::{grid} 1 1 3 3
:gutter: 3

:::{grid-item-card} 🎯 Detection
Predict a bounding box (`BoxCoordinate`) around a target structure. Scored with
IoU, precision, recall and F1.
:::

:::{grid-item-card} 📏 Tumour / Lesion size
Estimate the physical size of a tumour or lesion in millimetres
(`TumorLesionSize`). Scored with MAE and MRE.
:::

:::{grid-item-card} 📐 Angle / Distance
Measure an angle (degrees) or distance (mm) between landmarks
(`BiometricsFromLandmarks`). Scored with MAE and MRE.
:::
::::

## Quickstart

Install the package and run a small evaluation on one task, then aggregate the scores:

```bash
# 1. Install from PyPI
pip install medvision-bm

# 2. Point the loader at a data directory and pin a dataset version
export MedVision_DATA_DIR=/path/to/Data
export MedVision_PLANNER_VERSION=1.1.1

# 3. Evaluate an open-weight VLM on a handful of Angle/Distance samples
python -m medvision_bm.benchmark.eval__qwen2_5_vl \
    --model_hf_id Qwen/Qwen2.5-VL-7B-Instruct \
    --model_name Qwen2.5-VL-7B-Instruct \
    --data_dir /path/to/Data \
    --tasks_list_json_path tasks_list/tasks_MedVision-AD-CoT.json \
    --task_status_json_path completed_tasks/AD.json \
    --results_dir Results/MedVision-AD-CoT \
    --sample_limit 10

# 4. Parse per-sample outputs, then summarise per-anatomy metrics
python -m medvision_bm.benchmark.parse_outputs --task_type AD --task_dir Results/MedVision-AD-CoT -p 8
python -m medvision_bm.benchmark.summarize_AD_task --task_dir Results/MedVision-AD-CoT -p 8
```

See {doc}`getting-started/quickstart` for the annotated walkthrough, and
{doc}`getting-started/installation` for Docker and the full dependency setup.

## How the documentation is organised

- **Getting started** — install the package, then run your first end-to-end evaluation.
- **Dataset** — the data model (config naming, annotation types, physical units) and how to load it.
- **Benchmarking** — the three-step `eval → parse → summarize` pipeline and the metrics.
- **Fine-tuning** — supervised (SFT) and reinforcement (RFT) post-training recipes.
- **Reference** — the complete command-line reference and the autogenerated Python API.
- **Extending** — how to add a new model or a new task to the harness.

The benchmarking codebase is published on PyPI as
[`medvision-bm`](https://pypi.org/project/medvision-bm/) (`pip install medvision-bm`).

## Citation

If MedVision, the `medvision_bm` code, or the MedVision-V0 models are useful in your research, please cite the preprint:

```bibtex
@misc{yao2026medvisionbenchmarkingquantitativemedical,
      title={MedVision: Benchmarking Quantitative Medical Image Analysis},
      author={Yongcheng Yao and Yongshuo Zong and Raman Dutt and Yongxin Yang and Sotirios A Tsaftaris and Timothy Hospedales},
      year={2026},
      eprint={2511.18676},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2511.18676},
}
```

## Links

Everything the project publishes lives at one of these entry points:

- **Paper (arXiv)** — <https://arxiv.org/abs/2511.18676>
- **Project page & open leaderboard** — <https://medvision-vlm.github.io> (per-task score tables, an interactive case viewer, and a frontier API-model pilot study)
- **Dataset (Hugging Face)** — <https://huggingface.co/datasets/YongchengYAO/MedVision>
- **PyPI package** — <https://pypi.org/project/medvision-bm/>
- **MedVision-V0 model collection** — <https://huggingface.co/collections/YongchengYAO/medvision-v0>
- **Interactive demo (HF Space)** — <https://huggingface.co/spaces/YongchengYAO/MedVision-V0-demo>
- **Source code (GitHub)** — <https://github.com/YongchengYAO/MedVision>
- **Docker images** — <https://hub.docker.com/r/vincentycyao/medvision/tags>
- **verl fork for RFT** — <https://github.com/YongchengYAO/verl/tree/medvision-rl>

```{toctree}
:hidden:
:caption: Release notes
:maxdepth: 1

releases/index
```

```{toctree}
:hidden:
:caption: Getting started
:maxdepth: 2

getting-started/installation
getting-started/quickstart
```

```{toctree}
:hidden:
:caption: Dataset
:maxdepth: 2

dataset/concepts
dataset/loading
```

```{toctree}
:hidden:
:caption: Benchmarking
:maxdepth: 2

benchmarking/overview
benchmarking/running-evaluations
benchmarking/parsing-and-summarizing
```

```{toctree}
:hidden:
:caption: Fine-tuning
:maxdepth: 2

fine-tuning/sft
fine-tuning/rft
```

```{toctree}
:hidden:
:caption: Reference
:maxdepth: 2

reference/cli
reference/api/index
```

```{toctree}
:hidden:
:caption: Extending
:maxdepth: 2

extending/add-a-model
extending/add-a-task
```

```{toctree}
:hidden:
:caption: About
:maxdepth: 1

resources
```