# Release v1.1.1

Release notes for the `medvision_bm` codebase, **v1.1.1**. This is the version this
documentation set describes; use the version switcher (bottom-right on Read the Docs)
to read the docs for a different release.

---
## New Model Evaluations

- Claude (Anthropic API / OpenRouter): adaptive thinking + a 28px-grid pre-resize so the prompt's stated image/pixel size matches what the model perceives (ead5ae0); finalized eval scripts and pinned `requirements_eval_claude.txt` (91e14d9)
- Gemini (Google Gemini API / OpenRouter): pass-through resize with a 3072px client-side guard and per-model image caps (47210fb)
- OpenAI/GPT (OpenAI API / OpenRouter): patch-budget resize rule and per-model image caps (a0c5d84); added non-Pro GPT-5.5 scripts and capped `--reasoning_effort`/`--max_tokens` on all 6 GPT scripts to control runaway cost (cace62c)
- Kimi K2.6: OpenAI-compatible API model with MoonViT floor-28 fixed-point resize and a per-model patch-budget table (e3cf881)
- GLM-4.6V and GLM-4.6V-Flash: vLLM wrapper covering both the MoE and dense checkpoints, with reasoning-model sampling and a perceived-size resize probe (238f84b); fix GLM-4.6V environment setup (a76f382)
- MiniMax-M3: vLLM wrapper with a perceived-size probe (a070f93); MiniMax-M3-INT4 (AWQ) on 4xH100 with CUDA-13 forward-compat fixes (1e67af7); experimental MiniMax-M3 (MXFP8) for >=8x80GB GPUs with a fail-fast aggregate-VRAM guard (91a0cc8)
- Gemma 4 and Qwen3-VL(-Thinking): vLLM wrappers with thinking-mode support and reasoning-content merge (98b0aa9)
- Unify HealthGPT-L14/XL32 into a single `healthgpt` model key, selectable via a new `--model_choice` CLI arg (cf52d7d, 7efb72a)
- MedVision-V0-7B: benchmark eval scripts and environment (ae972d8); full HF model card covering the two-stage SFT+RFT recipe, `<think>`/`<answer>` output contract, and licensing, later revised for clarity and CC-BY 4.0 (575254e, 1002189, 84e8aa7)

---
## New Metrics

- F1/Precision/Recall for Detection and per-sample nMAE for TL/AD added to the parsed JSONL output (1f4cc6c)
- Step 1/2 raw L2 distance replaced by normalized L2 distance (÷√2) in the TL/AD process-accuracy scripts, making localization error comparable to the step 3/4 nMAE (986d4ea)
- nMAE for `-scaledPS` files reconstructed via a BLAKE2B hash when `pixel_size_scale` wasn't stored in the raw JSONL (677ccc4)

---
## New Analyses

- Detection box-size analysis pipeline: per-bin Recall/Precision/F1 vs. a simulated random-detection baseline, plus a per-label × box-size visualization (4575d66, a9e2326, 9242e1b, 80508e0)
- Removed-samples filtering added to the TL equation-/process-accuracy scripts to exclude samples dropped between dataset v1.0.0→v1.1.0 (a4a835f); later dropped from the response-visualization scripts as obsolete once dataset v1.1.1 changed which samples were removed (0719acb)

---
## Visualization

- Per-sample overlay and response-visualization scripts for TL/AD/Detection tasks (c6bfea4, fcac4cf), made robust to missing or unparseable predictions (eb97ad0)
- Radar-chart and cross-model comparison-grid compilation scripts with batch runners (4739c79, c4ca65f)
- `plot_utils` extended with AD landmark plotting and a GT axis overlay for TL (130583f)
- `export_webpage_cases` for the project webpage's interactive case viewer, with dual-origin off-the-shelf rendering and a demo gallery seeded from MedVision-V0-7B (9a76fd4, 563e9c7, c205333, 6a9fbdf)
- Image-vs-real-space ellipse-fit comparison figures illustrating the v1.1.1 TL bug (0a9ca53)
- All figures capped at arXiv's 34-megapixel per-image limit; transparent backgrounds and `--save_as_png`/`--save_as_pdf` output flags added (c0afb27, d4bc9d3)

---
## Key Bugfixes — Image Perception & Generation Truncation

- Critical: `get_resized_img_shape()` returned a single shape used for both the stated image size and the pixel size; split into a (perceived-canvas, content) pair so padding models — LLaVA-OneVision, InternVL3, Llama-3.2-Vision, LLaVA-Med, HuatuoGPT-Vision, HealthGPT-L14 — report the correct TL/AD image size and pixel scale; off-the-shelf runs on non-square inputs should be re-evaluated (0a4c5e2)
- Critical regression: lmms-eval's fewshot-delimiter `"\n\n"` was being forwarded as a vLLM generation stop for every model, truncating CoT answers before `<answer>` (0% parseable answers observed for InternVL3-38B/Llama-3.2-Vision); removed the default `until`-forwarding and gated string stops on `--stop_strings` only, generalizing the earlier Gemma 4-specific fix (2eb7706, 18aa18a)
- `--stop_strings` wired end-to-end through HuatuoGPT-Vision and LLaVA-Med to stop truncated outputs (de6ae15, a766b35)
- `min_new_tokens=16` floor prevents empty LLaVA-Med responses from greedy premature-EOS (556f0b1)
- Fix scaledPS process-accuracy scoring against the scaled ground truth instead of the unscaled one; restore the missing MedVision-V0 scaledPS eval scripts (f6fc5e6)
- Balance parentheses in the angle-formula CoT prompt (ae29149)

---
## Key Bugfixes — Metrics & Summarization Pipeline

- Exit nonzero when `lmms_eval` swallows an evaluation error, so vLLM-driver eval scripts correctly mark crashed tasks as incomplete instead of "done" (813b350; paired with raising eval subprocess verbosity to `INFO` across the 9 remaining vLLM drivers in ee7d5f9)
- Detection-metric parse failures now count as 0 (not NaN) in IoU/F1/Precision/Recall, matching the authoritative summarizer; fixed a crash in the no-CoT verl detection formatter (6e90af4)
- Exclude `_proc_acc`/`_eq_acc`/`_filtered` sidecar JSONLs and cross-analysis outputs from the summarize/collection steps, preventing crashes and double-counted results (8644f91, 313ff45, 641517b, ba111a2)

---
## Robustness Hardening (Audit-Driven)

- Replace `exec()` of model-generated code in tool-execution with an AST-restricted evaluator that rejects dunder/attribute/subscript/import escapes; redact secrets from evaluation-tracker logs; deterministic per-task dataset ordering to fix non-deterministic RFT crash-resume; report angle pixel size in millimeters, not degrees (742c484)
- Replace `eval()` with `ast.literal_eval` for JSONL targets, add a robust numeric parser, and seed the weighted sampler and `SFTConfig` for reproducibility (28f6c67)

---
## Dataset v1.1.1 Integration

- Release the **MedVision** dataset v1.1.1, fixing a TL anisotropic ellipse-fit bug (transposed in-plane voxel spacing); ~22% fewer TL samples on anisotropic data (fe27464)
- Set `MedVision_ACK_RELEASE=1.1.1` across benchmark/SFT scripts still pinned to older `MedVision_PLANNER_VERSION`s, acknowledging the new release-acknowledgement gate (5f35017); set `MedVision_PLANNER_VERSION` explicitly in SFT training scripts, required since medvision_ds v1.1.0 (01557f3)
- Add versioned v1.1.1 task lists (A/D, T/L, Detection) and a per-task pixel-size-distribution summarizer over MedVision configs (3673558, 77a00ea, a9947a3)
- Share `lmms_eval_specific_kwargs` via a YAML `include` across landmark base configs; drop stale TL-OOD task lists (7a18805)

---
## OOD Ablation

- Add MedVision-V0-7B eval scripts for plane-OOD and task-OOD ablation on TL and Detection tasks, evaluating generalization to unseen slicing planes and unseen anatomical targets; renamed the corresponding task-list JSONs to the `-CoT-<split>` naming convention used elsewhere (2673ce5)

---
## SFT / RFT

- Fix: SFT loss masked to train only on the assistant response (not the full prompt+response sequence) in the Qwen2.5-VL collate (9945b05)
- Feat (experimental): full-parameter and LoRA SFT scripts for MedGemma-27B, Gemma-4-31B, and Qwen3-VL-27B (0e2c0eb)
- Feat: parquet dataset building scripts for RFT in verl (7f638e3)

---
## Infrastructure / Install

- Persist per-sample model outputs to a resumable, prompt-hash-keyed JSONL cache so an interrupted eval run only re-does the in-flight sample (783dddb)
- Build the `medvision_bm` wheel in a private temp dir instead of the repo tree, fixing intermittent build failures on shared CephFS checkouts (473a4fd)
- Write task-status JSON atomically (temp file + fsync + `os.replace`) to survive full-disk writes (3c01bea)
- Standardize install method and pin `MedVision_PLANNER_VERSION='1.0.0'` across all 38 benchmark eval scripts; rename eval script filenames to consistent Title Case and drop dead gemini keys (702c392, 7ba7f9a)
- Relax scipy/nibabel/matplotlib version pins to fix environment setup conflicts (69f7afe)
- Auto-format all Python and shell sources with black/isort/shfmt (cosmetic only) (bfcf773)

---
## Documentation

- Add a Read the Docs documentation site (Sphinx + MyST) with autodoc/autosummary API reference over all importable modules (f1102c5)
- Change codebase license to CC-BY 4.0 (3633fea, 1002189)
- Document each model's perceived-resolution strategy in `docs/Model-Image-Processing.md`; restructure the doc and drop the superseded perceived-size bugfix page (e930941, 8573eca)
- Refresh README News, Quick Start, benchmarked-models list, and reference links throughout the release cycle (a168350, cb40b43, 7564b9a, 0b7106d, 58a63e3, 4a184f0, 9fe1f96, 6ffe47f, 0719acb)