KonfAI Scout

An agent that looks at your segmentation, registration or synthesis the way you would, and tells you where it is wrong. A score says how good a result is; Scout says where and why. It inspects one case at a viewer, slice by slice, zooms on what looks off, switches overlays to tell the image from the result, asks KonfAI Eval to measure the suspect region, and writes a report of localised defects with their type, severity, uncertainty, the views that show them and the measurements that support them.

pip install konfai-scout[anthropic]        # Claude through the Anthropic API
pip install konfai-scout[openai]           # GPT, or any OpenAI-compatible server (vLLM, Ollama, LM Studio)
pip install konfai-scout[claude-code]      # Claude under your Claude Code login, no API key
pip install konfai-scout[codex]            # GPT under your ChatGPT login (the Codex CLI), no API key

How it works

It explores. Coverage first, a montage per plane, then targeted views: slices, zooms, checkerboards, difference maps, displacement fields, Jacobians, deformed grids, landmarks. Every view carries its position in millimetres.
It measures with KonfAI Eval, not with a scorer of its own. A region is cropped and scored by konfai.evaluate with KonfAI's criteria: Dice per label, MAE, PSNR, SSIM, TRE where a reference exists; a field's folding, Jacobian and displacement statistics without one. Reading, warping and geometry are KonfAI's too.
It keeps its statements apart, and enforces it. A measured property, a suspected error (seen, not proven) and a confirmed error (a cited measurement) are distinct, and without a truth nothing is confirmed from appearance. The note tool refuses an error that no cited view shows, a figure that no cited measurement holds, a confirmation resting on measurements taken elsewhere. "Insufficient evidence" is a legitimate conclusion. On request, a fresh session of the model re-reads every error blind, from crops of its box and the claim alone, and its answer stands in the report next to the first reader's.
It leaves a record. A versioned JSON report with versions, configuration, prompts, every tool call, every view, every measurement, cost and duration, and a Markdown rendition with the evidence images. Two modes, fixed views and active exploration.
It works with Claude, GPT or a local vision model through an OpenAI-compatible server, and sends images to a cloud API only when told to.

konfai-scout inspect --task segmentation --name case01 \
    --input image=ct.nii.gz --input prediction=seg.nii.gz --input truth=gt.nii.gz \
    --labels 1=liver,2=spleen --evaluation Evaluations/run/Metric_TRAIN.json \
    --out reports/case01 --provider anthropic --allow-cloud-images

Scout is a package of the KonfAI family (konfai-scout/ in the repository), pinned to the konfai release it ships with: every volume is read through KonfAI's dataset backends (image files, OME-Zarr stores, DICOM series, ITK transform files), a registration is warped by KonfAI's own Resample, and every measurement is KonfAI Eval's. The MCP server runs it as a tracked job (run_scout_inspection, read_scout_report), and Studio shows its reports in a panel.