KonfAI MCP

konfai-mcp turns KonfAI into a backend an MCP client can drive. Give it a dataset and a goal in plain language and it inspects the data, picks the cheapest path that fits (reuse a published model, fine-tune one, or train from scratch), writes and validates the config, runs training, prediction and evaluation, monitors the jobs, and returns metrics with a record you can reproduce. Same framework, same YAML, exposed as structured, deterministic tools instead of hand-edited files; the core never depends on it.

pip install konfai-mcp        # beside a KonfAI able to read your data: pip install "konfai[imaging]"
konfai-mcp                    # the stdio server an MCP client launches

Fifty tools, one per object

A variant of an operation is an argument of its tool (session(action=...), check_config(level=...), get_job_status(wait=True)), never a sibling tool. Each tool is described in one short paragraph whose first sentence is what a client's tool search matches; the server's instructions say what KonfAI does and when to search its tools; six entry tools stay loaded under Claude Code's tool search. The whole list weighs about 15k tokens for a client that loads it upfront. Every call appends one line (tool, session, outcome, duration; no argument) to tool_calls.jsonl in the workspaces root: the usage data the next consolidation rests on.

Principles: structured tool contracts, explicit experiment workspaces, reproducible job manifests and config snapshots, staged validation before expensive runs, a clear separation between the KonfAI runtime and the MCP layer. Hardware is a resource, not a tool: server://capabilities reports per-GPU free VRAM and a recommended device, and every launch payload carries a VRAM pre-flight for the GPUs it will use.

The table below is the registry's own text, the first sentence of each tool; the full contracts live in the guide://tool-index resource and in the generated reference. Guide: Automate with MCP. Source: konfai-mcp/.

Discovery

What KonfAI can do and what a component is.

`describe_konfai_capabilities`	What KonfAI can do and which tool to reach for: the four workflows, the component kinds, the extension model, which actions are safe and which destroy data.
`list_components`	Built-in KonfAI components of one kind (model, loss, metric, transform, augmentation, scheduler, ...): name, where it goes in the YAML, the classpath to inspect.
`inspect_object_signature`	Signature, defaults, doc and contract hints of any configurable classpath: built-in, local File:Class, external package.module:Class, or a declarative .yml model (its hyperparameters, loss-attachable leaves and how to adapt it, with the file's content: no read_file needed).
`describe_config_schema`	Top-level schema of a workflow config (train, prediction, evaluation, transform), generated from KonfAI's own classes: each field's literal yaml_key, type, default, whether required, and the classpath to drill into with inspect_object_signature.
`describe_extension_points`	How to add a component of a kind (loss, metric, model, transform, augmentation, scheduler, pretrained model): the base class, the required methods, the YAML reference syntaxes (builtin, File:Class, package.module:Class) and the gotcha.
`check_component`	Pre-flight a component before wiring it into a config.

Dataset onboarding

What a dataset holds, before a config names it.

`inspect_dataset`	Groups, entry hints, ambiguities and sampled statistics (shapes channel-first) of one dataset root (include_stats=False for a fast structure scan, groups=[one] for focused statistics).
`read_file`	Read a bounded text range of one file.
`preview_volume`	One slice of a volume as a PNG (any SimpleITK-readable file), windowed between the 1st and 99th percentile and downsampled, for a visual check of a case or a prediction; the plane's min, max, mean and unique values come with it (a black image is a constant plane).
`prepare_dataset_aliases`	Copy, symlink or move dataset files under new group names when the content is right but the filenames do not match the intended config.

Config authoring

A session and its YAML.

`design_config_strategy`	A config-writing plan from the task, the dataset root(s), the group roles and the modeling intent: dataset summary, config plan, customisation options, unresolved questions, compatible example templates.
`initialize_session`	Create or reset the current session workspace, optionally seeded from an example template (read_file('templates://list') lists them; include_support_files=True copies its local .py).
`write_workflow_config`	Write one workflow YAML (train, prediction, evaluation, transform) into the session and check it against KonfAI's schema: schema_errors lists every mistake with the nearest valid key; the file is written either way.
`write_session_file`	Write one support file into the current session (Model.py, Loss.py, a transform, a manifest).

Validation and iteration

Checks before GPU time, comparisons after.

`check_config`	Check the session's workflow config.
`summarize_session`	One compact snapshot of the current session: readiness, jobs, metric summaries, and on request the leaderboard, a log tail and a config validation.
`leaderboard`	Ranked metrics across the session's completed runs, one metric and split, direction-aware, with the available metrics and splits.
`get_run_metrics`	Full evaluation metrics (per-case values and aggregates) of one run or app trial, from Evaluations/<run>/Metric_<split>.json.
`compare_runs`	Two runs metric by metric on aligned cases: means, per-case deltas, a direction-aware winner (loss-like metrics count lower as better).
`diff_run_configs`	Unified diff of the exact configs two jobs ran with, from their immutable launch-time snapshots.
`describe_model_outputs`	The addressable module paths of the session config's networks: the exact keys outputs_criterions and outputs_dataset accept (terminal=true marks output heads).
`read_training_curves`	A run's training curves (scalars over iterations) from its TensorBoard event files, downsampled to max_points.

Execution and monitoring

Jobs: launch, follow, steer, stop.

`run_train`	Launch a training job from the session Config.yml (after check_config).
`run_resume`	Resume a training from a checkpoint (model, optimizer, scheduler and counters restored), or warm-start a fine-tune with weights_only=True (weights only; prefer fine_tune_app for an app used as published).
`run_prediction`	Launch a prediction job from the session Prediction.yml with a checkpoint of the session (models picks which).
`run_evaluation`	Launch an evaluation job from the session Evaluation.yml on existing predictions.
`run_transform`	Batch-process a dataset with the session Transform.yml chain: read, transform, write.
`run_batch`	Run several training configs sequentially server-side (a sweep, fold training), blocking until the batch ends, with per-run outcomes.
`generate_folds`	Split a dataset into K cross-validation folds: one case-list file per fold in the session, plus the subset stanzas to paste into each config ('folds/fold_0.txt' keeps those cases, '~file' the others).
`get_job_status`	The state of one job (job_id) or of every job of the session (no job_id).
`cancel_job`	Stop a running job: requests cancellation and waits briefly for a clean shutdown.
`control_training`	Steer a RUNNING training without restarting it: lr and it_validation are applied at its next poll boundary and recorded in the run's config snapshot; validate_now=True runs a validation pass at the next iteration.
`read_live_metrics`	Parsed runtime metrics of a running or finished training (latest snapshot, recent entries, per-stage summaries) instead of raw log lines.
`read_job_log`	A job's console or runtime log: the last max_lines, or the lines matching a regex, to find the traceback.

Apps

Published models, used as they are, fine-tuned, or produced.

`list_apps`	Published KonfAI apps from the catalogue (shipped, workspace file, KONFAI_MCP_APP_CATALOG), HuggingFace repos expanded.
`describe_app`	One app's manifest: description, input and output modalities, inference, evaluation and uncertainty capabilities, checkpoints, terminology.
`run_app`	Run a published app as published on the user's data: action infer, evaluate (the app's own Evaluation.yml against gt), uncertainty, or pipeline (all three).
`fine_tune_app`	Fine-tune a published app's checkpoint on the user's dataset without authoring a config (epochs, it_validation, lr, batch_size as parameters; set_parameters for the model's tunables), as a tracked job that writes a runnable app bundle to output.
`fetch_app`	Bring a published app into the session.
`package_app_from_session`	Package a model trained in this session into a runnable app bundle (app.json, config, checkpoint, optional Model.py, requirements, support_files, ONNX export) that describe_app and run_app consume.
`app_sources`	Edit the workspace app catalogue that list_apps merges: action 'add' persists an app path or HuggingFace repo across sessions, 'remove' drops it.
`run_registration_evaluate`	Score a registration against what the pair has: labels (Dice), landmarks (TRE in mm, fixed points pushed through the transform), same-modality images (MAE), and a field's folded-voxel fraction.

Visual inspection

KonfAI Scout as a tracked job.

`run_scout_inspection`	Have KonfAI Scout inspect one result visually (segmentation, registration or synthesis): a vision model looks at slices, zooms and overlays, measures regions with KonfAI Eval and reports localised defects with evidence, as a tracked job under Scout/<name>.
`read_scout_report`	A finished Scout inspection by name: verdict, observations (kind, defect, severity, confidence, box in mm, cited views and measurements), measurements, views, usage, budget; no image bytes.

FidOpt

The benchmark through its own code.

`fidopt_catalog`	FidOpt's benchmark catalogue through its own code: one line per dataset (task, anatomy, modalities, cases; task= and anatomy= narrow it) and the methods of the task; with dataset=<name>, that dataset's descriptor (cases, truth, labels, licence) and the methods of its task.
`fidopt_table`	A FidOpt dataset's board (fidopt table --json): one line per method and machine, published scores and local rows.
`fidopt_run`	Launch one FidOpt operation as a tracked job: check, run or tune a method on a dataset's cases (only the operations KONFAI_MCP_FIDOPT_ALLOWED permits); scout='PROVIDER[:MODEL]' adds Scout's inspection of the scored outputs.

Session lifecycle

Workspaces, records, deletions.

`session`	Manage session workspaces: action 'list' (says whether a host pinned the session); 'create' (name; switches to it; from_example seeds it in the same call, else the workspace is empty until initialize_session); 'switch' (name); 'delete' (the current one; force cancels its jobs); 'import' (adopt an on-disk KonfAI experiment from source_dir: its configs and code, its artifacts linked, copied or left out).
`export_run_record`	The reproducibility record of one run in one payload: the job manifest (command, devices, environment with package versions), the launch-time config snapshots, the resolved config, every split's metrics and a log tail.
`delete_run`	Delete one run's outputs of a kind (train: its Statistics and Checkpoints; prediction, evaluation, uncertainty: its folder; transform: its plan and log only), jailed to the session.