Run Kapso on MLE-Bench: Kaggle competitions, ranked first

MLE-Bench is a benchmark for evaluating ML engineering agents on Kaggle competitions. Kapso achieved #1 among open-source systems on this benchmark.

MLE-Bench medal rate by difficulty split, comparing Kapso with R&D-Agent, AIRA-dojo, ML-Master and AIDE across four panels — Medal rate by difficulty split, against R&D-Agent, AIRA-dojo, ML-Master and AIDE

These results were submitted as an official submission to MLE-Bench.

Usage

# List available competitions
PYTHONPATH=. python -m benchmarks.mle.runner --list

# List lite benchmark competitions
PYTHONPATH=. python -m benchmarks.mle.runner --lite

# Solve a competition
PYTHONPATH=. python -m benchmarks.mle.runner -c tabular-playground-series-dec-2021

# With options
PYTHONPATH=. python -m benchmarks.mle.runner \
    -c tabular-playground-series-dec-2021 \
    -i 20 \
    -m MLE_GENERIC \
    -d codex

CLI options

Option	Description	Default
`-c, --competition`	Competition ID	Required
`-i, --iterations`	Max experiment iterations	20
`-m, --mode`	Config mode	`MLE_GENERIC`
`-d, --coding-agent`	Coding agent	From config
`--no-kg`	Disable knowledge graph	Enabled
`--list`	List all competitions	-
`--lite`	List lite competitions	-
`--list-agents`	List coding agents	-

Configuration modes

Both modes run the generic search strategy, but the MLE handler supplies the official evaluation: solutions are scored by mlebench grading rather than agent-built evaluation.

MLE_GENERIC
MINIMAL

The campaign standard. Ideation on Claude, implementation sessions on the Codex CLI.

search_strategy:
  type: "generic"
  params:
    idea_generation_model: "claude-opus-5"
    implementation_model: "gpt-5.6-sol"
    implementation_cli: "codex"
knowledge_search:
  enabled: false

Quick smoke testing, everything on Claude Code.

search_strategy:
  type: "generic"
  params:
    idea_generation_model: "claude-opus-5"
    implementation_model: "claude-opus-5"
coding_agent:
  type: "claude_code"
  model: "claude-opus-5"

Stages

The handler automatically adjusts strategy based on budget progress:

Stage	Budget	Behavior
MINI TRAINING	0-20%	Sample training data (for datasets >30GB)
FULL TRAINING	20-80%	Train on complete dataset
FINAL ENSEMBLING	80-100%	Ensemble best models from history

Output structure

The agent generates:

experiment_workspace/{uuid}/
├── main.py                    # Entry point
├── output_data_{branch}/
│   ├── final_submission.csv   # Kaggle submission file
│   └── checkpoints/           # Model checkpoints
└── sessions/                  # Experiment branches

Code requirements

Generated code must:

Support --debug flag for fast testing
Write final_submission.csv in the output directory
Print progress and metrics
Handle GPU efficiently (batch size, device selection)
Use early stopping and learning rate scheduling

Competition types

Type	Examples
Tabular	`tabular-playground-series-*`
Image	`dogs-vs-cats-`, `plant-pathology-`
Text	`spooky-author-identification`, `jigsaw-toxic-*`
Audio	`mlsp-2013-birds`

Environment variables

Variable	Default	Description
`CUDA_DEVICE`	`0`	GPU device ID
`MLE_SEED`	`1`	Random seed

Installation

Installing the harness

CLI reference

Running a campaign

Related pages: Installation · CLI reference · IOAI 2026 Kapso is an open-source framework by Leeroo that builds software toward measurable goals through experiment campaigns. Source code: github.com/Leeroo-AI/kapso · Install: pip install leeroo-kapso · Every page as plain text: llms.txt.

Getting started

Evolve system

Knowledge graph

Trajectory learning

Research

Deployment

Benchmarks

Run Kapso on MLE-Bench: Kaggle competitions, ranked first

Usage

CLI options

Configuration modes

Stages

Output structure

Code requirements

Competition types

Environment variables

Installation

CLI reference

​Usage

​CLI options

​Configuration modes

​Stages

​Output structure

​Code requirements

​Competition types

​Environment variables

​Related

Installation

CLI reference

Usage

CLI options

Configuration modes

Stages

Output structure

Code requirements

Competition types

Environment variables

Related