Five Claude agents — same model, different prompting strategies — compete on real ClinVar BRCA1/BRCA2 variants every 90 seconds. A deterministic ACMG/AMP scorer grades them, and an EMA leaderboard tracks which interpretive style actually wins on real-world evidence.
modal.Period (every 90 s)
│
▼
pick random fixture ┌──────────────────────────┐
from 25 BRCA1/BRCA2 variants ────────▶│ Modal Volume │
│ │ round.json │
▼ │ predictions.json │
run_round ── starmap × 5 agents ─────────│ scores.json │
│ (parallel Claude calls, │ leaderboard.json (EMA) │
│ prompt-cached, retry-armed) └──────────────────────────┘
▼ │
deterministic ACMG/AMP scorer │
│ ▼
▼ ┌──────────────────────────┐
EMA update on leaderboard ◀──────────│ FastAPI (asgi_app) │
│ GET /leaderboard │
│ GET /rounds │
│ GET /rounds/{id} │
│ POST /classify │
└──────────────────────────┘
│
▼
┌──────────────────────────┐
│ Next.js · Vercel │
│ pathoshunt.dev │
│ ⇣ │
│ Convex (user submits) │
└──────────────────────────┘Modal's starmap fans the round out across all five strategies concurrently, so a round's latency is roughly the slowest single agent — not the sum of five.
The ACMG/AMP reference is large and identical across agents. Caching keeps token cost flat as the tournament accumulates rounds.
Predictions are graded against ClinVar consensus by a hand-written scorer (partial credit for adjacent classes). Same input always grades the same way — no model drift in the metric.
Each agent's score is a recency-weighted exponential moving average. Recent rounds matter more, but the history is never unbounded — old artifacts can age out without altering the standings.
round.json / predictions.json / scores.json land on disk before the leaderboard EMA updates. A crashed run never produces a half-counted round.
One Modal web function exposes every endpoint via FastAPI, instead of one Modal function per route. Cleaner URL surface; one deploy artifact.
Everything the frontend renders is also available as JSON at the Modal endpoint. Type contracts live in frontend/lib/types.ts.
| Method | Path | Returns |
|---|---|---|
| GET | /leaderboard | LeaderboardEntry[] |
| GET | /rounds?limit=N | RoundSummary[] |
| GET | /rounds/{round_id} | Round |
| POST | /classify | {submission_id, predictions} |
base url · https://ch3mistocampus--pathos-fastapi-app.modal.run
Code: github.com/ch3mistocampus/pathos · Modal app: ch3mistocampus/pathos. Built solo over a weekend; everything you see is running live, not a recording.