Model Evolution Lab — public leaderboard

🧬 Model Evolution Lab a standing benchmark of real research tasks

View

Scope

X axis

What this is. Three tasks taken verbatim from real work in the Erwin lab (Molecular & Human Genetics, Baylor College of Medicine), run unchanged against each new frontier model as it ships, with every output archived. The point is the slope — how much better models get at actual research work, measured on tasks that were never written to be a benchmark.

Scoring. Task 01 is scored programmatically: the model's Python is executed by the harness, tracebacks are fed back until the figure renders, and the reported count of differentially expressed genes is checked against the paper. Task 03 is scored against a PMID answer key assembled from NCBI E-utilities, with a −25% penalty per fabricated reference. Task 02 is long-form writing and is not auto-scored. A blank cell means the task was not scored for that run, not that the model failed it.

Caveats. n = 1 per cell; these are single runs, not averaged trials, and the spread between two runs of the same model is real. Cost is the provider's own reported figure for that call and excludes search-tool spend billed separately. Latency is wall-clock and confounded by provider load; local runs are on one laptop and measure that laptop.