Meet AutoCuratorOur first autonomous training data researcher

AutoCurator

Improving Karpathy's AutoResearch with data curation

DiscardedSuccessful
AutoCurator experiment progressA stepped line chart showing two retained improvements across 161 experiments, ending with 5.6 percent of training data selected and a 5.2 percentage point benchmark gain.ACCURACY0326496128161EXPERIMENTS #
0

Experiments

0.0%

Training data selected

+0.0pts

Benchmark score

AnalyzeImproveTrainLearnAnalyzeImproveTrainLearnAnalyzeImproveTrainLearnAnalyzeImproveTrainLearn
What is Provena

Provena is buildingautonomous training data researchersto continuously improve your model

Provena continuous training data research loopA continuous four-stage workflow that analyzes training data, tests interventions, runs model experiments, and remembers what works.01AnalyzeExplore training data02ImproveTest data interventions03TrainRun model experiments04LearnRemember what worksPROVENAAutonomous training data researchStarting with AutoCurator
AnalyzeImproveTrainLearnAnalyzeImproveTrainLearnAnalyzeImproveTrainLearnAnalyzeImproveTrainLearn
Why Provena

Data shapes models.Yet the biggest gains remain undiscovered

Best practices are scattered across researchers

Build on what already works

Experimentation takes time

Explore more hypotheses autonomously

Learnings get lost

Make every experiment compound

AnalyzeImproveTrainLearnAnalyzeImproveTrainLearnAnalyzeImproveTrainLearnAnalyzeImproveTrainLearn
How It Works

Bring us a model objective.
We'll research the data

01

Define the objective

What capability do you want to improve?

02

Research what works

AutoCurator proposes and tests data interventions

03

Measure the impact

See what improves performance — and why

AnalyzeImproveTrainLearnAnalyzeImproveTrainLearnAnalyzeImproveTrainLearnAnalyzeImproveTrainLearn

Self-improving AI systems require
self-improving data systems

AutoCuratorAutoResearchAgentSafety refusal patternDedup near-matchesTag harm_categoryFix empty responsesAuto-merge paraphrasesDetect new harm categoryStandardize refusal styleAuto-translate gapsHUMAN-LED IMPROVEMENTSAGENT-LED IMPROVEMENTS
Ready