Eval Harness First
The Phase 0 gate for the whole plugin:
finetuning-method-selection and every downstream
skill assume this harness exists before a training
config gets written. The harness is not a run-end
side artifact — it is the data-curation engine. The
same labeled traces that build the goldens feed
training data, minus an explicit holdout.
Input: production/agent traces if they exist, or
a task spec if they don't, plus labelers willing to
grade ≥100 examples.
Output format: the eval/ directory below —
goldens, graders, drift suite, and the base-model
baseline that later phases gate on.
The Gate
No eval harness, no fine-tune. Skip to a training config and there is nothing to measure against, nothing to catch regressions, and no labeled data to train on. The flywheel: