Replace ad-hoc JSON parsing with a strict Pydantic model
(TrainingExample with typed OutputPair). All data loading goes
through load_examples() which fails loudly on invalid data.
- Convert v3_structured.jsonl from "searches" to "output" format
- Rewrite all consumer scripts (prepare, validate, score, analyze)
to load through the Pydantic schema
- Prepared train/val files are ephemeral build artifacts
- Restore LFM2 and GEPA experiments under experiments/
- Add pydantic>=2.0 to dependencies
- List all HuggingFace repos in CLAUDE.md (model, gguf, sft, grpo, train)
- Update jobs scripts to use tobil/qmd-query-expansion-train (no -v2)
- Clarify rules: no versioned repos, update in place
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove versioned files (sft_v4.yaml, prepare_v4_dataset.py, train_v2/)
- Update configs to use local data/train/ directory
- Add glob pattern support to prepare_data.py and train.py
- Update .gitignore to properly ignore outputs/ and data/train*/
- Document data preparation step in CLAUDE.md
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add finetune/CLAUDE.md documenting the training pipeline
- Update configs to output to local outputs/ directory (gitignored)
- Document that all data/*.jsonl files are training data
- Document local CUDA training vs HuggingFace Jobs cloud training
- Enforce eval requirement before any model upload
- Single model repo (no -v1, -v2, -v4 versioning)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
