qmd/finetune/dataset
Tobi Lütke 48f0917269
feat(finetune): hyde-first ordering, relative paths, structured format
Dataset improvements:
- Reorder output to put hyde first for better retrieval priming
- Convert absolute paths to relative paths in scripts
- Add convert_to_structured.py for structured data format
- Add qmd_expansion_v3_structured.jsonl with type/query objects
- Update schema.py with reorder_hyde_first() helper
- Verify data now validates hyde-first ordering

Training data regenerated with new ordering (100% validation success).
2026-02-17 06:31:35 -05:00
..
analyze_data.py lots of training stuff 2026-01-31 23:02:23 +00:00
clean_data.py lots of training stuff 2026-01-31 23:02:23 +00:00
generate_balanced.py lots of training stuff 2026-01-31 23:02:23 +00:00
generate_data_offline.py lots of training stuff 2026-01-31 23:02:23 +00:00
generate_data.py lots of training stuff 2026-01-31 23:02:23 +00:00
generate_diverse.py lots of training stuff 2026-01-31 23:02:23 +00:00
generate_ollama.py lots of training stuff 2026-01-31 23:02:23 +00:00
prepare_data.py lots of training stuff 2026-01-31 23:02:23 +00:00
schema.py feat(finetune): hyde-first ordering, relative paths, structured format 2026-02-17 06:31:35 -05:00
score_data.py lots of training stuff 2026-01-31 23:02:23 +00:00
validate_schema.py lots of training stuff 2026-01-31 23:02:23 +00:00