Switch from generic Qwen3-1.7B-Q8_0 (~2.2GB) to fine-tuned
qmd-query-expansion-1.7B-q4_k_m (~1.1GB). The fine-tuned Q4
scores 91.7% avg with 30/30 Excellent, outperforming the base Q8.
- Update default generate model in src/llm.ts
- Update README model table, architecture diagram, config block
- Add v2 training data, eval scripts, and quantize job
- Remove superseded v1 training data (5,742 → 1,000 examples)
- Update finetune README with v2 results and file structure
Co-Authored-By: Claude (claude-fudge-eap-cc) <noreply@anthropic.com>
- jobs/sft.py and jobs/grpo.py: self-contained scripts for
`hf jobs uv run` (no local GPU needed)
- 12 temporal/recency query examples in training data (e.g. "recent
news about Shopify" -> lex with years 2025/2026)
- 4 temporal test queries in evals/queries.txt
- README updated with HF Jobs workflow, training results, and
updated file structure
- Remove .beads tracking
SFT and GRPO successfully trained on A10G via HF Jobs:
SFT: eval loss 0.321, token accuracy 92.4%
GRPO: mean reward 0.757, 200 steps, KL 0.00048
Co-Authored-By: Claude (claude-fudge-eap-cc) <noreply@anthropic.com>
Consolidate ~2,800 lines of duplicated code across 12 files into 5 clean,
well-documented files targeting Qwen3-1.7B end-to-end.
Key changes:
- Extract reward function into single source of truth (reward.py)
Previously duplicated 3x with divergent bugs across rl.py,
train_1.7B_grpo.py, and train_4B_grpo.py
- Unify training into one script with sft/grpo subcommands (train.py)
Replaces train.py + rl.py + train_1.7B_grpo.py + train_4B_grpo.py
- Merge eval generate+score into single eval.py
Replaces evals/run.py + evals/score.py
- Parameterize GGUF conversion by --size (convert_gguf.py)
Replaces convert_1.7B_gguf.py + convert_4B_gguf.py
- Fix critical bug: rl.py silently ignored beta/temperature from config,
causing the exact catastrophic drift its own comments warned about
- Fix prompt consistency: all files use /no_think chat template format
- Retarget configs from 0.6B to 1.7B
- Comprehensive README documenting the full pipeline
Removed: rl.py, train_1.7B_grpo.py, train_4B_grpo.py, convert_1.7B_gguf.py,
convert_4B_gguf.py, tui.py, evals/run.py, evals/score.py
Net: -3,429 lines, +382 lines
Co-Authored-By: Claude (claude-fudge-eap-cc) <noreply@anthropic.com>
The SFT model was trained with chat template format but GRPO was
passing raw prompts. Now prompts are formatted with tokenizer.apply_chat_template()
so the model sees the same format it learned during SFT.
Also update extract_query_from_prompt to strip chat template artifacts.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fetch available qmd-query-expansion models from tobil/ on Hub
- Auto-detect model size (0.6B, 1.7B, 4B) and use correct base model
- Group models by type (SFT vs GRPO) in menu
- Skip GGUF repos in model listing
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Strip <|im_end|> token from completions (model output includes it)
- Change think_penalty to skipped_think bonus (+20 for not using think)
- Adjust max_possible to account for bonus (120/140)
- Fix typo in chat template artifact check
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Training scripts for GRPO fine-tuning:
- train_1.7B_grpo.py: GRPO training for Qwen3-1.7B
- train_4B_grpo.py: GRPO training for Qwen3-4B
GGUF conversion scripts:
- convert_1.7B_gguf.py: Merge SFT+GRPO adapters and convert to GGUF
- convert_4B_gguf.py: Merge SFT+GRPO adapters and convert to GGUF
All scripts use PEP 723 inline dependencies for HuggingFace Jobs.
Models published:
- tobil/qmd-query-expansion-1.7B-sft
- tobil/qmd-query-expansion-1.7B-grpo
- tobil/qmd-query-expansion-1.7B-gguf
- tobil/qmd-query-expansion-4B-sft
- tobil/qmd-query-expansion-4B-grpo
- tobil/qmd-query-expansion-4B-gguf
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Changed from subfolder approach to separate repos per model since
trainer.push_to_hub() doesn't support subfolder argument.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
All models (0.6B, 1.7B, 4B) with SFT and GRPO variants now go into
a single HuggingFace repo (tobil/qmd-query-expansion) with subfolders
for each size and training method.
Updated loading examples to show subfolder-based model loading.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
GRPO adapters were trained on merged SFT weights, so they need SFT
loaded and merged first before applying the GRPO adapter.
Updated MODELS config to include sft_base path for GRPO models,
and load_model() now handles the SFT -> merge -> GRPO flow.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The GRPO adapter was trained on merged SFT weights, so loading it
directly on the base model results in 0% score. Added --sft-model
parameter to evals/run.py to load SFT first, then apply GRPO adapter.
With correct loading: GRPO scores 89.7% (all 26 queries Excellent).
Updated README with correct GRPO score and loading instructions.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
New structure:
- evals/run.py: Generate model outputs to JSONL
- evals/score.py: Score outputs with detailed breakdown
- evals/queries.txt: Test queries (26 total)
Features:
- Supports both HF Hub and local model paths
- Named entity preservation scoring
- Chat template leakage detection
- Strict format validation (every line must be lex:/vec:/hyde:)
- Generic phrase detection
Usage:
uv run evals/run.py --model tobil/qmd-query-expansion-0.6B-v4
uv run evals/score.py evals/results_*.jsonl
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Any line that doesn't start with a valid prefix now returns 0.0
instead of just counting as a penalty. This prevents any prose,
explanations, bullet points, or other invalid content.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Zero reward for outputs containing:
- <|im_start|>, <|im_end|> tokens
- <think>, </think> tags (Qwen3 thinking mode)
- Role markers like \nassistant\n, \nuser\n
- <|endoftext|> token
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Major changes:
- train.py: Generic SFT training script using YAML config
- rl.py: Generic GRPO training script using YAML config
- configs/: YAML configs per training run (sft_v4.yaml, grpo_v4.yaml)
- dataset/: Data preparation scripts moved here
- tui.py: Interactive model testing interface
Training results:
- SFT v4: 98.8% avg score (all Excellent)
- GRPO v4: 0% (failed - model drifted to verbose explanations)
Removed per-model scripts (train_0.6B.py, train_1.7B.py, etc)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add explicit HF_TOKEN login before training
- Use SCORING.md criteria as RL reward function
- Conservative training: LR 1e-6, LoRA rank 4
- Reward scores: good=0.94, bad=0.38
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
New scoring criteria (0-100 points):
- Format (30): Must have lex: and vec: prefixes
- Diversity (30): Multiple types, no echoing query, diverse expansions
- Hyde (20): Optional, concise, no newlines, no word repetition
- Quality (20): Lex=keywords, vec=natural language
See SCORING.md for full documentation.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Use max_completion_length instead of max_new_tokens
- Use processing_class instead of tokenizer
- Use args instead of config for GRPOTrainer
- Add __name__ attribute to reward function class
- Accept **kwargs in reward function for extra TRL args
- Add new LoRA adapter after merging SFT weights
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
