ai-workspace-services/qmd
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: add explicit HuggingFace repo destinations

Browse Source 
- List all HuggingFace repos in CLAUDE.md (model, gguf, sft, grpo, train)
- Update jobs scripts to use tobil/qmd-query-expansion-train (no -v2)
- Clarify rules: no versioned repos, update in place

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Tobi Lutke 2026-01-31 12:26:02 -05:00
parent 38073799c0
commit 739038e1a7
No known key found for this signature in database
3 changed files with 14 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

18
finetune/CLAUDE.md
View File

@ -18,14 +18,20 @@ vec: another semantic variation
- `lex:` lines for BM25 keyword search (1-3 lines, short keywords)
- `vec:` lines for vector similarity search (1-3 lines, natural language)
## Model Repository
## HuggingFace Repositories
**Single destination**: `tobil/qmd-query-expansion-1.7B`
| Repository | Purpose |
|------------|---------|
| `tobil/qmd-query-expansion-1.7B` | Final merged model (SFT + GRPO) |
| `tobil/qmd-query-expansion-1.7B-gguf` | GGUF quantized versions for deployment |
| `tobil/qmd-query-expansion-1.7B-sft` | SFT adapter checkpoint (intermediate) |
| `tobil/qmd-query-expansion-1.7B-grpo` | GRPO adapter checkpoint (intermediate) |
| `tobil/qmd-query-expansion-train` | Prepared training dataset |
- No versioned directories (`-v1`, `-v2`, `-v4`, etc.)
- No separate `-sft` or `-grpo` repos for final models
- Update the main repo only when eval scores improve
- GGUF variants go to `tobil/qmd-query-expansion-1.7B-gguf`
**Rules:**
- No versioned repos (`-v1`, `-v2`, `-v4`, etc.) - update in place
- Only push when eval scores improve over current deployed model
- Always include eval results in model card when pushing
## Training Data

2
finetune/jobs/grpo.py
View File

@ -46,7 +46,7 @@ from eval_common import QMDRewardFunction, run_eval
BASE_MODEL = "Qwen/Qwen3-1.7B"
SFT_MODEL = "tobil/qmd-query-expansion-1.7B-sft"
OUTPUT_MODEL = "tobil/qmd-query-expansion-1.7B-grpo"
DATASET = "tobil/qmd-query-expansion-train-v2"
DATASET = "tobil/qmd-query-expansion-train"
def main():

2
finetune/jobs/sft.py
View File

@ -25,7 +25,7 @@ from huggingface_hub import login
# --- Config (inlined from configs/sft.yaml) ---
BASE_MODEL = "Qwen/Qwen3-1.7B"
OUTPUT_MODEL = "tobil/qmd-query-expansion-1.7B-sft"
DATASET = "tobil/qmd-query-expansion-train-v2"
DATASET = "tobil/qmd-query-expansion-train"
hf_token = os.environ.get("HF_TOKEN")
if hf_token: