- Add missing subprocess import (NameError on any quantize path)
- Replace broken optimum-cli quantize calls with direct onnxruntime:
Q4 uses MatMulNBitsQuantizer, Q8 uses quantize_dynamic
- Add onnxconverter-common to deps for FP16 (was silently swallowed)
- Make FP16 fail loudly on missing dep instead of silently uploading FP32
- README and transformers_js_config now reflect actual quantize_type
instead of always hardcoding Q4
- Remove dead _convert_fp16_external function
- Use no_post_process=True for ONNX export to avoid protobuf serialize error
- Add --validate and --validate-only flags for inference verification
- Fix position_ids in validation feed (required by Qwen3 ONNX export)
- Use optimum-cli for quantization to handle external data format
- Fix optimum dependency to optimum[onnxruntime]
Tested: export + validation passes on CPU, KV cache present (56 tensors).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add convert_onnx.py that mirrors convert_gguf.py's structure:
- Loads base Qwen3 model, merges SFT + GRPO adapters
- Exports to ONNX via Optimum (text-generation-with-past task)
- Supports Q4 (MatMulNBits), Q8, FP16, and FP32 output
- Uploads to separate HF repo (e.g. tobil/qmd-query-expansion-1.7B-ONNX)
- Writes Transformers.js compatibility config
- Includes model card with usage example
Usage:
uv run convert_onnx.py --size 1.7B
uv run convert_onnx.py --size 1.7B --quantize q4 --no-upload
Also adds `just convert-onnx` and `just convert-gguf` tasks.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
