On macOS with iCloud Drive (especially shared folders), some files may
appear in the filesystem but return EAGAIN (error -11) when read via
Node's readFileSync. This happens when iCloud has evicted the file
content but the file metadata remains visible.
Previously this crashed the entire update process. Now we catch the
error and skip the file, allowing the remaining files to index
successfully.
Affects: iCloud Drive shared folders on macOS
Error: 'Unknown system error -11: Unknown system error -11, read'
Reproduces with: Node.js v25.x, readFileSync on evicted iCloud files
- structured_search now accepts collections[] for OR filtering
- Updated skill docs with detailed query writing guidance
- lex: 2-5 keywords, include synonyms, exact names
- vec: full natural language questions with context
- hyde: 50-100 word hypothetical answer passages
BREAKING CHANGE: MCP tools search, vector_search, deep_search removed.
Use structured_search with lex/vec/hyde queries instead.
- Remove search, vector_search, deep_search MCP tool registrations
- Update MCP instructions to focus on structured_search
- Update skill docs to reflect simplified API
- Rename test describes to reflect they test store functions
- CLI commands (qmd search, vsearch, query) unchanged for backwards compat
Lines prefixed with lex:, vec:, or hyde: route directly to
structured search, skipping automatic query expansion.
Examples:
qmd query 'lex: CAP theorem'
qmd query $'lex: keywords\nvec: natural language question'
qmd query $'lex: terms\nvec: question\nhyde: hypothetical answer...'
Plain queries (single line, no prefix) still use automatic expansion.
Multiple plain lines without prefixes error with helpful message.
This lets CLI users leverage the same structured search as MCP,
useful when piping from scripts or when you know exactly what
query variations you want.
- New MCP tool: structured_search - lets capable LLMs provide their own
lex/vec/hyde query variations instead of using local expansion model
- New REST endpoint: POST /search - same functionality without MCP protocol
- Updated skill docs to prioritize structured_search for LLM callers
- Added installation instructions for Claude Code, Desktop, and OpenClaw
Pipeline: lex→FTS, vec/hyde→batch embed, RRF fusion (first query 2x weight),
chunk + rerank, position-aware blending, dedup.
This is the recommended endpoint for capable LLMs - they generate better
query variations than the small local model, especially for domain-specific
or nuanced queries.
collections-config.test.ts set currentIndexName to "myindex" in its
last test but only restored env vars in afterEach — not the module
variable. Under bun test (single process), this leaked into mcp.test.ts,
causing it to look for myindex.yml instead of index.yml.
Fix: reset setConfigIndexName("index") in afterEach, and add defensive
reset in mcp.test.ts beforeAll.
Previously, 'qmd search --json' would output plain text 'No results found.'
when no matches were found, which is invalid JSON. Now it correctly outputs
an empty JSON array [] when using --json format.
Fixed in all search commands: search, vsearch, and query.
When using --index with relative paths like './index/my-project', the path was
stored directly as the config filename, resulting in paths like:
/home/user/.config/qmd/./index/my-project.yml
This caused 'no such file or directory' errors. Now relative paths are resolved
and normalized by replacing path separators with underscores.
Dataset improvements:
- Reorder output to put hyde first for better retrieval priming
- Convert absolute paths to relative paths in scripts
- Add convert_to_structured.py for structured data format
- Add qmd_expansion_v3_structured.jsonl with type/query objects
- Update schema.py with reorder_hyde_first() helper
- Verify data now validates hyde-first ordering
Training data regenerated with new ordering (100% validation success).
Add training configuration and documentation for using LiquidAI's LFM2-1.2B
as an alternative base model for query expansion fine-tuning.
LFM2 benefits:
- 2x faster decode/prefill vs standard transformers
- Optimized for edge/on-device inference
- Good at agentic tasks, RAG, and data extraction
Changes:
- Add configs/sft_lfm2.yaml with LFM2-specific LoRA target modules
- Add jobs/sft_lfm2.py for HuggingFace Jobs training
- Update llm.ts with LFM2 GGUF model URIs
- Add documentation for LFM2 training workflow
LFM2 uses a hybrid architecture (convolutions + attention) requiring
different LoRA targets: q_proj, k_proj, v_proj, out_proj, in_proj, w1, w2, w3
Dataset improvements:
- fix_hyde.py: Replace generic template hyde entries with query-specific ones
using GPT-4o-mini (removed 'comprehensive guide covers everything' pattern)
- fix_lex_filler.py: Remove filler words (overview, tutorial, guide, examples,
documentation, best practices) that were padding rather than genuine search intent
- qmd_expansion_v3.jsonl: Improved dataset with 1,498 high-quality entries
Training data preparation:
- convert_to_chatml.py: Convert to ChatML format for LFM2.5 training
- verify_data.py: Validation script to ensure data quality
- train-lfm2/: Ready-to-use training data (90/10 train/val split)
Data quality metrics:
- 100% success rate (all entries properly formatted)
- Query length: 6-65 chars (avg: 29.3)
- Response length: 307-777 chars (avg: 539.5)
- All entries contain lex, vec, and hyde expansions
- Move model info from --help to `qmd status` with live HuggingFace
links derived from actual configured URIs
- Pre-push hook: handle non-interactive shells gracefully, resolve
annotated tags correctly for CI checks
- Add /release skill with full process: hook install, changelog
validation, git history review, preview, and release execution
- Skill auto-populates [Unreleased] from git history when empty
- Install hook script symlinks pre-push for tag validation
- Register skills/ dir in .pi/settings.json for pi discovery
- Add tsc build step (tsconfig.build.json) so npm package ships
compiled JS instead of raw TypeScript requiring tsx at runtime
- Update qmd wrapper and daemon spawn to use dist/qmd.js in
production while keeping tsx for development
- Add self-installing pre-push hook validating v* tag pushes:
package.json version match, changelog entry, CI status
- Add release.sh script that renames [Unreleased] to versioned
entry, bumps package.json, commits, and tags
- Add extract-changelog.sh for cumulative GitHub release notes
- Update publish workflow with build step and GitHub release creation
- Flesh out CHANGELOG.md with full history from 0.1.0 through 1.0.0
in Keep-a-Changelog format with PR/contributor attributions
- Add release standards and changelog guidelines to CLAUDE.md
- mcp.ts: add sessionIdGenerator to HTTP transport (fixes "stateless
transport cannot be reused" error in CI)
- test-preload.ts: set 30s default timeout for bun test runner (matches
vitest config, prevents CLI subprocess test timeouts)
- mcp.test.ts: use == null check instead of toBeUndefined for SQLite
get() result (bun:sqlite returns null, better-sqlite3 returns undefined)
- cli.test.ts: fix qmdScript path from <root>/qmd.ts to <root>/src/qmd.ts
(broke when tests moved from src/integration/ to test/)
- mcp.test.ts: forward Mcp-Session-Id header per MCP Streamable HTTP spec
No more src/models/ and src/integration/ subfolders to forget about.
All 9 test files live in test/, one command runs everything:
npx vitest run test/
bun test test/
Add src/db.ts that dynamically imports bun:sqlite under Bun and
better-sqlite3 under Node.js. Exports openDatabase(), loadSqliteVec(),
and a shared Database interface.
- sqlite-vec loading is now optional — FTS works without it, vector
ops throw a clear error if unavailable
- CI tests both runtimes: Node 22/23 via vitest, Bun via bun test
- All 104 unit tests pass on both Node and Bun
All test files now use vitest + better-sqlite3 imports.
bun test can't load the better-sqlite3 native addon (symbol
error on Linux, segfault on macOS). Run vitest on Node 22/23.
Split test suites for explicit runtime execution.
- Move model-related tests under `src/models/*`.
- Move CLI/integration tests under `src/integration/*`.
- Add `src/store.helpers.unit.test.ts` for helper unit coverage.
- Add shared Vitest config with default timeout and suite organization.
- Remove legacy flat test files from `src/` root.
- Keep core test commands in scripts supporting unit/models/integration runs.
Document both Node and Bun execution paths.
- Update install examples to `@tobilu/qmd` for npm and bun.
- Add npx/bunx one-off usage examples.
- Reflect Bun as first-class supported runtime in requirements.
Update README installation and quick-start commands to Node examples.
- replace bun install/link commands with npm-based Node workflow
- bump package version to 0.9.9 for CLI and MCP metadata
- keep Bun guidance as optional development/runtime note
