d383b5c226
- Replace Ollama HTTP API with node-llama-cpp for local GGUF models - Add structured query expansion using JSON schema grammar: - Generates lexical query (for BM25), vector query, and HyDE - Tree-style CLI output showing query types - Fix vector search: use cosine distance instead of L2 - Format queries with embeddinggemma nomic-style prompts - Rename ollama_cache table to llm_cache - Add disposeDefaultLlamaCpp() for clean process exit 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
141 lines
4.6 KiB
Markdown
141 lines
4.6 KiB
Markdown
# QMD - Quick Markdown Search
|
|
|
|
**Note**: This project uses [bd (beads)](https://github.com/steveyegge/beads) for issue tracking. Use `bd` commands instead of markdown TODOs. See AGENTS.md for workflow details.
|
|
|
|
Use Bun instead of Node.js (`bun` not `node`, `bun install` not `npm install`).
|
|
|
|
## Commands
|
|
|
|
```sh
|
|
qmd collection add . --name <n> # Create/index collection
|
|
qmd collection list # List all collections with details
|
|
qmd collection remove <name> # Remove a collection by name
|
|
qmd collection rename <old> <new> # Rename a collection
|
|
qmd ls [collection[/path]] # List collections or files in a collection
|
|
qmd context add [path] "text" # Add context for path (defaults to current dir)
|
|
qmd context list # List all contexts
|
|
qmd context check # Check for collections/paths missing context
|
|
qmd context rm <path> # Remove context
|
|
qmd get <file> # Get document by path or docid (#abc123)
|
|
qmd multi-get <pattern> # Get multiple docs by glob or comma-separated list
|
|
qmd status # Show index status and collections
|
|
qmd update [--pull] # Re-index all collections (--pull: git pull first)
|
|
qmd embed # Generate vector embeddings (uses node-llama-cpp)
|
|
qmd search <query> # BM25 full-text search
|
|
qmd vsearch <query> # Vector similarity search
|
|
qmd query <query> # Hybrid search with reranking (best quality)
|
|
```
|
|
|
|
## Collection Management
|
|
|
|
```sh
|
|
# List all collections
|
|
qmd collection list
|
|
|
|
# Create a collection with explicit name
|
|
qmd collection add ~/Documents/notes --name mynotes --mask '**/*.md'
|
|
|
|
# Remove a collection
|
|
qmd collection remove mynotes
|
|
|
|
# Rename a collection
|
|
qmd collection rename mynotes my-notes
|
|
|
|
# List all files in a collection
|
|
qmd ls mynotes
|
|
|
|
# List files with a path prefix
|
|
qmd ls journals/2025
|
|
qmd ls qmd://journals/2025
|
|
```
|
|
|
|
## Context Management
|
|
|
|
```sh
|
|
# Add context to current directory (auto-detects collection)
|
|
qmd context add "Description of these files"
|
|
|
|
# Add context to a specific path
|
|
qmd context add /subfolder "Description for subfolder"
|
|
|
|
# Add global context to all collections (system message)
|
|
qmd context add / "Always include this context"
|
|
|
|
# Add context using virtual paths
|
|
qmd context add qmd://journals/ "Context for entire journals collection"
|
|
qmd context add qmd://journals/2024 "Journal entries from 2024"
|
|
|
|
# List all contexts
|
|
qmd context list
|
|
|
|
# Check for collections or paths without context
|
|
qmd context check
|
|
|
|
# Remove context
|
|
qmd context rm qmd://journals/2024
|
|
qmd context rm / # Remove global context
|
|
```
|
|
|
|
## Document IDs (docid)
|
|
|
|
Each document has a unique short ID (docid) - the first 6 characters of its content hash.
|
|
Docids are shown in search results as `#abc123` and can be used with `get` and `multi-get`:
|
|
|
|
```sh
|
|
# Search returns docid in results
|
|
qmd search "query" --json
|
|
# Output: [{"docid": "#abc123", "score": 0.85, "file": "docs/readme.md", ...}]
|
|
|
|
# Get document by docid
|
|
qmd get "#abc123"
|
|
qmd get abc123 # Leading # is optional
|
|
|
|
# Docids also work in multi-get comma-separated lists
|
|
qmd multi-get "#abc123, #def456"
|
|
```
|
|
|
|
## Options
|
|
|
|
```sh
|
|
# Search & retrieval
|
|
-c, --collection <name> # Restrict search to a collection (matches pwd suffix)
|
|
-n <num> # Number of results
|
|
--all # Return all matches
|
|
--min-score <num> # Minimum score threshold
|
|
--full # Show full document content
|
|
--line-numbers # Add line numbers to output
|
|
|
|
# Multi-get specific
|
|
-l <num> # Maximum lines per file
|
|
--max-bytes <num> # Skip files larger than this (default 10KB)
|
|
|
|
# Output formats (search and multi-get)
|
|
--json, --csv, --md, --xml, --files
|
|
```
|
|
|
|
## Development
|
|
|
|
```sh
|
|
bun src/qmd.ts <command> # Run from source
|
|
bun link # Install globally as 'qmd'
|
|
```
|
|
|
|
## Architecture
|
|
|
|
- SQLite FTS5 for full-text search (BM25)
|
|
- sqlite-vec for vector similarity search
|
|
- node-llama-cpp for embeddings (embeddinggemma), reranking (qwen3-reranker), and query expansion (Qwen3)
|
|
- Reciprocal Rank Fusion (RRF) for combining results
|
|
- Token-based chunking: 800 tokens/chunk with 15% overlap
|
|
|
|
## Important: Do NOT run automatically
|
|
|
|
- Never run `qmd collection add`, `qmd embed`, or `qmd update` automatically
|
|
- Never modify the SQLite database directly
|
|
- Write out example commands for the user to run manually
|
|
- Index is stored at `~/.cache/qmd/index.sqlite`
|
|
|
|
## Do NOT compile
|
|
|
|
- Never run `bun build --compile` - it overwrites the shell wrapper and breaks sqlite-vec
|
|
- The `qmd` file is a shell script that runs `bun src/qmd.ts` - do not replace it |