Collections can now specify an optional 'update:' key with a bash command
that will be executed during 'qmd update' before indexing files.
Features:
- Runs command in collection directory (cwd)
- Uses /usr/bin/env bash -c "..."
- Always prints stdout/stderr output
- Stops with exit code if command fails (!= 0)
- Legacy --pull flag still works for git repos without update command
Example YAML:
```yaml
collections:
archive:
path: /path/to/repo
pattern: "**/*.md"
update: git pull
```
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Handle both empty string and '/' as root context in path display logic.
YAML stores root contexts as '/' but old code expected empty string.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Fixed YAML syntax: Quote all context strings containing colons
- Fixed context grouping: Use collection names instead of IDs
- contextsByCollection now uses Map<string, ...> keyed by name
- Removed collection.id references (collections from YAML have no ID)
YAML now properly parses and contexts display correctly per collection.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Remove collections and path_contexts table creation from initializeDatabase()
- Update all queries to work without these tables
- Remove JOINs with collections table from document retrieval functions
- Compute absolute file paths from YAML collections in application code
- Update deleteContext() to use collection names instead of IDs
- Update qmd.ts status command to use listCollections() from YAML
- Deprecate cleanupDuplicateCollections() (returns 0)
All collection and context metadata is now managed exclusively in YAML
at ~/.config/qmd/index.yml. Database contains only documents and content
(content-addressable storage).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Changed removeCollection() call to use collection name instead of ID
- Changed renameCollection() call to use oldName, newName instead of ID
- Both commands now work with YAML config via store.ts
All collection commands now fully integrated with YAML configuration.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Context system changes (qmd-oxy):
- getContextForPath() now uses collections.findContextForPath()
- getContextForFile() uses collections.ts to find context
- insertContext(), deleteContext(), deleteGlobalContexts() write to YAML
- listPathContexts() reads from collectionsListAllContexts()
- getCollectionsWithoutContext() checks YAML for missing contexts
- getTopLevelPathsWithoutContext() uses YAML config
- No longer queries path_contexts table
Collection management changes (qmd-u84):
- getCollectionByName() reads from YAML (id field removed)
- listCollections() merges YAML config with DB stats
- removeCollection() takes collectionName instead of ID
- renameCollection() takes oldName, newName instead of ID
- All collection queries now use YAML as source of truth
Bug fixes:
- Fixed qmd.ts status command to use d.collection instead of d.collection_id
All context and collection metadata now stored in ~/.config/qmd/index.yml
Database retains only documents and content (content-addressable storage)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Schema changes:
- documents.collection_id (INTEGER FK) → documents.collection (TEXT)
- Update UNIQUE constraint to (collection, path)
- Update indices to use collection name
- Update FTS triggers to compute filepath from collection || '/' || path
Code changes in store.ts:
- Change all function parameters from collectionId: number to collectionName: string
- Update all SQL queries to use d.collection instead of d.collection_id
- Remove unnecessary JOINs where collection name is already available
- Update DocumentResult type: collectionId → collectionName
- Update renameCollection() to also update documents.collection
Successfully migrated 2309 documents across 6 collections.
This prepares for YAML-based collection configuration where collections
table will be removed and collection names will be the primary identifier.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Export collections and contexts from SQLite to ~/.config/qmd/index.yml.
Successfully exported 6 collections and 10 contexts.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Create src/collections.ts module for managing collections in YAML
- Collections defined in ~/.config/qmd/index.yml instead of SQLite
- Support for nested contexts at any path level
- Global context applies to all collections
- Functions: load/save config, add/remove/rename collections
- Context management: add, remove, find best match for path
- Add yaml package dependency
- Include example-index.yml showing the clean YAML format
This is the foundation for removing collections and path_contexts
tables from SQLite, moving all configuration to YAML.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add file size and modification date columns
- Format dates like ls -l (recent: Month Day Time, old: Month Day Year)
- Add colors: cyan filenames, dimmed qmd:// prefix
- Join with content table to get file sizes
- Format sizes in human-readable units (B, KB, MB)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit implements a diagnostic command to help users find collections
and paths that don't have context strings defined.
Changes:
1. New 'qmd context check' command that:
- Lists collections without any context configured
- Identifies top-level directories in collections missing context
- Provides actionable suggestions with example commands
2. Updated help text and command documentation in CLAUDE.md
3. Enhanced error messages to include the new 'check' subcommand
The command provides helpful output showing:
- Collections without context (with document counts and suggestions)
- Top-level paths within collections that lack context
- Suggestions for adding context using virtual path syntax
This addresses the issue where users had contexts but couldn't find them,
by providing a clear diagnostic tool to identify what's missing.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit adds git integration to the qmd update command:
1. Git repository detection: Checks for .git directory in each collection
2. Git status display: Shows short status for git repositories during update
3. --pull flag: Added optional --pull flag to execute git pull before reindexing
4. Error handling: Gracefully handles git errors without failing the update
When a collection is a git repository:
- Displays "Git repository detected"
- If --pull is specified, runs git pull and shows output (dimmed)
- Shows git status --short output (dimmed)
- If status is clean, shows "Git status: clean"
Usage:
qmd update # Update all collections, show git status
qmd update --pull # Pull changes first, then update
All git operations are non-blocking - failures are shown as warnings
but don't prevent the update from continuing.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Lists all path contexts under their respective collections
- Shows path prefix and context description
- Truncates long context descriptions (>60 chars) for readability
- Clean, indented display format matching existing status style
Fixes qmd-0ic
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Move all .ts files to src/ to clean up the project root:
- Created src/ directory and moved all TypeScript source and test files
- Updated qmd shell wrapper to point to src/qmd.ts
- Updated package.json scripts to use src/ paths
- Updated documentation (CLAUDE.md, README.md) to reflect new structure
- All imports remain relative within src/, no changes needed
- Tests pass with same results (192 pass, 75 fail - existing issues)
This improves project organization and makes the root directory cleaner.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Removed:
- archive symlink (pointing to Shopify archive context)
- texts symlink (pointing to Shopify articles)
Updated:
- .gitignore: Added .github/ to ignore list
These symlinks were mistakenly tracked in git despite being in .gitignore.
They pointed to external directories and should not be in version control.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This refactoring extracts all document indexing database operations from
the indexFiles() function in qmd.ts into dedicated methods in store.ts:
- insertContent(): Insert content into content-addressable storage
- insertDocument(): Insert new document record
- findActiveDocument(): Find active document by collection and path
- updateDocumentTitle(): Update document title only
- updateDocument(): Update document hash, title, and modified_at
- deactivateDocument(): Mark document as inactive
- getActiveDocumentPaths(): Get all active document paths for collection
- cleanupOrphanedContent(): Remove unreferenced content hashes
Key improvements:
- Better separation of concerns (DB layer vs business logic)
- Reusable DB operations accessible throughout the codebase
- Fixed bug where document updates would fail with UNIQUE constraint error
- Fixed bug in updateCollections() that passed wrong parameters to indexFiles()
- Removed obsolete updateDisplayPaths() function from old schema
The indexing logic now uses updateDocument() instead of deactivate + insert
when content changes, which properly updates the existing document record
rather than violating the UNIQUE(collection_id, path) constraint.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Expose the new updateDocument() helper through the Store interface
for use by consumers.
Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add new helper function to update document hash and title simultaneously.
This is useful when file content changes but the path stays the same.
Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Update context management commands to use store.ts functions instead
of inline SQL queries. This change is part of the larger refactoring
effort to move all DB operations into store.ts.
Changes:
- contextAdd: Use insertContext() and getAllCollections()
- contextList: Use listPathContexts()
- contextRemove: Use deleteContext() and deleteGlobalContexts()
This simplifies the CLI code and centralizes database logic.
Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This change consolidates all database cleanup and maintenance operations
into store.ts for better organization and reusability.
Changes:
- Added deleteOllamaCache() - Remove cached Ollama API responses
- Added deleteInactiveDocuments() - Remove inactive document records
- Added cleanupOrphanedContent() - Remove unreferenced content hashes
- Added cleanupOrphanedVectors() - Remove orphaned vector embeddings
- Added cleanupDuplicateCollections() - Remove duplicate collections
- Added vacuumDatabase() - Run VACUUM to reclaim space
- Updated Store type and createStore() to expose these methods
- Updated qmd.ts cleanup command to use new store methods
- Removed local cleanupDuplicateCollections() from qmd.ts
- Removed duplicate cleanupOrphanedContent() definition
All cleanup operations now follow the pattern of returning counts
for reporting, making the cleanup command output more informative.
Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Refactor vector indexing by extracting database operations from vectorIndex()
in qmd.ts into three new store methods:
- getHashesForEmbedding(): Returns content hashes needing embeddings
- clearAllEmbeddings(): Clears all vectors for force re-indexing
- insertEmbedding(): Inserts a single embedding into both tables
This continues the refactoring effort to consolidate all database operations
in store.ts, making the codebase more maintainable and testable.
Changes:
- Add new embedding operation methods to Store type and factory
- Export getHashesForEmbedding(), clearAllEmbeddings(), insertEmbedding()
- Update vectorIndex() to use new store methods instead of direct SQL
- Remove inline SQL from embedding logic in qmd.ts
Related: qmd-4u4
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Added listCollections(), removeCollection(), renameCollection() to store.ts
- Updated qmd.ts to use store methods instead of direct DB operations
- Removed ~15 direct DB calls from qmd.ts collection functions
- All 9 collection command tests passing
Part of larger refactoring to move all DB operations to store.ts
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Updated vectorIndex() to work with new schema:
* Join with content table to get document body
* Use 'path' column instead of 'filepath'/'display_path'
* Query now: SELECT d.hash, c.doc as body, MIN(d.path) as path
- Documents table no longer has body column (in content table)
- Documents table uses 'path' not 'filepath' or 'display_path'
- Embedding functionality now compatible with hash-based storage
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Added collectionRename() function to rename collections
- Updates collection name in database (changes virtual path prefix)
- Added CLI handler for "qmd collection rename <old> <new>"
- Supports alias "mv" for rename command
- Includes comprehensive error handling:
* Checks if old collection exists
* Prevents renaming to existing collection name
* Validates required arguments
- Fixed bug in collectionAdd where --name flag was ignored
* indexFiles now accepts pwd and name parameters
* Collection name is properly passed through to getOrCreateCollection
- Updated help text and CLAUDE.md documentation
- Added 4 new tests for rename functionality (all passing)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Updated createTestCollection() to support new schema with name field
- Updated insertTestDocument() to match new schema (path, not filepath)
- Content is now stored in separate content table with hash deduplication
- Added comprehensive test suite for content-addressable storage:
* Same content gets same hash from multiple collections
* Removing one collection preserves shared content
* Deduplication works across many collections
* Different content gets different hashes
- All 4 content-addressable tests passing
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Display collections by name instead of full path
- Show virtual path format: qmd://name/
- Add context count per collection
- Include helpful command examples (ls, get, search)
- Update "no collections" message to reference collection add
- Sort collections alphabetically by name
Closes qmd-rhd
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Remove qmd add command (replaced by qmd collection add)
- Remove --drop flag (use collection remove + add instead)
- Update all help text to reflect new command structure
- Update CLAUDE.md documentation
- Update all tests to use collection add (39/42 passing)
- Reorganize help to put collection commands first
- Remove add-context from "do not run automatically" list
The CLI now has consistent command structure:
- qmd collection {add,list,remove} for collection management
- qmd context {add,list,rm} for context management
- qmd ls for browsing collections
- qmd get/multi-get for document retrieval
Closes qmd-c0m
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add collectionList() to display all collections with details
- Add collectionAdd() with --name and --mask options
- Add collectionRemove() to delete collections by name
- Add CLI command handlers for collection subcommands
- Add --name and --mask CLI options
- Update help text with collection commands
- Add comprehensive tests (5 test cases, all passing)
- Properly clean up orphaned content hashes on removal
Closes qmd-dmi
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add listFiles() function to list collections and files
- Support three modes:
- qmd ls - list all collections with file counts
- qmd ls <collection> - list all files in collection
- qmd ls <collection>/path - list files under path
- Support both virtual paths (qmd://) and short form
- Add CLI command handler and help text
- Add comprehensive tests (5 test cases, all passing)
Closes qmd-e2c
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Update getDocument() to support qmd:// virtual paths and filesystem paths
- Update multiGet() to handle virtual paths in patterns and comma-separated lists
- Update matchFilesByGlob() in store.ts to return virtual paths
- Remove duplicate getContextForFile() function from qmd.ts
- Use collection-scoped getContextForPath() instead of legacy function
- All get and multi-get tests now passing
Closes qmd-vro
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Remove mimeType from TextContent (not in spec, only valid on EmbeddedResource)
- Add isError: true to all error responses for proper error detection
- Replace CSV output with structuredContent for machine-readable results
- Add name and title fields to all embedded resources
- Fix URI encoding: preserve slashes, encode special chars (spaces → %20)
- Change template to {+path} for proper nested path support
- Rename tools: qmd_search → search, qmd_get → get, etc.
- Update tests for new response format and spec compliance
- Extract store.ts: database operations, search, document retrieval
- createStore() factory pattern for clean DB lifecycle management
- Unified DocumentResult type with optional body loading
- Snippet extraction with diff-style headers (@@ -line,count @@)
- Extract llm.ts: LLM abstraction layer with Ollama implementation
- Clean interface for embed, generate, rerank operations
- High-level rerankerLogprobsCheck with logprob-based scoring
- Query expansion support
- Extract formatter.ts: output formatting utilities
- Support for CLI, JSON, CSV, MD, XML formats
- MCP-specific CSV formatting
- Extract mcp.ts: MCP server using createStore() pattern
- Single DB connection for server lifetime (fixes closed DB errors)
- URL-decode resource paths for proper space/special char handling
- Add comprehensive test suites (215 tests total)
- store.test.ts: 96 tests covering all store operations
- llm.test.ts: 60 tests for LLM abstraction
- mcp.test.ts: 59 tests for MCP endpoints and resources
- All tests use mocked Ollama (errors on unmocked calls)
- Add bun run inspector script for MCP debugging
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Package QMD as a Nix derivation with bun and sqlite dependencies
- Support `nix profile install`, `nix run`, and `nix develop`
- Set library paths for sqlite extension support
- Add `qmd mcp` command to start stdio-based MCP server
- Expose tools: qmd_search, qmd_vsearch, qmd_query, qmd_get, qmd_status
- Add index health warnings for unembedded docs and stale indexes
- Return CSV format with text/csv mime type for search results
- Add MCP documentation and configuration examples to README
- Add @modelcontextprotocol/sdk and zod dependencies
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add new tagline: on-device search engine for everything you need to remember
- Add Quick Start section with walkthrough of indexing multiple directories
- Add "Using with AI Agents" section showing --json and --files workflows
- Update output format example to reflect new CLI format with Title/Context/Score
- Document --all flag for returning all matches
- Add --all flag to search/vsearch for returning all matches
- Works with --min-score to filter by relevance threshold
- Useful for bulk exports: qmd search "term" --all --files
- Display paths now always include parent folder for better context
- e.g., pages/file.md instead of just file.md
- qmd get now outputs folder context header when available
- Format: "Folder Context: <description>\n---\n<content>"
- Add display_path column with unique index for collection-relative paths
- Computes shortest unique path starting from filename
- Auto-migrates existing documents via update-all
- Add title field to all search result types (FTS, vector, hybrid)
- Improve CLI output format:
- Filepath on first line
- Title and Context on labeled lines
- Score on separate line
- Remove | prefix from snippets for better word wrap
- Double newline between results
- Add line range support to get command:
- qmd get file.md:100 (start at line 100)
- qmd get file.md -l 20 --from 50
- Include title in JSON, CSV, XML output formats
- Fix update-all crash when same file exists in multiple collections
- Documents matching multiple chunks get +0.02 per additional chunk
- Bonus capped at +0.1 (5+ chunks)
- Still uses max chunk score as base
- Rewards documents with broader relevance across content
🤖 Generated with [Claude Code](https://claude.com/claude-code)
- Fix architecture diagram: show BM25+Vector for all query variations
- Add position-aware blending percentages to diagram
- Update CLI commands: add → index, add-context, cleanup, status
- Document chunked embeddings (~6KB pieces with hash/seq/pos)
- Update schema section with new tables (path_contexts, ollama_cache)
- Rewrite How It Works flows with accurate pipeline details
- Fix output format examples to show ~/... paths
- Add --files and --json output options
🤖 Generated with [Claude Code](https://claude.com/claude-code)
