On macOS /tmp is a symlink to /private/tmp. mkdtemp returns /tmp/...
but getRealPath(resolve(pwd)) in collectionAdd resolves symlinks and
stores /private/tmp/... in the DB. toVirtualPath and --full-path
resolution then fail because the test-side collectionDir path doesn't
match the DB-side path. Fix by calling realpathSync on the test
collectionDir before passing it to CLI commands and assertions.
Filenames with special characters (#, &, spaces, [], (), etc.) now
round-trip correctly through index → search → get → full-path.
Root cause: reindexCollection() called handelize() on the relative path
before storing it in documents.path, turning
'# Meeting - 234232 3432 __ 5.md' → 'Meeting-234232-3432-5.md'
This broke all downstream operations that needed to reconstruct the
real filesystem path from the DB record.
Changes:
- Remove handelize() from reindexCollection() in store.ts (index time)
- Remove handelize() from update command path in cli/qmd.ts
- findOrMigrateLegacyDocument now tries both raw path and handalized
variant so existing indexes auto-migrate on next qmd update
- resolveVirtualPath, toVirtualPath, detectCollectionFromPath all work
correctly once the DB stores literal paths
Tests (test/path-fidelity.test.ts — 10/10):
- Store level: DB contains literal paths, not handalized slugs
- toVirtualPath returns non-null for crazy-named files
- (1) search --json file field shows literal path
- (2) get --full-path resolves to a real on-disk path
- (3) get <actual-fs-path> finds the document
- (3b) subdir file with crazy name also works
- (4) ls shows literal paths
- (5) search docid can be fetched back
- Normal filenames still work (regression)
- Migration: qmd update on handalized index rewrites paths to literal
The libggml-metal static destructor asserts on a non-empty residency-set
collection during __cxa_finalize_ranges, dumping a multi-kB GGML backtrace
after successful output (ggml-org/llama.cpp#22593, one-line fix open as
PR #22595). The assertion only trips when process.exit() skips Node's
beforeExit hook — which is exactly the hook node-llama-cpp registers to
auto-dispose its native handles.
Primary fix: finishSuccessfulCliCommand now sets process.exitCode = 0
and returns instead of calling process.exit(0). The event loop drains,
beforeExit fires, native Metal resources tear down in order, and the
process exits cleanly even without the workaround env var.
Defense-in-depth retained: bin/qmd and scripts/test-all.mjs still export
GGML_METAL_NO_RESIDENCY=1 on darwin for error paths and tests that
terminate via process.exit(). Opt back in with QMD_METAL_KEEP_RESIDENCY=1.
Also: correct upstream issue refs (was #17869 → now #22593/#22595).
Add scripts/repro-metal-rsets-crash.mjs minimal reproduction.
The libggml-metal static device destructor asserts on a non-empty
residency set during libc `exit()` → `__cxa_finalize_ranges`
(ggml-org/llama.cpp#17869). The residency set's 180 s keep_alive timer
hasn't expired by exit, so `GGML_ASSERT([rsets->data count] == 0)`
fails and `ggml_abort` dumps a multi-kB backtrace to stderr after the
user-visible output. Every llama-using CLI command (`query`,
`vsearch`, `embed`) was affected, plus the `bun test` runner.
No JS-side dispose path can prevent it: the static destructor runs
after every JS-reachable cleanup, and Node's `reallyExit` calls libc
`exit()` not `_exit()` (verified in node/src/api/environment.cc),
so it does NOT skip C++ static destructors as we'd assumed.
The actual fix is to disable residency sets via
`GGML_METAL_NO_RESIDENCY=1` before the native binding loads. For
QMD's short-lived CLI workflow there's no measurable cost
(benchmarked: identical wall time with and without on M3 Pro).
Three propagation points are needed:
- `bin/qmd` exports the env var before spawning node/bun. This
covers all production CLI invocations.
- `src/test-preload.ts` mirrors the launcher for `bun test` runs.
Bun does NOT sync `process.env` mutations to libc `setenv()`
(verified empirically — Node does, via uv_os_setenv), so on Bun we
reach for `bun:ffi` to call `setenv()` directly. vitest forks
per-test-file so its parent never loads the binding.
- `qmd doctor` reports the mitigation state via the new
`isDarwinMetalMitigationActive()` predicate so users can verify it
in their environment.
Opt back in with `QMD_METAL_KEEP_RESIDENCY=1` (long-lived qmd
processes, MCP daemon hot reload, upstream fix triage). The old
`QMD_DISABLE_DARWIN_QUERY_JSON_SAFE_EXIT` is removed — its per-command
bypass mechanism didn't actually work on Node (it called
`process.reallyExit` which goes through libc exit) and is fully
replaced by the launcher env var.
Removed the old broken `installDarwinExitGuard()` mechanism from
LlamaCpp; kept the function name as a no-op shim for back-compat.
Source-mode runner selection now mirrors the dist-mode 'npm priority' rule:
if both package-lock.json and bun.lock are present in the package root,
use Node + tsx instead of Bun. pnpm/npm installs ship Node-ABI native
modules (better-sqlite3, sqlite-vec), and routing through Bun produces
ABI mismatches.
This also fixes pnpm-global installs, which copy the entire working tree
(including .git and bun.lock) into <prefix>/node_modules/@tobilu/qmd/.
The old logic saw .git + bun.lock + bun-on-PATH and routed to Bun
against the Node-installed native modules.
Adds a regression test covering the both-lockfiles source-checkout case.
--full-path now ./-prefixes any path that resolves under $PWD, both for
search/query results and for get/multi-get headers. This makes the
output unambiguously a filesystem path — a bare 'notes/foo.md' could be
misread as a collection-relative qmd:// fragment, but './notes/foo.md'
cannot. Absolute realpaths (when the file is outside $PWD) are
unchanged. Extracted as renderFullPath() and reused across the three
call sites so the policy stays consistent.
New --format <kind> flag selects output format for search/query and
multi-get (cli|json|csv|md|xml|files). The legacy boolean aliases
(--json/--csv/--md/--xml/--files) still work for back-compat but are
removed from --help; the skill is updated to use --format.
ANSI colors and OSC 8 hyperlinks are already gated on process.stdout
.isTTY, so piped/agentic invocations get clean plain-text output with
no escape sequences. Verified via od -c on a piped 'qmd search' run.
`qmd://` URIs remain the default identifier in search and query output
(across all formats: cli, --json, --md, --csv, --xml, --files). The
default CLI view now consistently prints the full qmd:// URI as the
visible label so it can be piped straight into `qmd get`, and --md
output gains a **file:** line for the same reason.
--full-path (already on get/multi-get) now also applies to search and
query: the per-result label becomes the file's on-disk path — relative
to $PWD when the file is in a subfolder of the current directory,
absolute realpath otherwise — and the per-result #docid is dropped
because the path is the identifier. Falls back to qmd:// when the file
is no longer resolvable on disk.
Also locks in @@ -line,count @@ header arithmetic with a regression test
that mirrors the user-reported 77-line / '1 before, 72 after' scenario.
- Make structured `qmd query` with intent:/lex:/vec:/hyde: the default
search mode, and emphasize that the caller authors the expansion
rather than leaning on the built-in query-expansion model.
- Tell the caller to cite the #docid and exact line numbers now
printed by get/multi-get, and to slice files with the :from:count
suffix or --from/-l instead of piping through sed/head/tail.
- Document --full-path for handing the on-disk path to editor tools.
- Bump skill version to 2.2.0 and record the behavior changes under
## [Unreleased] in CHANGELOG.md.
- Update the package smoke test that pinned the old 'structured
queries' wording to match the new, more specific intro phrasing.
Redesign the get/multi-get retrieval surface so callers can cite what
they retrieved and request follow-up slices without piping through sed:
- Output is line-numbered by default; opt out with --no-line-numbers.
- Header always identifies the document by qmd:// path + #docid. The
MCP get/multi_get tools default lineNumbers=true to match.
- qmd get and the MCP get tool accept a :from:count suffix on a path
or docid (e.g. '#abc123:120:40' reads 40 lines from line 120).
Explicit --from/-l flags still override the suffix.
- qmd multi-get now includes #docid in every output format (--md,
--json, --csv, --xml, --files, default CLI), matching qmd search.
- New --full-path flag swaps the qmd:// + docid header for the
document's on-disk path (handy for piping into Read/Edit/editors);
falls back to the canonical header when the file no longer exists.
The query tool description tells agents to compute fromLine = line - 20
for context around a hit. For hits in lines 1 through 20 that yields a
negative fromLine, which propagated unchanged through:
MCP get handler -> store.getDocumentBody -> Array.prototype.slice
A negative slice start offsets from the end of the array rather than
clamping to the beginning, so a top-of-file hit on a long document
returned an empty string and on a short document returned content from
the wrong region (e.g. lines 11-30 of a 30-line file in response to a
request for the head of the document). The lineNumbers branch was the
same shape: addLineNumbers(text, -19) emitted "-19:", "-18:" prefixes.
Same buggy slice lived in the CLI getDocument path independently.
Fix in three layers, plus the docstring:
- src/mcp/server.ts: clamp parsedFromLine to >= 1 after parsing input
args and the :line suffix, before it reaches getDocumentBody and
addLineNumbers. Also tighten the query tool's recommendation to
`fromLine = max(1, line - 20)` so following the docstring literally
produces a valid value.
- src/cli/qmd.ts: same clamp on the CLI getDocument fromLine after
the colon-suffix parse.
- src/store.ts: defensive Math.max(0, ...) on the slice start in
getDocumentBody so SDK callers and any future entry points are
protected without relying on every caller remembering to clamp.
- test/store.test.ts: regression test on getDocumentBody with
fromLine = -19 returns the head of the document, not the tail.
- test/cli.test.ts: regression test on `qmd get --from -19` matches
the no-flag baseline (head of document).
The MCP `query` tool, HTTP `/query` endpoint, and CLI `qmd query`
all returned chunk-local line numbers in their snippet output, so
the line could not be passed back to `qmd_get` as `fromLine`
without an out-of-band lookup. Pass the full document body plus
`bestChunkPos` to `extractSnippet` instead of the chunk text alone
so it can compute absolute line offsets while still scoping the
keyword scan to the reranker-chosen chunk window (preserves #149).
Also restores documented behavior of `qmd query --full`, which was
emitting the best chunk (~3.6KB max) instead of the full document.
extractSnippet now also falls back to a full-body scan when given a
chunkPos but the chunk window contains no positive matches. The
upstream chunk selector leaves bestIdx=0 as its initialization
default whenever scoring fails to find a winner (e.g. queryTerms
filtered to empty by the length>2 guard, or semantic-only matches
with no lex overlap), so an unconditional chunk-scoped scan would
land on chunk 0 instead of where the actual match lives.
- src/mcp/server.ts: SearchResultItem gains `line: number`; both MCP
and HTTP `/query` handlers populate it
- src/cli/qmd.ts: OutputRow.body now sources from r.body
- src/store.ts: extractSnippet falls back to full-body scan when
chunk-scoped pass finds no positive match
- test/mcp.test.ts: new fixture asserts absolute line 301 for a
marker placed past the first chunk boundary
- test/store.test.ts: regression test for the bestScore<=0 fallback
