The libggml-metal static destructor asserts on a non-empty residency-set
collection during __cxa_finalize_ranges, dumping a multi-kB GGML backtrace
after successful output (ggml-org/llama.cpp#22593, one-line fix open as
PR #22595). The assertion only trips when process.exit() skips Node's
beforeExit hook — which is exactly the hook node-llama-cpp registers to
auto-dispose its native handles.
Primary fix: finishSuccessfulCliCommand now sets process.exitCode = 0
and returns instead of calling process.exit(0). The event loop drains,
beforeExit fires, native Metal resources tear down in order, and the
process exits cleanly even without the workaround env var.
Defense-in-depth retained: bin/qmd and scripts/test-all.mjs still export
GGML_METAL_NO_RESIDENCY=1 on darwin for error paths and tests that
terminate via process.exit(). Opt back in with QMD_METAL_KEEP_RESIDENCY=1.
Also: correct upstream issue refs (was #17869 → now #22593/#22595).
Add scripts/repro-metal-rsets-crash.mjs minimal reproduction.
The libggml-metal static device destructor asserts on a non-empty
residency set during libc `exit()` → `__cxa_finalize_ranges`
(ggml-org/llama.cpp#17869). The residency set's 180 s keep_alive timer
hasn't expired by exit, so `GGML_ASSERT([rsets->data count] == 0)`
fails and `ggml_abort` dumps a multi-kB backtrace to stderr after the
user-visible output. Every llama-using CLI command (`query`,
`vsearch`, `embed`) was affected, plus the `bun test` runner.
No JS-side dispose path can prevent it: the static destructor runs
after every JS-reachable cleanup, and Node's `reallyExit` calls libc
`exit()` not `_exit()` (verified in node/src/api/environment.cc),
so it does NOT skip C++ static destructors as we'd assumed.
The actual fix is to disable residency sets via
`GGML_METAL_NO_RESIDENCY=1` before the native binding loads. For
QMD's short-lived CLI workflow there's no measurable cost
(benchmarked: identical wall time with and without on M3 Pro).
Three propagation points are needed:
- `bin/qmd` exports the env var before spawning node/bun. This
covers all production CLI invocations.
- `src/test-preload.ts` mirrors the launcher for `bun test` runs.
Bun does NOT sync `process.env` mutations to libc `setenv()`
(verified empirically — Node does, via uv_os_setenv), so on Bun we
reach for `bun:ffi` to call `setenv()` directly. vitest forks
per-test-file so its parent never loads the binding.
- `qmd doctor` reports the mitigation state via the new
`isDarwinMetalMitigationActive()` predicate so users can verify it
in their environment.
Opt back in with `QMD_METAL_KEEP_RESIDENCY=1` (long-lived qmd
processes, MCP daemon hot reload, upstream fix triage). The old
`QMD_DISABLE_DARWIN_QUERY_JSON_SAFE_EXIT` is removed — its per-command
bypass mechanism didn't actually work on Node (it called
`process.reallyExit` which goes through libc exit) and is fully
replaced by the launcher env var.
Removed the old broken `installDarwinExitGuard()` mechanism from
LlamaCpp; kept the function name as a no-op shim for back-compat.
Source-mode runner selection now mirrors the dist-mode 'npm priority' rule:
if both package-lock.json and bun.lock are present in the package root,
use Node + tsx instead of Bun. pnpm/npm installs ship Node-ABI native
modules (better-sqlite3, sqlite-vec), and routing through Bun produces
ABI mismatches.
This also fixes pnpm-global installs, which copy the entire working tree
(including .git and bun.lock) into <prefix>/node_modules/@tobilu/qmd/.
The old logic saw .git + bun.lock + bun-on-PATH and routed to Bun
against the Node-installed native modules.
Adds a regression test covering the both-lockfiles source-checkout case.
The bin/qmd wrapper checks for bun.lock to select the runtime, but since
bun.lock is committed to the repo, source builds using npm install are
incorrectly routed to Bun — causing native module ABI mismatches (#381)
and sqlite-vec crashes (#380).
Add package-lock.json as a higher-priority signal: if it exists, npm
installed the dependencies and Node should be used. Also fix
cleanupOrphanedVectors() to use the existing isSqliteVecAvailable()
guard instead of checking sqlite_master, which can report the virtual
table even when the vec0 module isn't loaded.
Fixes#381, fixes#380
Continuation of #362 (runtime detection false positives)
When Bun is installed on the system but QMD was installed via npm,
$BUN_INSTALL is always set (typically to ~/.bun), causing the launcher
to incorrectly run QMD under Bun. This leads to ABI mismatches with
native modules (better-sqlite3, sqlite-vec) that were compiled for Node,
breaking vector operations with "no such module: vec0".
Only check for bun.lock/bun.lockb files, which reliably indicate that
QMD was actually installed with Bun.
Fixes#361
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Bump better-sqlite3 from ^11 to ^12.4.5 for Node 25 support (prebuilds
+ V8 API compat). Closes#257.
- Add bin/qmd shell wrapper that detects bun vs node install and execs
with the matching runtime, preventing native module ABI mismatches
when installed via bun. Closes#319.
