c162ed1319
The libggml-metal static device destructor asserts on a non-empty residency set during libc `exit()` → `__cxa_finalize_ranges` (ggml-org/llama.cpp#17869). The residency set's 180 s keep_alive timer hasn't expired by exit, so `GGML_ASSERT([rsets->data count] == 0)` fails and `ggml_abort` dumps a multi-kB backtrace to stderr after the user-visible output. Every llama-using CLI command (`query`, `vsearch`, `embed`) was affected, plus the `bun test` runner. No JS-side dispose path can prevent it: the static destructor runs after every JS-reachable cleanup, and Node's `reallyExit` calls libc `exit()` not `_exit()` (verified in node/src/api/environment.cc), so it does NOT skip C++ static destructors as we'd assumed. The actual fix is to disable residency sets via `GGML_METAL_NO_RESIDENCY=1` before the native binding loads. For QMD's short-lived CLI workflow there's no measurable cost (benchmarked: identical wall time with and without on M3 Pro). Three propagation points are needed: - `bin/qmd` exports the env var before spawning node/bun. This covers all production CLI invocations. - `src/test-preload.ts` mirrors the launcher for `bun test` runs. Bun does NOT sync `process.env` mutations to libc `setenv()` (verified empirically — Node does, via uv_os_setenv), so on Bun we reach for `bun:ffi` to call `setenv()` directly. vitest forks per-test-file so its parent never loads the binding. - `qmd doctor` reports the mitigation state via the new `isDarwinMetalMitigationActive()` predicate so users can verify it in their environment. Opt back in with `QMD_METAL_KEEP_RESIDENCY=1` (long-lived qmd processes, MCP daemon hot reload, upstream fix triage). The old `QMD_DISABLE_DARWIN_QUERY_JSON_SAFE_EXIT` is removed — its per-command bypass mechanism didn't actually work on Node (it called `process.reallyExit` which goes through libc exit) and is fully replaced by the launcher env var. Removed the old broken `installDarwinExitGuard()` mechanism from LlamaCpp; kept the function name as a no-op shim for back-compat.
50 lines
2.0 KiB
TypeScript
50 lines
2.0 KiB
TypeScript
/**
|
|
* Test preload file to ensure proper cleanup of native resources.
|
|
*
|
|
* Uses bun:test afterAll to properly dispose of llama.cpp Metal
|
|
* resources before the process exits, avoiding GGML_ASSERT failures.
|
|
*/
|
|
|
|
// Mirror bin/qmd's darwin Metal residency mitigation so `bun test` and
|
|
// `vitest` runs exit cleanly. The test runners load node-llama-cpp directly
|
|
// without going through the launcher, so the libggml-metal static destructor
|
|
// asserts on a non-empty residency set during __cxa_finalize_ranges and dumps
|
|
// a multi-kB backtrace at process exit (ggml-org/llama.cpp#17869). Opt back
|
|
// in with QMD_METAL_KEEP_RESIDENCY=1 if you're triaging the upstream Metal
|
|
// teardown bug.
|
|
//
|
|
// Two-step propagation, because:
|
|
// - Native code in libggml-metal reads via C getenv() at module load time.
|
|
// - Node syncs process.env mutations to libc via uv_os_setenv automatically.
|
|
// - Bun does NOT — `process.env.X = "1"` only updates the JS-level object,
|
|
// so getenv() in the C++ binding still sees nothing (verified empirically
|
|
// with bun:ffi). We have to call setenv() ourselves on Bun.
|
|
if (process.platform === "darwin" && process.env.QMD_METAL_KEEP_RESIDENCY !== "1") {
|
|
process.env.GGML_METAL_NO_RESIDENCY = process.env.GGML_METAL_NO_RESIDENCY || "1";
|
|
|
|
if (typeof (globalThis as { Bun?: unknown }).Bun !== "undefined") {
|
|
try {
|
|
const { dlopen, FFIType, suffix } = await import("bun:ffi");
|
|
const libc = dlopen(`libSystem.${suffix}`, {
|
|
setenv: { args: [FFIType.cstring, FFIType.cstring, FFIType.i32], returns: FFIType.i32 },
|
|
});
|
|
libc.symbols.setenv(
|
|
Buffer.from("GGML_METAL_NO_RESIDENCY\0", "utf8"),
|
|
Buffer.from("1\0", "utf8"),
|
|
1,
|
|
);
|
|
} catch {
|
|
// FFI unavailable on this Bun build — the backtrace dump at exit is
|
|
// cosmetic; tests still pass.
|
|
}
|
|
}
|
|
}
|
|
|
|
import { afterAll } from "bun:test";
|
|
import { disposeDefaultLlamaCpp } from "./llm";
|
|
|
|
// Global afterAll runs after all test files complete
|
|
afterAll(async () => {
|
|
await disposeDefaultLlamaCpp();
|
|
});
|