feat: add intent parameter for query disambiguation

Add optional `intent` parameter that steers query expansion, reranking,
chunk selection, and snippet extraction without searching on its own.

When a query like "performance" is ambiguous (web-perf vs team health vs
fitness), intent provides background context that disambiguates results
across all pipeline stages:

- expandQuery: includes intent in LLM prompt ("Query intent: {intent}")
- rerank: prepends intent to rerank query for Qwen3-Reranker
- chunk selection: intent terms scored at 0.5x weight vs query terms
- snippet extraction: intent terms scored at 0.3x weight
- strong-signal bypass: disabled when intent provided

Available via CLI (--intent flag or intent: line in query documents),
MCP (intent field on query tool), and programmatic API.

Adapted from PR #180 (thanks @vyalamar).
This commit is contained in:
Tobi Lutke 2026-03-07 19:27:29 -04:00
parent b838f74c8c
commit ad38c1f698
No known key found for this signature in database
8 changed files with 721 additions and 43 deletions

View File

@ -8,7 +8,8 @@ QMD queries are structured documents with typed sub-queries. Each line specifies
query = expand_query | query_document ;
expand_query = text | explicit_expand ;
explicit_expand= "expand:" text ;
query_document = { typed_line } ;
query_document = [ intent_line ] { typed_line } ;
intent_line = "intent:" text newline ;
typed_line = type ":" text newline ;
type = "lex" | "vec" | "hyde" ;
text = quoted_phrase | plain_text ;
@ -101,11 +102,28 @@ error handling best practices
Both forms call the local query expansion model, which generates lex, vec, and hyde variations automatically.
## Intent
An optional `intent:` line provides background context to disambiguate ambiguous queries. It steers query expansion, reranking, and snippet extraction but does not search on its own.
- At most one `intent:` line per query document
- `intent:` cannot appear alone — at least one `lex:`, `vec:`, or `hyde:` line is required
- Intent is also available via the `--intent` CLI flag or MCP `intent` parameter
```
intent: web page load times and Core Web Vitals
lex: performance
vec: how to improve performance
```
Without intent, "performance" is ambiguous (web-perf? team health? fitness?). With intent, the search pipeline preferentially selects and ranks web-performance content.
## Constraints
- Top-level query must be either a standalone expand query or a multi-line document
- Query documents allow only `lex`, `vec`, and `hyde` typed lines (no `expand:` inside)
- Query documents allow only `lex`, `vec`, `hyde`, and `intent` typed lines (no `expand:` inside)
- `lex` syntax (`-term`, `"phrase"`) only works in lex queries
- At most one `intent:` line per query document; cannot appear alone
- Empty lines are ignored
- Leading/trailing whitespace is trimmed
@ -132,6 +150,17 @@ Or structured format:
}
```
With intent:
```json
{
"searches": [
{ "type": "lex", "query": "performance" }
],
"intent": "web page load times and Core Web Vitals"
}
```
## CLI
```bash
@ -143,4 +172,10 @@ qmd query $'lex: auth token\nvec: how does authentication work'
# Structured
qmd query $'lex: keywords\nvec: question\nhyde: hypothetical answer...'
# With intent (inline)
qmd query $'intent: web performance and latency\nlex: performance\nvec: how to improve performance'
# With intent (flag)
qmd query --intent "web performance and latency" "performance"
```

View File

@ -60,6 +60,21 @@ Local search engine for markdown content.
- Lets the local LLM generate lex/vec/hyde variations
- Do not mix `expand:` with other typed lines — it's either a standalone expand query or a full query document
### Intent (Disambiguation)
When a query term is ambiguous, add `intent` to steer results:
```json
{
"searches": [
{ "type": "lex", "query": "performance" }
],
"intent": "web page load times and Core Web Vitals"
}
```
Intent affects expansion, reranking, chunk selection, and snippet extraction. It does not search on its own — it's a steering signal that disambiguates queries like "performance" (web-perf vs team health vs fitness).
### Combining Types
| Goal | Approach |
@ -68,6 +83,7 @@ Local search engine for markdown content.
| Don't know vocabulary | Use a single-line query (implicit `expand:`) or `vec` |
| Best recall | `lex` + `vec` |
| Complex topic | `lex` + `vec` + `hyde` |
| Ambiguous query | Add `intent` to any combination above |
First query gets 2x weight in fusion — put your best guess first.
@ -104,6 +120,7 @@ Omit to search all collections.
qmd query "question" # Auto-expand + rerank
qmd query $'lex: X\nvec: Y' # Structured
qmd query $'expand: question' # Explicit expand
qmd query --json --explain "q" # Show score traces (RRF + rerank blend)
qmd search "keywords" # BM25 only (no LLM)
qmd get "#abc123" # By docid
qmd multi-get "journals/2026-*.md" -l 40 # Batch pull snippets by glob

View File

@ -40,6 +40,7 @@ export type FormatOptions = {
query?: string; // Query for snippet extraction and highlighting
useColor?: boolean; // Enable terminal colors (default: false for non-CLI)
lineNumbers?: boolean;// Add line numbers to output
intent?: string; // Domain intent for snippet extraction disambiguation
};
// =============================================================================
@ -101,7 +102,7 @@ export function searchResultsToJson(
const output = results.map(row => {
const bodyStr = row.body || "";
let body = opts.full ? bodyStr : undefined;
let snippet = !opts.full ? extractSnippet(bodyStr, query, 300, row.chunkPos).snippet : undefined;
let snippet = !opts.full ? extractSnippet(bodyStr, query, 300, row.chunkPos, undefined, opts.intent).snippet : undefined;
if (opts.lineNumbers) {
if (body) body = addLineNumbers(body);
@ -132,7 +133,7 @@ export function searchResultsToCsv(
const header = "docid,score,file,title,context,line,snippet";
const rows = results.map(row => {
const bodyStr = row.body || "";
const { line, snippet } = extractSnippet(bodyStr, query, 500, row.chunkPos);
const { line, snippet } = extractSnippet(bodyStr, query, 500, row.chunkPos, undefined, opts.intent);
let content = opts.full ? bodyStr : snippet;
if (opts.lineNumbers && content) {
content = addLineNumbers(content);
@ -175,7 +176,7 @@ export function searchResultsToMarkdown(
if (opts.full) {
content = bodyStr;
} else {
content = extractSnippet(bodyStr, query, 500, row.chunkPos).snippet;
content = extractSnippet(bodyStr, query, 500, row.chunkPos, undefined, opts.intent).snippet;
}
if (opts.lineNumbers) {
content = addLineNumbers(content);
@ -196,7 +197,7 @@ export function searchResultsToXml(
const items = results.map(row => {
const titleAttr = row.title ? ` title="${escapeXml(row.title)}"` : "";
const bodyStr = row.body || "";
let content = opts.full ? bodyStr : extractSnippet(bodyStr, query, 500, row.chunkPos).snippet;
let content = opts.full ? bodyStr : extractSnippet(bodyStr, query, 500, row.chunkPos, undefined, opts.intent).snippet;
if (opts.lineNumbers) {
content = addLineNumbers(content);
}

View File

@ -970,7 +970,7 @@ export class LlamaCpp implements LLM {
// High-level abstractions
// ==========================================================================
async expandQuery(query: string, options: { context?: string, includeLexical?: boolean } = {}): Promise<Queryable[]> {
async expandQuery(query: string, options: { context?: string, includeLexical?: boolean, intent?: string } = {}): Promise<Queryable[]> {
// Ping activity at start to keep models alive during this operation
this.touchActivity();
@ -989,7 +989,10 @@ export class LlamaCpp implements LLM {
`
});
const prompt = `/no_think Expand this search query: ${query}`;
const intent = options.intent;
const prompt = intent
? `/no_think Expand this search query: ${query}\nQuery intent: ${intent}`
: `/no_think Expand this search query: ${query}`;
// Create a bounded context for expansion to prevent large default VRAM allocations.
const genContext = await this.generateModel!.createContext({

View File

@ -126,10 +126,13 @@ function buildInstructions(store: Store): string {
lines.push(" - type:'vec' — semantic vector search (meaning-based)");
lines.push(" - type:'hyde' — hypothetical document (write what the answer looks like)");
lines.push("");
lines.push(" Always provide `intent` on every search call to disambiguate and improve snippets.");
lines.push("");
lines.push("Examples:");
lines.push(" Quick keyword lookup: [{type:'lex', query:'error handling'}]");
lines.push(" Semantic search: [{type:'vec', query:'how to handle errors gracefully'}]");
lines.push(" Best results: [{type:'lex', query:'error'}, {type:'vec', query:'error handling best practices'}]");
lines.push(" With intent: searches=[{type:'lex', query:'performance'}], intent='web page load times'");
// --- Retrieval workflow ---
lines.push("");
@ -312,9 +315,12 @@ Intent-aware lex (C++ performance, not sports):
"Maximum candidates to rerank (default: 40, lower = faster but may miss results)"
),
collections: z.array(z.string()).optional().describe("Filter to collections (OR match)"),
intent: z.string().optional().describe(
"Background context to disambiguate the query. Example: query='performance', intent='web page load times and Core Web Vitals'. Does not search on its own."
),
},
},
async ({ searches, limit, minScore, candidateLimit, collections }) => {
async ({ searches, limit, minScore, candidateLimit, collections, intent }) => {
// Map to internal format
const subSearches: StructuredSubSearch[] = searches.map(s => ({
type: s.type,
@ -329,6 +335,7 @@ Intent-aware lex (C++ performance, not sports):
limit,
minScore,
candidateLimit,
intent,
});
// Use first lex or vec query for snippet extraction
@ -337,7 +344,7 @@ Intent-aware lex (C++ performance, not sports):
|| searches[0]?.query || "";
const filtered: SearchResultItem[] = results.map(r => {
const { line, snippet } = extractSnippet(r.bestChunk, primaryQuery, 300);
const { line, snippet } = extractSnippet(r.bestChunk, primaryQuery, 300, undefined, undefined, intent);
return {
docid: `#${r.docid}`,
file: r.displayPath,

View File

@ -1771,6 +1771,7 @@ type OutputOptions = {
explain?: boolean; // Include retrieval score traces (query only)
context?: string; // Optional context for query expansion
candidateLimit?: number; // Max candidates to rerank (default: 40)
intent?: string; // Domain intent for disambiguation
};
// Highlight query terms in text (skip short words < 3 chars)
@ -1863,7 +1864,7 @@ function outputResults(results: OutputRow[], query: string, opts: OutputOptions)
const output = filtered.map(row => {
const docid = row.docid || (row.hash ? row.hash.slice(0, 6) : undefined);
let body = opts.full ? row.body : undefined;
let snippet = !opts.full ? extractSnippet(row.body, query, 300, row.chunkPos).snippet : undefined;
let snippet = !opts.full ? extractSnippet(row.body, query, 300, row.chunkPos, undefined, opts.intent).snippet : undefined;
if (opts.lineNumbers) {
if (body) body = addLineNumbers(body);
if (snippet) snippet = addLineNumbers(snippet);
@ -1891,7 +1892,7 @@ function outputResults(results: OutputRow[], query: string, opts: OutputOptions)
for (let i = 0; i < filtered.length; i++) {
const row = filtered[i];
if (!row) continue;
const { line, snippet } = extractSnippet(row.body, query, 500, row.chunkPos);
const { line, snippet } = extractSnippet(row.body, query, 500, row.chunkPos, undefined, opts.intent);
const docid = row.docid || (row.hash ? row.hash.slice(0, 6) : undefined);
// Line 1: filepath with docid
@ -1954,7 +1955,7 @@ function outputResults(results: OutputRow[], query: string, opts: OutputOptions)
if (!row) continue;
const heading = row.title || row.displayPath;
const docid = row.docid || (row.hash ? row.hash.slice(0, 6) : undefined);
let content = opts.full ? row.body : extractSnippet(row.body, query, 500, row.chunkPos).snippet;
let content = opts.full ? row.body : extractSnippet(row.body, query, 500, row.chunkPos, undefined, opts.intent).snippet;
if (opts.lineNumbers) {
content = addLineNumbers(content);
}
@ -1967,7 +1968,7 @@ function outputResults(results: OutputRow[], query: string, opts: OutputOptions)
const titleAttr = row.title ? ` title="${row.title.replace(/"/g, '&quot;')}"` : "";
const contextAttr = row.context ? ` context="${row.context.replace(/"/g, '&quot;')}"` : "";
const docid = row.docid || (row.hash ? row.hash.slice(0, 6) : "");
let content = opts.full ? row.body : extractSnippet(row.body, query, 500, row.chunkPos).snippet;
let content = opts.full ? row.body : extractSnippet(row.body, query, 500, row.chunkPos, undefined, opts.intent).snippet;
if (opts.lineNumbers) {
content = addLineNumbers(content);
}
@ -1977,7 +1978,7 @@ function outputResults(results: OutputRow[], query: string, opts: OutputOptions)
// CSV format
console.log("docid,score,file,title,context,line,snippet");
for (const row of filtered) {
const { line, snippet } = extractSnippet(row.body, query, 500, row.chunkPos);
const { line, snippet } = extractSnippet(row.body, query, 500, row.chunkPos, undefined, opts.intent);
let content = opts.full ? row.body : snippet;
if (opts.lineNumbers) {
content = addLineNumbers(content, line);
@ -2036,7 +2037,12 @@ function filterByCollections<T extends { filepath?: string; file?: string }>(res
* "lex: CAP\nvec: consistency" -> [{ type: 'lex', ... }, { type: 'vec', ... }]
* "CAP\nconsistency" -> throws (multiple plain lines)
*/
function parseStructuredQuery(query: string): StructuredSubSearch[] | null {
interface ParsedStructuredQuery {
searches: StructuredSubSearch[];
intent?: string;
}
function parseStructuredQuery(query: string): ParsedStructuredQuery | null {
const rawLines = query.split('\n').map((line, idx) => ({
raw: line,
trimmed: line.trim(),
@ -2047,7 +2053,9 @@ function parseStructuredQuery(query: string): StructuredSubSearch[] | null {
const prefixRe = /^(lex|vec|hyde):\s*/i;
const expandRe = /^expand:\s*/i;
const intentRe = /^intent:\s*/i;
const typed: StructuredSubSearch[] = [];
let intent: string | undefined;
for (const line of rawLines) {
if (expandRe.test(line.trimmed)) {
@ -2061,6 +2069,19 @@ function parseStructuredQuery(query: string): StructuredSubSearch[] | null {
return null; // treat as standalone expand query
}
// Parse intent: lines
if (intentRe.test(line.trimmed)) {
if (intent !== undefined) {
throw new Error(`Line ${line.number}: only one intent: line is allowed per query document.`);
}
const text = line.trimmed.replace(intentRe, '').trim();
if (!text) {
throw new Error(`Line ${line.number}: intent: must include text.`);
}
intent = text;
continue;
}
const match = line.trimmed.match(prefixRe);
if (match) {
const type = match[1]!.toLowerCase() as 'lex' | 'vec' | 'hyde';
@ -2080,10 +2101,15 @@ function parseStructuredQuery(query: string): StructuredSubSearch[] | null {
return null;
}
throw new Error(`Line ${line.number} is missing a lex:/vec:/hyde: prefix. Each line in a query document must start with one.`);
throw new Error(`Line ${line.number} is missing a lex:/vec:/hyde:/intent: prefix. Each line in a query document must start with one.`);
}
return typed.length > 0 ? typed : null;
// intent: alone is not a valid query — must have at least one search
if (intent && typed.length === 0) {
throw new Error('intent: cannot appear alone. Add at least one lex:, vec:, or hyde: line.');
}
return typed.length > 0 ? { searches: typed, intent } : null;
}
function search(query: string, opts: OutputOptions): void {
@ -2152,6 +2178,7 @@ async function vectorSearch(query: string, opts: OutputOptions, _model: string =
collection: singleCollection,
limit: opts.all ? 500 : (opts.limit || 10),
minScore: opts.minScore || 0.3,
intent: opts.intent,
hooks: {
onExpand: (original, expanded) => {
logExpansionTree(original, expanded);
@ -2197,17 +2224,23 @@ async function querySearch(query: string, opts: OutputOptions, _embedModel: stri
checkIndexHealth(store.db);
// Check for structured query syntax (lex:/vec:/hyde: prefixes)
const structuredQueries = parseStructuredQuery(query);
// Check for structured query syntax (lex:/vec:/hyde:/intent: prefixes)
const parsed = parseStructuredQuery(query);
// Intent can come from --intent flag or from intent: line in query document
const intent = opts.intent || parsed?.intent;
await withLLMSession(async () => {
let results;
if (structuredQueries) {
if (parsed) {
const structuredQueries = parsed.searches;
// Structured search — user provided their own query expansions
const typeLabels = structuredQueries.map(s => s.type).join('+');
process.stderr.write(`${c.dim}Structured search: ${structuredQueries.length} queries (${typeLabels})${c.reset}\n`);
if (intent) {
process.stderr.write(`${c.dim}├─ intent: ${intent}${c.reset}\n`);
}
// Log each sub-query
for (const s of structuredQueries) {
let preview = s.query.replace(/\n/g, ' ');
@ -2222,6 +2255,7 @@ async function querySearch(query: string, opts: OutputOptions, _embedModel: stri
minScore: opts.minScore || 0,
candidateLimit: opts.candidateLimit,
explain: !!opts.explain,
intent,
hooks: {
onEmbedStart: (count) => {
process.stderr.write(`${c.dim}Embedding ${count} ${count === 1 ? 'query' : 'queries'}...${c.reset}`);
@ -2247,6 +2281,7 @@ async function querySearch(query: string, opts: OutputOptions, _embedModel: stri
minScore: opts.minScore || 0,
candidateLimit: opts.candidateLimit,
explain: !!opts.explain,
intent,
hooks: {
onStrongSignal: (score) => {
process.stderr.write(`${c.dim}Strong BM25 signal (${score.toFixed(2)}) — skipping expansion${c.reset}\n`);
@ -2293,6 +2328,7 @@ async function querySearch(query: string, opts: OutputOptions, _embedModel: stri
}
// Use first lex/vec query for output context, or original query
const structuredQueries = parsed?.searches;
const displayQuery = structuredQueries
? (structuredQueries.find(s => s.type === 'lex')?.query || structuredQueries.find(s => s.type === 'vec')?.query || query)
: query;
@ -2354,6 +2390,7 @@ function parseCLI() {
"line-numbers": { type: "boolean" }, // add line numbers to output
// Query options
"candidate-limit": { type: "string", short: "C" },
intent: { type: "string" },
// MCP HTTP transport options
http: { type: "boolean" },
daemon: { type: "boolean" },
@ -2393,6 +2430,7 @@ function parseCLI() {
lineNumbers: !!values["line-numbers"],
candidateLimit: values["candidate-limit"] ? parseInt(String(values["candidate-limit"]), 10) : undefined,
explain: !!values.explain,
intent: values.intent as string | undefined,
};
return {
@ -2457,7 +2495,8 @@ function showHelp(): void {
`query = expand_query | query_document ;`,
`expand_query = text | explicit_expand ;`,
`explicit_expand= "expand:" text ;`,
`query_document = { typed_line } ;`,
`query_document = [ intent_line ] { typed_line } ;`,
`intent_line = "intent:" text newline ;`,
`typed_line = type ":" text newline ;`,
`type = "lex" | "vec" | "hyde" ;`,
`text = quoted_phrase | plain_text ;`,

View File

@ -892,8 +892,8 @@ export function createStore(dbPath?: string): Store {
searchVec: (query: string, model: string, limit?: number, collectionName?: string, session?: ILLMSession, precomputedEmbedding?: number[]) => searchVec(db, query, model, limit, collectionName, session, precomputedEmbedding),
// Query expansion & reranking
expandQuery: (query: string, model?: string) => expandQuery(query, model, db),
rerank: (query: string, documents: { file: string; text: string }[], model?: string) => rerank(query, documents, model, db),
expandQuery: (query: string, model?: string, intent?: string) => expandQuery(query, model, db, intent),
rerank: (query: string, documents: { file: string; text: string }[], model?: string, intent?: string) => rerank(query, documents, model, db, intent),
// Document retrieval
findDocument: (filename: string, options?: { includeBody?: boolean }) => findDocument(db, filename, options),
@ -2346,9 +2346,9 @@ export function insertEmbedding(
// Query expansion
// =============================================================================
export async function expandQuery(query: string, model: string = DEFAULT_QUERY_MODEL, db: Database): Promise<ExpandedQuery[]> {
export async function expandQuery(query: string, model: string = DEFAULT_QUERY_MODEL, db: Database, intent?: string): Promise<ExpandedQuery[]> {
// Check cache first — stored as JSON preserving types
const cacheKey = getCacheKey("expandQuery", { query, model });
const cacheKey = getCacheKey("expandQuery", { query, model, ...(intent && { intent }) });
const cached = getCachedResult(db, cacheKey);
if (cached) {
try {
@ -2360,7 +2360,7 @@ export async function expandQuery(query: string, model: string = DEFAULT_QUERY_M
const llm = getDefaultLlamaCpp();
// Note: LlamaCpp uses hardcoded model, model parameter is ignored
const results = await llm.expandQuery(query);
const results = await llm.expandQuery(query, { intent });
// Map Queryable[] → ExpandedQuery[] (same shape, decoupled from llm.ts internals).
// Filter out entries that duplicate the original query text.
@ -2379,7 +2379,10 @@ export async function expandQuery(query: string, model: string = DEFAULT_QUERY_M
// Reranking
// =============================================================================
export async function rerank(query: string, documents: { file: string; text: string }[], model: string = DEFAULT_RERANK_MODEL, db: Database): Promise<{ file: string; score: number }[]> {
export async function rerank(query: string, documents: { file: string; text: string }[], model: string = DEFAULT_RERANK_MODEL, db: Database, intent?: string): Promise<{ file: string; score: number }[]> {
// Prepend intent to rerank query so the reranker scores with domain context
const rerankQuery = intent ? `${intent}\n\n${query}` : query;
const cachedResults: Map<string, number> = new Map();
const uncachedDocsByChunk: Map<string, RerankDocument> = new Map();
@ -2389,7 +2392,7 @@ export async function rerank(query: string, documents: { file: string; text: str
// File path is excluded from the new cache key because the reranker score
// depends on the chunk content, not where it came from.
for (const doc of documents) {
const cacheKey = getCacheKey("rerank", { query, model, chunk: doc.text });
const cacheKey = getCacheKey("rerank", { query: rerankQuery, model, chunk: doc.text });
const legacyCacheKey = getCacheKey("rerank", { query, file: doc.file, model, chunk: doc.text });
const cached = getCachedResult(db, cacheKey) ?? getCachedResult(db, legacyCacheKey);
if (cached !== null) {
@ -2403,13 +2406,13 @@ export async function rerank(query: string, documents: { file: string; text: str
if (uncachedDocsByChunk.size > 0) {
const llm = getDefaultLlamaCpp();
const uncachedDocs = [...uncachedDocsByChunk.values()];
const rerankResult = await llm.rerank(query, uncachedDocs, { model });
const rerankResult = await llm.rerank(rerankQuery, uncachedDocs, { model });
// Cache results by chunk text so identical chunks across files are scored once.
const textByFile = new Map(uncachedDocs.map(d => [d.file, d.text]));
for (const result of rerankResult.results) {
const chunk = textByFile.get(result.file) || "";
const cacheKey = getCacheKey("rerank", { query, model, chunk });
const cacheKey = getCacheKey("rerank", { query: rerankQuery, model, chunk });
setCachedResult(db, cacheKey, result.score.toString());
cachedResults.set(chunk, result.score);
}
@ -2885,7 +2888,45 @@ export type SnippetResult = {
snippetLines: number; // Number of lines in snippet
};
export function extractSnippet(body: string, query: string, maxLen = 500, chunkPos?: number, chunkLen?: number): SnippetResult {
/** Weight for intent terms relative to query terms (1.0) in snippet scoring */
export const INTENT_WEIGHT_SNIPPET = 0.3;
/** Weight for intent terms relative to query terms (1.0) in chunk selection */
export const INTENT_WEIGHT_CHUNK = 0.5;
// Common stop words filtered from intent strings before tokenization.
// Seeded from finetune/reward.py KEY_TERM_STOPWORDS, extended with common
// 2-3 char function words so the length threshold can drop to >1 and let
// short domain terms (API, SQL, LLM, CPU, CDN, …) survive.
const INTENT_STOP_WORDS = new Set([
// 2-char function words
"am", "an", "as", "at", "be", "by", "do", "he", "if",
"in", "is", "it", "me", "my", "no", "of", "on", "or", "so",
"to", "up", "us", "we",
// 3-char function words
"all", "and", "any", "are", "but", "can", "did", "for", "get",
"has", "her", "him", "his", "how", "its", "let", "may", "not",
"our", "out", "the", "too", "was", "who", "why", "you",
// 4+ char common words
"also", "does", "find", "from", "have", "into", "more", "need",
"show", "some", "tell", "that", "them", "this", "want", "what",
"when", "will", "with", "your",
// Search-context noise
"about", "looking", "notes", "search", "where", "which",
]);
/**
* Extract meaningful terms from an intent string, filtering stop words and punctuation.
* Uses Unicode-aware punctuation stripping so domain terms like "API" survive.
* Returns lowercase terms suitable for text matching.
*/
export function extractIntentTerms(intent: string): string[] {
return intent.toLowerCase().split(/\s+/)
.map(t => t.replace(/^[^\p{L}\p{N}]+|[^\p{L}\p{N}]+$/gu, ""))
.filter(t => t.length > 1 && !INTENT_STOP_WORDS.has(t));
}
export function extractSnippet(body: string, query: string, maxLen = 500, chunkPos?: number, chunkLen?: number, intent?: string): SnippetResult {
const totalLines = body.split('\n').length;
let searchBody = body;
let lineOffset = 0;
@ -2904,13 +2945,17 @@ export function extractSnippet(body: string, query: string, maxLen = 500, chunkP
const lines = searchBody.split('\n');
const queryTerms = query.toLowerCase().split(/\s+/).filter(t => t.length > 0);
const intentTerms = intent ? extractIntentTerms(intent) : [];
let bestLine = 0, bestScore = -1;
for (let i = 0; i < lines.length; i++) {
const lineLower = (lines[i] ?? "").toLowerCase();
let score = 0;
for (const term of queryTerms) {
if (lineLower.includes(term)) score++;
if (lineLower.includes(term)) score += 1.0;
}
for (const term of intentTerms) {
if (lineLower.includes(term)) score += INTENT_WEIGHT_SNIPPET;
}
if (score > bestScore) {
bestScore = score;
@ -2926,7 +2971,7 @@ export function extractSnippet(body: string, query: string, maxLen = 500, chunkP
// If we focused on a chunk window and it produced an empty/whitespace-only snippet,
// fall back to a full-document snippet so we always show something useful.
if (chunkPos && chunkPos > 0 && snippetText.trim().length === 0) {
return extractSnippet(body, query, maxLen, undefined);
return extractSnippet(body, query, maxLen, undefined, undefined, intent);
}
if (snippetText.length > maxLen) snippetText = snippetText.substring(0, maxLen - 3) + "...";
@ -2998,6 +3043,7 @@ export interface HybridQueryOptions {
minScore?: number; // default 0
candidateLimit?: number; // default RERANK_CANDIDATE_LIMIT
explain?: boolean; // include backend/RRF/rerank score traces
intent?: string; // domain intent hint for disambiguation
hooks?: SearchHooks;
}
@ -3043,6 +3089,7 @@ export async function hybridQuery(
const candidateLimit = options?.candidateLimit ?? RERANK_CANDIDATE_LIMIT;
const collection = options?.collection;
const explain = options?.explain ?? false;
const intent = options?.intent;
const hooks = options?.hooks;
const rankedLists: RankedResult[][] = [];
@ -3053,11 +3100,14 @@ export async function hybridQuery(
).get();
// Step 1: BM25 probe — strong signal skips expensive LLM expansion
// When intent is provided, disable strong-signal bypass — the obvious BM25
// match may not be what the caller wants (e.g. "performance" with intent
// "web page load times" should NOT shortcut to a sports-performance doc).
// Pass collection directly into FTS query (filter at SQL level, not post-hoc)
const initialFts = store.searchFTS(query, 20, collection);
const topScore = initialFts[0]?.score ?? 0;
const secondScore = initialFts[1]?.score ?? 0;
const hasStrongSignal = initialFts.length > 0
const hasStrongSignal = !intent && initialFts.length > 0
&& topScore >= STRONG_SIGNAL_MIN_SCORE
&& (topScore - secondScore) >= STRONG_SIGNAL_MIN_GAP;
@ -3068,7 +3118,7 @@ export async function hybridQuery(
const expandStart = Date.now();
const expanded = hasStrongSignal
? []
: await store.expandQuery(query);
: await store.expandQuery(query, undefined, intent);
hooks?.onExpand?.(query, expanded, Date.now() - expandStart);
@ -3157,6 +3207,7 @@ export async function hybridQuery(
// Step 5: Chunk documents, pick best chunk per doc for reranking.
// Reranking full bodies is O(tokens) — the critical perf lesson that motivated this refactor.
const queryTerms = query.toLowerCase().split(/\s+/).filter(t => t.length > 2);
const intentTerms = intent ? extractIntentTerms(intent) : [];
const chunksToRerank: { file: string; text: string }[] = [];
const docChunkMap = new Map<string, { chunks: { text: string; pos: number }[]; bestIdx: number }>();
@ -3165,11 +3216,15 @@ export async function hybridQuery(
if (chunks.length === 0) continue;
// Pick chunk with most keyword overlap (fallback: first chunk)
// Intent terms contribute at INTENT_WEIGHT_CHUNK (0.5) relative to query terms (1.0)
let bestIdx = 0;
let bestScore = -1;
for (let i = 0; i < chunks.length; i++) {
const chunkLower = chunks[i]!.text.toLowerCase();
const score = queryTerms.reduce((acc, term) => acc + (chunkLower.includes(term) ? 1 : 0), 0);
let score = queryTerms.reduce((acc, term) => acc + (chunkLower.includes(term) ? 1 : 0), 0);
for (const term of intentTerms) {
if (chunkLower.includes(term)) score += INTENT_WEIGHT_CHUNK;
}
if (score > bestScore) { bestScore = score; bestIdx = i; }
}
@ -3180,7 +3235,7 @@ export async function hybridQuery(
// Step 6: Rerank chunks (NOT full bodies)
hooks?.onRerankStart?.(chunksToRerank.length);
const rerankStart = Date.now();
const reranked = await store.rerank(query, chunksToRerank);
const reranked = await store.rerank(query, chunksToRerank, undefined, intent);
hooks?.onRerankDone?.(Date.now() - rerankStart);
// Step 7: Blend RRF position score with reranker score
@ -3251,6 +3306,7 @@ export interface VectorSearchOptions {
collection?: string;
limit?: number; // default 10
minScore?: number; // default 0.3
intent?: string; // domain intent hint for disambiguation
hooks?: Pick<SearchHooks, 'onExpand'>;
}
@ -3281,6 +3337,7 @@ export async function vectorSearchQuery(
const limit = options?.limit ?? 10;
const minScore = options?.minScore ?? 0.3;
const collection = options?.collection;
const intent = options?.intent;
const hasVectors = !!store.db.prepare(
`SELECT name FROM sqlite_master WHERE type='table' AND name='vectors_vec'`
@ -3289,7 +3346,7 @@ export async function vectorSearchQuery(
// Expand query — filter to vec/hyde only (lex queries target FTS, not vector)
const expandStart = Date.now();
const allExpanded = await store.expandQuery(query);
const allExpanded = await store.expandQuery(query, undefined, intent);
const vecExpanded = allExpanded.filter(q => q.type !== 'lex');
options?.hooks?.onExpand?.(query, vecExpanded, Date.now() - expandStart);
@ -3343,7 +3400,7 @@ export interface StructuredSearchOptions {
minScore?: number; // default 0
candidateLimit?: number; // default RERANK_CANDIDATE_LIMIT
explain?: boolean; // include backend/RRF/rerank score traces
/** Future: domain intent hint for routing/boosting */
/** Domain intent hint for disambiguation — steers reranking and chunk selection */
intent?: string;
hooks?: SearchHooks;
}
@ -3375,6 +3432,7 @@ export async function structuredSearch(
const minScore = options?.minScore ?? 0;
const candidateLimit = options?.candidateLimit ?? RERANK_CANDIDATE_LIMIT;
const explain = options?.explain ?? false;
const intent = options?.intent;
const hooks = options?.hooks;
const collections = options?.collections;
@ -3489,6 +3547,7 @@ export async function structuredSearch(
|| searches.find(s => s.type === 'vec')?.query
|| searches[0]?.query || "";
const queryTerms = primaryQuery.toLowerCase().split(/\s+/).filter(t => t.length > 2);
const intentTerms = intent ? extractIntentTerms(intent) : [];
const chunksToRerank: { file: string; text: string }[] = [];
const docChunkMap = new Map<string, { chunks: { text: string; pos: number }[]; bestIdx: number }>();
@ -3497,11 +3556,15 @@ export async function structuredSearch(
if (chunks.length === 0) continue;
// Pick chunk with most keyword overlap
// Intent terms contribute at INTENT_WEIGHT_CHUNK (0.5) relative to query terms (1.0)
let bestIdx = 0;
let bestScore = -1;
for (let i = 0; i < chunks.length; i++) {
const chunkLower = chunks[i]!.text.toLowerCase();
const score = queryTerms.reduce((acc, term) => acc + (chunkLower.includes(term) ? 1 : 0), 0);
let score = queryTerms.reduce((acc, term) => acc + (chunkLower.includes(term) ? 1 : 0), 0);
for (const term of intentTerms) {
if (chunkLower.includes(term)) score += INTENT_WEIGHT_CHUNK;
}
if (score > bestScore) { bestScore = score; bestIdx = i; }
}
@ -3512,7 +3575,7 @@ export async function structuredSearch(
// Step 5: Rerank chunks
hooks?.onRerankStart?.(chunksToRerank.length);
const rerankStart2 = Date.now();
const reranked = await store.rerank(primaryQuery, chunksToRerank);
const reranked = await store.rerank(primaryQuery, chunksToRerank, undefined, intent);
hooks?.onRerankDone?.(Date.now() - rerankStart2);
// Step 6: Blend RRF position score with reranker score

513
test/intent.test.ts Normal file
View File

@ -0,0 +1,513 @@
/**
* intent.test.ts - Tests for the intent feature
*
* Tests cover:
* - extractIntentTerms: stop word filtering, punctuation, acronyms, edge cases
* - extractSnippet with intent: disambiguation across multiple document sections
* - parseStructuredQuery with intent: lines (parsing, validation, error cases)
* - Chunk selection scoring with intent
* - Strong-signal bypass when intent is present
* - Intent constants
*
* Run with: npx vitest run test/intent.test.ts
*/
import { describe, test, expect } from "vitest";
import {
extractSnippet,
extractIntentTerms,
INTENT_WEIGHT_SNIPPET,
INTENT_WEIGHT_CHUNK,
type StructuredSubSearch,
} from "../src/store.js";
// =============================================================================
// parseStructuredQuery — duplicated from src/qmd.ts for unit testing
// (qmd.ts doesn't export it since it's a CLI internal)
// =============================================================================
interface ParsedStructuredQuery {
searches: StructuredSubSearch[];
intent?: string;
}
function parseStructuredQuery(query: string): ParsedStructuredQuery | null {
const rawLines = query.split('\n').map((line, idx) => ({
raw: line,
trimmed: line.trim(),
number: idx + 1,
})).filter(line => line.trimmed.length > 0);
if (rawLines.length === 0) return null;
const prefixRe = /^(lex|vec|hyde):\s*/i;
const expandRe = /^expand:\s*/i;
const intentRe = /^intent:\s*/i;
const typed: StructuredSubSearch[] = [];
let intent: string | undefined;
for (const line of rawLines) {
if (expandRe.test(line.trimmed)) {
if (rawLines.length > 1) {
throw new Error(`Line ${line.number} starts with expand:, but query documents cannot mix expand with typed lines. Submit a single expand query instead.`);
}
const text = line.trimmed.replace(expandRe, '').trim();
if (!text) {
throw new Error('expand: query must include text.');
}
return null;
}
if (intentRe.test(line.trimmed)) {
if (intent !== undefined) {
throw new Error(`Line ${line.number}: only one intent: line is allowed per query document.`);
}
const text = line.trimmed.replace(intentRe, '').trim();
if (!text) {
throw new Error(`Line ${line.number}: intent: must include text.`);
}
intent = text;
continue;
}
const match = line.trimmed.match(prefixRe);
if (match) {
const type = match[1]!.toLowerCase() as 'lex' | 'vec' | 'hyde';
const text = line.trimmed.slice(match[0].length).trim();
if (!text) {
throw new Error(`Line ${line.number} (${type}:) must include text.`);
}
if (/\r|\n/.test(text)) {
throw new Error(`Line ${line.number} (${type}:) contains a newline. Keep each query on a single line.`);
}
typed.push({ type, query: text, line: line.number });
continue;
}
if (rawLines.length === 1) {
return null;
}
throw new Error(`Line ${line.number} is missing a lex:/vec:/hyde:/intent: prefix. Each line in a query document must start with one.`);
}
if (intent && typed.length === 0) {
throw new Error('intent: cannot appear alone. Add at least one lex:, vec:, or hyde: line.');
}
return typed.length > 0 ? { searches: typed, intent } : null;
}
// =============================================================================
// extractIntentTerms
// =============================================================================
describe("extractIntentTerms", () => {
test("filters stop words", () => {
// "looking", "for", "notes", "about" are stop words
expect(extractIntentTerms("looking for notes about latency optimization"))
.toEqual(["latency", "optimization"]);
});
test("filters common function words", () => {
// "what", "is", "the", "to", "find" are stop words; "best", "way" survive
expect(extractIntentTerms("what is the best way to find"))
.toEqual(["best", "way"]);
});
test("preserves domain terms", () => {
expect(extractIntentTerms("web performance latency page load times"))
.toEqual(["web", "performance", "latency", "page", "load", "times"]);
});
test("handles surrounding punctuation with Unicode awareness", () => {
expect(extractIntentTerms("personal health, fitness, and endurance"))
.toEqual(["personal", "health", "fitness", "endurance"]);
});
test("preserves internal hyphens", () => {
expect(extractIntentTerms("self-hosted real-time (decision-making)"))
.toEqual(["self-hosted", "real-time", "decision-making"]);
});
test("short domain terms survive (API, SQL, LLM)", () => {
expect(extractIntentTerms("API design for LLM agents"))
.toEqual(["api", "design", "llm", "agents"]);
});
test("returns empty for empty input", () => {
expect(extractIntentTerms("")).toEqual([]);
expect(extractIntentTerms(" ")).toEqual([]);
});
test("filters single-char terms", () => {
const terms = extractIntentTerms("a b c web");
expect(terms).toEqual(["web"]);
});
test("all stop words returns empty", () => {
const terms = extractIntentTerms("the and or but in on at to for of with by");
expect(terms).toEqual([]);
});
test("preserves 2-char domain terms (CI, CD, DB)", () => {
const terms = extractIntentTerms("SQL CI CD DB");
expect(terms).toContain("sql");
expect(terms).toContain("ci");
expect(terms).toContain("cd");
expect(terms).toContain("db");
});
test("lowercases all terms", () => {
const terms = extractIntentTerms("WebSocket HTTP REST");
expect(terms).toContain("websocket");
expect(terms).toContain("http");
expect(terms).toContain("rest");
});
test("handles C++ style punctuation", () => {
const terms = extractIntentTerms("C++, performance! optimization.");
expect(terms).toContain("performance");
expect(terms).toContain("optimization");
});
});
// =============================================================================
// extractSnippet with intent — disambiguation
// =============================================================================
describe("extractSnippet with intent", () => {
// Each section contains "performance" so the query score is tied (1.0 each).
// Intent terms (INTENT_WEIGHT_SNIPPET) then break the tie toward the relevant section.
const body = [
"# Notes on Various Topics",
"",
"## Web Performance Section",
"Web performance means optimizing page load times and Core Web Vitals.",
"Reduce latency, improve rendering speed, and measure performance budgets.",
"",
"## Team Performance Section",
"Team performance depends on trust, psychological safety, and feedback.",
"Build culture where performance reviews drive growth not fear.",
"",
"## Health Performance Section",
"Health performance comes from consistent exercise, sleep, and endurance.",
"Track fitness metrics, optimize recovery, and monitor healthspan.",
].join("\n");
test("without intent, anchors on query terms only", () => {
const result = extractSnippet(body, "performance", 500);
// "performance" appears in title and multiple sections — should anchor on first match
expect(result.snippet).toContain("Performance");
});
test("with web-perf intent, prefers web performance section", () => {
const result = extractSnippet(
body, "performance", 500,
undefined, undefined,
"Looking for notes about web performance, latency, and page load times"
);
expect(result.snippet).toMatch(/latency|page.*load|Core Web Vitals/i);
});
test("with health intent, prefers health section", () => {
const result = extractSnippet(
body, "performance", 500,
undefined, undefined,
"Looking for notes about personal health, fitness, and endurance"
);
expect(result.snippet).toMatch(/health|fitness|endurance|exercise/i);
});
test("with team intent, prefers team section", () => {
const result = extractSnippet(
body, "performance", 500,
undefined, undefined,
"Looking for notes about building high-performing teams and culture"
);
expect(result.snippet).toMatch(/team|culture|trust|feedback/i);
});
test("intent does not override strong query match", () => {
// Query "Core Web Vitals" is very specific — intent shouldn't pull away from it
const result = extractSnippet(
body, "Core Web Vitals", 500,
undefined, undefined,
"Looking for notes about health and fitness"
);
expect(result.snippet).toContain("Core Web Vitals");
});
test("absent intent produces same result as undefined", () => {
const withoutIntent = extractSnippet(body, "performance", 500);
const withUndefined = extractSnippet(body, "performance", 500, undefined, undefined, undefined);
expect(withoutIntent.line).toBe(withUndefined.line);
expect(withoutIntent.snippet).toBe(withUndefined.snippet);
});
test("intent with no matching terms falls back to query-only scoring", () => {
const result = extractSnippet(
body, "performance", 500,
undefined, undefined,
"quantum computing and entanglement"
);
expect(result.snippet).toContain("Performance");
expect(result.snippet.length).toBeGreaterThan(0);
});
test("intent works with chunk position", () => {
const webPerfStart = body.indexOf("## Web Performance");
const result = extractSnippet(
body, "performance", 500,
webPerfStart, 200,
"web page load times"
);
expect(result.snippet).toMatch(/Web Performance|Core Web Vitals|Page load/i);
});
});
// =============================================================================
// extractSnippet — intent weight verification
// =============================================================================
describe("extractSnippet intent weight behavior", () => {
// Document where query term appears on every line but intent terms differ
const body = [
"performance metrics for team velocity",
"performance metrics for web latency",
"performance metrics for athletic endurance",
].join("\n");
test("intent breaks tie when query matches all lines equally", () => {
const noIntent = extractSnippet(body, "performance metrics", 500);
// Without intent, first line wins (all equal score)
expect(noIntent.line).toBe(1);
const withIntent = extractSnippet(
body, "performance metrics", 500,
undefined, undefined,
"web latency and page speed"
);
// Intent terms "web", "latency" match line 2
expect(withIntent.snippet).toContain("web latency");
});
});
// =============================================================================
// Chunk selection scoring with intent
// =============================================================================
describe("intent keyword extraction logic", () => {
// Mirrors the chunk selection scoring in hybridQuery, using the shared
// extractIntentTerms helper and INTENT_WEIGHT_CHUNK constant.
function scoreChunk(text: string, query: string, intent?: string): number {
const queryTerms = query.toLowerCase().split(/\s+/).filter(t => t.length > 2);
const intentTerms = intent ? extractIntentTerms(intent) : [];
const lower = text.toLowerCase();
const qScore = queryTerms.reduce((acc, term) => acc + (lower.includes(term) ? 1 : 0), 0);
const iScore = intentTerms.reduce((acc, term) => acc + (lower.includes(term) ? INTENT_WEIGHT_CHUNK : 0), 0);
return qScore + iScore;
}
const chunks = [
"Web performance: optimize page load times, reduce latency, improve rendering pipeline.",
"Team performance: build trust, give feedback, set clear expectations for the group.",
"Health performance: exercise regularly, sleep 8 hours, manage stress for endurance.",
];
test("without intent, all chunks score equally on 'performance'", () => {
const scores = chunks.map(c => scoreChunk(c, "performance"));
// All contain "performance", so all score 1
expect(scores[0]).toBe(scores[1]);
expect(scores[1]).toBe(scores[2]);
});
test("with web intent, web chunk scores highest", () => {
const intent = "looking for notes about page load times and latency optimization";
const scores = chunks.map(c => scoreChunk(c, "performance", intent));
expect(scores[0]).toBeGreaterThan(scores[1]!);
expect(scores[0]).toBeGreaterThan(scores[2]!);
});
test("with health intent, health chunk scores highest", () => {
const intent = "looking for notes about exercise, sleep, and endurance";
const scores = chunks.map(c => scoreChunk(c, "performance", intent));
expect(scores[2]).toBeGreaterThan(scores[0]!);
expect(scores[2]).toBeGreaterThan(scores[1]!);
});
test("intent terms have lower weight than query terms (1.0)", () => {
const intent = "looking for latency";
// Chunk 0 has "performance" (query: 1.0) + "latency" (intent: INTENT_WEIGHT_CHUNK) = 1.5
const withBoth = scoreChunk(chunks[0]!, "performance", intent);
const queryOnly = scoreChunk(chunks[0]!, "performance");
expect(withBoth).toBe(queryOnly + INTENT_WEIGHT_CHUNK);
});
test("stop words are filtered, short domain terms survive", () => {
const intent = "the art of web performance";
// "the" (stop word), "art" (survives), "of" (stop word),
// "web" (survives), "performance" (survives)
// intent terms after filtering: ["art", "web", "performance"]
// Chunk 0 has "web" + "performance" → 2 intent hits (no "art")
// Chunks 1,2 have "performance" only → 1 intent hit
const scores = chunks.map(c => scoreChunk(c, "test", intent));
expect(scores[0]).toBe(INTENT_WEIGHT_CHUNK * 2); // "web" + "performance"
expect(scores[1]).toBe(INTENT_WEIGHT_CHUNK); // "performance" only
expect(scores[2]).toBe(INTENT_WEIGHT_CHUNK); // "performance" only
});
});
// =============================================================================
// Strong-signal bypass with intent
// =============================================================================
describe("strong-signal bypass logic", () => {
// Mirrors the logic in hybridQuery:
// const hasStrongSignal = !intent && topScore >= STRONG_SIGNAL_MIN_SCORE && gap >= STRONG_SIGNAL_MIN_GAP
function hasStrongSignal(topScore: number, secondScore: number, intent?: string): boolean {
return !intent
&& topScore >= 0.85
&& (topScore - secondScore) >= 0.15;
}
test("strong signal detected without intent", () => {
expect(hasStrongSignal(0.90, 0.70)).toBe(true);
});
test("strong signal bypassed when intent provided", () => {
expect(hasStrongSignal(0.90, 0.70, "looking for health performance")).toBe(false);
});
test("weak signal not affected by intent", () => {
expect(hasStrongSignal(0.50, 0.45)).toBe(false);
expect(hasStrongSignal(0.50, 0.45, "some intent")).toBe(false);
});
test("close scores not strong even without intent", () => {
expect(hasStrongSignal(0.90, 0.80)).toBe(false); // gap < 0.15
});
});
// =============================================================================
// parseStructuredQuery with intent
// =============================================================================
describe("parseStructuredQuery with intent", () => {
test("parses intent + lex query", () => {
const result = parseStructuredQuery("intent: web performance\nlex: performance");
expect(result).not.toBeNull();
expect(result!.intent).toBe("web performance");
expect(result!.searches).toHaveLength(1);
expect(result!.searches[0]!.type).toBe("lex");
expect(result!.searches[0]!.query).toBe("performance");
});
test("parses intent + multiple typed lines", () => {
const result = parseStructuredQuery(
"intent: web page load times\nlex: performance\nvec: how to improve performance"
);
expect(result).not.toBeNull();
expect(result!.intent).toBe("web page load times");
expect(result!.searches).toHaveLength(2);
expect(result!.searches[0]!.type).toBe("lex");
expect(result!.searches[1]!.type).toBe("vec");
});
test("intent can appear after typed lines", () => {
const result = parseStructuredQuery(
"lex: performance\nintent: web page load times\nvec: latency"
);
expect(result).not.toBeNull();
expect(result!.intent).toBe("web page load times");
expect(result!.searches).toHaveLength(2);
});
test("intent is case-insensitive prefix", () => {
const result = parseStructuredQuery("Intent: web perf\nlex: performance");
expect(result).not.toBeNull();
expect(result!.intent).toBe("web perf");
});
test("no intent returns undefined", () => {
const result = parseStructuredQuery("lex: performance\nvec: speed");
expect(result).not.toBeNull();
expect(result!.intent).toBeUndefined();
});
test("intent alone throws error", () => {
expect(() => parseStructuredQuery("intent: web performance")).toThrow(
/intent: cannot appear alone/
);
});
test("multiple intent lines throw error", () => {
expect(() =>
parseStructuredQuery("intent: web perf\nintent: team health\nlex: performance")
).toThrow(/only one intent: line is allowed/);
});
test("empty intent text throws error", () => {
expect(() =>
parseStructuredQuery("intent:\nlex: performance")
).toThrow(/intent: must include text/);
});
test("intent with whitespace-only text throws error", () => {
expect(() =>
parseStructuredQuery("intent: \nlex: performance")
).toThrow(/intent: must include text/);
});
test("single plain line still returns null (expand mode)", () => {
const result = parseStructuredQuery("how does auth work");
expect(result).toBeNull();
});
test("expand: line still returns null", () => {
const result = parseStructuredQuery("expand: auth stuff");
expect(result).toBeNull();
});
test("intent with expand throws error (expand can't mix)", () => {
expect(() =>
parseStructuredQuery("intent: web\nexpand: performance")
).toThrow(/cannot mix expand/);
});
test("empty query returns null", () => {
expect(parseStructuredQuery("")).toBeNull();
expect(parseStructuredQuery(" \n \n ")).toBeNull();
});
test("intent with blank lines is fine", () => {
const result = parseStructuredQuery(
"intent: web perf\n\nlex: performance\n\nvec: speed"
);
expect(result).not.toBeNull();
expect(result!.intent).toBe("web perf");
expect(result!.searches).toHaveLength(2);
});
test("intent preserves full text including colons", () => {
const result = parseStructuredQuery(
"intent: web performance: LCP, FID, CLS\nlex: performance"
);
expect(result).not.toBeNull();
expect(result!.intent).toBe("web performance: LCP, FID, CLS");
});
});
// =============================================================================
// Constants exported
// =============================================================================
describe("intent constants", () => {
test("INTENT_WEIGHT_SNIPPET is 0.3", () => {
expect(INTENT_WEIGHT_SNIPPET).toBe(0.3);
});
test("INTENT_WEIGHT_CHUNK is 0.5", () => {
expect(INTENT_WEIGHT_CHUNK).toBe(0.5);
});
});