This refactoring extracts all document indexing database operations from
the indexFiles() function in qmd.ts into dedicated methods in store.ts:
- insertContent(): Insert content into content-addressable storage
- insertDocument(): Insert new document record
- findActiveDocument(): Find active document by collection and path
- updateDocumentTitle(): Update document title only
- updateDocument(): Update document hash, title, and modified_at
- deactivateDocument(): Mark document as inactive
- getActiveDocumentPaths(): Get all active document paths for collection
- cleanupOrphanedContent(): Remove unreferenced content hashes
Key improvements:
- Better separation of concerns (DB layer vs business logic)
- Reusable DB operations accessible throughout the codebase
- Fixed bug where document updates would fail with UNIQUE constraint error
- Fixed bug in updateCollections() that passed wrong parameters to indexFiles()
- Removed obsolete updateDisplayPaths() function from old schema
The indexing logic now uses updateDocument() instead of deactivate + insert
when content changes, which properly updates the existing document record
rather than violating the UNIQUE(collection_id, path) constraint.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Expose the new updateDocument() helper through the Store interface
for use by consumers.
Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add new helper function to update document hash and title simultaneously.
This is useful when file content changes but the path stays the same.
Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Update context management commands to use store.ts functions instead
of inline SQL queries. This change is part of the larger refactoring
effort to move all DB operations into store.ts.
Changes:
- contextAdd: Use insertContext() and getAllCollections()
- contextList: Use listPathContexts()
- contextRemove: Use deleteContext() and deleteGlobalContexts()
This simplifies the CLI code and centralizes database logic.
Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This change consolidates all database cleanup and maintenance operations
into store.ts for better organization and reusability.
Changes:
- Added deleteOllamaCache() - Remove cached Ollama API responses
- Added deleteInactiveDocuments() - Remove inactive document records
- Added cleanupOrphanedContent() - Remove unreferenced content hashes
- Added cleanupOrphanedVectors() - Remove orphaned vector embeddings
- Added cleanupDuplicateCollections() - Remove duplicate collections
- Added vacuumDatabase() - Run VACUUM to reclaim space
- Updated Store type and createStore() to expose these methods
- Updated qmd.ts cleanup command to use new store methods
- Removed local cleanupDuplicateCollections() from qmd.ts
- Removed duplicate cleanupOrphanedContent() definition
All cleanup operations now follow the pattern of returning counts
for reporting, making the cleanup command output more informative.
Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Refactor vector indexing by extracting database operations from vectorIndex()
in qmd.ts into three new store methods:
- getHashesForEmbedding(): Returns content hashes needing embeddings
- clearAllEmbeddings(): Clears all vectors for force re-indexing
- insertEmbedding(): Inserts a single embedding into both tables
This continues the refactoring effort to consolidate all database operations
in store.ts, making the codebase more maintainable and testable.
Changes:
- Add new embedding operation methods to Store type and factory
- Export getHashesForEmbedding(), clearAllEmbeddings(), insertEmbedding()
- Update vectorIndex() to use new store methods instead of direct SQL
- Remove inline SQL from embedding logic in qmd.ts
Related: qmd-4u4
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Added listCollections(), removeCollection(), renameCollection() to store.ts
- Updated qmd.ts to use store methods instead of direct DB operations
- Removed ~15 direct DB calls from qmd.ts collection functions
- All 9 collection command tests passing
Part of larger refactoring to move all DB operations to store.ts
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Updated vectorIndex() to work with new schema:
* Join with content table to get document body
* Use 'path' column instead of 'filepath'/'display_path'
* Query now: SELECT d.hash, c.doc as body, MIN(d.path) as path
- Documents table no longer has body column (in content table)
- Documents table uses 'path' not 'filepath' or 'display_path'
- Embedding functionality now compatible with hash-based storage
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Added collectionRename() function to rename collections
- Updates collection name in database (changes virtual path prefix)
- Added CLI handler for "qmd collection rename <old> <new>"
- Supports alias "mv" for rename command
- Includes comprehensive error handling:
* Checks if old collection exists
* Prevents renaming to existing collection name
* Validates required arguments
- Fixed bug in collectionAdd where --name flag was ignored
* indexFiles now accepts pwd and name parameters
* Collection name is properly passed through to getOrCreateCollection
- Updated help text and CLAUDE.md documentation
- Added 4 new tests for rename functionality (all passing)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Updated createTestCollection() to support new schema with name field
- Updated insertTestDocument() to match new schema (path, not filepath)
- Content is now stored in separate content table with hash deduplication
- Added comprehensive test suite for content-addressable storage:
* Same content gets same hash from multiple collections
* Removing one collection preserves shared content
* Deduplication works across many collections
* Different content gets different hashes
- All 4 content-addressable tests passing
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Display collections by name instead of full path
- Show virtual path format: qmd://name/
- Add context count per collection
- Include helpful command examples (ls, get, search)
- Update "no collections" message to reference collection add
- Sort collections alphabetically by name
Closes qmd-rhd
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Remove qmd add command (replaced by qmd collection add)
- Remove --drop flag (use collection remove + add instead)
- Update all help text to reflect new command structure
- Update CLAUDE.md documentation
- Update all tests to use collection add (39/42 passing)
- Reorganize help to put collection commands first
- Remove add-context from "do not run automatically" list
The CLI now has consistent command structure:
- qmd collection {add,list,remove} for collection management
- qmd context {add,list,rm} for context management
- qmd ls for browsing collections
- qmd get/multi-get for document retrieval
Closes qmd-c0m
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add collectionList() to display all collections with details
- Add collectionAdd() with --name and --mask options
- Add collectionRemove() to delete collections by name
- Add CLI command handlers for collection subcommands
- Add --name and --mask CLI options
- Update help text with collection commands
- Add comprehensive tests (5 test cases, all passing)
- Properly clean up orphaned content hashes on removal
Closes qmd-dmi
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add listFiles() function to list collections and files
- Support three modes:
- qmd ls - list all collections with file counts
- qmd ls <collection> - list all files in collection
- qmd ls <collection>/path - list files under path
- Support both virtual paths (qmd://) and short form
- Add CLI command handler and help text
- Add comprehensive tests (5 test cases, all passing)
Closes qmd-e2c
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Update getDocument() to support qmd:// virtual paths and filesystem paths
- Update multiGet() to handle virtual paths in patterns and comma-separated lists
- Update matchFilesByGlob() in store.ts to return virtual paths
- Remove duplicate getContextForFile() function from qmd.ts
- Use collection-scoped getContextForPath() instead of legacy function
- All get and multi-get tests now passing
Closes qmd-vro
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Remove mimeType from TextContent (not in spec, only valid on EmbeddedResource)
- Add isError: true to all error responses for proper error detection
- Replace CSV output with structuredContent for machine-readable results
- Add name and title fields to all embedded resources
- Fix URI encoding: preserve slashes, encode special chars (spaces → %20)
- Change template to {+path} for proper nested path support
- Rename tools: qmd_search → search, qmd_get → get, etc.
- Update tests for new response format and spec compliance
- Extract store.ts: database operations, search, document retrieval
- createStore() factory pattern for clean DB lifecycle management
- Unified DocumentResult type with optional body loading
- Snippet extraction with diff-style headers (@@ -line,count @@)
- Extract llm.ts: LLM abstraction layer with Ollama implementation
- Clean interface for embed, generate, rerank operations
- High-level rerankerLogprobsCheck with logprob-based scoring
- Query expansion support
- Extract formatter.ts: output formatting utilities
- Support for CLI, JSON, CSV, MD, XML formats
- MCP-specific CSV formatting
- Extract mcp.ts: MCP server using createStore() pattern
- Single DB connection for server lifetime (fixes closed DB errors)
- URL-decode resource paths for proper space/special char handling
- Add comprehensive test suites (215 tests total)
- store.test.ts: 96 tests covering all store operations
- llm.test.ts: 60 tests for LLM abstraction
- mcp.test.ts: 59 tests for MCP endpoints and resources
- All tests use mocked Ollama (errors on unmocked calls)
- Add bun run inspector script for MCP debugging
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Package QMD as a Nix derivation with bun and sqlite dependencies
- Support `nix profile install`, `nix run`, and `nix develop`
- Set library paths for sqlite extension support
- Add `qmd mcp` command to start stdio-based MCP server
- Expose tools: qmd_search, qmd_vsearch, qmd_query, qmd_get, qmd_status
- Add index health warnings for unembedded docs and stale indexes
- Return CSV format with text/csv mime type for search results
- Add MCP documentation and configuration examples to README
- Add @modelcontextprotocol/sdk and zod dependencies
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add new tagline: on-device search engine for everything you need to remember
- Add Quick Start section with walkthrough of indexing multiple directories
- Add "Using with AI Agents" section showing --json and --files workflows
- Update output format example to reflect new CLI format with Title/Context/Score
- Document --all flag for returning all matches
- Add --all flag to search/vsearch for returning all matches
- Works with --min-score to filter by relevance threshold
- Useful for bulk exports: qmd search "term" --all --files
- Display paths now always include parent folder for better context
- e.g., pages/file.md instead of just file.md
- qmd get now outputs folder context header when available
- Format: "Folder Context: <description>\n---\n<content>"
- Add display_path column with unique index for collection-relative paths
- Computes shortest unique path starting from filename
- Auto-migrates existing documents via update-all
- Add title field to all search result types (FTS, vector, hybrid)
- Improve CLI output format:
- Filepath on first line
- Title and Context on labeled lines
- Score on separate line
- Remove | prefix from snippets for better word wrap
- Double newline between results
- Add line range support to get command:
- qmd get file.md:100 (start at line 100)
- qmd get file.md -l 20 --from 50
- Include title in JSON, CSV, XML output formats
- Fix update-all crash when same file exists in multiple collections
- Documents matching multiple chunks get +0.02 per additional chunk
- Bonus capped at +0.1 (5+ chunks)
- Still uses max chunk score as base
- Rewards documents with broader relevance across content
🤖 Generated with [Claude Code](https://claude.com/claude-code)
- Fix architecture diagram: show BM25+Vector for all query variations
- Add position-aware blending percentages to diagram
- Update CLI commands: add → index, add-context, cleanup, status
- Document chunked embeddings (~6KB pieces with hash/seq/pos)
- Update schema section with new tables (path_contexts, ollama_cache)
- Rewrite How It Works flows with accurate pipeline details
- Fix output format examples to show ~/... paths
- Add --files and --json output options
🤖 Generated with [Claude Code](https://claude.com/claude-code)
- Truncate documents > 64KB with warning showing filenames
- Show document title in error messages instead of hash
- Format total time as "15m 4s" instead of "904.2s"
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Visual progress bar with filled/empty blocks
- Calculate ETA based on bytes processed (larger files = longer time)
- Show throughput in bytes/sec
- Skip empty documents
- Fix UNIQUE constraint with INSERT OR REPLACE
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Rename 'collections' to 'status' with richer output:
- Index size
- Documents count and vector embedding status
- Time since last update
- Per-collection stats
- Fix `qmd add .` to use default glob pattern
- Fix duplicate collections with cleanup and INSERT OR IGNORE
- Improve update-all with colored progress output
- Fix 'qmd vector' → 'qmd embed' in help messages
- Implement weighted RRF (2x weight for original query)
- Simplify CLAUDE.md for project-specific instructions
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
