9718d3767c
When a chunk exceeds the embedding model's context window (trainContextSize), node-llama-cpp's getEmbeddingFor() triggers a native SIGABRT in GGML/Metal, crashing the entire process. Fix: Add truncateToContextSize() guard in embed() and embedBatch() that uses the model's own tokenizer to check token count before calling getEmbeddingFor(). Oversized text is truncated to (trainContextSize - 4) tokens with a warning, preserving partial embedding coverage instead of crashing. Fixes #303 |
||
|---|---|---|
| .. | ||
| bench-rerank.ts | ||
| collections.ts | ||
| db.ts | ||
| formatter.ts | ||
| llm.ts | ||
| mcp.ts | ||
| qmd.ts | ||
| store.ts | ||
| test-preload.ts | ||