Hybrid Search and Why Pure Vector Search Is Overrated

Six months ago you wrote a note about why you chose SQLite for a side project. You remember the word "SQLite" was in it. You search your notes. Semantic search, doing its thing, returns documents about "database design," "local persistence," and "storage trade-offs." None of them are the note you want. The one that mentioned SQLite is on page two, buried under conceptually adjacent content that never once used the word.

This is not a hypothetical. It's a failure that happens constantly in systems built around pure semantic search, and it's why a ranking algorithm from 1994 is still running inside most of the world's search infrastructure.

BM25 (Best Match 25) was developed by Stephen Robertson and Karen Spärck Jones as part of the Okapi system at City University London. The version most people recognize was presented at TREC-3 in 1994. Thirty years later, it is the default retrieval algorithm in Elasticsearch, OpenSearch, and Apache Lucene. It has no idea what words mean. But it knows exactly where "SQLite" appears, and it treats that specificity as a signal worth trusting.

It survived not because of inertia. It's good.

The hype and what's actually true

When dense vector embeddings arrived — and then the vector database explosion of 2022 to 2024 — the pitch was that BM25 was obsolete. You didn't need to match words anymore. You could match meaning. A knowledge base powered by vectors would find your SQLite note even if you searched for "embedded database that fits in a file."

That's genuinely useful when it works. The problem shows up at the edges. Vector search is surprisingly bad at specificity. If you have a unique proper noun — a product name, a person's name, a rare technical term — the embedding tends to smear it into neighboring concepts. The vector for "BM25" ends up near "ranking" and "retrieval" and "information retrieval." So searching for "BM25" might surface documents that are generally about retrieval but never mention BM25 at all.

In personal knowledge, this matters more than you'd think. Your notes are full of specific names, dates, products, and decisions. You're usually not searching for vibes. You're searching for a thing you wrote, in a document, at some point.

A 2026 paper posted to arXiv found that tool-augmented LLM agents using simple keyword search could reach over 90% of the benchmark performance of vector RAG systems. Cursor and Claude Code, two of the most-used AI coding tools, rely substantially on grep rather than embeddings for code retrieval. When exact match is the right tool, the right engineers reach for it.

What vectors actually get right

None of this means vector search is useless. It does things keyword search cannot.

Synonyms are handled without configuration. It finds content where someone wrote "recurring commitment" when you searched for "meeting," and it works across paraphrasing and reformulation in ways that term matching can't touch. For fuzzy, conceptual queries — "what did I think about this situation last year?" — semantic retrieval often outperforms keyword search substantially. There are real classes of queries where BM25 produces garbage and embeddings nail it.

The error is treating vector search as a replacement for keyword retrieval rather than a complement. They fail in different directions. Keyword search fails when you don't know the exact words used in a document. Vector search fails when you do know the exact words — specifically when the word is a rare or specialized term that the embedding model has learned to generalize away from.

Why you need both

Hybrid search runs both signals in parallel and merges the results. The standard technique is Reciprocal Rank Fusion: a function that scores documents based on their rank in each list, rewarding documents that appear near the top of both. A document that shows up third in keyword results and fourth in semantic results beats one that ranks first in one and doesn't appear at all in the other.

In almost every comparison on standard benchmarks like BEIR or MS MARCO, hybrid retrieval outperforms either approach alone, typically by 10 to 20 points on precision metrics, and more on queries involving specific entities. Weaviate, Elasticsearch, and Meilisearch have all shipped hybrid search as a first-class feature because the benchmarks are hard to argue with.

What a knowledge retrieval system actually needs is three things that rarely come packaged together: full-text search for exact terms, semantic search for conceptual queries, and structured queries for typed facts — dates, tags, linked entities, names. Most tools give you one. A few give you two. Getting all three to coexist without creating a configuration nightmare is where most of the real engineering work lives.

The infrastructure argument nobody makes

There's one more thing worth naming. Vector databases add significant infrastructure complexity. You need an embedding model, which means an API call or a locally hosted model with its own latency and cold-start overhead. You need to re-embed documents when they change. You need to manage index drift as your embedding model is updated. For a personal knowledge base with a few thousand documents — notes, contacts, project records — this overhead rarely pays for itself.

BM25 over SQLite's FTS5 extension is fast, lightweight, runs offline, and has no external dependencies. Adding vector search on top of that is a reasonable next step. Replacing keyword search with it is not.

The algorithm presented at TREC-3 in 1994 is running inside the search box of most enterprise tools you use every day. That's not a legacy waiting to be replaced. It's a hint about what actually works.

---

Asgeir Albretsen is the founder of Harbor.