Transforming search for the AI world with vector databases

For twenty years, search meant matching keywords. In the AI world it means matching meaning — and that changes the engine underneath. Keyword indexes are giving way to vector databases: systems that store the numerical embeddings a model produces and retrieve by semantic similarity, not string overlap. It is the same shift that powers retrieval-augmented AI, and it is quietly rewriting product discovery and recommendations in e-commerce.

We recently put this to work: building AI-ready shopping recommendations and complementary-product discovery for Shopify merchants, on top of Qdrant. Here is what we built, and why the vector database was the difference.

The project: recommendations that understand the catalogue

The brief was familiar to any online retailer: help shoppers find what they want, and surface what goes with it. “More like this.” “Complete the look.” “Customers also considered.” Done well, these lift conversion and average order value; done with keyword rules, they are brittle and generic.

We embedded each product — its text and its imagery — into vectors that capture what the product actually is, and stored them in Qdrant. Now “a lightweight waterproof jacket for hiking” finds the right products even if none of those words appear in the title, and a checkout basket can be matched to genuinely complementary items rather than whatever shares a tag. The recommendations understand the catalogue the way a good salesperson would.

The value came not from any single model, but from what the vector database let us do around it.

The vector database wins

AI-native retrieval. At the core, Qdrant finds the nearest embeddings to a query vector in milliseconds. That is the primitive the whole AI era is built on — retrieval by meaning — and it is what makes semantic search, recommendations and complementary discovery share one engine instead of three bespoke systems.

Effectively limitless scaling. Catalogues grow, and a recommendations platform serving many merchants grows faster. Qdrant scales horizontally — sharding and replication across nodes — so millions of products across many stores fit without a ceiling, with multi-tenancy handled cleanly so one merchant’s data never bleeds into another’s.

Production-grade from day one. This was never a demo. HNSW indexing for fast approximate nearest-neighbour, vector quantization to keep memory and cost down at scale, rich payload filtering, snapshots and replication for resilience — the properties that separate a notebook prototype from something you can put in front of real shoppers and real revenue.

Making retrieval accurate: dense and sparse

Pure semantic search has a blind spot: it is brilliant at meaning and weak at specifics. Ask for an exact SKU, a brand name or a model number and a dense embedding can drift. So we ran hybrid search — dense vectors for semantic understanding alongside sparse vectors for lexical precision — and fused the two.

Qdrant handles both natively and combines them in a single query, so “meaning” and “exact match” reinforce each other instead of competing. The shopper searching by vibe and the shopper searching by part number both land on the right products.

Infusing business logic: score boosting

Here is where a recommendations engine either serves the business or fights it. Raw similarity does not know that an item is out of stock, low-margin, end-of-line or off-brand. Score boosting fixes that. Using Qdrant’s ability to re-score results with a formula over payload data, we blended semantic relevance with commercial signals — promoting in-stock and higher-margin items, lifting promoted lines, damping down what the merchant did not want to push, and filtering out what could not be sold.

The result is recommendations that are both relevant and aligned with the merchant’s commercial goals. This is the point where technology stops being a demo and starts delivering business value.

Token-level precision: late-interaction models

For the final shortlist — where precision matters most — we reranked with late-interaction models. Where a standard embedding squashes an entire product into one vector, a late-interaction model (ColBERT-style) keeps a vector per token and scores query and document token-against-token. That token-level matching catches fine distinctions a single vector blurs — “waterproof” versus “water-resistant”, “merino” versus “wool-blend” — and Qdrant’s multi-vector support runs it as a reranking pass over the top candidates, buying accuracy exactly where it counts without paying for it across the whole catalogue.

Diversifying results: Maximum Marginal Relevance

The last problem is a subtle one: the most similar results are often too similar. Show a shopper ten near-identical black t-shirts and you have technically nailed relevance and completely failed at discovery — and utterly failed at complementary recommendation, which is diversity by definition.

We used Maximum Marginal Relevance (MMR) to rebalance the final set — trading a little raw similarity for variety, so results span the useful range rather than clustering on one point. For “more like this” it keeps the set fresh; for “complete the look” it is essential, spreading recommendations across the categories that genuinely go together.

The pipeline, end to end

Put together, each product recommendation runs through a funnel:

1. Retrieve a broad candidate set with hybrid dense + sparse search.
2. Boost with business logic — margin, stock, promotions, brand strategy.
3. Rerank the shortlist with a late-interaction model for token-level precision.
4. Diversify the final set with MMR.

Each stage is cheap where it can be and expensive only where it must be, and the whole thing returns in the time a page takes to render.

Why it matters

None of this is search for search’s sake. Better, more diverse, commercially-aware recommendations mean shoppers find more, baskets grow, and discovery improves — measurable lift in conversion and average order value. And because the foundation is a vector database, the same infrastructure is ready for what comes next: conversational search, and the agentic commerce systems that will shop on customers’ behalf.

That is the theme in all our work: we are not here to ship technology for its own sake. We are here to make it deliver value to your business. A vector database is a powerful tool — the win is in how you wield it.

---

Exploring AI-ready search, recommendations or retrieval for your platform? Get in touch.