Skip to content
Inseed

RAG and semantic search

Search that understands intent, added to the app you already ship.

Retrieval-augmented generation and semantic search let people find things by meaning, not exact keywords. We add it as a layer beside your current search, so nothing you already shipped has to be rewritten.

Book a 30-minute call

Is this the right tool

Semantic search is a specific tool, not a default.

RAG and embeddings earn their place when the query is vague and the corpus is yours. When the query is exact, plain search is faster, cheaper, and easier to reason about.

Reach for it when

  • People search by vibe or description ("cozy spots to study", "something for a rainy afternoon") and keyword search returns nothing useful.
  • You have a body of your own content (docs, places, listings, past tickets) that a general model does not know about, and answers must be grounded in it.
  • You want an assistant that cites back to a source your user can open and verify, rather than a model guessing from training data.
  • Users describe the thing they want in their own words instead of typing the exact title.

Do not use it when

  • The query is exact: an SKU, an email, a username, an order number. A database index beats an embedding on both speed and cost here.
  • Your catalogue is small enough that a filtered list with good facets already does the job. Vectors add moving parts you would then have to maintain.
  • The result must be exhaustive and provable (legal, financial, compliance). Embedding recall is approximate, and "roughly relevant" is the wrong bar.
  • You have no evaluation set. Without a way to measure whether results got better, semantic search is a guess dressed up as a feature.

The honest default: try keyword and structured filters first. If they hold up, ship them and move on. Reach for embeddings only when a real query pattern falls through plain search, and keep the plain path as the fallback so a cold cache or a model outage never returns an empty screen.

How we build it

The real pipeline, with the failure modes named.

A good semantic search feature is a pipeline, not one embedding call. Each stage has a job, and each stage can fail in a way you plan for.

Query enrichment

A short LLM pass turns a vague query into concrete intent before any retrieval runs. On Playlists we used GPT-4o to expand terms like "cozy vibe" into attributes the index can actually match. It also parses structured intent (price band, distance) out of free text.

Embeddings and the vector store

Text is embedded and compared by cosine similarity. We have shipped this three ways: OpenAI text-embedding-3-small on Playlists, Gemini text-embedding-004 into Supabase pgvector on Epiphra, and a Milvus store on Artizan. The right choice depends on corpus size and how often it changes, not fashion.

The fallback pipeline

Pure embedding search misses. So the pipeline is staged: enrich, then semantic match, then keyword and fuzzy matching, then a filtered default. The staging guarantees the user always sees a usable result, even when the top stage comes up short.

Grounding and citations

When the feature answers rather than just ranks, retrieved passages are injected into the prompt and the answer cites them. The model is told to say it does not know rather than invent, and the source is one tap away so a person can check it.

Trade-offs and failure modes we design around

  • Stale embeddings: when your content changes, its vector has to be recomputed, or search silently answers from an old version. We wire re-embedding into the write path.
  • In-memory versus persistent: a small MVP corpus can live in memory (fast to ship, cheap to run), but it does not survive a restart or scale past one box. We call that trade-off out before we pick it, not after.
  • Hallucination on thin retrieval: if retrieval returns weak matches, a generative answer will confidently fill the gap. The guardrail is a relevance threshold that refuses to answer rather than guess.
  • Cost per query scales with tokens: enrichment plus generation on every search adds up. We cache enriched queries and embeddings so a repeat search does not pay twice.

What it costs to run

Roughly what a semantic search feature costs to run.

There are three cost lines: embedding the corpus once (and re-embedding on change), the per-query LLM calls for enrichment and generation, and the vector store itself. For a small-to-mid corpus the embedding cost is a rounding error paid mostly up front. The recurring bill is dominated by the per-query LLM calls.

That is why the enrichment and generation steps are where caching pays off. A repeated or near-repeated query should hit a cache, not the model. On an MVP corpus, in-memory similarity keeps the vector-store line at zero; a persistent store (pgvector, Milvus) adds a modest fixed cost that only matters once the data outgrows a single process.

We sketch this as a formula against your expected monthly active users and searches per user before writing code, so the model bill is a number you signed off on, not a quarter-one surprise.

Read the mobile AI cost model

Proof

Semantic search we have actually shipped.

Three live examples, three different vector strategies, and precise attribution on who delivered each one.

Playlists

Delivered by our team via Toptal

A social app for saving and sharing places, with an AI "vibe search" for natural-language queries like "cozy vibe restaurants". Our founder and team built the Python recommendation engine: GPT-4o query enrichment, then a staged matching pipeline ending in OpenAI text-embedding-3-small cosine similarity, with vibe and price-aware filtering and a fallback that always returns usable results.

  • GPT-4o enrichment plus text-embedding-3-small (1536-dim) semantic match
  • Staged fallback pipeline: enrich, embed, fuzzy, filtered default
  • Friend-aware "meet up" search fusing two users' interest profiles
Read the Playlists case study

Epiphra

Inseed client, led by our founder

A live AI reflection app whose companion "remembers" past conversations. We built long-term semantic memory: messages are embedded with Gemini text-embedding-004 and stored in Supabase pgvector, then the most similar past messages plus last-conversation context are retrieved and injected into the prompt so the assistant has continuity across sessions.

  • Gemini text-embedding-004 (768-dim) into Supabase pgvector
  • HNSW index, cosine similarity, retrieved context injected into the prompt
  • Live on the App Store and Play Store

Artizan

Inseed client via Upwork, full-stack build

A live art marketplace and market-intelligence app. We took the app over at roughly half-built and shipped it to production on both stores, owning the mobile, web, Supabase backend, and the AI valuation layer, with a Milvus vector store backing embeddings for comparable-sales and pricing intelligence.

  • Milvus vector database for embeddings
  • AI price valuation on top of a comparable-sales dataset
  • Live on the App Store and Play Store

One honest boundary: on Playlists the vibe search used in-memory cosine similarity for the MVP, not a scaled persistent vector database, and Artizan's auction-data scraping pipeline was mostly built by another engineer (we owned the app and the valuation layer, not that pipeline). Epiphra is the example where the persistent pgvector retrieval ran in production.

FAQ

Semantic search questions, answered straight.

Do I need a dedicated vector database?

Not always. For a small MVP corpus, in-memory cosine similarity ships fast and costs nothing to host, which is what we did on Playlists. You need a persistent store like pgvector or Milvus once the corpus outgrows a single process, changes often, or has to survive restarts. We pick based on your data size and update rate, not on what is trendy.

Will this hallucinate answers?

Ranking-only semantic search cannot hallucinate; it just orders your own items. The risk appears when you add a generative answer on top. We control it with a relevance threshold that refuses to answer on weak retrieval, a prompt that tells the model to say it does not know, and citations back to the source so a person can verify. Thin retrieval should return "no confident match", not a confident guess.

Can you add this without touching my existing search?

Yes, that is the point. Semantic search sits beside your current keyword or filter search as a separate path, and we keep the plain path as the fallback. If the embedding layer is cold or a provider is down, users still get keyword results, so you are adding a capability, not betting the whole search box on it.

How do you keep results fresh when my content changes?

An embedding is a snapshot of the text at the moment it was computed. If content changes and the vector does not, search answers from a stale version. We wire re-embedding into the write path so that updating an item updates its vector, and for large corpora we batch re-embeds on a schedule. This is a common failure mode we design out up front.

Where this goes next

Where semantic search tends to land, and what it costs.

These are the app categories where natural-language discovery moves a real number, plus our pricing so you can size the work.

AI for marketplace apps

Vibe search, natural-language discovery, and buyer-to-listing matching for two-sided marketplaces.

View

AI for creator platforms

Search and recommendation over creator content, so audiences find the right thing by describing it.

View

Pricing

Audit, Sprint, and MVP bands, with the cost model sketched before any code is written.

View

Think your app needs search that understands intent?

30 minutes, free. We will look at your real queries, tell you honestly whether embeddings beat plain search for your case, what it costs to run, and what we would ship first. Shuhel will be on the call.

Book a 30-minute call