// AI — 2026-11-13 — 8 min

What Is a Vector Database and Why Does RAG Depend on It?

Vektör VeritabanıRAGEmbeddingAI Altyapısı

What a vector database actually is, how it differs from a regular database, and why it becomes unavoidable once a RAG system grows past a handful of documents.

You built a RAG chatbot over your company's documents, and with 200 of them it works great: ask the right question, get the right paragraph back. Then the document count grows to 50,000 chunks, and the same questions start returning irrelevant results while search noticeably slows down. The problem isn't the model you're using — it's that you're trying to search at this scale without a vector database, the piece of infrastructure built to find meaning-similar text in milliseconds.

This is the first unpleasant surprise most RAG projects run into: everything works perfectly in the demo, and then the moment real data volume hits, retrieval quality collapses and the team starts blaming the model or the prompt — when the piece that's actually missing is a completely different layer.

##What Is a Vector Database and How Is It Different from a Regular Database?

An embedding model turns a piece of text (or an image) into a list of numbers that represents its meaning — that list is a vector, typically between 384 and 3,072 numbers long depending on the model. A relational database is fast at exact-match and range queries like 'id = 5' or 'date > 2026-01-01', using row-based index structures built for that. A vector database answers a completely different question: 'which of the stored vectors are closest to this one?' That means searching for proximity in a space with hundreds or thousands of dimensions, and it has nothing to do with exact matching.

>A simple example

"The dog was running in the park" and "my puppy was playing in the yard" share no common words, yet an embedding model places both close together in meaning-space because both describe 'an animal moving outdoors.' "The stock closed higher today" lands far away from both. A vector database is the infrastructure that finds this kind of closeness across millions of records quickly — keyword search can't catch this paraphrase at all, because there's no shared word to match on.

##What a Vector Database Actually Does (the Technical Part)

The simplest approach is comparing the query vector against every stored vector one by one — brute force. It's always correct, but it scales linearly, and on ordinary hardware it starts to feel slow for a real-time application somewhere past the tens of thousands of vectors. So vector databases build an index — graph-based structures like HNSW, or cluster-based ones like IVF — and shift from 'find the exact closest' to 'find vectors that are very likely the closest, extremely fast.' This is called approximate nearest neighbor (ANN) search; tuned well, it hits a 95-99% recall rate at a small fraction of brute force's time cost.

  • Dimension: how many numbers make up the vector — typically 384 to 3,072 depending on the model
  • Distance metric: how similarity is measured — cosine, dot product, or Euclidean
  • Top-k: how many results come back per query
  • Metadata filtering: narrowing the search by category, date, or user before the similarity search runs
  • Index type: the lever that trades speed, memory use, and accuracy against each other

Put in numbers, the gap is obvious: on a store of 500,000 vectors, brute-force comparison can take 200-800 milliseconds per query depending on hardware — not something a user will wait for in a chatbot. On the same store, a well-tuned HNSW index typically answers the same query in 5-20 milliseconds, and what you give up is a small chance that the second-best match comes back instead of the absolute best one in rare cases. For most applications, that trade is well worth taking.

Index choice is a genuine trade-off: a more aggressive ANN setting speeds up queries but occasionally misses the actual best match; a more conservative setting is more accurate but costs more memory and time. Shipping with the default setting, without measuring it against a small test set of known question-answer pairs, means you won't notice when production starts quietly returning the wrong answers.

##Why It Sits at the Heart of RAG

A RAG system follows a fixed sequence: split a document into chunks, turn each chunk into an embedding, store the vector alongside the original text and metadata; at query time, embed the question too, ask the store for the closest chunks, and feed those chunks to the LLM as context. Without a vector database, the 'find the relevant chunk' step can only happen two ways: brute-force comparison, which holds up for hundreds of chunks but not tens of thousands, or keyword search, which misses the question entirely the moment the user phrases it differently from the source text. The vector database is the piece that makes RAG's 'retrieval' step actually work at real scale — it's why RAG holds up in production and not just in a demo.

To make the gap concrete: a user asks "how do I invite my teammate" while the documentation uses the phrase "adding a member." Keyword search finds nothing here, because there's no shared word; vector search catches that both phrases mean 'adding someone to the workspace' and surfaces the right article anyway. This small difference shows up constantly in day-to-day RAG use, because real users never phrase their questions in the exact words the documentation happens to use.

##A Real Scenario: Finding the Right Three Articles Out of 50,000

A SaaS company wants to wire its 3,000 support articles into RAG. Each article averages 1,200 words, split into chunks of 500-800 tokens with roughly 15% overlap between them — the overlap exists so a sentence that would otherwise get cut in half between two chunks doesn't lose its context. That produces around 50,000 chunks total. Each chunk is run through a 768-dimension embedding model and written to the vector database alongside metadata: article title, product category, and last-updated date.

The first test produces odd results: a user asks 'how do I cancel my invoice' and the system surfaces the 'how to generate an invoice' article — close in meaning, wrong door. This usually traces back to one of three causes: chunks were left too large and several topics got mixed into one chunk; the embedding model is general-purpose and can't distinguish the product's own feature names from generic terms; or no metadata filter was applied at all, even though articles in the 'cancellation' category could have been narrowed down first. The fix gets applied in order: shrink the chunk size, add a category filter to the query, and finally swap in a more recent version of the embedding model. Done together, the hit rate for surfacing the right article improves noticeably — but testing all three changes at once means you'll never know which one actually mattered.

##Which One: pgvector or a Dedicated Vector Database?

If you're already on Postgres, the pgvector extension adds vector search without running a separate system — it performs well at small-to-medium scale, from a few tens of thousands up to a few million records, with close to zero added operational overhead. Dedicated vector databases like Pinecone, Weaviate, Qdrant, or Milvus earn their place once you're dealing with tens to hundreds of millions of vectors, very high write throughput, or advanced filtering and multi-tenant isolation — in exchange, you're now running (or paying for) a separate system.

  • Data size: how many vectors you'll actually store
  • Latency requirement: how tight the millisecond budget is
  • Existing stack: whether you're already running Postgres
  • Ops capacity: whether you have the bandwidth to run a separate system
  • Filtering complexity: how many metadata conditions your queries carry

Practical ceiling for brute force

~10,000-50,000 vectors (hardware-dependent)

pgvector's comfortable range

a few hundred thousand to a few million vectors

Typical query latency with an ANN index

5-50 ms

Cost shapes the decision too: adding pgvector to Postgres costs close to nothing extra in practice — it rides on database resources you're already paying for. A managed dedicated vector database service is usually priced by vector count and query volume; a small project might run a few tens of dollars a month, while a large, high-traffic setup can climb into the thousands. Self-hosting an open-source vector database (Qdrant, Milvus) lowers that bill but shifts the operational load onto your own team — choosing between them really comes down to whether you'd rather spend money or engineering time.

There's also an honest limit worth naming: vector search is good at finding what's semantically close, but it can be surprisingly bad when you're looking for an exact product code, an error number, or a specific date — those need exact matching, not meaning. That's why mature RAG setups usually run hybrid search: vector search for semantic similarity, classic keyword or metadata filtering for exact matches, combined.

##FAQ

>What exactly is the difference between a vector database and a regular relational database?

A relational database answers 'is this equal' and 'is this in range' quickly; a vector database answers 'which one is closest.' Neither replaces the other — they're usually run together, with structured data (user, date, category) on the relational side and meaning representation (embeddings) on the vector side.

>After how many records do you actually need a vector database?

Up to a few hundred thousand records, brute-force comparison (even inside a simple query function) stays fast enough in practice. The real difference shows up once you're past the tens of thousands and need real-time, sub-second responses. So the question isn't 'how many records' — it's 'how fast does the answer actually need to come back.'

>Is pgvector enough, or do you need a dedicated vector database?

If you're already on Postgres and your data stays under a few million vectors, starting with pgvector makes sense — you avoid running an extra system. Once data crosses into the tens of millions, filtering gets genuinely complex, or you're handling thousands of queries per second, it's worth evaluating a dedicated vector database.

>What does an embedding's 'dimension' mean, and why does it matter?

Dimension is how many numbers the text gets compressed into — commonly between 384 and 3,072 depending on the model. A higher dimension usually carries more nuance of meaning, but it also costs more to store and search. Mixing embeddings of different dimensions in the same project (for example, switching models without re-embedding the old data) silently breaks results, because distance can't be meaningfully measured between vectors of different lengths.

>How often do you need to update the embedding model or re-embed your data?

When you switch embedding models (moving to a better version), you need to re-embed all existing data with the new model — old and new vectors don't live in the same meaning-space, and mixing them silently produces wrong results. If you keep the model fixed, you only need to embed documents that are new or changed; for a large content library, that's usually set up as a daily or weekly incremental job, and re-embedding everything from scratch is rarely needed.

A vector database alone isn't magic — the chunking and embedding side of RAG matters just as much. You can read how the full RAG setup works and what an embedding actually is through the links here; if you want to talk through what setup makes sense for your own dataset, reach out through the contact page.

// LET'S WORK

Planning a similar SaaS product?

We can define scope, MVP milestones, and a realistic delivery timeline together.

> CONTACT