Vector databases are specialized systems designed to store, index, and query high-dimensional vector embeddings for AI applications like semantic search and RAG.
If you're building AI-powered features, you'll eventually need to retrieve information based on meaning, not just keywords. That's where Vector Databases come in. They solve the core problem of similarity search at scale, allowing your application to find data points "close" to a given query in a multi-dimensional space. As a full-stack developer, understanding them is becoming as fundamental as knowing your primary SQL or NoSQL store.
Why Vector Databases Matter (and When to Skip Them)
Vector databases matter because traditional databases fail at semantic search. A LIKE '%query%' clause can't find documents discussing "canine companionship" when a user searches for "dogs." Vector databases enable this by comparing the mathematical "meaning" encoded in embeddings.
However, be opinionated about their use. You can skip a dedicated vector database entirely if your scale is small. For a simple prototype or a filtered dataset under 10,000 embeddings, you can often get away with PostgreSQL using the pgvector extension or even an in-memory library like hnswlib in your application layer. Adding a complex new infrastructure piece is premature optimization if you're just testing an AI feature's viability.
Getting Started with Vector Databases
The fastest way to experiment is with a local, open-source option. We'll use ChromaDB for its simplicity and in-memory mode. First, install the client library.
npm install chromadb
Next, here's a minimal script to create a collection, add text embeddings, and query it. We'll use OpenAI's embedding API for simplicity, but the concept applies to any embedding model.
import { ChromaClient } from 'chromadb';
import OpenAI from 'openai';
const client = new ChromaClient({ path: "http://localhost:8000" });
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
async function vectorDemo() {
// 1. Create or get a collection
const collection = await client.getOrCreateCollection({
name: "demo_docs"
});
// 2. Create embeddings for your documents
const documents = [
"Dogs are loyal and friendly pets.",
"Programming in TypeScript improves code quality.",
"Paris is the capital city of France."
];
const embeddings = await Promise.all(
documents.map(async (doc) => {
const response = await openai.embeddings.create({
model: "text-embedding-3-small",
input: doc,
});
return response.data[0].embedding;
})
);
// 3. Add documents with their embeddings and IDs
await collection.add({
ids: ["id1", "id2", "id3"],
embeddings: embeddings,
documents: documents,
});
console.log("Data indexed.");
// 4. Query the collection
const queryText = "What are pets like?";
const queryEmbedding = await openai.embeddings.create({
model: "text-embedding-3-small",
input: queryText,
});
const results = await collection.query({
queryEmbeddings: [queryEmbedding.data[0].embedding],
nResults: 2,
});
console.log("Most similar documents:", results.documents);
}
vectorDemo();
This script outlines the universal workflow: embed your data, store the vectors, then query with an embedded prompt.
Core Vector Databases Concepts Every Developer Should Know
1. Embeddings: Turning Data into Vectors Everything—text, images, audio—must be converted into a numerical vector (a list of floats) using a model. This vector represents the data's semantic features in a high-dimensional space (often 1536 dimensions).
// Example: Generating an embedding vector
async function generateEmbedding(text: string): Promise<number[]> {
const response = await openai.embeddings.create({
model: "text-embedding-3-small",
input: text,
});
return response.data[0].embedding; // An array of ~1536 numbers
}
const vector = await generateEmbedding("Example sentence");
console.log(`Vector dimension: ${vector.length}`);
2. Similarity Search: Finding Nearest Neighbors The primary operation is finding the stored vectors closest to your query vector. Distance is measured by metrics like cosine similarity or Euclidean distance (L2). Lower distance means higher semantic similarity.
3. Indexing: The Magic of Fast Search Scanning every vector is O(n) and far too slow. Vector databases use approximate nearest neighbor (ANN) indexes like HNSW or IVF. These trade perfect accuracy for massive speed gains, enabling millisecond searches over billions of vectors.
4. Metadata Filtering: Combining Semantic and Traditional Search
You rarely want pure similarity search. You need to filter results by traditional attributes like user_id or date. A robust vector database lets you run a similarity search within a filtered subset.
// Querying with metadata filtering in Chroma
const filteredResults = await collection.query({
queryEmbeddings: [queryEmbedding],
nResults: 5,
where: { "category": { "$eq": "technology" } } // Filter for tech docs only
});
Common Vector Databases Mistakes and How to Fix Them
Mistake 1: Not Normalizing Embeddings for Consistent Distance Metrics If your embedding model doesn't output normalized vectors (unit length), your similarity calculations will be skewed. Cosine similarity, a common metric, assumes normalized vectors.
Fix: Normalize your vectors before insertion if needed. For cosine similarity, ensure each vector's magnitude is 1.
function normalizeVector(vec: number[]): number[] {
const magnitude = Math.sqrt(vec.reduce((sum, val) => sum + val * val, 0));
return vec.map(v => v / magnitude);
}
Mistake 2: Ignoring Index Tuning Using default index parameters for all datasets hurts performance. An index built for 1M dense vectors won't be optimal for 10K sparse ones.
Fix: Understand your data scale and access patterns. For HNSW, tune parameters like M (connections per node) and efConstruction (build-time search scope) based on your recall/latency requirements.
Mistake 3: Storing Large Original Documents in the Vector DB Vector databases are optimized for vectors and metadata, not bulk storage. Storing full PDF text or images blobs can degrade performance and increase costs.
Fix: Adopt a hybrid storage pattern. Store the vector embedding and a minimal metadata (e.g., a unique ID and source type) in the vector DB. Keep the full original content in your primary database (PostgreSQL, S3) and join them via the ID when returning results.
When Should You Use Vector Databases?
Use a vector database when you have a clear need for semantic or similarity-based retrieval over unstructured data. The most common triggers are: implementing Retrieval-Augmented Generation (RAG) for an LLM, building a recommendation system based on item content, creating a semantic search engine for internal documents, or detecting anomalies in complex datasets. If your search needs are purely keyword-based or your dataset is highly structured with clear IDs, a traditional database is likely sufficient and simpler.
Vector Databases in Production
First, separate your read and write concerns. Indexing (writing) is computationally heavy and can block queries. Many managed services (like Pinecone) handle this automatically, but if you're self-hosting, plan for asynchronous index updates during low-traffic periods.
Second, implement a robust embedding versioning strategy. If you change your embedding model (e.g., from text-embedding-ada-002 to text-embedding-3-large), all your old vectors become incompatible. Store the model name and version as metadata with each vector, and maintain separate collections or prefixes for each model version to allow safe migrations.
Finally, monitor what matters: query latency, recall rate (accuracy of your ANN index), and error rates from your application layer. These metrics are more telling than simple uptime for a vector database's health.
Start your next AI feature by prototyping semantic search with a local vector database before committing to a managed service.