How to Build Embeddings and Vector Search

Every vector search system has two halves: an ingestion pipeline that runs when content changes, and a query path that runs on every search.

Ingestion:

1. Extract clean text from the source, removing navigation, boilerplate and markup.
2. Chunk the text into passages small enough to embed and specific enough to be useful.
3. Embed each chunk with the chosen model.
4. Store the vector alongside the chunk text, the source document ID, and metadata such as title, URL, date and access permissions.
5. Index the vectors once the table is large enough to need it.

Query:

1. Embed the user's query with the same model.
2. Search for the nearest chunks, applying any metadata filters.
3. Optionally combine the results with keyword search and rerank.
4. Return the chunks, or pass them to a language model as context.

Keep the model name and version stored with every vector. If you change embedding models, vectors from the old and new models are not comparable, and everything must be re-embedded.

Pinecone's guide gives two reasons to chunk: so that embedding models can fit the data into their context windows, and so that the chunks themselves contain the information necessary for search. Text beyond a model's context window is usually truncated before embedding, and a chunk that mixes several topics produces a vector that represents none of them well.

The same guide offers a test that works better than any formula: if a chunk makes sense to a human without the surrounding context, it will make sense to the model too. If chunks are too small, they lose context. If they are too large, the meaning of any single passage is diluted.

The common strategies, roughly from simplest to most involved:

Pinecone recommends starting with fixed-size chunking: it will be the best path in most cases, and you should iterate only after finding it insufficient. A recursive splitter, such as the one LangChain implements, is a good middle ground because it tries paragraph and sentence boundaries before falling back to words.

Two habits improve almost any strategy.

• Prepend context. Add the document title and section heading to each chunk before embedding, so "Step 3: restart the service" carries the name of the product and procedure it belongs to.
• Use overlap. A small overlap between consecutive chunks keeps a sentence that straddles a boundary findable.

If you already run Postgres, the pgvector extension lets you store vectors next to your other data, join them with normal tables, and filter with ordinary SQL. The pgvector README lists support for exact and approximate nearest neighbor search and for L2, inner product, cosine, L1, Hamming and Jaccard distances.

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE chunks (
  id          bigserial PRIMARY KEY,
  document_id bigint NOT NULL REFERENCES documents(id),
  content     text NOT NULL,
  model       text NOT NULL,
  embedding   vector(1024) NOT NULL
);

-- Nearest chunks by cosine distance
SELECT id, content, 1 - (embedding <=> $1) AS similarity
FROM chunks
ORDER BY embedding <=> $1
LIMIT 10;

The operators are <-> for L2 distance, <#> for negative inner product and <=> for cosine distance. The vector size in the column definition must match the dimension count of your embedding model; the value above is only an example.

An important default: pgvector performs exact nearest neighbor search, which provides perfect recall, until you add an index. For tens of thousands of rows, exact search is often fast enough, and it is always correct. Add an index when query latency, not accuracy, becomes the problem. The README also lists the maximum dimensions each vector type can index, which matters if your model produces very large vectors.

Approximate indexes speed up search by not comparing the query with every vector. The pgvector README warns that this trades some recall for speed and that, unlike typical indexes, you will see different results for queries after adding an approximate index.

pgvector offers two index types.

HNSW is the usual default because it gives better results per millisecond and can be created on an empty table. IVFFlat suits very large tables where build time or memory is the constraint. The README suggests starting IVFFlat with rows / 1000 lists for up to a million rows and probes around the square root of the list count.

CREATE INDEX ON chunks USING hnsw (embedding vector_cosine_ops);
SET hnsw.ef_search = 100;  -- higher means better recall, slower queries

Filtering needs special care. With approximate indexes, the README, as read on 2026-10-17, explains that filtering is applied after the index is scanned, so with the default ef_search of 40 and a filter matching 10% of rows, only 4 rows will match on average. The fixes it lists are iterative index scans, which keep scanning until enough rows match, partial indexes for a few common filter values, and partitioning for many values. A B-tree index on a selective filter column can also make exact search fast enough to skip the vector index entirely.

Vector search is weak at exact matches. pgEdge's hybrid search article shows the problem with a technical query: searching for a specific configuration parameter, the model may rank a general replication article above the one that specifically explains that parameter, because it treats all replication content as similar. BM25, the classic keyword ranking algorithm, has the opposite profile: it finds exact terms reliably and fails at conceptual questions.

Hybrid search runs both and merges the lists. The difficulty is that the scores are in different units. Reciprocal rank fusion (RRF) avoids the problem by ignoring scores and using only positions. OpenSearch's announcement of RRF explains that, unlike min-max or L2 normalization, it uses a rank-based aggregation strategy, which keeps one method's score distribution from dominating and favors documents that rank well in several lists.

The formula is short. Each document's score is the sum, over every result list it appears in, of 1 / (k + rank). A hybrid search reference from Digital Applied notes that k = 60 is Elasticsearch's production default and recommends starting there.

def rrf(result_lists, k=60):
    scores = {}
    for results in result_lists:
        for rank, doc_id in enumerate(results, start=1):
            scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank)
    return sorted(scores, key=scores.get, reverse=True)

merged = rrf([vector_results, keyword_results])

In Postgres, the keyword side can use built-in full-text search with a GIN index on a tsvector column, so both searches run in the same database. A cross-encoder reranker over the top merged results can add further precision when latency allows.

Changing chunk sizes, models, index settings or fusion weights without measurement is guesswork. Digital Applied's practical recommendation is to evaluate on your own query set with NDCG or MRR before adding complexity such as reranking.

A small evaluation set goes a long way.

1. Collect 50 to 200 real queries from search logs, support tickets or users.
2. For each, record which documents or chunks a correct answer should include.
3. Measure recall at k (did the right chunk appear in the top results?) and a rank-aware metric such as MRR or NDCG.
4. Change one thing at a time and rerun the set.
5. Compare approximate index results with exact search on the same queries to see how much recall the index costs.

Keep the set under version control and rerun it whenever you change the model, chunking or index settings. Quanta Intelligence routes requests to the right agent with this same approach, embeddings plus curated terms, and evaluation queries are what show whether a change helped.