Choosing Between HNSW and IVFFlat After Your Data Grows

Three signals do most of the work: row count, update rate and the recall your application actually needs. The table below converts them into a decision, drawing on the published thresholds and the parameter behavior each source describes.

The first three rows summarize published guidance from BigData Boutique. The memory row and the quantization advice come from pgvector's maintainer, who recommends halfvec or binary quantization with re-ranking to shrink HNSW index size the pgvector maintainer on index builds.

The row-count boundary deserves a worked example rather than a slogan. Suppose a table starts at 200,000 chunks of 768-dimension embeddings, the configuration in one reported production case the pgvector issue thread. With IVFFlat and lists set to a common rule of thumb (rows divided by 1,000 for tables up to a million), that is 200 lists. Ingest then adds 50,000 new chunks a month. After a year the table is 800,000 rows but the index still has 200 lists, so each list now averages four times the vectors it was trained on. A query that probes 10 lists scans a much larger fraction of the data than it was tuned for, latency climbs, and recall against the drifted distribution falls. The fix is a rebuild, and the rebuild cadence now tracks ingest rate rather than anything you control. Past a million rows that arithmetic gets worse every month.

Here is the same reasoning expressed as a decision rule you can run against your own table:

1. If your p95 recall target is below about 0.90 and your table is mostly static between batch loads, IVFFlat with tuned lists and probes is the cheaper answer.
2. If any two of these are true, budget for HNSW: more than a million rows, recall above 0.95, writes arriving continuously.
3. If the table is past roughly 50 million vectors and memory is the binding constraint, neither default wins; go to IVFFlat with quantization or HNSW with halfvec plus re-ranking.

For IVFFlat, the parameter that matters is lists, set at build time and frozen afterwards. The pgvector maintainer's rule of thumb is rows divided by 1,000 for tables up to a million rows and the square root of rows above that the BigData Boutique comparison. Also populate the table with at least 50,000 to 100,000 representative vectors before you build, because the cluster boundaries are derived from whatever data exists at that moment the BigData Boutique comparison.

For HNSW, m controls the number of connections per node and ef_construction controls how hard the build searches when inserting each node. Raising both improves accuracy but slows the build, and the effect is steep: the Crunchy Data guide warns that higher values slow index build drastically while not necessarily improving query performance the Crunchy Data guide. pgvector's maintainer's advice for a faster build is to use the default ef_construction of 64 the pgvector maintainer on index builds.

At query time, HNSW's knob is ef_search. To return more than 40 nearest neighbors you must raise SET hnsw.ef_search to at least the number of neighbors you want back, per the Crunchy Data guide, which also flags the most common silent failure: queries that quietly do not use the index at all, diagnosed with EXPLAIN the Crunchy Data guide.

A build that takes hours is usually a memory setting, not an algorithm problem. In the reported case, a 40-million-row partition build warned that the HNSW graph no longer fit into maintenance_work_mem after 570,905 tuples and was still at 29.2 percent after 19 hours. Two causes were identified: SET LOCAL was used with CREATE INDEX CONCURRENTLY, which runs outside the transaction LOCAL applies to, and max_parallel_workers was too low for the requested 11 maintenance workers the pgvector issue thread. The non-local form is what you want:

SET maintenance_work_mem = '40GB';
SET max_parallel_maintenance_workers = 8;
SET max_parallel_workers = 8;
CREATE INDEX CONCURRENTLY items_embedding_hnsw
  ON items USING hnsw (embedding vector_cosine_ops)
  WITH (m = 16, ef_construction = 64);

Parallel index builds are the single biggest lever on HNSW build time; one cloud provider measured a 30x improvement on a 10-million-row, 1,536-dimension dataset on a 64 vCPU instance the Neon build-time report.

The honest case against HNSW is memory. HNSW uses roughly 2 to 5 times more memory than IVFFlat, so if the graph does not fit in RAM, query latency collapses because the graph traversal becomes disk-bound rather than logically slow the BigData Boutique comparison. A managed instance with a fixed memory ceiling is a legitimate reason to stay on IVFFlat.

The second case is build time against a shutdown window. HNSW indexes always build slower and use more memory than IVFFlat the Scaling Postgres episode summary, and that summary also captures the structural charm of IVFFlat's flat storage: it is simple, fast to build, and usable as a coarse quantizer on very large datasets Jonathan Katz on pgvector indexes. If your data lands in weekly batch loads and queries tolerate recall near 0.90, a scheduled rebuild is a small, predictable cost.

The third case is very large static corpora. Past roughly 50 million vectors with memory as the binding constraint, the recommended path is IVFFlat with product or binary quantization rather than plain HNSW the BigData Boutique comparison. If you later need more recall at the same memory budget, HNSW over halfvec with a re-ranking pass over the compressed candidates is the documented escape hatch the pgvector maintainer on index builds.

One question worth asking before you migrate: is the index the bottleneck, or is the retrieval quality problem actually in how the corpus was chunked? Small, focused chunks produce precise embeddings, while oversized chunks average several ideas into one noisy vector that no index can rescue the Weaviate chunking guide. Chunking directly determines how effectively an agent can retrieve, reason over and reuse information, and when a RAG system performs poorly the issue is often the chunks rather than the retriever the Weaviate chunking guide. If you are also running keyword search alongside vectors, the fusion layer matters as much as the index: reciprocal rank fusion combines ranked lists by position rather than raw score, which sidesteps the problem that BM25 scores and cosine similarity are on incompatible scales the OpenSearch RRF writeup.

The migration is a drop and recreate, not an in-place conversion. Plan for a maintenance window or a concurrent build, and measure recall before and after so you can prove the change helped.

1. Capture a baseline. Sample 100 to 1,000 real queries, compute exact ground truth with a sequential scan (SET enable_indexscan = off), and record recall at your current probes setting.
2. Size the memory. Estimate the HNSW graph cost before you build, and confirm maintenance_work_mem can hold the graph; the "graph no longer fits" notice is your early warning that the build will slow dramatically the pgvector issue thread.
3. Build concurrently. Use CREATE INDEX CONCURRENTLY with the non-local SET statements shown earlier so parallel workers actually apply, and set ef_construction at the default 64 for the first pass the pgvector maintainer on index builds.
4. Drop the old index in the same change window. In pgvector both methods live on the same column and operator class, so the planner picks the single usable index; keeping the IVFFlat index alongside HNSW wastes space and can confuse planning.
5. Re-measure recall against the same sampled queries and tune ef_search until you clear the target.
6. Evidence the result. Record the recall number you accepted so future drift is detectable; a tool like the pg_recall_guard extension measures recall of any vector index against exact ground truth by discovering indexes through the catalog and reports when recall drifts from an approved baseline the pg_recall_guard release notes. Its framing is worth keeping: a degraded ANN index does not fail, it returns k plausible neighbors and never mentions the good ones it left out the pg_recall_guard release notes.

Going the other direction, HNSW to IVFFlat, is the same steps with the parameters swapped: build on a fully populated table with at least 50,000 to 100,000 representative vectors so the k-means centroids are meaningful, then tune probes back up to recover recall the BigData Boutique comparison. Expect to lose a few points of recall and gain memory headroom and a shorter build.

If your team is running several embedding models from different providers across these pipelines, consolidating the calls behind one endpoint keeps the re-embedding step in a migration script to a single credential and billing line; Quanta Inference exposes text, image, video, audio and embedding models through one OpenAI-compatible key.

Three things change after the switch, and each shows up on a different dashboard.

Vacuum and bloat. HNSW indexes are more expensive to maintain under churn. The Scaling Postgres episode pairs HNSW index behavior with autovacuum tuning for high-churn tables precisely because the two interact the Scaling Postgres episode. If you switch to HNSW on a table with heavy update traffic, lower the autovacuum scale factors and check bloat on the index as well as the heap.

Tuning at query time. IVFFlat has one query knob (probes) and one build knob (lists). HNSW has one query knob (ef_search) and two build knobs (m, ef_construction). More knobs means more to document, and the ef_search default caps how many neighbors you can retrieve until you raise it the Crunchy Data guide.

Recall drift. With IVFFlat, drift is structural and the answer is a scheduled rebuild. With HNSW, drift is subtler: it comes from deletions leaving tombstones and from the graph slowly losing quality as the distribution moves. Either way, monitoring recall on a fixed query set is what turns "search feels worse" into an alert the pg_recall_guard release notes.

A defensive checklist for the maintenance runbook, derived from the sources above:

• Check pg_stat_progress_create_index during any build, and act on the maintenance_work_mem notice rather than waiting it out the pgvector issue thread.
• Run EXPLAIN on the top five production queries after any index change to confirm the index is used the Crunchy Data guide.
• Keep a pinned query sample with exact ground truth and record the recall number you signed off the pg_recall_guard release notes.
• Tune autovacuum for high-churn vector tables before, not after, the first bloat incident the Scaling Postgres episode.
• If you also serve lexical search from the same corpus, compare the fused result quality, not just vector recall, since rank-based fusion and score normalization behave very differently under drift the OpenSearch RRF writeup.