Overview — What this analysis covers and why it matters

As of November 2026, enterprises operating vector search at 100 million vectors and above face a new balance of performance, cost and operational complexity. Improvements in storage hardware, wider adoption of learned-quantization techniques, and richer observability have changed practical choices since mid‑2026. This update synthesizes field experience, vendor feature trajectories and verifiable heuristics so architects can choose among HNSW-style graphs, IVF+PQ, and SSD-first (DiskANN-style) designs with current constraints and SLAs in mind.

Background — What changed since the first pilots

Three material shifts accelerated during 2025–2026 and are now operational norms:

  • Embedding engineering matured: Teams increasingly use task-tuned embeddings and smaller student models produced by distillation and quantization-aware training. The practical sweet spot moved toward 1,024–2,048 dimensions for many production retrieval tasks, reducing raw storage and improving compressibility without material quality loss.
  • Storage and compute co-design: NVMe performance gains, NVMe-over-Fabrics (NVMe-oF) maturation, and early deployments of computational-storage (programmable SSD prototypes) have reduced SSD-first tail latency. Cloud providers and vendors now offer instance classes and managed tiers explicitly optimized for vector workloads.
  • Observability and per-query adaptation: Per-query recall estimation, difficulty classification and adaptive probe schedulers have moved from research into standard production tooling. Operators can trade recall and latency per query rather than using a single global policy.

Data and evidence — updated numbers and practical ranges (late‑2026)

Use the ranges below as planning heuristics. Replace dimensionality and hardware specs with your values to re-run the math.

Storage and memory math (updated examples)

  • Raw float32 storage per vector: dims × 4 bytes. For 1,024 dims ≈ 4 KB; for 2,048 dims ≈ 8 KB. At 100M vectors that’s ~400 GB and ~800 GB respectively for raw arrays (pre-compression).
  • PQ / OPQ and learned quantizers: production codebook sizes still commonly range 32–128 bytes per vector. With OPQ + 32B PQ a 100M × 1,024-dim corpus can be stored in the low tens of GB for codes plus index metadata; 128B codes push that toward the low hundreds of GB once metadata and inverted-list overhead are included.
  • HNSW memory: modern compressed-HNSW implementations (compressed edges + 8‑bit quantized node vectors) often require hundreds of bytes to ~2 KB per vector depending on degree and metadata. That means ~30 GB–200 GB per 100M vectors at the very aggressive low end, and multiple terabytes where full precision and higher degrees are used.
  • SSD-first / DiskANN-style: these hybrids commonly target tens to a few hundred GB of RAM for metadata/cache and use NVMe for codes. With modern NVMe and hot-set caching sizes of 10–30% many deployments report p95 latencies in the tens of milliseconds for typical queries; mixed workloads and low cache hit rates still push p95 into the low hundreds of ms.

Performance axes to measure (updated)

  1. Recall at tail latencies: Measure P@k or nDCG for your query mix at p95/p99 under realistic filtering and concurrency.
  2. End-to-end cost per query: Include RAM, NVMe, CPU/GPU embedding time, network (NVMe-oF/RDMA) and operational labor; include cloud instance on-demand vs reserved pricing for a 12‑month horizon.
  3. Update and rebuild economics: Record full rebuild time, incremental write latency, and the operational cost of background re-quantization or compaction.
  4. Filter and multi-tenant worst cases: Attribute filters and high-cardinality predicates remain the dominant source of tail latency increases—benchmarks must include realistic distribution of filters.

Index families and what’s changed in 2026

HNSW (graph-based, primarily in-memory)

  • Strengths: Still the best raw latency and top-tier recall for workloads where the working graph fits in RAM. Recent improvements—compressed edge encodings, 8-bit / learned vector quantization, and incremental compaction—have lowered RAM requirements by a meaningful factor for many deployments.
  • Weaknesses: Memory and rebuild costs remain limiting at 100M+ vectors if you require high-degree graphs or full-precision vectors. Insert/delete churn still drives compaction work and periodic rebuilds.
  • When to pick: Consumer-facing experiences where p99 latency must be single-digit to low‑double-digit milliseconds and the budget supports TBs of RAM across a sharded fleet.

IVF + PQ (inverted lists + product quantization)

  • Strengths: Predictable storage economics and better support for high update velocity and attribute-aware sharding. Adaptive probe scheduling has become standard, narrowing latency gaps with HNSW for many enterprise query mixes.
  • Weaknesses: Sensitive to probe budget and PQ granularity; high-cardinality filters or partitions orthogonal to IVF keys can force scanning many lists and increase tail latency.
  • 2026 nuance: Learned codebooks and online re-quantization pipelines let teams get better recall with equal or smaller code sizes, making IVF+PQ more attractive where cost and update velocity matter.

DiskANN and SSD-optimized hybrids

  • Strengths: Scale to hundreds of millions to billions by keeping most data on NVMe and a compact in-RAM working set. NVMe-oF and tuned caching policies now make p95 latency competitive for many production use cases.
  • Weaknesses: Engineering complexity—cache sizing, I/O scheduling, and network topology—remains high. Tail latency still depends on hot-set locality and SSD behavior under load.
  • When to pick: Massive archives, read-heavy corpora, or tight RAM budgets where p95/p99 latencies in the tens-to-low-hundreds of milliseconds are acceptable.

Multiple perspectives — who prioritizes what in 2026

Stakeholders still emphasize different axes:

  • Product teams: Demand strict p99 SLAs and consistent user experience; they often accept higher cost for HNSW or RAM-heavy compressed graphs.
  • SRE/Platform engineers: Favor predictable TCO and operational simplicity—often IVF+PQ or SSD-first hybrids with automated tiering, observability and autoscaling policies.
  • Data scientists: Prioritize retrieval quality and freshness; they push for architectures that support fast reindexing, online vector updates, or controlled retraining of quantizers.
  • Security & Compliance: Now a first-class concern—vector encryption-at-rest, access control per-tenant, and provenance for embeddings must be part of architecture choices, especially for regulated industries.

Practical architectures and current examples

Patterns validated in late‑2026:

  • Low-latency consumer search: Compressed HNSW on RAM-optimized nodes, OPQ + 32–64B learned codes, per-query adaptive pruning and a warm NVMe cache for cold starts. Targets: p99 80–120 ms with top-10 recall > 0.9 on many mixes; cost: multi-node TB RAM clusters.
  • Multi-tenant catalog with frequent updates: Sharded IVF+PQ partitioned by tenant/time with append-only writes and background re-quantization. Typical p95: 40–150 ms depending on probe budget and filter density; this pattern minimizes rebuilds while keeping per-tenant isolation.
  • Billion-scale archive search: NVMe-first hybrid with a compact RAM graph for hot items and distributed NVMe (local NVMe + NVMe-oF) for codes. Typical p95: 50–250 ms depending on cache hit rates; large savings on RAM vs full in-memory graphs.

Operational checklist — what to benchmark before committing (updated)

  1. Define recall targets at p95/p99 for your production query mix (include heavy filters and cold-start queries).
  2. Benchmark compression (OPQ, PQ, learned quantizers, 8-bit/4-bit encodings) on representative subsets; validate end-to-end quality after quantizer retraining and incremental updates.
  3. Simulate update/delete churn and measure rebuild/compaction cost under expected write loads and retention policies.
  4. Run filtered and multi-tenant benchmarks using realistic filter distributions; measure worst-case shard fanout and cross-shard join costs.
  5. Instrument per-query recall estimates, cache hit rates, and probe budget adaptation; use these signals to route hard queries to higher-cost but higher-recall tiers.
  6. Measure end-to-end cost per useful result (not just per query): include embedding compute, network egress, and DAO (data ops) overhead for reindexing.
  7. Validate compliance needs—encryption at rest for vectors, retention controls, and access auditing for multi-tenant scenarios.

Implications — what this means for enterprise teams

Choosing an index family at 100M+ vectors is now a combined engineering, financial and governance decision. Recent advances push hybrid and tiered architectures into the default toolkit: use RAM/graph where UX demands tight tails, SSD-first for massive scale and cold archives, and IVF+PQ for multi-tenant, update-heavy catalogs. Observability—per-query recall prediction, cache telemetry and rebuild metrics—has become the deciding factor in whether a lower-cost architecture satisfies SLAs. Teams that automate probe adaptation and routing can often reduce infra spend without sacrificing perceived quality.

Outlook — what to watch for next

  • Hardware-software co-design: Wider commercial availability of computational SSDs and tighter NVMe-oF/RDMA stacks could further shrink SSD-first tail latency and simplify hybrid architectures.
  • Learned quantization & model co-training: Expect more production use of task-specific codebooks and quantizers trained jointly with retrieval objectives, reducing code sizes while preserving quality.
  • Managed services and migration paths: Vendors continue to add adaptive presets and transparent tiering; look for services that can auto-migrate corpus segments between index types as access patterns change.
  • Standardized observability: Industry tooling is converging on per-query recall telemetry and standardized benchmark suites—adopting these will accelerate trustworthy comparisons across index types.

Frequently asked questions

Is HNSW still worth it at 100M vectors?

Yes—if your product requires tight tail latency and high recall and you can budget for the RAM and operational overhead. Compression and compressed-edge encodings have reduced the required RAM in many deployments, but HNSW still typically demands more memory than IVF or SSD-first hybrids unless you accept aggressive quantization.

When should I prefer DiskANN-style or SSD-first architectures?

Choose SSD-first when you need to scale to hundreds of millions or billions of vectors, you have a constrained RAM budget, and you can tolerate p95/p99 latencies in the tens-to-low-hundreds of milliseconds. They also fit read-heavy archives and corpora with infrequent updates.

Can IVF+PQ handle complex attribute filtering at scale?

Yes—if you partition or shard inverted lists by the most common filter attributes (tenant, date, category). When filters are high-cardinality or orthogonal to your IVF partitioning, expect increased tail latency because more lists must be scanned; realistic filter-distribution tests are essential.

Should I start with a managed service or build in-house?

For most teams uncertain about scale, start with a managed vector DB that exposes multiple index types and realistic scaling presets. However, run representative benchmarks and simulate update patterns early—managed presets can mask nonlinear costs that appear at 100M+ vectors and when heavy filtering is required.

How important is per-query adaptability and observability?

Crucial. Adaptive probe budgets, per-query difficulty classification, and routing hard queries to higher-quality indexes allow you to meet strict SLAs while reducing cost overall. Invest early in per-query recall estimators, cache telemetry and automated routing policies—these are now production best practices.

At enterprise scale, aim for an architecture that transparently trades cost and latency per query, automates rebuilds or compactions, secures vectors for compliance, and instruments recall. That combination will keep vector search predictable and maintainable as you move from 100M vectors toward billions.

Further reading and vendor materials

  • FAISS, HNSW and PQ implementation notes for compression tradeoffs
  • DiskANN research and commercial notes for NVMe-first patterns
  • Vendor guides and managed options (Pinecone, Qdrant, Milvus, Weaviate, Redis) to test presets at scale