Scaling for Query Throughput (QPS)

Throughput scaling means handling more parallel queries per second. This is different from latency. Segment count pulls throughput and latency in opposite directions, so pick a priority per collection.

High throughput favors fewer, larger segments so each query touches less overhead.

Performance Tuning for Higher RPS

• Use fewer, larger segments (default_segment_number: 2) Maximizing throughput
• Enable quantization pinned in RAM to reduce disk IO: memory: pinned on Qdrant 1.19 or newer, always_ram: true on 1.18 or older Quantization
• Use batch search API to amortize overhead Batch search

Minimize impact of Update Workloads

• Configure update throughput control (v1.17+) to prevent unoptimized searches degrading reads Low latency search
• Set optimizer_cpu_budget to limit indexing CPUs (e.g. 2 on an 8-CPU node reserves 6 for queries)
• Configure delayed read fan-out (v1.17+) for tail latency Delayed fan-outs

Horizontal Scaling for Throughput

If a single node is saturated on CPU after applying the tuning above, scale horizontally with read replicas.

• Shard replicas serve queries from replicated shards, distributing read load across nodes
• Each replica adds independent query capacity without re-sharding
• Use replication_factor: 2+ and route reads to replicas Distributed deployment

See also Horizontal Scaling for general horizontal scaling guidance.

Disk I/O Bottlenecks

If it is not possible to keep all vectors in RAM, disk I/O can become the bottleneck for throughput. In this case:

• Upgrade to provisioned IOPS or local NVMe first. See impact of disk performance to vector search in Disk performance article
• Use io_uring on Linux (kernel 5.11+) io_uring article
• In case of quantized vectors, prefer global rescoring over per-segment rescoring to reduce disk reads. Example in the tutorial
• Configure higher number of search threads to parallelize disk reads. Default is cpu_count - 1, which is optimal for RAM-based search but may be too low for disk-based search. See configuration reference
• If still saturated, scale out horizontally (each node adds independent IOPS)

What NOT to Do

• Do not expect one segment configuration to maximize both throughput and latency: pick a priority per collection
• Do not use many small segments for throughput workloads (increases per-query overhead)
• Do not scale horizontally when IOPS-bound without also upgrading disk tier
• Do not run at >90% RAM (OS cache eviction = severe performance degradation)