You 🧑
Ok lets use int8 quant and 512d
Assistant 🤖
Great — 512-d + int8 changes the sizing a lot. That’s the configuration I’d probably choose for this workload unless a quality eval shows 1024-d float/scalar is materially better.
Assumptions:
Corpus: 100M searchable chunks/docs
Chunk length: 128–256 tokens
Embedding model: voyage-4-large
Embedding dimension: 512
Embedding dtype: int8
Query model: voyage-4-lite, 512-d int8 or compatible query vectors
Queries: 100k/day
Candidates: 50 vector + 50 lexical
Rerank: 100 candidates/query
Atlas DB cluster: M50 or M60
Search Nodes: 2 × S80 Low CPU
Search node RAM: 128 GB each
Search node storage: 855 GB each on AWS
Approx AWS annual base cost:
|
Component |
Size |
Annual cost |
|
Atlas database |
M50 |
~$17.5k/yr |
|
Search Nodes |
2 × S80 Low CPU |
~$29.4k/yr |
|
Atlas subtotal |
~$47k/yr |
With M60 instead of M50:
|
Component |
Size |
Annual cost |
|
Atlas database |
M60 |
~$34.6k/yr |
|
Search Nodes |
2 × S80 Low CPU |
~$29.4k/yr |
|
Atlas subtotal |
~$64k/yr |
This is the configuration I’d try first if the workload has modest peak QPS.
Atlas DB cluster: M60
Search Nodes: 2 × S90 Low CPU
Search node RAM: 256 GB each
Search node storage: 1.7 TB each on AWS
Approx annual cost:
|
Component |
Size |
Annual cost |
|
Atlas database |
M60 |
~$34.6k/yr |
|
Search Nodes |
2 × S90 Low CPU |
~$54.7k/yr |
|
Atlas subtotal |
~$89k/yr |
This is what I’d pick if you want more breathing room for lexical index size, stored source, filters, rebuilds, cache misses, and peak traffic.
Raw vector payload:
512 dimensions × 1 byte = 512 bytes/vector
100M vectors × 512 bytes = ~51.2 GB raw vector data
With HNSW graph overhead, metadata, filter fields, lexical index, stored source, JVM/headroom, etc., you should not think of this as only 51 GB. But compared with 1024-d float32:
1024-d float32 = ~409.6 GB raw vector data
512-d int8 = ~51.2 GB raw vector data
That’s roughly an 8× reduction in raw vector footprint.
For 100M chunks, I’d expect the practical Search Node working set to be more like:
Vector index + HNSW + metadata: maybe ~100–180 GB
Lexical index + stored fields: maybe ~50–150+ GB
So 2 × S80 Low CPU may work, but 2 × S90 Low CPU is the safer call.
Quantization and dimension reduction affect Atlas storage/memory/latency, but Voyage embedding is still billed by tokens, not dimensions.
voyage-4-large is $0.12 / 1M tokens.
|
Chunk size |
Tokens |
Cost |
|
128 tokens × 100M |
12.8B tokens |
~$1.5k |
|
256 tokens × 100M |
25.6B tokens |
~$3.1k |
So:
Initial corpus embedding: ~$1.5k–$3.1k
Possibly lower with Batch API discount.
voyage-4-lite is $0.02 / 1M tokens.
At 100k queries/day:
Query embedding: <$200/yr for typical query lengths
Basically irrelevant in the budget.
This remains your main recurring Voyage cost.
Assuming 100 candidates/query:
|
Candidate tokens |
Query tokens |
rerank-2.5-lite annual |
rerank-2.5 annual |
|
128 |
32 |
~$11.7k |
~$29.2k |
|
128 |
128 |
~$18.7k |
~$46.7k |
|
256 |
32 |
~$21.0k |
~$52.6k |
|
256 |
128 |
~$28.0k |
~$70.1k |
So I’d budget:
Reranking with rerank-2.5-lite: ~$12k–$28k/yr
Reranking with rerank-2.5: ~$29k–$70k/yr
Atlas M50 + 2 × S80 Low CPU: ~$47k/yr
Voyage rerank-2.5-lite: ~$12k–$28k/yr
Voyage initial corpus embedding: ~$1.5k–$3.1k first year
Query embeddings: negligible
------------------------------------------------------
First-year total: ~$61k–$78k
Ongoing after initial embedding: ~$59k–$75k/yr
Atlas M60 + 2 × S90 Low CPU: ~$89k/yr
Voyage rerank-2.5-lite: ~$12k–$28k/yr
Voyage initial corpus embedding: ~$1.5k–$3.1k first year
Query embeddings: negligible
------------------------------------------------------
First-year total: ~$103k–$120k
Ongoing after initial embedding: ~$101k–$117k/yr
Add roughly:
+$17k–$42k/yr versus rerank-2.5-lite
So the safer setup with full rerank becomes roughly:
~$120k–$162k/yr
I’d start with:
Embedding:
model: voyage-4-large
output_dimension: 512
output_dtype: int8
input_type: document
Query embedding:
model: voyage-4-lite
output_dimension: 512
output_dtype: int8
input_type: query
Atlas Vector Search:
numDimensions: 512
similarity: dotProduct
vector storage: BSON BinData int8 vector subtype
quantization: none at Atlas index level if pre-quantized int8
Atlas Search Nodes:
Start: 2 × S80 Low CPU
Safer: 2 × S90 Low CPU
Important: if you ingest pre-quantized int8 vectors, don’t also think of this as Atlas automatic scalar quantization. You’re storing/querying int8 directly. Make sure the query vector subtype and indexed vector subtype are compatible.
If you want one planning number, I’d use:
~$75k/year lean
~$115k/year comfortable production
~$150k/year if using full rerank and extra Atlas headroom
For your described traffic, I’d probably begin with M60 + 2 × S80 Low CPU, benchmark, and scale Search Nodes to S90 only if Search memory/page faults or p95 latency say you need it.