💡
We used an instance with 1875 GB NVMe storage, though only about 1014 GB was needed. This was the minimum AWS option providing sufficient fast local SSD. Finding a machine with less storage could reduce costs.
For the PostgreSQL setup within this instance, we adjusted the `shared_buffers` parameter:
* Set to 48GB during the index building process.
* Increased to 54GB for search evaluation to allow more data caching for search performance.
## **Uploading and Indexing**
The VectorChord VChordRQ index uses an Index-based Vector File (IVF) structuring. While VChordRQ can compute the required centroids during the index building process, the building of an index of this size (400M vectors) pre-calculating the centroids out of the system significantly accelerates the process of building an index.
For LAION-400M, we performed an external k-means clustering step to determine these centroids. This was done on an instance with an `A10 GPU` for faster computation.
* The clustering was configured for `nlist = 160000` clusters.
* The clustering took approximately 220 seconds per iteration for 25 iterations, totaling about 1.5 hours.
Once we have computed the centroids, we set up the VectorChord-enabled PostgreSQL database. We used a pre-built Docker image for convenience:
```sql
-- Mount PGDATA path to the NVMe SSD
-- sudo mount /dev/nvme1n1 /data
docker run --name vchord-pg17 \
-e POSTGRES_PASSWORD=mysecretpassword -p 5432:5432 -v /data/pg:/var/lib/postgresql/data \
-d tensorchord/vchord-postgres:pg17-v0.3.0
```
This Docker image contains `PostgreSQL 17` with `pgvector 0.8.0` and `VectorChord 0.3.0` installed.
The core of the upload process involves inserting the pre-computed centroids into a dedicated table and then inserting the vectors from the LAION dataset chunks into the main data table. This can be done efficiently using a client library like psycopg in Python with batched inserts.
First, insert the centroids into a table (e.g., laion\_centroids):
```sql
-- The centroids table we created is:
-- CREATE TABLE laion_centroids (id SERIAL PRIMARY KEY, vector vector(512));
INSERT INTO laion_centroids (vector) VALUES ('[...centroid vector...]'); -- Insert all 160000 centroids
```
Then, insert the vectors from the dataset into the data table (e.g., laion):
```sql
-- The data table we created is:
-- CREATE TABLE laion (id BIGINT PRIMARY KEY, embedding vector(512));
-- Note: VectorChord is compatible with pgvector's vector insert syntax.
INSERT INTO laion (id, embedding) VALUES (0, '[...vector data...]'); -- Batch inserts for chunks
```
After inserting all data, we build the VectorChord index (vchordrq). vchordrq stands for [RabitQ](https://arxiv.org/abs/2405.12497), which is VectorChord's highly optimized index type based on the IVF structure combined with quantization and efficient reranking.
```sql
-- Note: The metric operator vector_cosine_ops is specified here, similar to pgvector.
-- The $$ syntax is used to pass build options as a text block.
CREATE INDEX laion_embedding_idx ON laion USING vchordrq (embedding vector_cosine_ops) WITH (options = $$
[build.external]
table = 'public.laion_centroids' -- Link to the pre-computed centroids
$$);
```
The image below shows the system resource utilization (CPU, memory, and disk usage) on the instance during the upload and indexing process. The disk usage graph shows the data being written (initial steep increase) and the subsequent index building (steady increase), resulting in a final size over 1TB. The CPU and memory graphs indicate the resources consumed throughout this process.
![](https://cdn.hashnode.com/res/hashnode/image/upload/v1745721010613/c5d20e21-6c6f-43d4-a915-aa121497d496.png align="center")
## **Ground Truth Data**
To correctly measure search performance and accuracy, comparing to ground truth is vital. The LAION data set comes without inherent ground truth. **We are in debt to the Qdrant team for having published the** [**ground truth data**](https://github.com/qdrant/laion-400m-benchmark/blob/master/expected.py) **they had prepared for their** [**tutorial**](https://qdrant.tech/documentation/database-tutorials/large-scale-search/) **on this dataset.**
Their methodology involved performing a full-scan nearest neighbor search for the first 100 vectors in the dataset to find the top 50 true nearest neighbors for each query. We **used this same exact ground truth file** to evaluate the VectorChord search accuracy.
## **Search Query**
VectorChord search employs two main parameters to control the performance-accuracy trade-off: nprobe and epsilon.
* **nprobe**: Configured using the `vchord.probes` setting. This parameter determines how many clusters (partitions) of the IVF index are searched in the initial phase. A higher nprobe increases the search scope, potentially finding better results but also increasing latency.
* **epsilon**: Configured using the `vchord.epsilon` setting. This parameter controls the reranking precision. After fetching the initial candidates from the nprobed clusters, VectorChord performs a reranking step using more precise vector data. A larger epsilon means more candidates are considered and potentially reranked for higher recall, impacting latency.
These parameters are set as PostgreSQL session variables before executing the search query:
```sql
-- Example settings
SET vchord.probes = 100; -- Configure nprobe
SET vchord.epsilon = 0.8; -- Configure epsilon
​
-- The search query itself uses the standard pgvector distance operator
SELECT id FROM laion ORDER BY embedding <=> '[...query_vector...]' LIMIT 50;
```
The ORDER BY embedding `<=> '[...]'` clause leverages the index to find the approximate nearest neighbors based on the specified distance metric (cosine distance, like CLIP). The LIMIT 50 retrieves the top 50 results.
## **Running Search Requests**
After the index built is finished, we ran queries over the 100 ground truth vectors with varying nprobe and epsilon settings to try out performance versus precision trade-offs.
We expressed search latency in milliseconds (ms). "Cold" latency refers to instances where vector data of interest may need to be read from disk, while "warm" latency implies that data will be in memory.
Here are the average latencies and corresponding Precision@50 scores across the 100 queries:
| **nprob / epsilon** | **Cold Latency (ms)** | **Warm Latency (ms)** | **Precision@50** |
| --- | --- | --- | --- |
| 10 / 0.8 | **112.5** | **7.0** | 0.8192 |
| 100 / 0.8 | 194.2 | 15.0 | 0.9216 |
| 200 / 1.0 | 331.2 | 24.9 | 0.9438 |
| 400 / 1.5 | 970.9 | 51.6 | **0.9608** |
| 800 / 1.9 | 2174.0 | 2174.0 | **0.9662** |
The table clearly illustrates the trade-off:
* Increasing `nprobe` and `epsilon` generally leads to higher Precision@50.
* However, this comes at the cost of increased search latency, especially noticeable in cold scenarios where more disk I/O is required to fetch data from more clusters and perform more extensive reranking.
* Warm cache performance is significantly faster, highlighting the importance of sufficient RAM for caching frequently accessed data.
* It is possible to have over 90% precision through warm latencies less than 50ms, and increasing precision above this yields hundreds of milliseconds or even more than a second latency for best recall, depending on the parameters selected.
The graph below shows the trade-off between Queries Per Second (QPS) and Precision@50 for VectorChord (cold and warm cache) alongside Qdrant, with Qdrant’s metrics taken from their [published benchmarks](https://qdrant.tech/documentation/database-tutorials/large-scale-search/#running-search-requests) on a comparable 8 vCPU, 64 GB machine rather than run on the same hardware.
![](https://cdn.hashnode.com/res/hashnode/image/upload/v1745753692001/731cb592-86e6-4eef-ae0f-c699df421a33.png align="center")
## **Conclusion**
In this tutorial, we illustrated how VectorChord as an extension to PostgreSQL can be utilized to upload, index, and search a gigantic 400 million vector dataset cost-efficiently on relatively small hardware with fast NVMe SSD.
By using the VectorChord VChordRQ index with external centroids, we were able to handle the scale. Having carefully set `nprobe` and `epsilon` at query time provides precise control over the critical trade-off between search speed and accuracy, allowing users to tune the system to their specific needs.
VectorChord's compatibility with the pgvector interface simplifies adoption for current pgvector users, and its vchordrq index provides powerful capabilities for large-scale vector search directly within the robust and familiar PostgreSQL environment.