How to build a Qdrant index with Sentence Transformers

Vector retrieval needs a searchable index that survives the Python process which created it. Sentence Transformers produces document and query embeddings, while Qdrant persists those vectors, payloads, and its HNSW graph as one collection.

A named Docker volume keeps the local Qdrant storage outside the container layer. The service listens only on the host loopback address, so the Python client can use http://localhost:6333 without exposing the unauthenticated development service to the local network.

The collection lowers its indexing threshold and supplies 512 deterministic records so Qdrant builds an HNSW segment instead of serving the corpus only by full scan. The program waits for a green collection with an idle optimizer, closes the client, reconnects, and limits the final nearest-neighbor query to indexed segments.

1. Create the named Docker volume for persistent Qdrant storage.

   $ docker volume create sentence-transformers-qdrant-data
   sentence-transformers-qdrant-data

2. Start Qdrant 1.18.3 on the host loopback address with the named volume mounted at /qdrant/storage.

   $ docker run --detach --name sentence-transformers-qdrant --publish 127.0.0.1:6333:6333 --volume sentence-transformers-qdrant-data:/qdrant/storage qdrant/qdrant:v1.18.3

   The container can be replaced without deleting the named volume, while the loopback binding keeps this unauthenticated local service off other network interfaces.

3. Install the CPU build of PyTorch in the active Python environment.

   $ python -m pip install --upgrade torch --index-url https://download.pytorch.org/whl/cpu

4. Install Sentence Transformers and the Qdrant Python client in the same environment.

   $ python -m pip install --upgrade sentence-transformers qdrant-client

   The first model run downloads sentence-transformers/all-MiniLM-L6-v2 when it is not already cached.
   Related: How to install Sentence Transformers with pip

5. Create build_qdrant_index.py with the deterministic corpus and document-embedding stage.

   build_qdrant_index.py

   from time import monotonic, sleep
    
   from qdrant_client import QdrantClient, models
   from sentence_transformers import SentenceTransformer
    
    
   qdrant_url = "http://localhost:6333"
   collection_name = "sentence_transformers_support_docs"
   expected_points = 512
   query = "How can I reset a forgotten account password?"
    
   corpus = [
       {
           "doc_id": "doc-000",
           "title": "Reset a forgotten password",
           "text": "Reset a forgotten account password from account settings and confirm the recovery email.",
       }
   ]
   corpus.extend(
       {
           "doc_id": f"doc-{point_id:03d}",
           "title": f"Archive retention record {point_id:03d}",
           "text": (
               f"Archive record {point_id:03d} describes retention schedules, "
               "storage labels, and routine document review."
           ),
       }
       for point_id in range(1, expected_points)
   )
    
   model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
   document_embeddings = model.encode_document(
       [item["text"] for item in corpus],
       normalize_embeddings=True,
       convert_to_numpy=True,
       show_progress_bar=False,
   )
   vector_size = document_embeddings.shape[1]

   encode_document() applies the document route for retrieval models that distinguish corpus text from queries.

6. Append the persistent collection, HNSW configuration, and point-upload stage after the vector_size assignment.

   client = QdrantClient(url=qdrant_url)
   if client.collection_exists(collection_name):
       raise SystemExit(
           f"collection {collection_name!r} already exists; choose a new collection name"
       )
    
   client.create_collection(
       collection_name=collection_name,
       vectors_config=models.VectorParams(
           size=vector_size,
           distance=models.Distance.COSINE,
       ),
       hnsw_config=models.HnswConfigDiff(
           m=16,
           ef_construct=100,
           full_scan_threshold=10,
       ),
       optimizers_config=models.OptimizersConfigDiff(
           indexing_threshold=1,
       ),
   )
   client.upload_points(
       collection_name=collection_name,
       points=[
           models.PointStruct(
               id=point_id,
               vector=vector.tolist(),
               payload=item,
           )
           for point_id, (item, vector) in enumerate(
               zip(corpus, document_embeddings),
           )
       ],
       wait=True,
   )

   The one-kilobyte indexing threshold is intentionally low for this compact demonstration corpus. The existing-collection check prevents an accidental overwrite when the program is run again.

7. Append the index wait, client reconnect, indexed-only query, and verification stage after the upload call.

   stored_points = client.count(
       collection_name=collection_name,
       exact=True,
   ).count
   if stored_points != expected_points:
       raise SystemExit(f"expected {expected_points} points, found {stored_points}")
    
   deadline = monotonic() + 120
   while True:
       collection = client.get_collection(collection_name)
       collection_status = getattr(collection.status, "value", str(collection.status))
       optimizer_status = getattr(
           collection.optimizer_status,
           "value",
           str(collection.optimizer_status),
       )
       indexed_vectors = collection.indexed_vectors_count or 0
       if (
           collection_status == "green"
           and optimizer_status == "ok"
           and indexed_vectors >= expected_points
       ):
           break
       if monotonic() >= deadline:
           raise SystemExit(
               "Qdrant did not finish the HNSW index before the 120-second deadline"
           )
       sleep(1)
    
   client.close()
   client = QdrantClient(url=qdrant_url)
   reconnected_collection = client.get_collection(collection_name)
   reconnected_status = getattr(
       reconnected_collection.status,
       "value",
       str(reconnected_collection.status),
   )
   reconnected_optimizer_status = getattr(
       reconnected_collection.optimizer_status,
       "value",
       str(reconnected_collection.optimizer_status),
   )
   reconnected_indexed_vectors = reconnected_collection.indexed_vectors_count or 0
   reconnected_points = client.count(
       collection_name=collection_name,
       exact=True,
   ).count
    
   query_embedding = model.encode_query(
       query,
       normalize_embeddings=True,
       convert_to_numpy=True,
       show_progress_bar=False,
   )
   hits = client.query_points(
       collection_name=collection_name,
       query=query_embedding.tolist(),
       search_params=models.SearchParams(
           hnsw_ef=64,
           indexed_only=True,
       ),
       limit=3,
       with_payload=True,
   ).points
    
   if reconnected_status != "green":
       raise SystemExit(f"reconnected collection status is {reconnected_status}")
   if reconnected_optimizer_status != "ok":
       raise SystemExit(
           f"reconnected optimizer status is {reconnected_optimizer_status}"
       )
   if reconnected_indexed_vectors < expected_points:
       raise SystemExit(
           f"expected {expected_points} indexed vectors after reconnect, "
           f"found {reconnected_indexed_vectors}"
       )
   if reconnected_points != expected_points:
       raise SystemExit(
           f"expected {expected_points} reconnected points, found {reconnected_points}"
       )
   if not hits or hits[0].payload["doc_id"] != "doc-000":
       raise SystemExit("password reset document was not the top indexed match")
    
   print(f"collection status: {reconnected_status}")
   print(f"optimizer status: {reconnected_optimizer_status}")
   print(f"stored points: {stored_points}")
   print(f"indexed vectors: {reconnected_indexed_vectors}")
   print(f"reconnected points: {reconnected_points}")
   print(f"query: {query}")
   print(
       f"top indexed match: {hits[0].payload['doc_id']} "
       f"score={hits[0].score:.4f} title={hits[0].payload['title']}"
   )
    
   client.close()

   The indexed-only query excludes segments without a vector index, while the assertions fail if the collection is not green, the optimizer is not idle, any vectors remain unindexed, or the expected payload is not ranked first after reconnecting.

8. Run build_qdrant_index.py to build the persistent HNSW collection.

   $ python build_qdrant_index.py
   collection status: green
   optimizer status: ok
   stored points: 512
   indexed vectors: 512
   reconnected points: 512
   query: How can I reset a forgotten account password?
   top indexed match: doc-000 score=0.7494 title=Reset a forgotten password