How to configure an OpenAI text embedder in Haystack

Embedding retrieval in Haystack depends on query vectors that use the same model settings as the indexed documents. OpenAITextEmbedder creates those vectors through an OpenAI embeddings model or an OpenAI-compatible endpoint, so a query can move from plain text into an embedding retriever.

The component reads credentials through Secret objects and accepts the model name, output dimensions, and optional API base URL at construction time. Leaving api_base_url unset uses OpenAI's hosted API, while setting it points the same component at a compatible gateway that exposes the OpenAI embeddings path.

Use the same model and dimensions for document embeddings and query embeddings. The smoke test embeds one query string, checks the vector length, and passes that vector to InMemoryEmbeddingRetriever so the configuration is proven beyond initialization.

1. Open the Python environment where haystack-ai is installed.

   Related: How to install Haystack with pip

2. Set the OpenAI API key for the current shell.

   $ export OPENAI_API_KEY="<OpenAI API key>"

   Do not hard-code a real API key in the Python file or commit it to source control. Use an environment variable, a secret manager, or another runtime secret surface.
   Related: How to use API key secrets in Haystack

3. Set a base URL only when using an OpenAI-compatible gateway instead of OpenAI's hosted API.

   $ export OPENAI_API_BASE_URL="https://gateway.example.net/v1"

   Leave OPENAI_API_BASE_URL unset for the default OpenAI API. Compatible endpoints must implement the embeddings request shape used by the OpenAI API.

4. Create haystack-openai-text-embedder.py with the embedder configuration and a retriever smoke test.

   haystack-openai-text-embedder.py

   import os
    
   from haystack import Document
   from haystack.components.embedders import OpenAITextEmbedder
   from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
   from haystack.document_stores.in_memory import InMemoryDocumentStore
   from haystack.utils import Secret
    
    
   model = os.getenv("OPENAI_EMBEDDING_MODEL", "text-embedding-3-small")
   dimensions_env = os.getenv("OPENAI_EMBEDDING_DIMENSIONS", "512")
   dimensions = int(dimensions_env) if dimensions_env else None
   api_base_url = os.getenv("OPENAI_API_BASE_URL")
    
   embedder = OpenAITextEmbedder(
       api_key=Secret.from_env_var("OPENAI_API_KEY"),
       model=model,
       dimensions=dimensions,
       api_base_url=api_base_url,
       timeout=30.0,
       max_retries=1,
   )
    
   result = embedder.run(text="password reset approval policy")
   embedding = result["embedding"]
   meta = result["meta"]
    
   document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
   document_store.write_documents(
       [
           Document(
               content="Password reset requests require help desk approval.",
               meta={"name": "password-reset-policy"},
               embedding=embedding,
           )
       ]
   )
    
   retriever = InMemoryEmbeddingRetriever(document_store=document_store, top_k=1)
   match = retriever.run(query_embedding=embedding)["documents"][0]
   usage = meta.get("usage", {})
    
   print(f"model={meta.get('model', model)}")
   print(f"embedding_dimensions={len(embedding)}")
   print(f"first_values={embedding[0]:.4f},{embedding[1]:.4f},{embedding[2]:.4f}")
   print(f"usage_total_tokens={usage.get('total_tokens', 'unknown')}")
   print(f"retriever_top_document={match.meta['name']}")

   OPENAI_EMBEDDING_MODEL and OPENAI_EMBEDDING_DIMENSIONS are optional overrides. The dimensions parameter is intended for embedding models that support adjustable output size, such as OpenAI's third-generation embedding models.

5. Run the smoke test.

   $ python3 haystack-openai-text-embedder.py
   model=text-embedding-3-small
   embedding_dimensions=512
   first_values=0.0100,0.0200,0.0300
   usage_total_tokens=5
   retriever_top_document=password-reset-policy

   The exact vector values and token count vary by endpoint, model, and input text. embedding_dimensions=512 confirms the requested vector length, and retriever_top_document=password-reset-policy confirms the vector can be passed to an embedding retriever.

6. Remove the smoke-test file if it was only created to check the embedder configuration.

   $ rm haystack-openai-text-embedder.py