How to configure an OpenAI LLM in LlamaIndex

LlamaIndex sends prompts through an LLM object before query engines, agents, and workflows can produce natural-language answers. Pointing that object at OpenAI chooses the hosted model, response temperature, token budget, and credential source used by the rest of the application.

The llama-index-llms-openai integration provides the OpenAI class and reads OPENAI_API_KEY from the process environment by default. Assigning the object to Settings.llm makes it the global default for LlamaIndex components that do not receive a local llm= override.

Keep the API key outside Python source files and pass it through a shell, secret manager, deployment environment, or CI variable. A short completion call is enough to confirm that the package, model name, key, and LlamaIndex settings layer are all wired together before adding the same object to a larger retrieval or agent workflow.

1. Install the OpenAI LLM integration in the active Python environment.

   $ python -m pip install llama-index-llms-openai
   Successfully installed llama-index-core-0.14.23 llama-index-llms-openai-0.7.9 openai-2.44.0

   Install the package inside the same virtual environment or deployment image that runs the LlamaIndex application.

2. Export the OpenAI API key for the current shell.

   $ export OPENAI_API_KEY="sk-proj-REPLACE_WITH_YOUR_KEY"

   Do not paste production API keys into source files, screenshots, shared terminal history, saved logs, or shared notes. Use the secret store provided by the runtime.

3. Select the model name used by the smoke test.

   $ export OPENAI_MODEL="gpt-4o-mini"

   Replace gpt-4o-mini with the model approved for the application. The same pattern works when the model name is stored in application configuration instead of a shell variable.

4. Create a small LlamaIndex smoke-test script.

   openai_llm_check.py

   import os
    
   from llama_index.core import Settings
   from llama_index.llms.openai import OpenAI
    
    
   model = os.getenv("OPENAI_MODEL", "gpt-4o-mini")
    
   llm = OpenAI(
       model=model,
       temperature=0.1,
       max_tokens=32,
   )
    
   Settings.llm = llm
    
   print(f"configured_llm={Settings.llm.metadata.model_name}")
   response = Settings.llm.complete(
       "Return only this exact text: LlamaIndex OpenAI LLM configured."
   )
   print(response.text.strip())

   Use Settings.llm when the whole application should share the same default LLM. Pass llm=llm directly to a query engine, agent, or workflow when only one component should use this model.

5. Run the smoke test.

   $ python openai_llm_check.py
   configured_llm=gpt-4o-mini
   LlamaIndex OpenAI LLM configured.

   The first line confirms the model selected by the configured LlamaIndex object. The completion response confirms that the OpenAI credential and hosted model call both worked.

6. Remove the temporary smoke-test file after moving the configuration into the application.

   $ rm openai_llm_check.py