How to build a FunctionAgent in LlamaIndex

Agent code becomes useful when a user request can reach a typed tool and return the tool result through the model loop. A LlamaIndex FunctionAgent handles that handoff for models with function-calling support, making the selected Python function and its arguments observable during a first agent test.

A deterministic FunctionCallingLLM test double keeps the build credential-free while exercising the same tools, system_prompt, and asynchronous run() interfaces used with provider integrations. Its first response requests the invoice tool, while its second reads the resulting tool message and returns that content as the agent answer.

The tool name, docstring, and type hints form the schema that a function-calling model receives. A provider-backed project can replace only DemoFunctionLLM after this local path works, leaving the typed tool and FunctionAgent wiring unchanged.

1. Install the LlamaIndex core package in the active Python environment.

   $ python3 -m pip install --upgrade llama-index-core

   A project virtual environment keeps LlamaIndex dependencies separate from the system Python environment.
   Related: How to install LlamaIndex with pip

2. Start llamaindex-function-agent.py with the required imports and a typed invoice tool.

   llamaindex-function-agent.py

   import asyncio
   from typing import Any
    
   from llama_index.core.agent.workflow import FunctionAgent
   from llama_index.core.base.llms.types import (
       ChatMessage,
       ChatResponse,
       CompletionResponse,
       LLMMetadata,
       MessageRole,
   )
   from llama_index.core.llms.function_calling import FunctionCallingLLM
   from llama_index.core.llms.llm import ToolSelection
    
   TOOL_CALL_LOG: list[str] = []
   TOOL_RESULT_LOG: list[str] = []
    
    
   def get_invoice_total(invoice_id: str) -> str:
       """Return the approved total for an invoice ID."""
       total = {"INV-1024": "$184.50"}[invoice_id]
       result = f"The approved total for {invoice_id} is {total}."
       TOOL_CALL_LOG.append(f"get_invoice_total(invoice_id={invoice_id})")
       TOOL_RESULT_LOG.append(result)
       return result

3. Append the deterministic LLM metadata and tool-call parser after get_invoice_total().

   class DemoFunctionLLM(FunctionCallingLLM):
       @property
       def metadata(self) -> LLMMetadata:
           return LLMMetadata(
               is_chat_model=True,
               is_function_calling_model=True,
               model_name="demo-function-llm",
           )
    
       def _prepare_chat_with_tools(
           self,
           tools: list[Any],
           user_msg: str | ChatMessage | None = None,
           chat_history: list[ChatMessage] | None = None,
           **kwargs: Any,
       ) -> dict[str, Any]:
           messages = list(chat_history or [])
           if isinstance(user_msg, str):
               messages.append(ChatMessage(role=MessageRole.USER, content=user_msg))
           elif isinstance(user_msg, ChatMessage):
               messages.append(user_msg)
           return {"messages": messages}
    
       def get_tool_calls_from_response(
           self,
           response: ChatResponse,
           error_on_no_tool_call: bool = True,
           **kwargs: Any,
       ) -> list[ToolSelection]:
           calls = response.message.additional_kwargs.get("tool_calls", [])
           if not calls and error_on_no_tool_call:
               raise ValueError("No tool calls returned")
           return calls

4. Add the two chat responses inside DemoFunctionLLM after get_tool_calls_from_response().

       def chat(self, messages: list[ChatMessage], **kwargs: Any) -> ChatResponse:
           if messages and messages[-1].role == MessageRole.TOOL:
               tool_result = messages[-1].content
               if tool_result is None:
                   raise ValueError("Tool message did not contain text")
               return ChatResponse(
                   message=ChatMessage(
                       role=MessageRole.ASSISTANT,
                       content=tool_result,
                   )
               )
           return ChatResponse(
               message=ChatMessage(
                   role=MessageRole.ASSISTANT,
                   content="",
                   additional_kwargs={
                       "tool_calls": [
                           ToolSelection(
                               tool_id="call_invoice_total",
                               tool_name="get_invoice_total",
                               tool_kwargs={"invoice_id": "INV-1024"},
                           )
                       ]
                   },
               )
           )

   The first call requests get_invoice_total() with a typed argument. The next call reads the text carried by the TOOL message instead of repeating a fixed invoice amount.

5. Complete the required async, streaming, and completion methods inside DemoFunctionLLM.

       async def achat(self, messages: list[ChatMessage], **kwargs: Any) -> ChatResponse:
           return self.chat(messages, **kwargs)
    
       def stream_chat(self, messages: list[ChatMessage], **kwargs: Any):
           yield self.chat(messages, **kwargs)
    
       async def astream_chat(self, messages: list[ChatMessage], **kwargs: Any):
           yield self.chat(messages, **kwargs)
    
       def complete(self, prompt: str, **kwargs: Any) -> CompletionResponse:
           return CompletionResponse(text="")
    
       async def acomplete(self, prompt: str, **kwargs: Any) -> CompletionResponse:
           return self.complete(prompt, **kwargs)
    
       def stream_complete(self, prompt: str, **kwargs: Any):
           yield CompletionResponse(text="")
    
       async def astream_complete(self, prompt: str, **kwargs: Any):
           yield CompletionResponse(text="")

6. Append the asynchronous FunctionAgent entry point after DemoFunctionLLM.

   async def main() -> None:
       agent = FunctionAgent(
           tools=[get_invoice_total],
           llm=DemoFunctionLLM(),
           system_prompt="Use the invoice tool when a user asks for an approved total.",
           streaming=False,
       )
       response = await agent.run(
           "What is the approved total for invoice INV-1024?"
       )
       agent_response = str(response)
       result_flow_match = agent_response == TOOL_RESULT_LOG[-1]
       print(f"agent_response={agent_response}")
       print(f"tool_calls={TOOL_CALL_LOG}")
       print(f"tool_results={TOOL_RESULT_LOG}")
       print(f"result_flow_match={result_flow_match}")
       assert result_flow_match
    
    
   if __name__ == "__main__":
       asyncio.run(main())

   streaming=False keeps this deterministic test compatible with LLM implementations that do not stream. A configured provider integration can replace DemoFunctionLLM() when hosted model access is required.
   Related: How to configure an OpenAI LLM in LlamaIndex

7. Run the completed FunctionAgent script to verify the returned tool text becomes the agent response.

   $ python3 llamaindex-function-agent.py
   agent_response=The approved total for INV-1024 is $184.50.
   tool_calls=['get_invoice_total(invoice_id=INV-1024)']
   tool_results=['The approved total for INV-1024 is $184.50.']
   result_flow_match=True