Language models got a lot more capable with gpt-35-turbo and gpt-4, but on their own they can only produce text. They cannot look up your order status, query a database, or check the weather. In July 2023, Microsoft brought function calling to Azure OpenAI Service on the 0613 versions of these two models, and this single addition changed how most of us design applications on top of Azure OpenAI.
Function calling lets you describe one or more functions to the model as part of your request. When the model decides that answering the user needs one of those functions, it responds with a structured JSON object containing the function name and the arguments to call it with. The model never executes anything on its own. Your application code reads that JSON, runs the actual function, and decides what happens next. That separation matters a lot for security and it is worth keeping in mind throughout this article.
How function calling actually works
You describe your functions using a JSON schema: name, description, and parameters with types. The model was fine-tuned specifically to recognise when a user request maps to one of these definitions. If it decides a function applies, it stops generating a plain text answer and instead returns a function_call object with the name and a JSON string of arguments.
Working with functions breaks down into three steps in practice. First, you call the chat completions API and pass your functions array alongside the user’s message. Second, if the model responds with a function_call, your code parses the arguments and actually invokes your API, database query, or calculation. Third, you call the chat completions API again, this time including the function’s result as a new message with role set to function, so the model can turn that raw result into a natural language answer for the user.
The most common mistake I see teams make when they first wire this up is stopping after step two. They get the function_call JSON back, run their code, and then just format the raw result themselves instead of sending it back to the model. That works, but you lose the model’s ability to phrase the answer naturally and to decide if a follow-up function call is needed. If you want a conversational answer rather than a raw data dump, the round trip in step three is not optional.
Retrieving data from an external source
The most frequent use case for function calling is pulling in data the model has no way of knowing on its own, whether that is a search index, a product catalogue, or a live API. Here is a search_hotels function definition and a user query about beachfront hotels in San Diego.
messages = [
{"role": "user", "content": "Find beachfront hotels in San Diego for less than $300 a month with free breakfast."}
]
functions = [
{
"name": "search_hotels",
"description": "Retrieves hotels from the search index based on the parameters provided",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The location of the hotel (i.e. Seattle, WA)"
},
"max_price": {
"type": "number",
"description": "The maximum price for the hotel"
},
"features": {
"type": "string",
"description": "A comma separated list of features (i.e. beachfront, free wifi, etc.)"
}
},
"required": ["location"]
}
}
]
response = openai.ChatCompletion.create(
engine="gpt-35-turbo-0613",
messages=messages,
functions=functions,
function_call="auto",
)
print(response['choices'][0]['message'])This code does not query anything by itself. It sends the user’s message and the function schema to the chat completions endpoint and asks the model to decide, through function_call set to auto, whether search_hotels applies here. Nothing in your search index gets touched at this point.
The model responds with this instead of plain text:
{
"role": "assistant",
"function_call": {
"name": "search_hotels",
"arguments": "{\n \"location\": \"San Diego\",\n \"max_price\": 300, \n \"features\": \"beachfront, free breakfast\"\n}"
}
}The arguments field is a JSON string, not a JSON object, so you need to run json.loads on it before you can use it. It is worth guarding this with a try/except in production, because a model can occasionally return arguments that do not parse cleanly, especially with edge case inputs. Once parsed, you call your own search_hotels function with those parameters, get back the actual hotel results, and send them to the model in a follow-up call so it can write the final response to the user.
Giving the model tools it is bad at doing itself
Language models are notoriously unreliable at arithmetic, especially with larger numbers. Function calling lets you hand off calculations to actual code instead of trusting the model to compute them in its head, so to speak.
messages = [
{"role": "user", "content": "Last month Fabrikam made $73,846 in sales. Based on that, what would the annual run rate be?"}
]
functions = [
{
"name": "calculator",
"description": "A simple calculator",
"parameters": {
"type": "object",
"properties": {
"num1": {"type": "number"},
"num2": {"type": "number"},
"operator": {"type": "string", "enum": ["+", "-", "*", "/", "**", "sqrt"]}
},
"required": ["num1", "num2", "operator"]
}
}
]
response = openai.ChatCompletion.create(
engine="gpt-35-turbo-0613",
messages=messages,
functions=functions,
function_call="auto",
)
print(response['choices'][0]['message'])The model reasons that an annual run rate means multiplying monthly sales by twelve, then asks your calculator function to do the actual multiplication:
{
"role": "assistant",
"function_call": {
"name": "calculator",
"arguments": "{\n \"num1\": 73846,\n \"num2\": 12,\n \"operator\": \"*\"\n}"
}
}This pattern generalises well beyond a calculator. Anything the model is unreliable at, or anything that needs to touch a live system such as writing a database row, sending an email, or placing an order, belongs in a function rather than in the model’s own output. The trade-off is that every action-taking function needs its own validation and, ideally, a human confirmation step before it actually runs, because the model can call a function with plausible-looking arguments that are still wrong for your business logic.
Extracting structured data from free text
The third common pattern uses function calling purely to force a clean JSON shape out of unstructured text, without any external system involved at all. Before function calling existed, getting reliable JSON out of a chat model meant prompt engineering tricks and still occasionally getting a stray sentence like “Here is the JSON:” glued to the front of the output.
messages = [
{"role": "system", "content": "Assistant is a large language model designed to extract structured data from text."},
{"role": "user", "content": "There are many fruits that were found on the recently discovered planet Goocrux. There are neoskizzles that grow there, which are purple and taste like candy. There are also loheckles, which are a grayish blue fruit and are very tart, a little bit like a lemon. Pounits are a bright green color and are more savory than sweet. There are also plenty of loopnovas which are a neon pink flavor and taste like cotton candy. Finally, there are fruits called glowls, which have a very sour and bitter taste which is acidic and caustic, and a pale orange tinge to them."}
]
functions = [
{
"name": "extract_fruit",
"description": "Extract fruit names from text.",
"parameters": {
"type": "object",
"properties": {
"fruits": {
"type": "array",
"items": {
"type": "object",
"properties": {
"fruit": {"type": "string", "description": "The name of the fruit."},
"color": {"type": "string", "description": "The color of the fruit."},
"flavor": {"type": "string", "description": "The flavor of the fruit."}
},
"required": ["fruit", "color", "flavor"]
}
}
},
"required": ["fruits"]
}
}
]
response = openai.ChatCompletion.create(
engine="gpt-35-turbo-0613",
messages=messages,
functions=functions,
function_call={"name": "extract_fruit"},
)
print(response['choices'][0]['message'])Notice the difference in the function_call parameter here. Instead of auto, it names extract_fruit directly, which forces the model to call that specific function rather than deciding for itself. This is the setting to reach for whenever you know exactly which function should run and just want the model to fill in the structured arguments, such as this extraction case. The response comes back as a clean array with no explanation text wrapped around it.
{
"fruits": [
{"fruit": "neoskizzles", "color": "purple", "flavor": "candy"},
{"fruit": "loheckles", "color": "grayish blue", "flavor": "tart"},
{"fruit": "pounits", "color": "bright green", "flavor": "savory"},
{"fruit": "loopnovas", "color": "neon pink", "flavor": "cotton candy"},
{"fruit": "glowls", "color": "pale orange", "flavor": "sour and bitter"}
]
}This response no longer needs a third round trip back to the model, since there is no external system to call and no natural language reply to construct. You can parse this JSON directly and hand it to whatever downstream system needs the structured fruit records.
Security considerations before you ship this
Because function calling can trigger real actions, treat the model’s output the same way you would treat any untrusted input coming from a user. Validate the function name and arguments before executing anything, and do not assume the model will only ever request functions that make sense for the current context.
Give each function the least privilege it needs to do its job. A function that queries a database for read-only lookups should use a database credential that literally cannot write, rather than relying on your function’s code to just not issue write statements. If a function performs an action with real consequences, such as sending an email or placing an order, add a confirmation step where a human approves the specific call before it executes. Also be careful about what data flows into your functions from outside sources, since untrusted data returned by a function’s own output could be used to steer the model into generating calls you did not intend.
Where this sits in the bigger picture
It is worth remembering that this was the first version of tool use on Azure OpenAI, released well before the Assistants API existed. That means the three-step loop described above, tracking conversation state, deciding when to stop calling functions, and stitching results back into the message history, was entirely your application’s responsibility. There was no framework managing threads or runs for you yet.
The functions and function_call parameters used in the examples here were later superseded by the tools and tool_choice parameters, which support requesting multiple function calls in a single model turn instead of one at a time. If you are starting a new project today, use the tools parameter, since functions is kept around mainly for backward compatibility. The underlying concepts in this article, the three-step cycle, the auto versus forced function selection, and the security guidance, still apply directly to the newer parameter names.
One limitation worth planning around at this stage of the feature: only the 0613 versions of gpt-35-turbo and gpt-4 supported function calling at launch, so if you were pinned to an older model version in production you had to upgrade before any of this worked. It is also worth testing function calling against edge cases in your own domain early, since the model’s judgment about when to call a function versus when to just answer in plain text is not perfect, and a well-written function description makes a bigger difference to accuracy than most people expect on their first attempt.
Getting started
- Apply for access to Azure OpenAI Service if you do not already have it.
- Read through the official documentation on function calling for the exact parameter reference.
- Try the end-to-end samples that walk through a working function calling setup before you build your own from scratch.
Leave a Reply