I’m trying to fine-tune a Llama model (any model) on Vertex AI and I’d like to enable tool calling (a.k.a. function calling) so the fine-tuned model can call structured tools.
However, I can’t find any schema or data format in the documentation that shows how to format the training examples for tool-augmented Llama fine-tuning e.g., how to include the “tools” definitions, “tool_calls”, and “responses” fields in the JSONL format accepted by Vertex AI fine-tuning jobs.
Is there an officially supported training data schema for Llama fine-tuning with tool calling in Vertex AI yet?
Example of what I’m trying to achieve (it’s a Gemini 2.5 schema):
{
"systemInstruction": {
"parts": \[
{
"text": "You are a helpful assistant."
}
\]
},
"contents": \[
{
"role": "user",
"parts": \[
{
"text": "In our 'Project Zeus' project, add a new task 'Post-Launch Review'."
}
\]
},
{
"role": "model",
"parts": \[
{
"text": "Adding the 'Post-Launch Review' task to 'Project Zeus'."
},
{
"function_call": {
"name": "addTask",
"args": {
"planId": "PZ",
"itemType": "theme",
"name": "Post-Launch Review",
"id": "PZ-5",
"description": "Initiative to review project performance and gather feedback post-launch.",
"endDate": "2024-12-31"
}
}
}
\]
}
\]
}
Any guidance, examples, or clarification from Google Cloud engineers or anyone who has tried this would be greatly appreciated!
Check out the fine-tuned chat completion template with ENG support – it already integrates tool calling, meaning you can adapt the model using the same format for further tuning.
I’m also planning to create a tutorial demonstrating how to fine-tune Llama for function calls.
Thanks for the update, @ilnardo92 ! I’ll go through the chat completion template and I’m looking forward to see that Llama function-calling fine-tuning tutorial when you publish it.
Really appreciate you taking the time to look into this!
But I got this error: “Dataset example 1 of 25 contains an unsupported modality [function call] in the “parts” field, which is not supported by this model. The supported modalities are: [image, PDF document, audio, video]…..”
My dataset still worked well with Gemini 2.0, but did not with 2.5. Could you guys give me any clues to fix? Thank you.