Hi @arthurbrcuni ,
Welcome to Google Cloud Community!
The problem you’re facing with Vertex AI caching is due to the contents parameter in caching.CachedContent.create requiring at least one user-generated content object, which means the system prompt can’t be added directly to this list. The contents parameter must consist of user-generated content, not system prompts.
While there’s no direct way to include the system prompt in the contents list, Here are three workarounds you might explore:
- Instead of using system_instruction_content in contents, extract the system prompt text (system_instruction).
- Modify your code to include the extracted text in the request body outside the contents list.
- Modify your application logic to prepend the system prompt text to user input before creating the Content object for the contents list.
In addition, you may also check these documents which might help you:
- Overview of context caching
- How to get information about a context cache
- Configuring execution caching
Note: Some of these products or features are subject to the “Pre-GA Offerings Terms” outlined in the General Service Terms section. Pre-GA products and features are offered “as is” and may have limited support.
I hope the above information is helpful.