Howdy y’all 
I am interested in trying out the sec-palm@001 model to evaluate its usefulness in cybersecurity applications. I can see it in the Vertex AI Studio under the freeform/experimental models, but I get the dreaded 429 everyone is coming to know and love. Does use dynamic quota’s? What would be the process of requesting quota’s for it, and in the event I do get the quota raised, should I expect it to be something in the 5-10 RPM ballpark? Because that may render it virtually unusable for testing, so I may just give it up before its began on this one.
also, there is no mention of that model in the release notes or documentation, just stuff from RSA23 and the Cloud Sec podcast talk about sec-palm vs SecLM, which I am not holding my breath on being GA for a bit, so I was hoping to tinker with in the meantime,
Any help would be appreciated, a link to any kind of documentation on sec-palm@001, official or otherwise, would be even more appreciated
Here is my 429 error:
Failed to submit prompt
Error message: "Quota exceeded for aiplatform.googleapis.com/online_prediction_requests_per_base_model with base model: secpalm. Please submit a quota increase request. https://cloud.google.com/vertex-ai/docs/generative-ai/quotas-genai."
Status: 429 Error code: 429
Tracking number: c2968650322827753
Thanks!
Hi @S_Gehman,
Welcome to Google Cloud Community!
It appears that you are hitting the quota limit for online predictions with the sec-palm@001 model, which is a common challenge with experimental models. Unfortunately, Google does not publicly disclose quotas for these models, and the request limit (requests per minute or RPM) is likely to be quite low initially, possibly even in the single digits, as you’ve noted. These models are intended for experimentation, rather than for production environments. Furthermore, since sec-palm@001 is an experimental model, there is no specific documentation for it.
In addition, You can confirm whether the quota limit in your Google Cloud project has indeed been exceeded. You can navigate to the Google Cloud Console, and, in the left-hand navigation panel, click on “IAM & Admin” and then select “Quotas & System Limits." You can filter by the specific service that might be exceeded.
If you want to increase any of your quotas on Vertex AI, you can use the Google Cloud Console to request a quota increase. You may follow the steps in this documentation. Keep in mind that these requests are subject to review and approval and may take some time to process. Additionally, quota increase requests are typically evaluated based on the validity of the business case provided.
You may refer to the following documentation, which will provide you with an overview of the updates and help you understand how to access and use the sec-palm@001 model in Google Vertex AI:
Was this helpful? If so, please accept this answer as “Solution”. If you need additional assistance, reply here within 2 business days and I’ll be happy to help.
Marvin,
Thank you for your response. I had identified the article you linked in your response previously. Unfortunately, I am unable to glean any particularly meaningful information from it regarding using SecLM, specifically the sec-palm@001 experimental freeform model in Vertex AI. I was also able to work out how to see my available quota’s such as the aiplatform.googleapis.com/online_prediction_requests_per_base_model and I can see that is it set to 0. One thing that is interesting, is that when I try to filter in the quota search/filter utility for the base_model:secpalm tag, it returns no results. Ultimately, my question still stands. In the event I submit a quota increase request in order to experiment with the model and evaluate its potential applications for my orgs needs it seems from your response that no documentation would be provided.
So to recap;
I am interested in gauging the likelihood of success in requesting a quota increase for:
aiplatform.googleapis.com/online_prediction_requests_per_base_model with base model: secpalm
I am interested in hearing from others in the community who have worked with this model and what their findings are
I am interested in any insider programs or additional information that can shed light on whether someone could leverage the sec-palm@001 model and vertex to develop their own orchestrator system, similar to what I presume is happening in the back end of the currently available SecLM as is it works currently, as part of GCP, SecOps, and any other implementation I am not currently aware of.
Basically just trying to kick the tires on the big boy toys and barring that, I would love some first hand testimonials, specifically from smaller orgs or individuals who are using it as part of a larger flow. Any additional info would be super appreciated,
Thanks!