Hello,
I don’t see anywhere in the documentation of GCP that accelerator type L4 are not available for private endpoint with custom container.
Yet i get this error when i try to deploy them:
(gcloud.beta.ai.endpoints.deploy-model) FAILED_PRECONDITION: Private Endpoint (PSA) does not support machine families [C3], accelerators [NVIDIA_L4] and TPU.
Where can i keep track of this limitation ? Thanks
Hi @mbtki,
Welcome to Google Cloud Community!
Based on the official documentation, NVIDIA_L4 accelerators are not listed to be used for private endpoints with custom containers. Hence, NVIDIA_L4 is currently not supported for Private Endpoint deployments with custom containers in Vertex AI.
As you can see in the image below, these accelerators are currently supported for Private Endpoint deployments. I recommend choosing one that best aligns with your specific workload requirements.
Additionally, you may refer to this page which outlines the general concepts and benefits of using private endpoints.
You may also regularly check the Google Cloud AI Platform release notes for any announcements regarding new features, changes, or limitations. These notes may occasionally include mentions of updates affecting Private Endpoint and accelerator support.
I hope the above information is helpful.
For people coming here for more precise information, Google confirmed that there is currently no release date for the L4 on endpoint, using previous mode (PSA). On should migrate to PSC connection (Preview mode) before, Use Private Service Connect for online prediction | Vertex AI | Google Cloud
