How to package custom prediction code and serve it using an Endpoint in Vertex AI ?

Google Cloud Build with AI Custom ML & MLOps
1

Goal: serve prediction request from a Vertex AI Endpoint by executing custom prediction logic.

Expected Workflow:

  1. Upload a pretrained image_quality.pb model (developed in a non vertex-ai pythonic environment) in a gcs bucket

  2. Port existing image inference logic into a container and serve the prediction functionality through a vertex AI endpoint.

  3. Use Vertex AI api for logging and capturing metrics inside the custom inference logic.

  4. Finally we want to pass a list of images (stored in another gcs bucket) to that endpoint.

  5. We also want to see the logs and metrics in tensorboard.

Existing Vertex AI code samples provide examples for custom training , invoking model.batch_predict / endpoint.predict , but don’t mention how to execute custom prediction code.

It would be great if someone can provide guidelines and links to documents/code in order to implement the above steps.

Thanks

2 Likes
2

Please find the following guides respective of the points

[1] Import model: https://cloud.google.com/vertex-ai/docs/general/import-model
[2] What cannot be migrated: https://cloud.google.com/vertex-ai/docs/start/migrating-to-vertex-ai#migration-exceptions
2.
[3] Custom containers: https://cloud.google.com/vertex-ai/docs/training/containers-overview
[4] https://cloud.google.com/vertex-ai/docs/training/create-custom-container
3.
[5] About metrics: https://cloud.google.com/vertex-ai/docs/general/monitoring-metrics
4.
[6] Passing list of images: https://cloud.google.com/vertex-ai/docs/datasets/create-dataset-api
5.
[7] Metrics in Tensorboard: https://cloud.google.com/architecture/ml-on-gcp-best-practices?hl=en#use-vertex-tensorboard-to-visualize-experiments

As there is no existing unifying guide for these operations, I created a documentation feature request to have one, and asked the documentation team to post updates here.