Error "failed to list the clusters" creating cloud serverless runtime template

Google Cloud Build with AI Custom ML & MLOps
1

Hello,

I try to create a dataproc cloud serverless runtime template to get a spark environment notebook in a vertex ai workbench instance.

If click on the “New Runtime Template” in category Dataproc Serverless Notebooks I get directly the error " failed to list the clusters" and i am not able to save the creation dialog.

What is the problem? Error makes not much sense because its a serverless thing. or?

jochen_hegele_0-1712659579754.png
jochen_hegele_0-1712659579754.png1128×975 55.7 KB

jochen_hegele_2-1712659624995.png
jochen_hegele_2-1712659624995.png1163×909 51.2 KB

Thanks for help.

Best Regards

6 Likes
2

Can someone provide information about this? Thanks

6 Likes
3

Hi @jochen_hegele ,

Thank you for joining our community.

I know it can be frustrating when things aren’t working as we expect it to be, coupled with the lack of information for troubleshooting. Consider following this document (“Use the Dataproc JupyterLab plugin for serverless batch and interactive notebook sessions”) as it can guide you in resolving your use case.

To perform this, you will need to enable the Cloud Dataproc API, then create a cluster and lastly enable the Dataproc metastore API.

I hope I was able to provide you with useful insights.

4

Hello,

thanks for answer. I tried to follow this guide:

https://cloud.google.com/vertex-ai/docs/workbench/instances/create-dataproc-enabled and use the plugin with dataproc severless for spark:

https://cloud.google.com/vertex-ai/docs/workbench/instances/create-dataproc-enabled#serverless-spark
Dataproc api was already active. the dataproc metadata API not, but this helped nothing error is still showing up.
This plugin should be pre-installed on vertex-ai. A dataproc cluster should not be necessary becaus I would like to use the “Dataproc Serverless Notebooks” → New Runtime Template icon. So no cluster should be necessary or?

5

I think a solved the problem. A couple of permission roles solved it for the compute engine service account. Also the VPC subnet of the default network hast to be configured that the “Private Google Access” Option is set to ON. A clear guideline that these things are necessary if you start your journey from Vertex AI as a strating point would be helpful:-)

jochen_hegele_0-1714632541854.png
jochen_hegele_0-1714632541854.png429×229 7.81 KB

2 Likes
6

My team has experienced the same issue failed to list the clusters when running the dataproc cluster on the 2.2.50-debian12 image.

The dataproc cluster was created from gcloud command without any issues, when trying to run on the jupyter lab on the cluster, after a few minutes the error failed to list the clusters appears and freezes the jupyter lab. After this event pops up, we can not even ssh to the master node. After some time the cluster is stale (jupyter lab not opened) you can login again, but the issue repeats.