GCP Memorystore Redis connection to Databricks

Google Cloud Database
1

Hi,

I have a requirement.

Databricks, which is not hosted on the Google Cloud Platform (GCP), needs to be connected to the GCP Memorystore Redis. However, the Google documentation only shows ways to connect to Redis from a VM or GKE cluster.

Can someone help me with the process?

Thank you!

2

To connect Databricks to Memorystore Redis from a non-GCP hosted environment, you will need to use a VPN or other tunneling mechanism to create a secure connection between Databricks and the GCP network. Once you have established a connection, you can then use the standard Redis client libraries to connect to Memorystore Redis.

To connect Databricks to Memorystore Redis from a non-GCP hosted environment using a VPN, follow these steps:

  1. Set up a Cloud VPN on GCP: This will securely connect your external network (where Databricks is hosted) to your VPC network in GCP.

  2. Configure Databricks to use the VPN: Once the VPN connection is established, this step will allow Databricks to access resources on the GCP network.

  3. Install the Redis client library: Depending on your programming language, install the appropriate Redis client library on Databricks.

  4. Configure Memorystore Redis: Allow connections from the VPN subnet to your Memorystore Redis instance.

  5. Connect to Memorystore Redis: Utilize the Redis client library and connect using the private IP address of the Memorystore Redis instance.

Additional notes:

  • Consider using a managed Redis service like Redis Cloud for direct connectivity to Databricks. Be aware of potential costs or other considerations.
  • If VPN setup isn’t feasible, consider using a bastion host to connect to Memorystore Redis. This method has security implications, so use it as a last resort.
  • Ensure the VPC network firewall rules are configured to allow traffic from the VPN’s IP range to the Memorystore Redis instance.

Python code example:

import redis

Replace <private-ip-address> with the actual private IP address of your Memorystore Redis instance.

conn = redis.StrictRedis(host=‘’, port=6379)

Set a value in Redis

conn.set(‘my_key’, ‘my_value’)

Get a value from Redis

value = conn.get(‘my_key’)

Print the value

print(value.decode(‘utf-8’))

3

Sure, let me give it a try and reply back to you.