ReadWriteMany PV on Kubernetes

Google Cloud Serverless Applications
1

I need PV on my GKE cluster with RWX mode. so multiple instances of a deployment can read write easily.
here’s what I did.
hosted the nfs server in kubernetes using this: gcr.io/google_containers/volume-nfs:0.8 for gcloud balanced persistent disk.

as this is exposed with service the pv can point to this easily like

nfs:
path: /
server: nfs-server..svc.cluster.local
persistentVolumeReclaimPolicy: Retain
volumeMode: Filesystem

now i can easily mount this PV on my deployments, and works fine.

the problem starts when the nfs server restarts for any one the resason in k8s cluster. when that happens pods throws error: OSError: [Errno 116] Stale file handle

As explained in this thread: https://github.com/ehough/docker-nfs-server/issues/25
is it generally bad idea to deploy nfs server on k8s. what is the best solutions/alternates for such RWX PV need on k8s?

4 Likes
2

Hello @surshrestha ,

You can use GCS Fuse that use Google Cloud Storage as filesystem, or you can use external softwares likes Netapp or Portworx. In addition some open source tools are compatible with RWX like Longhorn or Ceph.

3 Likes
3

@MaxImbrox thanks!
but how reliable is GCS Fuse, can it cause problem any time, especially when pod has some files opened for operations?

3 Likes
4

Hello again @surshrestha , IMHO GCS Fuse it’s not suitable for batch workloads, because I had a lot of problems using it in terms of failed and restarted pods for many reasons related to the number of jobs that I needed to launch in few seconds.

3 Likes