[Gluster-users] NFS clients show missing files while gluster volume rebalanced

[Erik Jacobson]

Closing the loop in case someone does a search on this...

I have an update. I am getting some time on 1,000 node soon so I have
started to validate jumping to gluster 7.2 on my small lab machine.

I switched the packages to my own build of gluster 7.2 with gnfs.
I re-installed my leader node (gluster/gnfs servers) and created
the volumes the same way as before. This includes heavy cache
optimization for the NFS services volume.

I can no longer duplicate this problem on gluster 7.2. I was able to
duplicate rebalance troubles on NFS clients every time on gluster
4.1.6.

I do have a couple questions on some rebalance errors, which I will send
in a separate email.

Erik

On Wed, Jan 29, 2020 at 06:20:34PM -0600, Erik Jacobson wrote:
> We are using gluster 4.1.6. We are using gluster NFS (not ganesha).
> 
> Distributed/replicated with subvolume size 3 (6 total servers, 2
> subvols).
> 
> The NFS clients use this for their root filesystem.
> 
> When I add 3 more gluster servers to add one more subvolume to the
> storage volumes (so now subvolume size 3, 9 total servers, 3 total
> subvolumes), the process gets started. 
> 
> ssh leader1 gluster volume add-brick cm_shared 172.23.0.9://data/brick_cm_shared 172.23.0.10://data/brick_cm_shared 172.23.0.11://data/brick_cm_shared
> 
> then
> 
> ssh leader1 gluster volume rebalance cm_shared start
> 
> The re-balance works. 'gluster volume status' shows re-balance in
> progress.
> 
> However, existing gluster-NFS clients now show missing files and I can
> no longer log into them (since NFS is their root). If you are logged in,
> you can find that libraries are missing and general unhappiness with
> random files now missing.
> 
> Is accessing a volume that is in the process of being re-balanced not
> supported from a gluster NFS client? Or have I made an error?
> 
> Thank you for any help,
> 
> Erik