[Gluster-users] Upgrade 7.8 -> 8.4: transport endpoint not connected

[Hu Bert]

Hi there,

i just did an upgrade on my test servers from version 7.8 to version
8.4 (debian buster, replicate 3 setup, 2 volumes), and noticed that
one of two mounts wasn't working afterwards.

procedure (same i used for 6.x -> 7.x):
- stop all gluster processes on the server
- apt update && apt upgrade
- check volume, services, mounts etc. afterwards

The volumes themselves are working, but one of the transport endpoints
is not connected.

Before the upgrade:
ls -lah /shared/
drwxr-xr-x  4 root root   75 Apr  1  2020 private
drwxr-xr-x  5 root root   46 Apr 16  2019 public

after the upgrade:

ls -lah /shared/
ls: cannot access '/shared/private': Transport endpoint is not connected
d?????????  ? ?    ?       ?            ? private
drwxr-xr-x  2 root root 4.0K Feb 11  2019 public

directory permissions and ownership screwed up? And even the date of
the 2nd dir (public) has changed.

Job for glusterfssharedstorage.service failed because the control
process exited with error code.
See "systemctl status glusterfssharedstorage.service" and "journalctl
-xe" for details.

systemctl status glusterfssharedstorage.service
● glusterfssharedstorage.service - Mount glusterfs sharedstorage
  Loaded: loaded (/lib/systemd/system/glusterfssharedstorage.service;
enabled; vendor preset: enabled)
  Active: activating (start) since Wed 2021-03-03 09:55:16 CET; 1s ago
Cntrl PID: 2148 (mount-shared-st)
   Tasks: 2 (limit: 4915)
  Memory: 4.1M
  CGroup: /system.slice/glusterfssharedstorage.service
          ├─2148 /bin/bash /usr/libexec/glusterfs/mount-shared-storage.sh
          └─2213 sleep 10

Mar 03 09:55:16 dirac systemd[1]: Starting Mount glusterfs sharedstorage...
Mar 03 09:55:16 dirac mount-shared-storage.sh[2148]: ERROR: Mount
point does not exist
Mar 03 09:55:16 dirac mount-shared-storage.sh[2148]: Please specify a
mount point
Mar 03 09:55:16 dirac mount-shared-storage.sh[2148]: Usage:
Mar 03 09:55:16 dirac mount-shared-storage.sh[2148]: man 8 /sbin/mount.glusterfs
Mar 03 09:55:26 dirac mount-shared-storage.sh[2148]: /shared/private
failed to mount
Mar 03 09:55:36 dirac mount-shared-storage.sh[2148]: /shared/public
has been mounted
Mar 03 09:55:36 dirac systemd[1]: glusterfssharedstorage.service:
Control process exited, code=exited, status=1/FAILURE

Well... does anyone have an idea what might have gone wrong?

While writing/thinking i was able to fix it:
- umount /shared/private -> directory looks normal again (no ???)
- systemctl start glusterfssharedstorage.service
- check if mount is done -> yes

ah ok, read it here...
https://docs.gluster.org/en/latest/Upgrade-Guide/generic-upgrade-procedure/

"Upgrade procedure for clients
Unmount all glusterfs mount points on the client"

My servers are mounting the volumes as clients as well. During the
last version upgrade i never had done the unmount before - is this v8
related? Just curious...


Best regards,
Hubert