[Gluster-users] gluster 3.7.6 volume set: failed: One or more connected clients cannot support the feature being set
Steve Dainard
sdainard at spd1.com
Fri Mar 4 16:53:23 UTC 2016
Except that even when I unmount gluster from both Debian clients, I'm still
getting the same error, and a dump of those clients after unmounting still
shows:
glusterd.client1.identifier=10.0.231.10:1023
glusterd.client1.volname=storage
glusterd.client1.max-op-version=30603
glusterd.client1.min-op-version=1
glusterd.client4.identifier=10.0.231.11:1022
glusterd.client4.volname=storage
glusterd.client4.max-op-version=30603
glusterd.client4.min-op-version=1
On Thu, Mar 3, 2016 at 5:28 PM, Atin Mukherjee <atin.mukherjee83 at gmail.com>
wrote:
> -Atin
> Sent from one plus one
> On 04-Mar-2016 3:35 am, "Steve Dainard" <sdainard at spd1.com> wrote:
> >
> > FYI Gluster storage node hostnames are gluster0[1-6].
> >
> > Full dump attached. I see a few clients not on 30706. Most notably the
> two debian 7 servers (using packages from gluster.org) seem to be running
> lower op versions than the centos7 machines (every other client in
> 10.0.231.0/24 subnet).
> >
> > glusterd.client1.identifier=10.0.231.10:1023 <-- debian 7, glusterfs
> 3.7.6 built on Feb 4 2016 06:25:19
> > glusterd.client1.volname=storage
> > glusterd.client1.max-op-version=30603
> This is running with 3.6.3 and that's why volume set fails. Unmount this
> client and upgrade and remount it back.
>
> > glusterd.client1.min-op-version=1
> >
> > glusterd.client2.identifier=10.0.231.51:65515 <-- gluster02
> (mounted localhost:storage on /run/gluster/storage type fuse.glusterfs)
> > glusterd.client2.volname=
> > glusterd.client2.max-op-version=0
> > glusterd.client2.min-op-version=0
> >
> > glusterd.client3.identifier=10.0.231.54:65521 <-- gluster05 (no actual
> mounts)
> > glusterd.client3.volname=
> > glusterd.client3.max-op-version=0
> > glusterd.client3.min-op-version=0
> >
> > glusterd.client4.identifier=10.0.231.11:1022 <--- debian 7, glusterfs
> 3.7.6 built on Feb 4 2016 06:25:19
> > glusterd.client4.volname=storage
> > glusterd.client4.max-op-version=30603
> > glusterd.client4.min-op-version=1
> >
> > glusterd.client5.identifier=10.0.231.55:65530 <-- gluster06 (no actual
> mounts)
> > glusterd.client5.volname=
> > glusterd.client5.max-op-version=0
> > glusterd.client5.min-op-version=0
> >
> > glusterd.client6.identifier=10.0.231.53:65516 <-- gluster04
> (mounted localhost:storage on /run/gluster/storage type fuse.glusterfs)
> > glusterd.client6.volname=
> > glusterd.client6.max-op-version=0
> > glusterd.client6.min-op-version=0
> >
> > glusterd.client7.identifier=10.0.231.50:65529
> >
> glusterd.client7.volname=export-domain-storage.10.0.231.50.mnt-lv-export-domain-storage-export-domain-storage
> > glusterd.client7.max-op-version=30706
> > glusterd.client7.min-op-version=1
> >
> > ...
> >
> > Debian package info:
> > apt-cache policy glusterfs-client
> > glusterfs-client:
> > Installed: 3.7.6-2
> > Candidate: 3.7.6-2
> > Version table:
> > *** 3.7.6-2 0
> > 500
> http://download.gluster.org/pub/gluster/glusterfs/3.7/3.7.6/Debian/wheezy/apt/
> wheezy/main amd64 Packages
> >
> >
> > Thanks
> >
> > On Wed, Mar 2, 2016 at 10:29 PM, Gaurav Garg <ggarg at redhat.com> wrote:
> >>
> >> Hi Steve,
> >>
> >> As atin pointed out to take statedump by running #kill -SIGUSR1 $(pidof
> glusterd) command. it will create .dump file in /var/run/gluster/
> directory. client-op-version information will be present in dump file.
> >>
> >> Thanks,
> >> ~Gaurav
> >>
> >> ----- Original Message -----
> >> From: "Steve Dainard" <sdainard at spd1.com>
> >> To: "Gaurav Garg" <ggarg at redhat.com>
> >> Cc: "gluster-users at gluster.org List" <gluster-users at gluster.org>
> >> Sent: Thursday, March 3, 2016 12:07:25 AM
> >> Subject: Re: [Gluster-users] gluster 3.7.6 volume set: failed: One or
> more connected clients cannot support the feature being set
> >>
> >> From the the client side logs I can see version info on mount:
> >>
> >> Final graph:
> >>
> +------------------------------------------------------------------------------+
> >> 1: volume storage-client-0
> >> 2: type protocol/client
> >> 3: option clnt-lk-version 1
> >> 4: option volfile-checksum 0
> >> 5: option volfile-key /storage
> >> 6: option client-version 3.7.6
> >> 7: option process-uuid
> >>
> template-centos7-compute.compute.domain-2773-2016/03/02-18:28:34:328100-storage-client-0-0-0
> >> 8: option fops-version 1298437
> >> 9: option ping-timeout 42
> >> 10: option remote-host 10.0.231.50
> >> 11: option remote-subvolume /mnt/raid6-storage/storage
> >> 12: option transport-type socket
> >> 13: option send-gids true
> >> 14: end-volume
> >> 15:
> >> 16: volume storage-client-1
> >> 17: type protocol/client
> >> 18: option clnt-lk-version 1
> >> 19: option volfile-checksum 0
> >> 20: option volfile-key /storage
> >> 21: option client-version 3.7.6
> >> 22: option process-uuid
> >>
> template-centos7-compute.compute.domain-2773-2016/03/02-18:28:34:328100-storage-client-1-0-0
> >> 23: option fops-version 1298437
> >> 24: option ping-timeout 42
> >> 25: option remote-host 10.0.231.51
> >> 26: option remote-subvolume /mnt/raid6-storage/storage
> >> 27: option transport-type socket
> >> 28: option send-gids true
> >> 29: end-volume
> >> 30:
> >> 31: volume storage-client-2
> >> 32: type protocol/client
> >> 33: option clnt-lk-version 1
> >> 34: option volfile-checksum 0
> >> 35: option volfile-key /storage
> >> 36: option client-version 3.7.6
> >> 37: option process-uuid
> >>
> template-centos7-compute.compute.domain-2773-2016/03/02-18:28:34:328100-storage-client-2-0-0
> >> 38: option fops-version 1298437
> >> 39: option ping-timeout 42
> >> 40: option remote-host 10.0.231.52
> >> 41: option remote-subvolume /mnt/raid6-storage/storage
> >> 42: option transport-type socket
> >> 43: option send-gids true
> >> 44: end-volume
> >> 45:
> >> 46: volume storage-client-3
> >> 47: type protocol/client
> >> 48: option clnt-lk-version 1
> >> 49: option volfile-checksum 0
> >> 50: option volfile-key /storage
> >> 51: option client-version 3.7.6
> >> 52: option process-uuid
> >>
> template-centos7-compute.compute.domain-2773-2016/03/02-18:28:34:328100-storage-client-3-0-0
> >> 53: option fops-version 1298437
> >> 54: option ping-timeout 42
> >> 55: option remote-host 10.0.231.53
> >> 56: option remote-subvolume /mnt/raid6-storage/storage
> >> 57: option transport-type socket
> >> 58: option send-gids true
> >> 59: end-volume
> >> 60:
> >> 61: volume storage-client-4
> >> 62: type protocol/client
> >> 63: option ping-timeout 42
> >> 64: option remote-host 10.0.231.54
> >> 65: option remote-subvolume /mnt/raid6-storage/storage
> >> 66: option transport-type socket
> >> 67: option send-gids true
> >> 68: end-volume
> >> 69:
> >> 70: volume storage-client-5
> >> 71: type protocol/client
> >> 72: option ping-timeout 42
> >> 73: option remote-host 10.0.231.55
> >> 74: option remote-subvolume /mnt/raid6-storage/storage
> >> 75: option transport-type socket
> >> 76: option send-gids true
> >> 77: end-volume
> >> 78:
> >> 79: volume storage-dht
> >> 80: type cluster/distribute
> >> 81: subvolumes storage-client-0 storage-client-1 storage-client-2
> >> storage-client-3 storage-client-4 storage-client-5
> >> 82: end-volume
> >>
> >>
> >> But not the client op-version, how can I retrieve this info?
> >>
> >> Thanks
> >>
> >> On Tue, Mar 1, 2016 at 10:19 PM, Gaurav Garg <ggarg at redhat.com> wrote:
> >>
> >> > Hi Steve,
> >> >
> >> > Which version you have upgraded client, could you tell us client
> >> > op-version after upgrade ?
> >> >
> >> >
> >> > have you upgraded all of your clients ?
> >> >
> >> >
> >> > Thanks,
> >> > Gaurav
> >> >
> >> >
> >> > ----- Original Message -----
> >> > From: "Steve Dainard" <sdainard at spd1.com>
> >> > To: "gluster-users at gluster.org List" <gluster-users at gluster.org>
> >> > Sent: Wednesday, March 2, 2016 1:10:27 AM
> >> > Subject: [Gluster-users] gluster 3.7.6 volume set: failed: One or more
> >> > connected clients cannot support the feature being set
> >> >
> >> > Gluster 3.7.6
> >> > 'storage' is a distributed volume
> >> >
> >> > # gluster volume set storage rebal-throttle lazy
> >> > volume set: failed: One or more connected clients cannot support the
> >> > feature being set. These clients need to be upgraded or disconnected
> before
> >> > running this command again
> >> >
> >> > I found a client connected using version 3.6.7 so I upgraded &
> >> > umount/mount the gluster volume on the client but I'm still getting
> this
> >> > error.
> >> >
> >> > I've run grep "accepted client from" /var/log/glusterfs/bricks/* |
> grep -v
> >> > 3.7.6 and I get a few returns from the client above, all dated last
> week.
> >> >
> >> > I've run 'gluster volume status storage clients' and checked the
> connected
> >> > clients manually, they're all running 3.7.6.
> >> >
> >> > /var/log/gluster/etc-glusterfs-glusterd.vol.log:
> >> > [2016-03-01 19:23:20.180821] E [MSGID: 106022]
> >> > [glusterd-utils.c:10154:glusterd_check_client_op_version_support]
> >> > 0-management: One or more c
> >> > lients don't support the required op-version
> >> > [2016-03-01 19:23:20.180853] E [MSGID: 106301]
> >> > [glusterd-syncop.c:1274:gd_stage_op_phase] 0-management: Staging of
> >> > operation 'Volume Set' fa
> >> > iled on localhost : One or more connected clients cannot support the
> >> > feature being set. These clients need to be upgraded or disconnected
> be
> >> > fore running this command again
> >> >
> >> > Also tried setting the diagnostics.brick-log-level logging level and
> got
> >> > the same error.
> >> >
> >> > /var/lib/glusterd/vols/storage/info:
> >> > type=0
> >> > count=6
> >> > status=1
> >> > sub_count=0
> >> > stripe_count=1
> >> > replica_count=1
> >> > disperse_count=0
> >> > redundancy_count=0
> >> > version=26
> >> > transport-type=0
> >> > volume-id=26d355cb-c486-481f-ac16-e25390e73775
> >> > username=eb9e2063-6ba8-4d16-a54f-2c7cf7740c4c
> >> > password=
> >> > op-version=3
> >> > client-op-version=3
> >> > quota-version=1
> >> > parent_volname=N/A
> >> > restored_from_snap=00000000-0000-0000-0000-000000000000
> >> > snap-max-hard-limit=256
> >> > features.quota-deem-statfs=on
> >> > features.inode-quota=on
> >> > diagnostics.brick-log-level=WARNING
> >> > features.quota=on
> >> > performance.readdir-ahead=on
> >> > performance.cache-size=1GB
> >> > performance.stat-prefetch=on
> >> > brick-0=10.0.231.50:-mnt-raid6-storage-storage
> >> > brick-1=10.0.231.51:-mnt-raid6-storage-storage
> >> > brick-2=10.0.231.52:-mnt-raid6-storage-storage
> >> > brick-3=10.0.231.53:-mnt-raid6-storage-storage
> >> > brick-4=10.0.231.54:-mnt-raid6-storage-storage
> >> > brick-5=10.0.231.55:-mnt-raid6-storage-storage
> >> >
> >> >
> >> > _______________________________________________
> >> > Gluster-users mailing list
> >> > Gluster-users at gluster.org
> >> > http://www.gluster.org/mailman/listinfo/gluster-users
> >> >
> >
> >
> >
> > _______________________________________________
> > Gluster-users mailing list
> > Gluster-users at gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160304/a1205a86/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: glusterdump.1590.dump.1457110194
Type: application/octet-stream
Size: 24836 bytes
Desc: not available
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160304/a1205a86/attachment.obj>
More information about the Gluster-users
mailing list