[Gluster-users] gluster 3.7.6 volume set: failed: One or more connected clients cannot support the feature being set
Steve Dainard
sdainard at spd1.com
Fri Mar 4 17:02:32 UTC 2016
And I'll mention again, those Debian clients are not running 3.6. As shown
in previous email, they are both running:
# apt-cache policy glusterfs-client
glusterfs-client:
Installed: 3.7.6-2
Candidate: 3.7.6-2
Version table:
*** 3.7.6-2 0
500
http://download.gluster.org/pub/gluster/glusterfs/3.7/3.7.6/Debian/wheezy/apt/
wheezy/main
amd64 Packages
# glusterfs --version
glusterfs 3.7.6 built on Feb 4 2016 06:25:19
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2013 Red Hat, Inc. <http://www.redhat.com/>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
It is licensed to you under your choice of the GNU Lesser
General Public License, version 3 or any later version (LGPLv3
or later), or the GNU General Public License, version 2 (GPLv2),
in all cases as published by the Free Software Foundation.
On Fri, Mar 4, 2016 at 8:53 AM, Steve Dainard <sdainard at spd1.com> wrote:
> Except that even when I unmount gluster from both Debian clients, I'm
> still getting the same error, and a dump of those clients after unmounting
> still shows:
>
> glusterd.client1.identifier=10.0.231.10:1023
> glusterd.client1.volname=storage
> glusterd.client1.max-op-version=30603
> glusterd.client1.min-op-version=1
>
> glusterd.client4.identifier=10.0.231.11:1022
> glusterd.client4.volname=storage
> glusterd.client4.max-op-version=30603
> glusterd.client4.min-op-version=1
>
>
>
>
>
> On Thu, Mar 3, 2016 at 5:28 PM, Atin Mukherjee <atin.mukherjee83 at gmail.com
> > wrote:
>
>> -Atin
>> Sent from one plus one
>> On 04-Mar-2016 3:35 am, "Steve Dainard" <sdainard at spd1.com> wrote:
>> >
>> > FYI Gluster storage node hostnames are gluster0[1-6].
>> >
>> > Full dump attached. I see a few clients not on 30706. Most notably the
>> two debian 7 servers (using packages from gluster.org) seem to be
>> running lower op versions than the centos7 machines (every other client in
>> 10.0.231.0/24 subnet).
>> >
>> > glusterd.client1.identifier=10.0.231.10:1023 <-- debian 7, glusterfs
>> 3.7.6 built on Feb 4 2016 06:25:19
>> > glusterd.client1.volname=storage
>> > glusterd.client1.max-op-version=30603
>> This is running with 3.6.3 and that's why volume set fails. Unmount this
>> client and upgrade and remount it back.
>>
>> > glusterd.client1.min-op-version=1
>> >
>> > glusterd.client2.identifier=10.0.231.51:65515 <-- gluster02
>> (mounted localhost:storage on /run/gluster/storage type fuse.glusterfs)
>> > glusterd.client2.volname=
>> > glusterd.client2.max-op-version=0
>> > glusterd.client2.min-op-version=0
>> >
>> > glusterd.client3.identifier=10.0.231.54:65521 <-- gluster05 (no actual
>> mounts)
>> > glusterd.client3.volname=
>> > glusterd.client3.max-op-version=0
>> > glusterd.client3.min-op-version=0
>> >
>> > glusterd.client4.identifier=10.0.231.11:1022 <--- debian 7, glusterfs
>> 3.7.6 built on Feb 4 2016 06:25:19
>> > glusterd.client4.volname=storage
>> > glusterd.client4.max-op-version=30603
>> > glusterd.client4.min-op-version=1
>> >
>> > glusterd.client5.identifier=10.0.231.55:65530 <-- gluster06 (no actual
>> mounts)
>> > glusterd.client5.volname=
>> > glusterd.client5.max-op-version=0
>> > glusterd.client5.min-op-version=0
>> >
>> > glusterd.client6.identifier=10.0.231.53:65516 <-- gluster04
>> (mounted localhost:storage on /run/gluster/storage type fuse.glusterfs)
>> > glusterd.client6.volname=
>> > glusterd.client6.max-op-version=0
>> > glusterd.client6.min-op-version=0
>> >
>> > glusterd.client7.identifier=10.0.231.50:65529
>> >
>> glusterd.client7.volname=export-domain-storage.10.0.231.50.mnt-lv-export-domain-storage-export-domain-storage
>> > glusterd.client7.max-op-version=30706
>> > glusterd.client7.min-op-version=1
>> >
>> > ...
>> >
>> > Debian package info:
>> > apt-cache policy glusterfs-client
>> > glusterfs-client:
>> > Installed: 3.7.6-2
>> > Candidate: 3.7.6-2
>> > Version table:
>> > *** 3.7.6-2 0
>> > 500
>> http://download.gluster.org/pub/gluster/glusterfs/3.7/3.7.6/Debian/wheezy/apt/
>> wheezy/main amd64 Packages
>> >
>> >
>> > Thanks
>> >
>> > On Wed, Mar 2, 2016 at 10:29 PM, Gaurav Garg <ggarg at redhat.com> wrote:
>> >>
>> >> Hi Steve,
>> >>
>> >> As atin pointed out to take statedump by running #kill -SIGUSR1
>> $(pidof glusterd) command. it will create .dump file in /var/run/gluster/
>> directory. client-op-version information will be present in dump file.
>> >>
>> >> Thanks,
>> >> ~Gaurav
>> >>
>> >> ----- Original Message -----
>> >> From: "Steve Dainard" <sdainard at spd1.com>
>> >> To: "Gaurav Garg" <ggarg at redhat.com>
>> >> Cc: "gluster-users at gluster.org List" <gluster-users at gluster.org>
>> >> Sent: Thursday, March 3, 2016 12:07:25 AM
>> >> Subject: Re: [Gluster-users] gluster 3.7.6 volume set: failed: One or
>> more connected clients cannot support the feature being set
>> >>
>> >> From the the client side logs I can see version info on mount:
>> >>
>> >> Final graph:
>> >>
>> +------------------------------------------------------------------------------+
>> >> 1: volume storage-client-0
>> >> 2: type protocol/client
>> >> 3: option clnt-lk-version 1
>> >> 4: option volfile-checksum 0
>> >> 5: option volfile-key /storage
>> >> 6: option client-version 3.7.6
>> >> 7: option process-uuid
>> >>
>> template-centos7-compute.compute.domain-2773-2016/03/02-18:28:34:328100-storage-client-0-0-0
>> >> 8: option fops-version 1298437
>> >> 9: option ping-timeout 42
>> >> 10: option remote-host 10.0.231.50
>> >> 11: option remote-subvolume /mnt/raid6-storage/storage
>> >> 12: option transport-type socket
>> >> 13: option send-gids true
>> >> 14: end-volume
>> >> 15:
>> >> 16: volume storage-client-1
>> >> 17: type protocol/client
>> >> 18: option clnt-lk-version 1
>> >> 19: option volfile-checksum 0
>> >> 20: option volfile-key /storage
>> >> 21: option client-version 3.7.6
>> >> 22: option process-uuid
>> >>
>> template-centos7-compute.compute.domain-2773-2016/03/02-18:28:34:328100-storage-client-1-0-0
>> >> 23: option fops-version 1298437
>> >> 24: option ping-timeout 42
>> >> 25: option remote-host 10.0.231.51
>> >> 26: option remote-subvolume /mnt/raid6-storage/storage
>> >> 27: option transport-type socket
>> >> 28: option send-gids true
>> >> 29: end-volume
>> >> 30:
>> >> 31: volume storage-client-2
>> >> 32: type protocol/client
>> >> 33: option clnt-lk-version 1
>> >> 34: option volfile-checksum 0
>> >> 35: option volfile-key /storage
>> >> 36: option client-version 3.7.6
>> >> 37: option process-uuid
>> >>
>> template-centos7-compute.compute.domain-2773-2016/03/02-18:28:34:328100-storage-client-2-0-0
>> >> 38: option fops-version 1298437
>> >> 39: option ping-timeout 42
>> >> 40: option remote-host 10.0.231.52
>> >> 41: option remote-subvolume /mnt/raid6-storage/storage
>> >> 42: option transport-type socket
>> >> 43: option send-gids true
>> >> 44: end-volume
>> >> 45:
>> >> 46: volume storage-client-3
>> >> 47: type protocol/client
>> >> 48: option clnt-lk-version 1
>> >> 49: option volfile-checksum 0
>> >> 50: option volfile-key /storage
>> >> 51: option client-version 3.7.6
>> >> 52: option process-uuid
>> >>
>> template-centos7-compute.compute.domain-2773-2016/03/02-18:28:34:328100-storage-client-3-0-0
>> >> 53: option fops-version 1298437
>> >> 54: option ping-timeout 42
>> >> 55: option remote-host 10.0.231.53
>> >> 56: option remote-subvolume /mnt/raid6-storage/storage
>> >> 57: option transport-type socket
>> >> 58: option send-gids true
>> >> 59: end-volume
>> >> 60:
>> >> 61: volume storage-client-4
>> >> 62: type protocol/client
>> >> 63: option ping-timeout 42
>> >> 64: option remote-host 10.0.231.54
>> >> 65: option remote-subvolume /mnt/raid6-storage/storage
>> >> 66: option transport-type socket
>> >> 67: option send-gids true
>> >> 68: end-volume
>> >> 69:
>> >> 70: volume storage-client-5
>> >> 71: type protocol/client
>> >> 72: option ping-timeout 42
>> >> 73: option remote-host 10.0.231.55
>> >> 74: option remote-subvolume /mnt/raid6-storage/storage
>> >> 75: option transport-type socket
>> >> 76: option send-gids true
>> >> 77: end-volume
>> >> 78:
>> >> 79: volume storage-dht
>> >> 80: type cluster/distribute
>> >> 81: subvolumes storage-client-0 storage-client-1 storage-client-2
>> >> storage-client-3 storage-client-4 storage-client-5
>> >> 82: end-volume
>> >>
>> >>
>> >> But not the client op-version, how can I retrieve this info?
>> >>
>> >> Thanks
>> >>
>> >> On Tue, Mar 1, 2016 at 10:19 PM, Gaurav Garg <ggarg at redhat.com> wrote:
>> >>
>> >> > Hi Steve,
>> >> >
>> >> > Which version you have upgraded client, could you tell us client
>> >> > op-version after upgrade ?
>> >> >
>> >> >
>> >> > have you upgraded all of your clients ?
>> >> >
>> >> >
>> >> > Thanks,
>> >> > Gaurav
>> >> >
>> >> >
>> >> > ----- Original Message -----
>> >> > From: "Steve Dainard" <sdainard at spd1.com>
>> >> > To: "gluster-users at gluster.org List" <gluster-users at gluster.org>
>> >> > Sent: Wednesday, March 2, 2016 1:10:27 AM
>> >> > Subject: [Gluster-users] gluster 3.7.6 volume set: failed: One or
>> more
>> >> > connected clients cannot support the feature being set
>> >> >
>> >> > Gluster 3.7.6
>> >> > 'storage' is a distributed volume
>> >> >
>> >> > # gluster volume set storage rebal-throttle lazy
>> >> > volume set: failed: One or more connected clients cannot support the
>> >> > feature being set. These clients need to be upgraded or disconnected
>> before
>> >> > running this command again
>> >> >
>> >> > I found a client connected using version 3.6.7 so I upgraded &
>> >> > umount/mount the gluster volume on the client but I'm still getting
>> this
>> >> > error.
>> >> >
>> >> > I've run grep "accepted client from" /var/log/glusterfs/bricks/* |
>> grep -v
>> >> > 3.7.6 and I get a few returns from the client above, all dated last
>> week.
>> >> >
>> >> > I've run 'gluster volume status storage clients' and checked the
>> connected
>> >> > clients manually, they're all running 3.7.6.
>> >> >
>> >> > /var/log/gluster/etc-glusterfs-glusterd.vol.log:
>> >> > [2016-03-01 19:23:20.180821] E [MSGID: 106022]
>> >> > [glusterd-utils.c:10154:glusterd_check_client_op_version_support]
>> >> > 0-management: One or more c
>> >> > lients don't support the required op-version
>> >> > [2016-03-01 19:23:20.180853] E [MSGID: 106301]
>> >> > [glusterd-syncop.c:1274:gd_stage_op_phase] 0-management: Staging of
>> >> > operation 'Volume Set' fa
>> >> > iled on localhost : One or more connected clients cannot support the
>> >> > feature being set. These clients need to be upgraded or disconnected
>> be
>> >> > fore running this command again
>> >> >
>> >> > Also tried setting the diagnostics.brick-log-level logging level and
>> got
>> >> > the same error.
>> >> >
>> >> > /var/lib/glusterd/vols/storage/info:
>> >> > type=0
>> >> > count=6
>> >> > status=1
>> >> > sub_count=0
>> >> > stripe_count=1
>> >> > replica_count=1
>> >> > disperse_count=0
>> >> > redundancy_count=0
>> >> > version=26
>> >> > transport-type=0
>> >> > volume-id=26d355cb-c486-481f-ac16-e25390e73775
>> >> > username=eb9e2063-6ba8-4d16-a54f-2c7cf7740c4c
>> >> > password=
>> >> > op-version=3
>> >> > client-op-version=3
>> >> > quota-version=1
>> >> > parent_volname=N/A
>> >> > restored_from_snap=00000000-0000-0000-0000-000000000000
>> >> > snap-max-hard-limit=256
>> >> > features.quota-deem-statfs=on
>> >> > features.inode-quota=on
>> >> > diagnostics.brick-log-level=WARNING
>> >> > features.quota=on
>> >> > performance.readdir-ahead=on
>> >> > performance.cache-size=1GB
>> >> > performance.stat-prefetch=on
>> >> > brick-0=10.0.231.50:-mnt-raid6-storage-storage
>> >> > brick-1=10.0.231.51:-mnt-raid6-storage-storage
>> >> > brick-2=10.0.231.52:-mnt-raid6-storage-storage
>> >> > brick-3=10.0.231.53:-mnt-raid6-storage-storage
>> >> > brick-4=10.0.231.54:-mnt-raid6-storage-storage
>> >> > brick-5=10.0.231.55:-mnt-raid6-storage-storage
>> >> >
>> >> >
>> >> > _______________________________________________
>> >> > Gluster-users mailing list
>> >> > Gluster-users at gluster.org
>> >> > http://www.gluster.org/mailman/listinfo/gluster-users
>> >> >
>> >
>> >
>> >
>> > _______________________________________________
>> > Gluster-users mailing list
>> > Gluster-users at gluster.org
>> > http://www.gluster.org/mailman/listinfo/gluster-users
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160304/bb06c899/attachment.html>
More information about the Gluster-users
mailing list