[Gluster-users] gluster 3.7.6 volume set: failed: One or more connected clients cannot support the feature being set
Atin Mukherjee
atin.mukherjee83 at gmail.com
Sat Mar 5 07:20:30 UTC 2016
This is really weird, if you unmount the clients then statedump shouldn't
be showing the same. Was unmount successful? Do you see any related error
log entry in mount & glusterd log?
-Atin
Sent from one plus one
On 04-Mar-2016 10:23 pm, "Steve Dainard" <sdainard at spd1.com> wrote:
> Except that even when I unmount gluster from both Debian clients, I'm
> still getting the same error, and a dump of those clients after unmounting
> still shows:
>
> glusterd.client1.identifier=10.0.231.10:1023
> glusterd.client1.volname=storage
> glusterd.client1.max-op-version=30603
> glusterd.client1.min-op-version=1
>
> glusterd.client4.identifier=10.0.231.11:1022
> glusterd.client4.volname=storage
> glusterd.client4.max-op-version=30603
> glusterd.client4.min-op-version=1
>
>
>
>
>
> On Thu, Mar 3, 2016 at 5:28 PM, Atin Mukherjee <atin.mukherjee83 at gmail.com
> > wrote:
>
>> -Atin
>> Sent from one plus one
>> On 04-Mar-2016 3:35 am, "Steve Dainard" <sdainard at spd1.com> wrote:
>> >
>> > FYI Gluster storage node hostnames are gluster0[1-6].
>> >
>> > Full dump attached. I see a few clients not on 30706. Most notably the
>> two debian 7 servers (using packages from gluster.org) seem to be
>> running lower op versions than the centos7 machines (every other client in
>> 10.0.231.0/24 subnet).
>> >
>> > glusterd.client1.identifier=10.0.231.10:1023 <-- debian 7, glusterfs
>> 3.7.6 built on Feb 4 2016 06:25:19
>> > glusterd.client1.volname=storage
>> > glusterd.client1.max-op-version=30603
>> This is running with 3.6.3 and that's why volume set fails. Unmount this
>> client and upgrade and remount it back.
>>
>> > glusterd.client1.min-op-version=1
>> >
>> > glusterd.client2.identifier=10.0.231.51:65515 <-- gluster02
>> (mounted localhost:storage on /run/gluster/storage type fuse.glusterfs)
>> > glusterd.client2.volname=
>> > glusterd.client2.max-op-version=0
>> > glusterd.client2.min-op-version=0
>> >
>> > glusterd.client3.identifier=10.0.231.54:65521 <-- gluster05 (no actual
>> mounts)
>> > glusterd.client3.volname=
>> > glusterd.client3.max-op-version=0
>> > glusterd.client3.min-op-version=0
>> >
>> > glusterd.client4.identifier=10.0.231.11:1022 <--- debian 7, glusterfs
>> 3.7.6 built on Feb 4 2016 06:25:19
>> > glusterd.client4.volname=storage
>> > glusterd.client4.max-op-version=30603
>> > glusterd.client4.min-op-version=1
>> >
>> > glusterd.client5.identifier=10.0.231.55:65530 <-- gluster06 (no actual
>> mounts)
>> > glusterd.client5.volname=
>> > glusterd.client5.max-op-version=0
>> > glusterd.client5.min-op-version=0
>> >
>> > glusterd.client6.identifier=10.0.231.53:65516 <-- gluster04
>> (mounted localhost:storage on /run/gluster/storage type fuse.glusterfs)
>> > glusterd.client6.volname=
>> > glusterd.client6.max-op-version=0
>> > glusterd.client6.min-op-version=0
>> >
>> > glusterd.client7.identifier=10.0.231.50:65529
>> >
>> glusterd.client7.volname=export-domain-storage.10.0.231.50.mnt-lv-export-domain-storage-export-domain-storage
>> > glusterd.client7.max-op-version=30706
>> > glusterd.client7.min-op-version=1
>> >
>> > ...
>> >
>> > Debian package info:
>> > apt-cache policy glusterfs-client
>> > glusterfs-client:
>> > Installed: 3.7.6-2
>> > Candidate: 3.7.6-2
>> > Version table:
>> > *** 3.7.6-2 0
>> > 500
>> http://download.gluster.org/pub/gluster/glusterfs/3.7/3.7.6/Debian/wheezy/apt/
>> wheezy/main amd64 Packages
>> >
>> >
>> > Thanks
>> >
>> > On Wed, Mar 2, 2016 at 10:29 PM, Gaurav Garg <ggarg at redhat.com> wrote:
>> >>
>> >> Hi Steve,
>> >>
>> >> As atin pointed out to take statedump by running #kill -SIGUSR1
>> $(pidof glusterd) command. it will create .dump file in /var/run/gluster/
>> directory. client-op-version information will be present in dump file.
>> >>
>> >> Thanks,
>> >> ~Gaurav
>> >>
>> >> ----- Original Message -----
>> >> From: "Steve Dainard" <sdainard at spd1.com>
>> >> To: "Gaurav Garg" <ggarg at redhat.com>
>> >> Cc: "gluster-users at gluster.org List" <gluster-users at gluster.org>
>> >> Sent: Thursday, March 3, 2016 12:07:25 AM
>> >> Subject: Re: [Gluster-users] gluster 3.7.6 volume set: failed: One or
>> more connected clients cannot support the feature being set
>> >>
>> >> From the the client side logs I can see version info on mount:
>> >>
>> >> Final graph:
>> >>
>> +------------------------------------------------------------------------------+
>> >> 1: volume storage-client-0
>> >> 2: type protocol/client
>> >> 3: option clnt-lk-version 1
>> >> 4: option volfile-checksum 0
>> >> 5: option volfile-key /storage
>> >> 6: option client-version 3.7.6
>> >> 7: option process-uuid
>> >>
>> template-centos7-compute.compute.domain-2773-2016/03/02-18:28:34:328100-storage-client-0-0-0
>> >> 8: option fops-version 1298437
>> >> 9: option ping-timeout 42
>> >> 10: option remote-host 10.0.231.50
>> >> 11: option remote-subvolume /mnt/raid6-storage/storage
>> >> 12: option transport-type socket
>> >> 13: option send-gids true
>> >> 14: end-volume
>> >> 15:
>> >> 16: volume storage-client-1
>> >> 17: type protocol/client
>> >> 18: option clnt-lk-version 1
>> >> 19: option volfile-checksum 0
>> >> 20: option volfile-key /storage
>> >> 21: option client-version 3.7.6
>> >> 22: option process-uuid
>> >>
>> template-centos7-compute.compute.domain-2773-2016/03/02-18:28:34:328100-storage-client-1-0-0
>> >> 23: option fops-version 1298437
>> >> 24: option ping-timeout 42
>> >> 25: option remote-host 10.0.231.51
>> >> 26: option remote-subvolume /mnt/raid6-storage/storage
>> >> 27: option transport-type socket
>> >> 28: option send-gids true
>> >> 29: end-volume
>> >> 30:
>> >> 31: volume storage-client-2
>> >> 32: type protocol/client
>> >> 33: option clnt-lk-version 1
>> >> 34: option volfile-checksum 0
>> >> 35: option volfile-key /storage
>> >> 36: option client-version 3.7.6
>> >> 37: option process-uuid
>> >>
>> template-centos7-compute.compute.domain-2773-2016/03/02-18:28:34:328100-storage-client-2-0-0
>> >> 38: option fops-version 1298437
>> >> 39: option ping-timeout 42
>> >> 40: option remote-host 10.0.231.52
>> >> 41: option remote-subvolume /mnt/raid6-storage/storage
>> >> 42: option transport-type socket
>> >> 43: option send-gids true
>> >> 44: end-volume
>> >> 45:
>> >> 46: volume storage-client-3
>> >> 47: type protocol/client
>> >> 48: option clnt-lk-version 1
>> >> 49: option volfile-checksum 0
>> >> 50: option volfile-key /storage
>> >> 51: option client-version 3.7.6
>> >> 52: option process-uuid
>> >>
>> template-centos7-compute.compute.domain-2773-2016/03/02-18:28:34:328100-storage-client-3-0-0
>> >> 53: option fops-version 1298437
>> >> 54: option ping-timeout 42
>> >> 55: option remote-host 10.0.231.53
>> >> 56: option remote-subvolume /mnt/raid6-storage/storage
>> >> 57: option transport-type socket
>> >> 58: option send-gids true
>> >> 59: end-volume
>> >> 60:
>> >> 61: volume storage-client-4
>> >> 62: type protocol/client
>> >> 63: option ping-timeout 42
>> >> 64: option remote-host 10.0.231.54
>> >> 65: option remote-subvolume /mnt/raid6-storage/storage
>> >> 66: option transport-type socket
>> >> 67: option send-gids true
>> >> 68: end-volume
>> >> 69:
>> >> 70: volume storage-client-5
>> >> 71: type protocol/client
>> >> 72: option ping-timeout 42
>> >> 73: option remote-host 10.0.231.55
>> >> 74: option remote-subvolume /mnt/raid6-storage/storage
>> >> 75: option transport-type socket
>> >> 76: option send-gids true
>> >> 77: end-volume
>> >> 78:
>> >> 79: volume storage-dht
>> >> 80: type cluster/distribute
>> >> 81: subvolumes storage-client-0 storage-client-1 storage-client-2
>> >> storage-client-3 storage-client-4 storage-client-5
>> >> 82: end-volume
>> >>
>> >>
>> >> But not the client op-version, how can I retrieve this info?
>> >>
>> >> Thanks
>> >>
>> >> On Tue, Mar 1, 2016 at 10:19 PM, Gaurav Garg <ggarg at redhat.com> wrote:
>> >>
>> >> > Hi Steve,
>> >> >
>> >> > Which version you have upgraded client, could you tell us client
>> >> > op-version after upgrade ?
>> >> >
>> >> >
>> >> > have you upgraded all of your clients ?
>> >> >
>> >> >
>> >> > Thanks,
>> >> > Gaurav
>> >> >
>> >> >
>> >> > ----- Original Message -----
>> >> > From: "Steve Dainard" <sdainard at spd1.com>
>> >> > To: "gluster-users at gluster.org List" <gluster-users at gluster.org>
>> >> > Sent: Wednesday, March 2, 2016 1:10:27 AM
>> >> > Subject: [Gluster-users] gluster 3.7.6 volume set: failed: One or
>> more
>> >> > connected clients cannot support the feature being set
>> >> >
>> >> > Gluster 3.7.6
>> >> > 'storage' is a distributed volume
>> >> >
>> >> > # gluster volume set storage rebal-throttle lazy
>> >> > volume set: failed: One or more connected clients cannot support the
>> >> > feature being set. These clients need to be upgraded or disconnected
>> before
>> >> > running this command again
>> >> >
>> >> > I found a client connected using version 3.6.7 so I upgraded &
>> >> > umount/mount the gluster volume on the client but I'm still getting
>> this
>> >> > error.
>> >> >
>> >> > I've run grep "accepted client from" /var/log/glusterfs/bricks/* |
>> grep -v
>> >> > 3.7.6 and I get a few returns from the client above, all dated last
>> week.
>> >> >
>> >> > I've run 'gluster volume status storage clients' and checked the
>> connected
>> >> > clients manually, they're all running 3.7.6.
>> >> >
>> >> > /var/log/gluster/etc-glusterfs-glusterd.vol.log:
>> >> > [2016-03-01 19:23:20.180821] E [MSGID: 106022]
>> >> > [glusterd-utils.c:10154:glusterd_check_client_op_version_support]
>> >> > 0-management: One or more c
>> >> > lients don't support the required op-version
>> >> > [2016-03-01 19:23:20.180853] E [MSGID: 106301]
>> >> > [glusterd-syncop.c:1274:gd_stage_op_phase] 0-management: Staging of
>> >> > operation 'Volume Set' fa
>> >> > iled on localhost : One or more connected clients cannot support the
>> >> > feature being set. These clients need to be upgraded or disconnected
>> be
>> >> > fore running this command again
>> >> >
>> >> > Also tried setting the diagnostics.brick-log-level logging level and
>> got
>> >> > the same error.
>> >> >
>> >> > /var/lib/glusterd/vols/storage/info:
>> >> > type=0
>> >> > count=6
>> >> > status=1
>> >> > sub_count=0
>> >> > stripe_count=1
>> >> > replica_count=1
>> >> > disperse_count=0
>> >> > redundancy_count=0
>> >> > version=26
>> >> > transport-type=0
>> >> > volume-id=26d355cb-c486-481f-ac16-e25390e73775
>> >> > username=eb9e2063-6ba8-4d16-a54f-2c7cf7740c4c
>> >> > password=
>> >> > op-version=3
>> >> > client-op-version=3
>> >> > quota-version=1
>> >> > parent_volname=N/A
>> >> > restored_from_snap=00000000-0000-0000-0000-000000000000
>> >> > snap-max-hard-limit=256
>> >> > features.quota-deem-statfs=on
>> >> > features.inode-quota=on
>> >> > diagnostics.brick-log-level=WARNING
>> >> > features.quota=on
>> >> > performance.readdir-ahead=on
>> >> > performance.cache-size=1GB
>> >> > performance.stat-prefetch=on
>> >> > brick-0=10.0.231.50:-mnt-raid6-storage-storage
>> >> > brick-1=10.0.231.51:-mnt-raid6-storage-storage
>> >> > brick-2=10.0.231.52:-mnt-raid6-storage-storage
>> >> > brick-3=10.0.231.53:-mnt-raid6-storage-storage
>> >> > brick-4=10.0.231.54:-mnt-raid6-storage-storage
>> >> > brick-5=10.0.231.55:-mnt-raid6-storage-storage
>> >> >
>> >> >
>> >> > _______________________________________________
>> >> > Gluster-users mailing list
>> >> > Gluster-users at gluster.org
>> >> > http://www.gluster.org/mailman/listinfo/gluster-users
>> >> >
>> >
>> >
>> >
>> > _______________________________________________
>> > Gluster-users mailing list
>> > Gluster-users at gluster.org
>> > http://www.gluster.org/mailman/listinfo/gluster-users
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160305/6187ed13/attachment.html>
More information about the Gluster-users
mailing list