[Gluster-users] Quota issue after upgrading from 3.6.4 to 3.6.6

[Jorick Astrego]

On 01/27/2016 05:44 PM, Atin Mukherjee wrote:
> Please share the glusterd log files along with cmd history log file from
> all the nodes.
>
> Thanks,
> Atin
>
> On 01/27/2016 08:33 PM, Jorick Astrego wrote:
>> Hi,
>>
>> After upgrading one of a 3 node glusterfs cluster to 3.6.6, I cannot get
>> the HDD-1 volume to start properly on this server. The other servers are
>> still running 3.6.4 because I don't know if they will all fail after the
>> upgrade. There appears to be an issue with the quota fixed in 3.6.6
>> (https://bugzilla.redhat.com/show_bug.cgi?id=1259578) that may be the cause:
>>
>>      gluster volume status HDD1
>>      Locking failed on 10.99.99.1. Please check log file for details.
>>      Locking failed on 10.99.99.3. Please check log file for details.
>>
>>      [2016-01-27 14:52:11.268901] W
>>      [marker-quota.c:1452:mq_release_parent_lock] (-->
>>      /lib64/libglusterfs.so.0(_gf_log_callingfn+0x186)[0x7f869c2094f6]
>>      (-->
>>      /usr/lib64/glusterfs/3.6.6/xlator/features/marker.so(mq_release_parent_lock+0x271)[0x7f868a2b0bd1]
>>      (-->
>>      /usr/lib64/glusterfs/3.6.6/xlator/features/marker.so(mq_fetch_child_size_and_contri+0x30b)[0x7f868a2b1e1b]
>>      (-->
>>      /lib64/libglusterfs.so.0(default_setxattr_cbk+0xa3)[0x7f869c210bc3]
>>      (-->
>>      /usr/lib64/glusterfs/3.6.6/xlator/features/access-control.so(posix_acl_setxattr_cbk+0xa3)[0x7f868acfb743]
>>      ))))) 0-HDD1-marker: An operation during quota updation of path
>>      (/52884f44-75f5-45ab-b909-a547e7c1c6f7/images/ce445275-4aa5-4f66-ad80-44259b425d98/33fd39ac-abbb-42df-8c9f-639c93e8c00d)
>>      failed (Invalid argument)
>>      [2016-01-27 14:52:11.284504] W
>>      [marker-quota.c:1452:mq_release_parent_lock] (-->
>>      /lib64/libglusterfs.so.0(_gf_log_callingfn+0x186)[0x7f869c2094f6]
>>      (-->
>>      /usr/lib64/glusterfs/3.6.6/xlator/features/marker.so(mq_release_parent_lock+0x271)[0x7f868a2b0bd1]
>>      (-->
>>      /usr/lib64/glusterfs/3.6.6/xlator/features/marker.so(mq_fetch_child_size_and_contri+0x30b)[0x7f868a2b1e1b]
>>      (-->
>>      /lib64/libglusterfs.so.0(default_setxattr_cbk+0xa3)[0x7f869c210bc3]
>>      (-->
>>      /usr/lib64/glusterfs/3.6.6/xlator/features/access-control.so(posix_acl_setxattr_cbk+0xa3)[0x7f868acfb743]
>>      ))))) 0-HDD1-marker: An operation during quota updation of path
>>      (/52884f44-75f5-45ab-b909-a547e7c1c6f7/images/ce445275-4aa5-4f66-ad80-44259b425d98/33fd39ac-abbb-42df-8c9f-639c93e8c00d)
>>      failed (Invalid argument)
>>      [2016-01-27 14:52:11.288115] W
>>      [marker-quota.c:1452:mq_release_parent_lock] (-->
>>      /lib64/libglusterfs.so.0(_gf_log_callingfn+0x186)[0x7f869c2094f6]
>>      (-->
>>      /usr/lib64/glusterfs/3.6.6/xlator/features/marker.so(mq_release_parent_lock+0x271)[0x7f868a2b0bd1]
>>      (-->
>>      /usr/lib64/glusterfs/3.6.6/xlator/features/marker.so(mq_fetch_child_size_and_contri+0x30b)[0x7f868a2b1e1b]
>>      (-->
>>      /lib64/libglusterfs.so.0(default_setxattr_cbk+0xa3)[0x7f869c210bc3]
>>      (-->
>>      /usr/lib64/glusterfs/3.6.6/xlator/features/access-control.so(posix_acl_setxattr_cbk+0xa3)[0x7f868acfb743]
>>      ))))) 0-HDD1-marker: An operation during quota updation of path
>>      (/52884f44-75f5-45ab-b909-a547e7c1c6f7/images/ce445275-4aa5-4f66-ad80-44259b425d98/33fd39ac-abbb-42df-8c9f-639c93e8c00d)
>>      failed (Invalid argument)
>>      [2016-01-27 14:52:11.292966] W
>>      [marker-quota.c:1452:mq_release_parent_lock] (-->
>>      /lib64/libglusterfs.so.0(_gf_log_callingfn+0x186)[0x7f869c2094f6]
>>      (-->
>>      /usr/lib64/glusterfs/3.6.6/xlator/features/marker.so(mq_release_parent_lock+0x271)[0x7f868a2b0bd1]
>>      (-->
>>      /usr/lib64/glusterfs/3.6.6/xlator/features/marker.so(mq_fetch_child_size_and_contri+0x30b)[0x7f868a2b1e1b]
>>      (-->
>>      /lib64/libglusterfs.so.0(default_setxattr_cbk+0xa3)[0x7f869c210bc3]
>>      (-->
>>      /usr/lib64/glusterfs/3.6.6/xlator/features/access-control.so(posix_acl_setxattr_cbk+0xa3)[0x7f868acfb743]
>>      ))))) 0-HDD1-marker: An operation during quota updation of path
>>      (/52884f44-75f5-45ab-b909-a547e7c1c6f7/images/ce445275-4aa5-4f66-ad80-44259b425d98/33fd39ac-abbb-42df-8c9f-639c93e8c00d)
>>      failed (Invalid argument)
>>
>>
>>
Hi,

Any progress on this? I hope you received the logs last week.

Currently we have a lot of stability problems with the cluster after I 
tried a downgrade to the original version without resolving the errors.

The node with the errors is still up as far as oVirt sees it and some 
hosts connect to it. After a couple of hours VM's start to pause with 
unkown storage errors.

What would be the best aproach as we now have a degraded storage 
cluster? Wipe and reinitialize the node or are there things I could try 
first?






Met vriendelijke groet, With kind regards,

Jorick Astrego

Netbulae Virtualization Experts 

----------------

	Tel: 053 20 30 270 	info at netbulae.eu 	Staalsteden 4-3A 	KvK 08198180
 	Fax: 053 20 30 271 	www.netbulae.eu 	7547 TA Enschede 	BTW NL821234584B01

----------------

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160203/57811043/attachment.html>