<div dir="ltr"><div>Hi Artem,</div><div><br></div><div>The above issue is depended on <a href="https://review.gluster.org/#/c/glusterfs/+/24289/">https://review.gluster.org/#/c/glusterfs/+/24289/</a> and <br></div><div><a href="https://review.gluster.org/#/c/glusterfs/+/24264/">https://review.gluster.org/#/c/glusterfs/+/24264/</a>. Both of the patches are available in release-7.5<br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Apr 27, 2020 at 3:56 AM Artem Russakovskii <<a href="mailto:archon810@gmail.com">archon810@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Looks like it's in 7.5, right?<div><br></div><div><a href="https://docs.gluster.org/en/latest/release-notes/7.5/" target="_blank">https://docs.gluster.org/en/latest/release-notes/7.5/</a></div><div><a href="https://github.com/gluster/glusterfs/issues/832" target="_blank">https://github.com/gluster/glusterfs/issues/832</a><br clear="all"><div><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><br>Sincerely,<br>Artem<br><br>--<br>Founder, <a href="http://www.androidpolice.com" target="_blank">Android Police</a>, <a href="http://www.apkmirror.com/" style="font-size:12.8px" target="_blank">APK Mirror</a><span style="font-size:12.8px">, Illogical Robot LLC</span></div><div dir="ltr"><a href="http://beerpla.net/" target="_blank">beerpla.net</a> | <a href="http://twitter.com/ArtemR" target="_blank">@ArtemR</a><br></div></div></div></div></div></div></div></div></div></div></div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Mar 23, 2020 at 10:26 PM Hari Gowtham <<a href="mailto:hgowtham@redhat.com" target="_blank">hgowtham@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>The patch is not there in 7.4. It is yet to be backported. It should soon be backported to the release branches.<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Mar 24, 2020 at 3:24 AM Artem Russakovskii <<a href="mailto:archon810@gmail.com" target="_blank">archon810@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Could you please confirm whether it made it into 7.4 or not? <a href="https://docs.gluster.org/en/latest/release-notes/7.4/" target="_blank">https://docs.gluster.org/en/latest/release-notes/7.4/</a> - I don't think I'm seeing it, but maybe it's one of these?<div><br></div><div>Thanks.<br clear="all"><div><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><br>Sincerely,<br>Artem<br><br>--<br>Founder, <a href="http://www.androidpolice.com" target="_blank">Android Police</a>, <a href="http://www.apkmirror.com/" style="font-size:12.8px" target="_blank">APK Mirror</a><span style="font-size:12.8px">, Illogical Robot LLC</span></div><div dir="ltr"><a href="http://beerpla.net/" target="_blank">beerpla.net</a> | <a href="http://twitter.com/ArtemR" target="_blank">@ArtemR</a><br></div></div></div></div></div></div></div></div></div></div></div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Mar 10, 2020 at 10:33 PM Hari Gowtham <<a href="mailto:hgowtham@redhat.com" target="_blank">hgowtham@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>As the patch is yet to be merged in master, release-7 I can't give a definite release version.</div><div>I expect it to be merged in 7.4 if they get merged in master and release-7 soon. Else it will be made available in 7.5 for sure.</div><div>I'm CCing Rinku who takes care of the release-7 cycles.</div><div><br></div><div>Rinku, before the release, once the availability is known for sure can you please respond here which version the patch makes it to?<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Mar 11, 2020 at 2:33 AM Artem Russakovskii <<a href="mailto:archon810@gmail.com" target="_blank">archon810@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">I'd actually be looking for this fix in v7 since it's the one preventing us from upgrading to it. Curious which exact version of 7 I should be looking out for with this fix.<br clear="all"><div><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><br>Sincerely,<br>Artem<br><br>--<br>Founder, <a href="http://www.androidpolice.com" target="_blank">Android Police</a>, <a href="http://www.apkmirror.com/" style="font-size:12.8px" target="_blank">APK Mirror</a><span style="font-size:12.8px">, Illogical Robot LLC</span></div><div dir="ltr"><a href="http://beerpla.net/" target="_blank">beerpla.net</a> | <a href="http://twitter.com/ArtemR" target="_blank">@ArtemR</a><br></div></div></div></div></div></div></div></div></div></div></div></div></div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Mar 9, 2020 at 1:41 AM Hari Gowtham <<a href="mailto:hgowtham@redhat.com" target="_blank">hgowtham@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>The release for 7 will happen every month while the release 5 and 6 are done once in every 2 months.</div><div>We are done for February, so the next cycle for 5 and 6 are in April. <br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Mar 7, 2020 at 1:00 PM Artem Russakovskii <<a href="mailto:archon810@gmail.com" target="_blank">archon810@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Great, I'll be glad to test and confirm once the release with this change is out. Would you please confirm when it is by replying here?<div><br><div>Thanks, Amar.<br clear="all"><div><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><br>Sincerely,<br>Artem<br><br>--<br>Founder, <a href="http://www.androidpolice.com" target="_blank">Android Police</a>, <a href="http://www.apkmirror.com/" style="font-size:12.8px" target="_blank">APK Mirror</a><span style="font-size:12.8px">, Illogical Robot LLC</span></div><div dir="ltr"><a href="http://beerpla.net/" target="_blank">beerpla.net</a> | <a href="http://twitter.com/ArtemR" target="_blank">@ArtemR</a><br></div></div></div></div></div></div></div></div></div></div></div></div></div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Mar 3, 2020 at 8:25 PM Amar Tumballi <<a href="mailto:amar@kadalu.io" target="_blank">amar@kadalu.io</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Feb 15, 2020 at 7:27 AM Amar Tumballi <<a href="mailto:amar@kadalu.io" target="_blank">amar@kadalu.io</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Feb 15, 2020 at 3:02 AM Artem Russakovskii <<a href="mailto:archon810@gmail.com" target="_blank">archon810@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Amar,<div><br></div><div>From what I can tell, there's no error actually using the referenced files, but the log messages all constantly pop up in the log when accessed through the web server. The files display OK in the browser.</div><div><br></div><div>I also tried renaming/touching them via shell as the user that owns them, and that worked fine (with no notices generated).</div></div></blockquote><div><br></div><div>Thanks for this update.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><br></div><div>I hope you guys figure it out because I really don't feel comfortable upgrading to 6 or 7 from 5 which has been stable.<br clear="all"><div><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><br></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></blockquote><div><br></div><div>Ack! Makes sense, upgrading to a version when you know there is a regression is not a good thing. We will check this out.</div><div> </div></div></div></blockquote><div><br></div><div><a href="https://review.gluster.org/#/c/glusterfs/+/24200/" target="_blank">https://review.gluster.org/#/c/glusterfs/+/24200/</a> would hopefully fix the issue.<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div></div><div>-Amar</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr">Sincerely,<br>Artem<br><br>--<br>Founder, <a href="http://www.androidpolice.com" target="_blank">Android Police</a>, <a href="http://www.apkmirror.com/" style="font-size:12.8px" target="_blank">APK Mirror</a><span style="font-size:12.8px">, Illogical Robot LLC</span></div><div dir="ltr"><a href="http://beerpla.net/" target="_blank">beerpla.net</a> | <a href="http://twitter.com/ArtemR" target="_blank">@ArtemR</a><br></div></div></div></div></div></div></div></div></div></div></div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Feb 12, 2020 at 11:02 PM Amar Tumballi <<a href="mailto:amarts@gmail.com" target="_blank">amarts@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div><br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Feb 7, 2020 at 1:56 PM Artem Russakovskii <<a href="mailto:archon810@gmail.com" target="_blank">archon810@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Still no response? Amar?<br clear="all"><div><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><br></div></div></div></div></div></div></div></div></div></div></div></div></div></div></blockquote><div><br></div><div>Sorry, this got down in my Inbox, and missed responding, and also I got busy with few other engagements. Reply inline.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr">Sincerely,<br>Artem<br><br></div></div></div></div></div></div></div></div></div></div></div></div></div></div><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Jan 14, 2020 at 12:57 PM Artem Russakovskii <<a href="mailto:archon810@gmail.com" target="_blank">archon810@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi,<div><br></div><div>Any updates here please?<br clear="all"><div><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><br>Sincerely,<br>Artem<br><br>--<br>Founder, <a href="http://www.androidpolice.com" target="_blank">Android Police</a>, <a href="http://www.apkmirror.com/" style="font-size:12.8px" target="_blank">APK Mirror</a><span style="font-size:12.8px">, Illogical Robot LLC</span></div><div dir="ltr"><a href="http://beerpla.net/" target="_blank">beerpla.net</a> | <a href="http://twitter.com/ArtemR" target="_blank">@ArtemR</a><br></div></div></div></div></div></div></div></div></div></div></div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Jan 6, 2020 at 4:18 PM Artem Russakovskii <<a href="mailto:archon810@gmail.com" target="_blank">archon810@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Thanks Amar,<div><br></div><div>Checking the brick logs, I'm seeing these and numerous other files with Permission denied. The question is why is the permission being denied? Gluster runs as root and the files are owned by wwwrun:www, which it should have full access to.</div><div><br></div><div>Is there any more documentation on what enabling or disabling this feature accomplishes, why it was added to 7.1 by default (and 7.0 as far as I can tell), and what side effects it may have? Basically, I'd like to understand the problem better instead of simply turning the flag off and fix the underlying issue (or help fix a bug if it's a bug in gluster).</div><div><br></div><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">==> bricks/mnt-pylon_block6-dev_SNIP_data.log <==<br>[2020-01-06 21:39:00.176476] I [MSGID: 115060] [server-rpc-fops_v2.c:897:server4_setxattr_cbk] 0-dev_SNIP_data-server: client: CTX_ID:3e0436a5-cdf8-4d3f-a4e9-fc681fe902d3-GRAPH_ID:2-PID:32069-HOST:pylon-PC_NAME:dev_SNIP_data-client-2-RECON_NO:-0, error-xlator: dev_SNIP_data-access-control [Permission denied]<br>[2020-01-06 21:39:00.325329] I [MSGID: 139001] [posix-acl.c:262:posix_acl_log_permit_denied] 0-dev_SNIP_data-access-control: client: CTX_ID:3e0436a5-cdf8-4d3f-a4e9-fc681fe902d3-GRAPH_ID:2-PID:32069-HOST:pylon-PC_NAME:dev_SNIP_data-client-2-RECON_NO:-0, gfid: e9bd83fd-b16e-44ef-8fde-2ae320ee54d0, req(uid:491,gid:488,perm:2,ngrps:1), ctx(uid:30,gid:8,in-groups:0,perm:664,updated-fop:LOOKUP, acl:-) [Permission denied]<br>[2020-01-06 21:39:00.325435] I [MSGID: 115060] [server-rpc-fops.c:937:_gf_server_log_setxattr_failure] 0-dev_SNIP_data-server: 709342: SETXATTR /uploads/2019/06/Anker-PowerPort-Atom-10-768x512.jpg (e9bd83fd-b16e-44ef-8fde-2ae320ee54d0) ==> set-ctime-mdata, client: CTX_ID:3e0436a5-cdf8-4d3f-a4e9-fc681fe902d3-GRAPH_ID:2-PID:32069-HOST:pylon-PC_NAME:dev_SNIP_data-client-2-RECON_NO:-0, error-xlator: dev_SNIP_data-access-control<br>[2020-01-06 21:39:00.325460] I [MSGID: 115060] [server-rpc-fops_v2.c:897:server4_setxattr_cbk] 0-dev_SNIP_data-server: client: CTX_ID:3e0436a5-cdf8-4d3f-a4e9-fc681fe902d3-GRAPH_ID:2-PID:32069-HOST:pylon-PC_NAME:dev_SNIP_data-client-2-RECON_NO:-0, error-xlator: dev_SNIP_data-access-control [Permission denied]<br>[2020-01-06 21:39:01.195012] I [MSGID: 139001] [posix-acl.c:262:posix_acl_log_permit_denied] 0-dev_SNIP_data-access-control: client: CTX_ID:3e0436a5-cdf8-4d3f-a4e9-fc681fe902d3-GRAPH_ID:2-PID:32069-HOST:pylon-PC_NAME:dev_SNIP_data-client-2-RECON_NO:-0, gfid: ff96827e-777b-45fb-a40d-e5089cd024fe, req(uid:491,gid:488,perm:2,ngrps:1), ctx(uid:30,gid:8,in-groups:0,perm:664,updated-fop:LOOKUP, acl:-) [Permission denied]<br>[2020-01-06 21:39:01.195091] I [MSGID: 115060] [server-rpc-fops.c:937:_gf_server_log_setxattr_failure] 0-dev_SNIP_data-server: 709349: SETXATTR /uploads/2019/06/Anker-PowerPort-Atom-768x512.jpg (ff96827e-777b-45fb-a40d-e5089cd024fe) ==> set-ctime-mdata, client: CTX_ID:3e0436a5-cdf8-4d3f-a4e9-fc681fe902d3-GRAPH_ID:2-PID:32069-HOST:pylon-PC_NAME:dev_SNIP_data-client-2-RECON_NO:-0, error-xlator: dev_SNIP_data-access-control<br>[2020-01-06 21:39:01.195291] I [MSGID: 115060] [server-rpc-fops_v2.c:897:server4_setxattr_cbk] 0-dev_SNIP_data-server: client: CTX_ID:3e0436a5-cdf8-4d3f-a4e9-fc681fe902d3-GRAPH_ID:2-PID:32069-HOST:pylon-PC_NAME:dev_SNIP_data-client-2-RECON_NO:-0, error-xlator: dev_SNIP_data-access-control [Permission denied]<br>[2020-01-06 21:39:04.184502] I [MSGID: 139001] [posix-acl.c:262:posix_acl_log_permit_denied] 0-dev_SNIP_data-access-control: client: CTX_ID:3e0436a5-cdf8-4d3f-a4e9-fc681fe902d3-GRAPH_ID:2-PID:32069-HOST:pylon-PC_NAME:dev_SNIP_data-client-2-RECON_NO:-0, gfid: 546305c8-7611-4752-9168-215e9eb03938, req(uid:491,gid:488,perm:2,ngrps:1), ctx(uid:30,gid:8,in-groups:0,perm:664,updated-fop:LOOKUP, acl:-) [Permission denied]<br>[2020-01-06 21:39:04.184554] I [MSGID: 115060] [server-rpc-fops.c:937:_gf_server_log_setxattr_failure] 0-dev_SNIP_data-server: 709360: SETXATTR /uploads/2019/06/Anker-PowerPort-Atom-10-1-768x512.jpg (546305c8-7611-4752-9168-215e9eb03938) ==> set-ctime-mdata, client: CTX_ID:3e0436a5-cdf8-4d3f-a4e9-fc681fe902d3-GRAPH_ID:2-PID:32069-HOST:pylon-PC_NAME:dev_SNIP_data-client-2-RECON_NO:-0, error-xlator: dev_SNIP_data-access-control<br>[2020-01-06 21:39:04.184576] I [MSGID: 115060] [server-rpc-fops_v2.c:897:server4_setxattr_cbk] 0-dev_SNIP_data-server: client: CTX_ID:3e0436a5-cdf8-4d3f-a4e9-fc681fe902d3-GRAPH_ID:2-PID:32069-HOST:pylon-PC_NAME:dev_SNIP_data-client-2-RECON_NO:-0, error-xlator: dev_SNIP_data-access-control [Permission denied]<br clear="all"></blockquote><div><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><br></div><div dir="ltr">Btw, cluster.op-version is 70000, which is the latest. I always rev it up after upgrading the cluster.<br></div><div dir="ltr"><br></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></blockquote></div></blockquote></div></blockquote><div><br></div><div>I don't have any reports of this error coming for other users, and not able to reproduce locally too. I am starting to think that this error and the issue Strahil faced after upgrading (saying there was an ACL issue) may be related. Will check and update. Meantime, is there any application error you are seeing? That would also give us some hints to reproduce the issue.</div><div><br></div><div>-Amar</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr">Sincerely,<br>Artem<br><br>--<br>Founder, <a href="http://www.androidpolice.com" target="_blank">Android Police</a>, <a href="http://www.apkmirror.com/" style="font-size:12.8px" target="_blank">APK Mirror</a><span style="font-size:12.8px">, Illogical Robot LLC</span></div><div dir="ltr"><a href="http://beerpla.net/" target="_blank">beerpla.net</a> | <a href="http://twitter.com/ArtemR" target="_blank">@ArtemR</a><br></div></div></div></div></div></div></div></div></div></div></div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Dec 28, 2019 at 1:19 AM Amar Tumballi <<a href="mailto:amarts@gmail.com" target="_blank">amarts@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">Answers inline.<div><br></div></div><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Dec 25, 2019 at 2:06 AM Artem Russakovskii <<a href="mailto:archon810@gmail.com" target="_blank">archon810@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi everyone,<div><br></div><div>I'm preparing the upgrade for our 5.10 gluster cluster running 1x4 replicate volumes to 7.X and decided to upgrade our test cluster first.</div><div><br></div><div>As soon as I upgraded to 7.0 (and now 7.1), I started seeing the following messages every 10 minutes in the log for one of the volumes:</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">[2019-12-19 21:27:55.041949] W [MSGID: 114031] [client-rpc-fops_v2.c:850:client4_0_setxattr_cbk] 0-dev_SNIP_data-client-0: remote operation failed [Permission denied]<br>[2019-12-19 21:27:55.042002] W [MSGID: 114031] [client-rpc-fops_v2.c:850:client4_0_setxattr_cbk] 0-dev_SNIP_data-client-2: remote operation failed [Permission denied]<br>[2019-12-19 21:27:55.042013] W [MSGID: 114031] [client-rpc-fops_v2.c:850:client4_0_setxattr_cbk] 0-dev_SNIP_data-client-1: remote operation failed [Permission denied]<br>[2019-12-19 21:27:55.042634] E [MSGID: 148002] [utime.c:146:gf_utime_set_mdata_setxattr_cbk] 0-dev_SNIP_data-utime: dict set of key for set-ctime-mdata failed [Permission denied]<br>The message "W [MSGID: 114031] [client-rpc-fops_v2.c:850:client4_0_setxattr_cbk] 0-dev_SNIP_data-client-0: remote operation failed [Permission denied]" repeated 2 times between [2019-12-19 21:27:55.041949] and [2019-12-19 21:27:55.047300]<br>The message "W [MSGID: 114031] [client-rpc-fops_v2.c:850:client4_0_setxattr_cbk] 0-dev_SNIP_data-client-2: remote operation failed [Permission denied]" repeated 2 times between [2019-12-19 21:27:55.042002] and [2019-12-19 21:27:55.047312]<br>The message "W [MSGID: 114031] [client-rpc-fops_v2.c:850:client4_0_setxattr_cbk] 0-dev_SNIP_data-client-1: remote operation failed [Permission denied]" repeated 2 times between [2019-12-19 21:27:55.042013] and [2019-12-19 21:27:55.047524]<br>The message "E [MSGID: 148002] [utime.c:146:gf_utime_set_mdata_setxattr_cbk] 0-dev_SNIP_data-utime: dict set of key for set-ctime-mdata failed [Permission denied]" repeated 2 times between [2019-12-19 21:27:55.042634] and [2019-12-19 21:27:55.047887]<br></blockquote><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">[2019-12-19 21:37:55.541329] W [MSGID: 114031] [client-rpc-fops_v2.c:850:client4_0_setxattr_cbk] 0-dev_SNIP_data-client-2: remote operation failed [Permission denied]<br>[2019-12-19 21:37:55.541644] W [MSGID: 114031] [client-rpc-fops_v2.c:850:client4_0_setxattr_cbk] 0-dev_SNIP_data-client-1: remote operation failed [Permission denied]<br>[2019-12-19 21:37:55.541681] W [MSGID: 114031] [client-rpc-fops_v2.c:850:client4_0_setxattr_cbk] 0-dev_SNIP_data-client-0: remote operation failed [Permission denied]<br>[2019-12-19 21:37:55.542067] E [MSGID: 148002] [utime.c:146:gf_utime_set_mdata_setxattr_cbk] 0-dev_SNIP_data-utime: dict set of key for set-ctime-mdata failed [Permission denied]<br>The message "W [MSGID: 114031] [client-rpc-fops_v2.c:850:client4_0_setxattr_cbk] 0-dev_SNIP_data-client-2: remote operation failed [Permission denied]" repeated 3 times between [2019-12-19 21:37:55.541329] and [2019-12-19 21:37:55.546695]<br>The message "W [MSGID: 114031] [client-rpc-fops_v2.c:850:client4_0_setxattr_cbk] 0-dev_SNIP_data-client-1: remote operation failed [Permission denied]" repeated 3 times between [2019-12-19 21:37:55.541644] and [2019-12-19 21:37:55.546711]<br>The message "W [MSGID: 114031] [client-rpc-fops_v2.c:850:client4_0_setxattr_cbk] 0-dev_SNIP_data-client-0: remote operation failed [Permission denied]" repeated 3 times between [2019-12-19 21:37:55.541681] and [2019-12-19 21:37:55.546761]<br>The message "E [MSGID: 148002] [utime.c:146:gf_utime_set_mdata_setxattr_cbk] 0-dev_SNIP_data-utime: dict set of key for set-ctime-mdata failed [Permission denied]" repeated 3 times between [2019-12-19 21:37:55.542067] and [2019-12-19 21:37:55.547042]<br></blockquote><div><br></div><div>etc.</div><div><br></div><div>The questions are:</div><div><ol><li>Is it a cause for concern? They weren't there before the upgrade.</li></ol></div></div></blockquote><div>Looks like all these logs are related to ctime feature which is by default enabled in 7.1 (and not in 5.x series)</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><ol><li>How can I determine what's causing the errors?</li></ol></div></div></blockquote><div><br></div><div>Also check the server logs.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><ol><li>How can I fix them and prevent them from spamming the logs?</li></ol></div></div></blockquote><div><br></div><div>Check op-version of the cluster after upgrade. Disable 'ctime' option (volume set <vol> ctime off).</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div>Thank you and happy holidays!</div></div></div></blockquote><div><br></div><div>+1</div><div><br></div><div>Regards,</div><div>Amar</div><div><a href="https://kadalu.io" target="_blank">https://kadalu.io</a></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><br>Sincerely,<br>Artem<br><br>--<br>Founder, <a href="http://www.androidpolice.com" target="_blank">Android Police</a>, <a href="http://www.apkmirror.com/" style="font-size:12.8px" target="_blank">APK Mirror</a><span style="font-size:12.8px">, Illogical Robot LLC</span></div><div dir="ltr"><a href="http://beerpla.net/" target="_blank">beerpla.net</a> | <a href="http://twitter.com/ArtemR" target="_blank">@ArtemR</a><br></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div>
________<br>
<br>
Community Meeting Calendar:<br>
<br>
APAC Schedule -<br>
Every 2nd and 4th Tuesday at 11:30 AM IST<br>
Bridge: <a href="https://bluejeans.com/441850968" rel="noreferrer" target="_blank">https://bluejeans.com/441850968</a><br>
<br>
NA/EMEA Schedule -<br>
Every 1st and 3rd Tuesday at 01:00 PM EDT<br>
Bridge: <a href="https://bluejeans.com/441850968" rel="noreferrer" target="_blank">https://bluejeans.com/441850968</a><br>
<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="https://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-users</a><br>
</blockquote></div></div>
</blockquote></div>
</blockquote></div>
</blockquote></div>
</blockquote></div></div>
</blockquote></div>
________<br>
<br>
Community Meeting Calendar:<br>
<br>
APAC Schedule -<br>
Every 2nd and 4th Tuesday at 11:30 AM IST<br>
Bridge: <a href="https://bluejeans.com/441850968" rel="noreferrer" target="_blank">https://bluejeans.com/441850968</a><br>
<br>
NA/EMEA Schedule -<br>
Every 1st and 3rd Tuesday at 01:00 PM EDT<br>
Bridge: <a href="https://bluejeans.com/441850968" rel="noreferrer" target="_blank">https://bluejeans.com/441850968</a><br>
<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="https://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-users</a><br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr">--<div><a href="https://kadalu.io" target="_blank">https://kadalu.io</a></div><div>Container Storage made easy!</div><div><br></div></div></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr">--<div><a href="https://kadalu.io" target="_blank">https://kadalu.io</a></div><div>Container Storage made easy!</div><div><br></div></div></div></div>
</blockquote></div>
________<br>
<br>
<br>
<br>
Community Meeting Calendar:<br>
<br>
Schedule -<br>
Every Tuesday at 14:30 IST / 09:00 UTC<br>
Bridge: <a href="https://bluejeans.com/441850968" rel="noreferrer" target="_blank">https://bluejeans.com/441850968</a><br>
<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="https://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-users</a><br>
</blockquote></div><br clear="all"><br>-- <br><div dir="ltr">Regards,<br>Hari Gowtham.</div>
</blockquote></div>
</blockquote></div><br clear="all"><br>-- <br><div dir="ltr">Regards,<br>Hari Gowtham.</div>
</blockquote></div>
</blockquote></div><br clear="all"><br>-- <br><div dir="ltr">Regards,<br>Hari Gowtham.</div>
</blockquote></div>
</blockquote></div><br clear="all"><br>-- <br><div dir="ltr" class="gmail_signature">Regards,<br>Hari Gowtham.</div>