[Gluster-users] remote operation failed [Permission denied] every 10 minutes after upgrading from 5.10 to 7.0

Hari Gowtham hgowtham at redhat.com
Mon Apr 27 09:56:29 UTC 2020 

Hi Artem,

The above issue is depended on
https://review.gluster.org/#/c/glusterfs/+/24289/ and
https://review.gluster.org/#/c/glusterfs/+/24264/. Both of the patches are
available in release-7.5


On Mon, Apr 27, 2020 at 3:56 AM Artem Russakovskii <archon810 at gmail.com>
wrote:

> Looks like it's in 7.5, right?
>
> https://docs.gluster.org/en/latest/release-notes/7.5/
> https://github.com/gluster/glusterfs/issues/832
>
> Sincerely,
> Artem
>
> --
> Founder, Android Police <http://www.androidpolice.com>, APK Mirror
> <http://www.apkmirror.com/>, Illogical Robot LLC
> beerpla.net | @ArtemR <http://twitter.com/ArtemR>
>
>
> On Mon, Mar 23, 2020 at 10:26 PM Hari Gowtham <hgowtham at redhat.com> wrote:
>
>> The patch is not there in 7.4. It is yet to be backported. It should soon
>> be backported to the release branches.
>>
>> On Tue, Mar 24, 2020 at 3:24 AM Artem Russakovskii <archon810 at gmail.com>
>> wrote:
>>
>>> Could you please confirm whether it made it into 7.4 or not?
>>> https://docs.gluster.org/en/latest/release-notes/7.4/ - I don't think
>>> I'm seeing it, but maybe it's one of these?
>>>
>>> Thanks.
>>>
>>> Sincerely,
>>> Artem
>>>
>>> --
>>> Founder, Android Police <http://www.androidpolice.com>, APK Mirror
>>> <http://www.apkmirror.com/>, Illogical Robot LLC
>>> beerpla.net | @ArtemR <http://twitter.com/ArtemR>
>>>
>>>
>>> On Tue, Mar 10, 2020 at 10:33 PM Hari Gowtham <hgowtham at redhat.com>
>>> wrote:
>>>
>>>> As the patch is yet to be merged in master, release-7 I can't give a
>>>> definite release version.
>>>> I expect it to be merged in 7.4 if they get merged in master and
>>>> release-7 soon. Else it will be made available in 7.5 for sure.
>>>> I'm CCing Rinku who takes care of the release-7 cycles.
>>>>
>>>> Rinku, before the release, once the availability is known for sure can
>>>> you please respond here which version the patch makes it to?
>>>>
>>>> On Wed, Mar 11, 2020 at 2:33 AM Artem Russakovskii <archon810 at gmail.com>
>>>> wrote:
>>>>
>>>>> I'd actually be looking for this fix in v7 since it's the one
>>>>> preventing us from upgrading to it. Curious which exact version of 7 I
>>>>> should be looking out for with this fix.
>>>>>
>>>>> Sincerely,
>>>>> Artem
>>>>>
>>>>> --
>>>>> Founder, Android Police <http://www.androidpolice.com>, APK Mirror
>>>>> <http://www.apkmirror.com/>, Illogical Robot LLC
>>>>> beerpla.net | @ArtemR <http://twitter.com/ArtemR>
>>>>>
>>>>>
>>>>> On Mon, Mar 9, 2020 at 1:41 AM Hari Gowtham <hgowtham at redhat.com>
>>>>> wrote:
>>>>>
>>>>>> The release for 7 will happen every month while the release 5 and 6
>>>>>> are done once in every 2 months.
>>>>>> We are done for February, so the next cycle for 5 and 6 are in April.
>>>>>>
>>>>>> On Sat, Mar 7, 2020 at 1:00 PM Artem Russakovskii <
>>>>>> archon810 at gmail.com> wrote:
>>>>>>
>>>>>>> Great, I'll be glad to test and confirm once the release with this
>>>>>>> change is out. Would you please confirm when it is by replying here?
>>>>>>>
>>>>>>> Thanks, Amar.
>>>>>>>
>>>>>>> Sincerely,
>>>>>>> Artem
>>>>>>>
>>>>>>> --
>>>>>>> Founder, Android Police <http://www.androidpolice.com>, APK Mirror
>>>>>>> <http://www.apkmirror.com/>, Illogical Robot LLC
>>>>>>> beerpla.net | @ArtemR <http://twitter.com/ArtemR>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Mar 3, 2020 at 8:25 PM Amar Tumballi <amar at kadalu.io> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sat, Feb 15, 2020 at 7:27 AM Amar Tumballi <amar at kadalu.io>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Sat, Feb 15, 2020 at 3:02 AM Artem Russakovskii <
>>>>>>>>> archon810 at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Amar,
>>>>>>>>>>
>>>>>>>>>> From what I can tell, there's no error actually using the
>>>>>>>>>> referenced files, but the log messages all constantly pop up in the log
>>>>>>>>>> when accessed through the web server. The files display OK in the browser.
>>>>>>>>>>
>>>>>>>>>> I also tried renaming/touching them via shell as the user that
>>>>>>>>>> owns them, and that worked fine (with no notices generated).
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks for this update.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I hope you guys figure it out because I really don't feel
>>>>>>>>>> comfortable upgrading to 6 or 7 from 5 which has been stable.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> Ack! Makes sense, upgrading to a version when you know there is a
>>>>>>>>> regression is not a good thing. We will check this out.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> https://review.gluster.org/#/c/glusterfs/+/24200/ would hopefully
>>>>>>>> fix the issue.
>>>>>>>>
>>>>>>>>
>>>>>>>>> -Amar
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Sincerely,
>>>>>>>>>> Artem
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Founder, Android Police <http://www.androidpolice.com>, APK
>>>>>>>>>> Mirror <http://www.apkmirror.com/>, Illogical Robot LLC
>>>>>>>>>> beerpla.net | @ArtemR <http://twitter.com/ArtemR>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Feb 12, 2020 at 11:02 PM Amar Tumballi <amarts at gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Feb 7, 2020 at 1:56 PM Artem Russakovskii <
>>>>>>>>>>> archon810 at gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Still no response? Amar?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> Sorry, this got down in my Inbox, and missed responding, and
>>>>>>>>>>> also I got busy with few other engagements. Reply inline.
>>>>>>>>>>>
>>>>>>>>>>> Sincerely,
>>>>>>>>>>>> Artem
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Jan 14, 2020 at 12:57 PM Artem Russakovskii <
>>>>>>>>>>>> archon810 at gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Any updates here please?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Sincerely,
>>>>>>>>>>>>> Artem
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Founder, Android Police <http://www.androidpolice.com>, APK
>>>>>>>>>>>>> Mirror <http://www.apkmirror.com/>, Illogical Robot LLC
>>>>>>>>>>>>> beerpla.net | @ArtemR <http://twitter.com/ArtemR>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Jan 6, 2020 at 4:18 PM Artem Russakovskii <
>>>>>>>>>>>>> archon810 at gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks Amar,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Checking the brick logs, I'm seeing these and numerous other
>>>>>>>>>>>>>> files with Permission denied. The question is why is the permission being
>>>>>>>>>>>>>> denied? Gluster runs as root and the files are owned by wwwrun:www, which
>>>>>>>>>>>>>> it should have full access to.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Is there any more documentation on what enabling or disabling
>>>>>>>>>>>>>> this feature accomplishes, why it was added to 7.1 by default (and 7.0 as
>>>>>>>>>>>>>> far as I can tell), and what side effects it may have? Basically, I'd like
>>>>>>>>>>>>>> to understand the problem better instead of simply turning the flag off and
>>>>>>>>>>>>>> fix the underlying issue (or help fix a bug if it's a bug in gluster).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ==> bricks/mnt-pylon_block6-dev_SNIP_data.log <==
>>>>>>>>>>>>>>> [2020-01-06 21:39:00.176476] I [MSGID: 115060]
>>>>>>>>>>>>>>> [server-rpc-fops_v2.c:897:server4_setxattr_cbk] 0-dev_SNIP_data-server:
>>>>>>>>>>>>>>> client:
>>>>>>>>>>>>>>> CTX_ID:3e0436a5-cdf8-4d3f-a4e9-fc681fe902d3-GRAPH_ID:2-PID:32069-HOST:pylon-PC_NAME:dev_SNIP_data-client-2-RECON_NO:-0,
>>>>>>>>>>>>>>> error-xlator: dev_SNIP_data-access-control [Permission denied]
>>>>>>>>>>>>>>> [2020-01-06 21:39:00.325329] I [MSGID: 139001]
>>>>>>>>>>>>>>> [posix-acl.c:262:posix_acl_log_permit_denied]
>>>>>>>>>>>>>>> 0-dev_SNIP_data-access-control: client:
>>>>>>>>>>>>>>> CTX_ID:3e0436a5-cdf8-4d3f-a4e9-fc681fe902d3-GRAPH_ID:2-PID:32069-HOST:pylon-PC_NAME:dev_SNIP_data-client-2-RECON_NO:-0,
>>>>>>>>>>>>>>> gfid: e9bd83fd-b16e-44ef-8fde-2ae320ee54d0,
>>>>>>>>>>>>>>> req(uid:491,gid:488,perm:2,ngrps:1),
>>>>>>>>>>>>>>> ctx(uid:30,gid:8,in-groups:0,perm:664,updated-fop:LOOKUP, acl:-)
>>>>>>>>>>>>>>> [Permission denied]
>>>>>>>>>>>>>>> [2020-01-06 21:39:00.325435] I [MSGID: 115060]
>>>>>>>>>>>>>>> [server-rpc-fops.c:937:_gf_server_log_setxattr_failure]
>>>>>>>>>>>>>>> 0-dev_SNIP_data-server: 709342: SETXATTR
>>>>>>>>>>>>>>> /uploads/2019/06/Anker-PowerPort-Atom-10-768x512.jpg
>>>>>>>>>>>>>>> (e9bd83fd-b16e-44ef-8fde-2ae320ee54d0) ==> set-ctime-mdata, client:
>>>>>>>>>>>>>>> CTX_ID:3e0436a5-cdf8-4d3f-a4e9-fc681fe902d3-GRAPH_ID:2-PID:32069-HOST:pylon-PC_NAME:dev_SNIP_data-client-2-RECON_NO:-0,
>>>>>>>>>>>>>>> error-xlator: dev_SNIP_data-access-control
>>>>>>>>>>>>>>> [2020-01-06 21:39:00.325460] I [MSGID: 115060]
>>>>>>>>>>>>>>> [server-rpc-fops_v2.c:897:server4_setxattr_cbk] 0-dev_SNIP_data-server:
>>>>>>>>>>>>>>> client:
>>>>>>>>>>>>>>> CTX_ID:3e0436a5-cdf8-4d3f-a4e9-fc681fe902d3-GRAPH_ID:2-PID:32069-HOST:pylon-PC_NAME:dev_SNIP_data-client-2-RECON_NO:-0,
>>>>>>>>>>>>>>> error-xlator: dev_SNIP_data-access-control [Permission denied]
>>>>>>>>>>>>>>> [2020-01-06 21:39:01.195012] I [MSGID: 139001]
>>>>>>>>>>>>>>> [posix-acl.c:262:posix_acl_log_permit_denied]
>>>>>>>>>>>>>>> 0-dev_SNIP_data-access-control: client:
>>>>>>>>>>>>>>> CTX_ID:3e0436a5-cdf8-4d3f-a4e9-fc681fe902d3-GRAPH_ID:2-PID:32069-HOST:pylon-PC_NAME:dev_SNIP_data-client-2-RECON_NO:-0,
>>>>>>>>>>>>>>> gfid: ff96827e-777b-45fb-a40d-e5089cd024fe,
>>>>>>>>>>>>>>> req(uid:491,gid:488,perm:2,ngrps:1),
>>>>>>>>>>>>>>> ctx(uid:30,gid:8,in-groups:0,perm:664,updated-fop:LOOKUP, acl:-)
>>>>>>>>>>>>>>> [Permission denied]
>>>>>>>>>>>>>>> [2020-01-06 21:39:01.195091] I [MSGID: 115060]
>>>>>>>>>>>>>>> [server-rpc-fops.c:937:_gf_server_log_setxattr_failure]
>>>>>>>>>>>>>>> 0-dev_SNIP_data-server: 709349: SETXATTR
>>>>>>>>>>>>>>> /uploads/2019/06/Anker-PowerPort-Atom-768x512.jpg
>>>>>>>>>>>>>>> (ff96827e-777b-45fb-a40d-e5089cd024fe) ==> set-ctime-mdata, client:
>>>>>>>>>>>>>>> CTX_ID:3e0436a5-cdf8-4d3f-a4e9-fc681fe902d3-GRAPH_ID:2-PID:32069-HOST:pylon-PC_NAME:dev_SNIP_data-client-2-RECON_NO:-0,
>>>>>>>>>>>>>>> error-xlator: dev_SNIP_data-access-control
>>>>>>>>>>>>>>> [2020-01-06 21:39:01.195291] I [MSGID: 115060]
>>>>>>>>>>>>>>> [server-rpc-fops_v2.c:897:server4_setxattr_cbk] 0-dev_SNIP_data-server:
>>>>>>>>>>>>>>> client:
>>>>>>>>>>>>>>> CTX_ID:3e0436a5-cdf8-4d3f-a4e9-fc681fe902d3-GRAPH_ID:2-PID:32069-HOST:pylon-PC_NAME:dev_SNIP_data-client-2-RECON_NO:-0,
>>>>>>>>>>>>>>> error-xlator: dev_SNIP_data-access-control [Permission denied]
>>>>>>>>>>>>>>> [2020-01-06 21:39:04.184502] I [MSGID: 139001]
>>>>>>>>>>>>>>> [posix-acl.c:262:posix_acl_log_permit_denied]
>>>>>>>>>>>>>>> 0-dev_SNIP_data-access-control: client:
>>>>>>>>>>>>>>> CTX_ID:3e0436a5-cdf8-4d3f-a4e9-fc681fe902d3-GRAPH_ID:2-PID:32069-HOST:pylon-PC_NAME:dev_SNIP_data-client-2-RECON_NO:-0,
>>>>>>>>>>>>>>> gfid: 546305c8-7611-4752-9168-215e9eb03938,
>>>>>>>>>>>>>>> req(uid:491,gid:488,perm:2,ngrps:1),
>>>>>>>>>>>>>>> ctx(uid:30,gid:8,in-groups:0,perm:664,updated-fop:LOOKUP, acl:-)
>>>>>>>>>>>>>>> [Permission denied]
>>>>>>>>>>>>>>> [2020-01-06 21:39:04.184554] I [MSGID: 115060]
>>>>>>>>>>>>>>> [server-rpc-fops.c:937:_gf_server_log_setxattr_failure]
>>>>>>>>>>>>>>> 0-dev_SNIP_data-server: 709360: SETXATTR
>>>>>>>>>>>>>>> /uploads/2019/06/Anker-PowerPort-Atom-10-1-768x512.jpg
>>>>>>>>>>>>>>> (546305c8-7611-4752-9168-215e9eb03938) ==> set-ctime-mdata, client:
>>>>>>>>>>>>>>> CTX_ID:3e0436a5-cdf8-4d3f-a4e9-fc681fe902d3-GRAPH_ID:2-PID:32069-HOST:pylon-PC_NAME:dev_SNIP_data-client-2-RECON_NO:-0,
>>>>>>>>>>>>>>> error-xlator: dev_SNIP_data-access-control
>>>>>>>>>>>>>>> [2020-01-06 21:39:04.184576] I [MSGID: 115060]
>>>>>>>>>>>>>>> [server-rpc-fops_v2.c:897:server4_setxattr_cbk] 0-dev_SNIP_data-server:
>>>>>>>>>>>>>>> client:
>>>>>>>>>>>>>>> CTX_ID:3e0436a5-cdf8-4d3f-a4e9-fc681fe902d3-GRAPH_ID:2-PID:32069-HOST:pylon-PC_NAME:dev_SNIP_data-client-2-RECON_NO:-0,
>>>>>>>>>>>>>>> error-xlator: dev_SNIP_data-access-control [Permission denied]
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Btw, cluster.op-version is 70000, which is the latest. I
>>>>>>>>>>>>>> always rev it up after upgrading the cluster.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>> I don't have any reports of this error coming for other users,
>>>>>>>>>>> and not able to reproduce locally too.  I am starting to think that this
>>>>>>>>>>> error and the issue Strahil faced after upgrading (saying there was an ACL
>>>>>>>>>>> issue) may be related. Will check and update. Meantime, is there any
>>>>>>>>>>> application error you are seeing? That would also give us some hints to
>>>>>>>>>>> reproduce the issue.
>>>>>>>>>>>
>>>>>>>>>>> -Amar
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Sincerely,
>>>>>>>>>>>>>> Artem
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Founder, Android Police <http://www.androidpolice.com>, APK
>>>>>>>>>>>>>> Mirror <http://www.apkmirror.com/>, Illogical Robot LLC
>>>>>>>>>>>>>> beerpla.net | @ArtemR <http://twitter.com/ArtemR>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Sat, Dec 28, 2019 at 1:19 AM Amar Tumballi <
>>>>>>>>>>>>>> amarts at gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Answers inline.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Dec 25, 2019 at 2:06 AM Artem Russakovskii <
>>>>>>>>>>>>>>> archon810 at gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi everyone,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I'm preparing the upgrade for our 5.10 gluster cluster
>>>>>>>>>>>>>>>> running 1x4 replicate volumes to 7.X and decided to upgrade our test
>>>>>>>>>>>>>>>> cluster first.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> As soon as I upgraded to 7.0 (and now 7.1), I started
>>>>>>>>>>>>>>>> seeing the following messages every 10 minutes in the log for one of the
>>>>>>>>>>>>>>>> volumes:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [2019-12-19 21:27:55.041949] W [MSGID: 114031]
>>>>>>>>>>>>>>>>> [client-rpc-fops_v2.c:850:client4_0_setxattr_cbk] 0-dev_SNIP_data-client-0:
>>>>>>>>>>>>>>>>> remote operation failed [Permission denied]
>>>>>>>>>>>>>>>>> [2019-12-19 21:27:55.042002] W [MSGID: 114031]
>>>>>>>>>>>>>>>>> [client-rpc-fops_v2.c:850:client4_0_setxattr_cbk] 0-dev_SNIP_data-client-2:
>>>>>>>>>>>>>>>>> remote operation failed [Permission denied]
>>>>>>>>>>>>>>>>> [2019-12-19 21:27:55.042013] W [MSGID: 114031]
>>>>>>>>>>>>>>>>> [client-rpc-fops_v2.c:850:client4_0_setxattr_cbk] 0-dev_SNIP_data-client-1:
>>>>>>>>>>>>>>>>> remote operation failed [Permission denied]
>>>>>>>>>>>>>>>>> [2019-12-19 21:27:55.042634] E [MSGID: 148002]
>>>>>>>>>>>>>>>>> [utime.c:146:gf_utime_set_mdata_setxattr_cbk] 0-dev_SNIP_data-utime: dict
>>>>>>>>>>>>>>>>> set of key for set-ctime-mdata failed [Permission denied]
>>>>>>>>>>>>>>>>> The message "W [MSGID: 114031]
>>>>>>>>>>>>>>>>> [client-rpc-fops_v2.c:850:client4_0_setxattr_cbk] 0-dev_SNIP_data-client-0:
>>>>>>>>>>>>>>>>> remote operation failed [Permission denied]" repeated 2 times between
>>>>>>>>>>>>>>>>> [2019-12-19 21:27:55.041949] and [2019-12-19 21:27:55.047300]
>>>>>>>>>>>>>>>>> The message "W [MSGID: 114031]
>>>>>>>>>>>>>>>>> [client-rpc-fops_v2.c:850:client4_0_setxattr_cbk] 0-dev_SNIP_data-client-2:
>>>>>>>>>>>>>>>>> remote operation failed [Permission denied]" repeated 2 times between
>>>>>>>>>>>>>>>>> [2019-12-19 21:27:55.042002] and [2019-12-19 21:27:55.047312]
>>>>>>>>>>>>>>>>> The message "W [MSGID: 114031]
>>>>>>>>>>>>>>>>> [client-rpc-fops_v2.c:850:client4_0_setxattr_cbk] 0-dev_SNIP_data-client-1:
>>>>>>>>>>>>>>>>> remote operation failed [Permission denied]" repeated 2 times between
>>>>>>>>>>>>>>>>> [2019-12-19 21:27:55.042013] and [2019-12-19 21:27:55.047524]
>>>>>>>>>>>>>>>>> The message "E [MSGID: 148002]
>>>>>>>>>>>>>>>>> [utime.c:146:gf_utime_set_mdata_setxattr_cbk] 0-dev_SNIP_data-utime: dict
>>>>>>>>>>>>>>>>> set of key for set-ctime-mdata failed [Permission denied]" repeated 2 times
>>>>>>>>>>>>>>>>> between [2019-12-19 21:27:55.042634] and [2019-12-19 21:27:55.047887]
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> [2019-12-19 21:37:55.541329] W [MSGID: 114031]
>>>>>>>>>>>>>>>>> [client-rpc-fops_v2.c:850:client4_0_setxattr_cbk] 0-dev_SNIP_data-client-2:
>>>>>>>>>>>>>>>>> remote operation failed [Permission denied]
>>>>>>>>>>>>>>>>> [2019-12-19 21:37:55.541644] W [MSGID: 114031]
>>>>>>>>>>>>>>>>> [client-rpc-fops_v2.c:850:client4_0_setxattr_cbk] 0-dev_SNIP_data-client-1:
>>>>>>>>>>>>>>>>> remote operation failed [Permission denied]
>>>>>>>>>>>>>>>>> [2019-12-19 21:37:55.541681] W [MSGID: 114031]
>>>>>>>>>>>>>>>>> [client-rpc-fops_v2.c:850:client4_0_setxattr_cbk] 0-dev_SNIP_data-client-0:
>>>>>>>>>>>>>>>>> remote operation failed [Permission denied]
>>>>>>>>>>>>>>>>> [2019-12-19 21:37:55.542067] E [MSGID: 148002]
>>>>>>>>>>>>>>>>> [utime.c:146:gf_utime_set_mdata_setxattr_cbk] 0-dev_SNIP_data-utime: dict
>>>>>>>>>>>>>>>>> set of key for set-ctime-mdata failed [Permission denied]
>>>>>>>>>>>>>>>>> The message "W [MSGID: 114031]
>>>>>>>>>>>>>>>>> [client-rpc-fops_v2.c:850:client4_0_setxattr_cbk] 0-dev_SNIP_data-client-2:
>>>>>>>>>>>>>>>>> remote operation failed [Permission denied]" repeated 3 times between
>>>>>>>>>>>>>>>>> [2019-12-19 21:37:55.541329] and [2019-12-19 21:37:55.546695]
>>>>>>>>>>>>>>>>> The message "W [MSGID: 114031]
>>>>>>>>>>>>>>>>> [client-rpc-fops_v2.c:850:client4_0_setxattr_cbk] 0-dev_SNIP_data-client-1:
>>>>>>>>>>>>>>>>> remote operation failed [Permission denied]" repeated 3 times between
>>>>>>>>>>>>>>>>> [2019-12-19 21:37:55.541644] and [2019-12-19 21:37:55.546711]
>>>>>>>>>>>>>>>>> The message "W [MSGID: 114031]
>>>>>>>>>>>>>>>>> [client-rpc-fops_v2.c:850:client4_0_setxattr_cbk] 0-dev_SNIP_data-client-0:
>>>>>>>>>>>>>>>>> remote operation failed [Permission denied]" repeated 3 times between
>>>>>>>>>>>>>>>>> [2019-12-19 21:37:55.541681] and [2019-12-19 21:37:55.546761]
>>>>>>>>>>>>>>>>> The message "E [MSGID: 148002]
>>>>>>>>>>>>>>>>> [utime.c:146:gf_utime_set_mdata_setxattr_cbk] 0-dev_SNIP_data-utime: dict
>>>>>>>>>>>>>>>>> set of key for set-ctime-mdata failed [Permission denied]" repeated 3 times
>>>>>>>>>>>>>>>>> between [2019-12-19 21:37:55.542067] and [2019-12-19 21:37:55.547042]
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> etc.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The questions are:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>    1. Is it a cause for concern? They weren't there before
>>>>>>>>>>>>>>>>    the upgrade.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Looks like all these logs are related to ctime feature
>>>>>>>>>>>>>>> which is by default enabled in 7.1 (and not in 5.x series)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>    1. How can I determine what's causing the errors?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Also check the server logs.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>    1. How can I fix them and prevent them from
>>>>>>>>>>>>>>>>    spamming the logs?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Check op-version of the cluster after upgrade. Disable
>>>>>>>>>>>>>>> 'ctime' option (volume set <vol> ctime off).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thank you and happy holidays!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> +1
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>> Amar
>>>>>>>>>>>>>>> https://kadalu.io
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Sincerely,
>>>>>>>>>>>>>>>> Artem
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Founder, Android Police <http://www.androidpolice.com>, APK
>>>>>>>>>>>>>>>> Mirror <http://www.apkmirror.com/>, Illogical Robot LLC
>>>>>>>>>>>>>>>> beerpla.net | @ArtemR <http://twitter.com/ArtemR>
>>>>>>>>>>>>>>>> ________
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Community Meeting Calendar:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> APAC Schedule -
>>>>>>>>>>>>>>>> Every 2nd and 4th Tuesday at 11:30 AM IST
>>>>>>>>>>>>>>>> Bridge: https://bluejeans.com/441850968
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> NA/EMEA Schedule -
>>>>>>>>>>>>>>>> Every 1st and 3rd Tuesday at 01:00 PM EDT
>>>>>>>>>>>>>>>> Bridge: https://bluejeans.com/441850968
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Gluster-users mailing list
>>>>>>>>>>>>>>>> Gluster-users at gluster.org
>>>>>>>>>>>>>>>> https://lists.gluster.org/mailman/listinfo/gluster-users
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ________
>>>>>>>>>>
>>>>>>>>>> Community Meeting Calendar:
>>>>>>>>>>
>>>>>>>>>> APAC Schedule -
>>>>>>>>>> Every 2nd and 4th Tuesday at 11:30 AM IST
>>>>>>>>>> Bridge: https://bluejeans.com/441850968
>>>>>>>>>>
>>>>>>>>>> NA/EMEA Schedule -
>>>>>>>>>> Every 1st and 3rd Tuesday at 01:00 PM EDT
>>>>>>>>>> Bridge: https://bluejeans.com/441850968
>>>>>>>>>>
>>>>>>>>>> Gluster-users mailing list
>>>>>>>>>> Gluster-users at gluster.org
>>>>>>>>>> https://lists.gluster.org/mailman/listinfo/gluster-users
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> --
>>>>>>>>> https://kadalu.io
>>>>>>>>> Container Storage made easy!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> --
>>>>>>>> https://kadalu.io
>>>>>>>> Container Storage made easy!
>>>>>>>>
>>>>>>>> ________
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Community Meeting Calendar:
>>>>>>>
>>>>>>> Schedule -
>>>>>>> Every Tuesday at 14:30 IST / 09:00 UTC
>>>>>>> Bridge: https://bluejeans.com/441850968
>>>>>>>
>>>>>>> Gluster-users mailing list
>>>>>>> Gluster-users at gluster.org
>>>>>>> https://lists.gluster.org/mailman/listinfo/gluster-users
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Regards,
>>>>>> Hari Gowtham.
>>>>>>
>>>>>
>>>>
>>>> --
>>>> Regards,
>>>> Hari Gowtham.
>>>>
>>>
>>
>> --
>> Regards,
>> Hari Gowtham.
>>
>

-- 
Regards,
Hari Gowtham.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20200427/4a002ef7/attachment.html>

More information about the Gluster-users mailing list