[Gluster-devel] gfid and volume-id extended attributes lost
Pranith Kumar Karampuri
pkarampu at redhat.com
Fri Jul 7 16:14:47 UTC 2017
On Fri, Jul 7, 2017 at 9:25 PM, Ankireddypalle Reddy <areddy at commvault.com>
wrote:
> 3.7.19
>
These are the only callers for removexattr and only _posix_remove_xattr has
the potential to do removexattr as posix_removexattr already makes sure
that it is not gfid/volume-id. And surprise surprise _posix_remove_xattr
happens only from healing code of afr/ec. And this can only happen if the
source brick doesn't have gfid, which doesn't seem to match with the
situation you explained.
# line filename / context / line
1 1234 xlators/mgmt/glusterd/src/glusterd-quota.c
<<glusterd_remove_quota_limit>>
ret = sys_lremovexattr (abspath, QUOTA_LIMIT_KEY);
2 1243 xlators/mgmt/glusterd/src/glusterd-quota.c
<<glusterd_remove_quota_limit>>
ret = sys_lremovexattr (abspath, QUOTA_LIMIT_OBJECTS_KEY);
3 6102 xlators/mgmt/glusterd/src/glusterd-utils.c
<<glusterd_check_and_set_brick_xattr>>
sys_lremovexattr (path, "trusted.glusterfs.test");
4 80 xlators/storage/posix/src/posix-handle.h <<REMOVE_PGFID_XATTR>>
op_ret = sys_lremovexattr (path, key); \
5 5026 xlators/storage/posix/src/posix.c <<_posix_remove_xattr>>
op_ret = sys_lremovexattr (filler->real_path, key);
6 5101 xlators/storage/posix/src/posix.c <<posix_removexattr>>
op_ret = sys_lremovexattr (real_path, name);
7 6811 xlators/storage/posix/src/posix.c <<init>>
sys_lremovexattr (dir_data->data, "trusted.glusterfs.test");
So there are only two possibilities:
1) Source directory in ec/afr doesn't have gfid
2) Something else removed these xattrs.
What is your volume info? May be that will give more clues.
PS: sys_fremovexattr is called only from posix_fremovexattr(), so that
doesn't seem to be the culprit as it also have checks to guard against
gfid/volume-id removal.
>
> Thanks and Regards,
>
> Ram
>
> *From:* Pranith Kumar Karampuri [mailto:pkarampu at redhat.com]
> *Sent:* Friday, July 07, 2017 11:54 AM
>
> *To:* Ankireddypalle Reddy
> *Cc:* Gluster Devel (gluster-devel at gluster.org); gluster-users at gluster.org
> *Subject:* Re: [Gluster-devel] gfid and volume-id extended attributes lost
>
>
>
>
>
>
>
> On Fri, Jul 7, 2017 at 9:20 PM, Ankireddypalle Reddy <areddy at commvault.com>
> wrote:
>
> Pranith,
>
> Thanks for looking in to the issue. The bricks were
> mounted after the reboot. One more thing that I noticed was when the
> attributes were manually set when glusterd was up then on starting the
> volume the attributes were again lost. Had to stop glusterd set attributes
> and then start glusterd. After that the volume start succeeded.
>
>
>
> Which version is this?
>
>
>
>
>
> Thanks and Regards,
>
> Ram
>
>
>
> *From:* Pranith Kumar Karampuri [mailto:pkarampu at redhat.com]
> *Sent:* Friday, July 07, 2017 11:46 AM
> *To:* Ankireddypalle Reddy
> *Cc:* Gluster Devel (gluster-devel at gluster.org); gluster-users at gluster.org
> *Subject:* Re: [Gluster-devel] gfid and volume-id extended attributes lost
>
>
>
> Did anything special happen on these two bricks? It can't happen in the
> I/O path:
> posix_removexattr() has:
> 0 if (!strcmp (GFID_XATTR_KEY, name))
> {
>
>
> 1 gf_msg (this->name, GF_LOG_WARNING, 0,
> P_MSG_XATTR_NOT_REMOVED,
> 2 "Remove xattr called on gfid for file %s",
> real_path);
> 3 op_ret = -1;
>
> 4 goto out;
>
> 5 }
>
> 6 if (!strcmp (GF_XATTR_VOL_ID_KEY, name))
> {
> 7 gf_msg (this->name, GF_LOG_WARNING, 0,
> P_MSG_XATTR_NOT_REMOVED,
> 8 "Remove xattr called on volume-id for file
> %s",
> 9 real_path);
>
> 10 op_ret = -1;
>
> 11 goto out;
>
> 12 }
>
> I just found that op_errno is not set correctly, but it can't happen in
> the I/O path, so self-heal/rebalance are off the hook.
>
> I also grepped for any removexattr of trusted.gfid from glusterd and
> didn't find any.
>
> So one thing that used to happen was that sometimes when machines reboot,
> the brick mounts wouldn't happen and this would lead to absence of both
> trusted.gfid and volume-id. So at the moment this is my wild guess.
>
>
>
>
>
> On Fri, Jul 7, 2017 at 8:39 PM, Ankireddypalle Reddy <areddy at commvault.com>
> wrote:
>
> Hi,
>
> We faced an issue in the production today. We had to stop the
> volume and reboot all the servers in the cluster. Once the servers
> rebooted starting of the volume failed because the following extended
> attributes were not present on all the bricks on 2 servers.
>
> 1) trusted.gfid
>
> 2) trusted.glusterfs.volume-id
>
>
>
> We had to manually set these extended attributes to start the volume. Are
> there any such known issues.
>
>
>
> Thanks and Regards,
>
> Ram
>
> ***************************Legal Disclaimer***************************
>
> "This communication may contain confidential and privileged material for
> the
>
> sole use of the intended recipient. Any unauthorized review, use or
> distribution
>
> by others is strictly prohibited. If you have received the message by
> mistake,
>
> please advise the sender by reply email and delete the message. Thank you."
>
> **********************************************************************
>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
>
>
>
>
> --
>
> Pranith
>
> ***************************Legal Disclaimer***************************
>
> "This communication may contain confidential and privileged material for
> the
>
> sole use of the intended recipient. Any unauthorized review, use or
> distribution
>
> by others is strictly prohibited. If you have received the message by
> mistake,
>
> please advise the sender by reply email and delete the message. Thank you."
>
> **********************************************************************
>
>
>
>
> --
>
> Pranith
> ***************************Legal Disclaimer***************************
> "This communication may contain confidential and privileged material for
> the
> sole use of the intended recipient. Any unauthorized review, use or
> distribution
> by others is strictly prohibited. If you have received the message by
> mistake,
> please advise the sender by reply email and delete the message. Thank you."
> **********************************************************************
>
--
Pranith
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-devel/attachments/20170707/b205dae7/attachment.html>
More information about the Gluster-devel
mailing list