[Gluster-devel] GETXATTR op pending on index xlator for more than 10 hours

[Xie Changlong]

Hi all,




Today, i found gnfs GETXATTR  bailing out on gluster release 3.12.0. I have a simple 4*2 Distributed-Rep volume. 




[2019-06-03 19:58:33.085880] E [rpc-clnt.c:185:Call_bail] 0-cl25vol01-client-4: bailing out frame type(GlusterFS 3.3) op(GETXATTR(18)) xid=0x21de4275 sent = 2019-06-03 19:28:30.552356. timeout = 1800 for 10.3.133.57:49153




xid= 0x21de4275 = 568214133




Then i try to dump brick 10.3.133.57:49153, and find the GETXATTR op pending on index xlator for more than 10 hours!




1111MicrosoftInternetExplorer402DocumentNotSpecified7.8 磅Normal0                                                                                                                                                                                                                                                                  

[root at node0001 gluster]# grep -rn 568214133 gluster-brick-1-cl25vol01.6078.dump.15596*

gluster-brick-1-cl25vol01.6078.dump.1559617125:5093:unique=568214133

gluster-brick-1-cl25vol01.6078.dump.1559618121:5230:unique=568214133

gluster-brick-1-cl25vol01.6078.dump.1559618912:5434:unique=568214133

gluster-brick-1-cl25vol01.6078.dump.1559628467:6921:unique=568214133




[root at node0001 gluster]# date -d @1559617125

Tue Jun  4 10:58:45 CST 2019

[root at node0001 gluster]# date -d @1559628467

Tue Jun  4 14:07:47 CST 2019

1111MicrosoftInternetExplorer402DocumentNotSpecified7.8 磅Normal0                                                                                                                                                                                                                                                                  

[root at node0001 gluster]# 








[global.callpool.stack.115]

stack=0x7f8b342623c0

uid=500

gid=500

pid=-6

unique=568214133

lk-owner=faffffff

op=stack

type=0

cnt=4




[global.callpool.stack.115.frame.1]

frame=0x7f8b1d6fb540

ref_count=0

translator=cl25vol01-index

complete=0

parent=cl25vol01-quota

wind_from=quota_getxattr

wind_to=(this->children->xlator)->fops->getxattr

unwind_to=default_getxattr_cbk




[global.callpool.stack.115.frame.2]

frame=0x7f8b30a14da0

ref_count=1

translator=cl25vol01-quota

complete=0

parent=cl25vol01-io-stats

wind_from=io_stats_getxattr

wind_to=(this->children->xlator)->fops->getxattr

unwind_to=io_stats_getxattr_cbk




[global.callpool.stack.115.frame.3]

frame=0x7f8b6debada0

ref_count=1

translator=cl25vol01-io-stats

complete=0

parent=cl25vol01-server

wind_from=server_getxattr_resume

wind_to=FIRST_CHILD(this)->fops->getxattr

unwind_to=server_getxattr_cbk




[global.callpool.stack.115.frame.4]

frame=0x7f8b21962a60

ref_count=1

translator=cl25vol01-server

complete=0




I've checked the code logic and got nothing, any advice? I still have the scene on my side, so we can dig more.







Thanks









-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-devel/attachments/20190604/6af3b277/attachment-0001.html>