[Gluster-users] Assertion failed: ec_get_inode_size

[Ashish Pandey]

Hi Serkan, 

I have gone through the logs and can see there are some blocked inode lock requests. 
We have observed that some other user have also faced this issue with similar logs. 
I think you have tried some rolling update on your setup or some NODES , on which you have collected these statedumps, must have gone down for one or other reason. 

We will further dig it up and will try to find out the root cause. Till than you can resolve this issue by restarting the volume which will restart nfs and shd and will release any locks taken by these process. 

"gluster volume start <volname> force" will do the same. 

Regards, 
Ashish 


----- Original Message -----

From: "Serkan Çoban" <cobanserkan at gmail.com> 
To: "Ashish Pandey" <aspandey at redhat.com> 
Cc: "Gluster Users" <gluster-users at gluster.org>, "Gluster Devel" <gluster-devel at gluster.org> 
Sent: Monday, April 18, 2016 11:51:37 AM 
Subject: Re: [Gluster-users] Assertion failed: ec_get_inode_size 

You can find the statedumps of server and client in below link. 
Gluster version is 3.7.10, 78x(16+4) disperse setup. 60 nodes named 
node185..node244 
https://www.dropbox.com/s/cc2dgsxwuk48mba/gluster_statedumps.zip?dl=0 


On Fri, Apr 15, 2016 at 9:52 PM, Ashish Pandey <aspandey at redhat.com> wrote: 
> 
> Actually it was my mistake I overlooked the configuration you provided..It 
> will be huge. 
> I would suggest to take statedump on all the nodes and try to grep for 
> "BLOCKED" in statedump files on all the nodes. 
> See if you can see any such line in any file and send those files. No need 
> to send statedump of all the bricks.. 
> 
> 
> 
> 
> ________________________________ 
> From: "Serkan Çoban" <cobanserkan at gmail.com> 
> To: "Ashish Pandey" <aspandey at redhat.com> 
> Cc: "Gluster Users" <gluster-users at gluster.org>, "Gluster Devel" 
> <gluster-devel at gluster.org> 
> Sent: Friday, April 15, 2016 6:07:00 PM 
> 
> Subject: Re: [Gluster-users] Assertion failed: ec_get_inode_size 
> 
> Hi Asish, 
> 
> Sorry for the question but do you want all brick statedumps from all 
> servers or all brick dumps from one server? 
> All server brick dumps is nearly 700MB zipped.. 
> 
> On Fri, Apr 15, 2016 at 2:16 PM, Ashish Pandey <aspandey at redhat.com> wrote: 
>> 
>> To get the state dump of fuse client- 
>> 1 - get the PID of fuse mount process 
>> 2 - kill -USR1 <PID> 
>> 
>> statedump can be found in the same directory where u get for brick 
>> process. 
>> 
>> Following link could be helpful for future reference - 
>> 
>> https://github.com/gluster/glusterfs/blob/master/doc/debugging/statedump.md 
>> 
>> Ashish 
>> 
>> ________________________________ 
>> From: "Serkan Çoban" <cobanserkan at gmail.com> 
>> To: "Ashish Pandey" <aspandey at redhat.com> 
>> Cc: "Gluster Users" <gluster-users at gluster.org>, "Gluster Devel" 
>> <gluster-devel at gluster.org> 
>> Sent: Friday, April 15, 2016 4:02:20 PM 
>> Subject: Re: [Gluster-users] Assertion failed: ec_get_inode_size 
>> 
>> Yes it is only one brick which error appears. I can send all other 
>> brick dumps too.. 
>> How can I get state dump in fuse client? There is no gluster command 
>> there.. 
>> _______________________________________________ 
>> Gluster-users mailing list 
>> Gluster-users at gluster.org 
>> http://www.gluster.org/mailman/listinfo/gluster-users 
>> 
> _______________________________________________ 
> Gluster-users mailing list 
> Gluster-users at gluster.org 
> http://www.gluster.org/mailman/listinfo/gluster-users 
> 
_______________________________________________ 
Gluster-users mailing list 
Gluster-users at gluster.org 
http://www.gluster.org/mailman/listinfo/gluster-users 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160418/66700c52/attachment.html>