[Gluster-users] Problem with files showing zero links

[Ravishankar N]

What version of gluster are you using? Was there a chance there were 
directory renames from the client?
There was a bug which Pranith fixed quite some time back: 
http://review.gluster.org/#/c/7879/ for missing .glusterfs link files.

-Ravi

On 01/29/2016 05:58 PM, Venky Shankar wrote:
> On Fri, Jan 29, 2016 at 11:34:37AM +0000, Ronny Adsetts wrote:
>> Venky Shankar wrote on 29/01/2016 11:09:
>>> On Fri, Jan 29, 2016 at 10:46:14AM +0000, Ronny Adsetts wrote:
>>>> No idea how it came about in the first place. Suspect it happened
>>>> during either an operating system upgrade or upgrading glusterfs
>>>> packages to those from gluster.org.
>>> If you're still willing to debug, please send across brick log file
>>> of the concerned node.
>> Of course. Attached is the current brick log.
> The log files does not have any clue about how some of the files ended up with missing .glusterfs
> linkage, though there are tons of errors of operations trying to access .glusterfs/.././.....
> (e.g. for gfid 7f0fa160-8e5d-44f5-88b3-8187b2397313). Probably an upgrade and restart of services
> in midst of an operation?
>
>> My previous comments about having fixed the problem were a little premature. I now have most of the files that were showing with zero links now in the 'failed-heal' state:
>>
>> # gluster volume heal software statistics
>> [...]
>> Starting time of crawl: Fri Jan 29 11:15:53 2016
>>
>> Ending time of crawl: Fri Jan 29 11:15:58 2016
>>
>> Type of crawl: INDEX
>> No. of entries healed: 0
>> No. of entries in split-brain: 0
>> No. of heal failed entries: 1976
> My guess would be, as soon as you fixed the backend linkages, self-heal was able to kick in as it could
> "find" these files on the replica. But why did the heal fail or why these files needed healing in the
> first place.
>
> Pranith, Ravi?
>
> Also, please provide self-heal logs.
>
>>
>> The volume log file shows all the details but the "E" error entries look like this:
>>
>> [2016-01-29 11:12:00.479724] E [client-rpc-fops.c:914:client3_3_flush_cbk] 0-software-client-1: remote operation failed: Operation not permitted
>>
>>
>> And warnings like this:
>>
>> [2016-01-29 11:11:33.747428] W [client-rpc-fops.c:2772:client3_3_lookup_cbk] 0-software-client-1: remote operation failed: No such file or directory. Path: /win_patches/NDP20SP2-KB2446704-v2-x86.exe (9aa6b79a-b18f-42f2-ae9a-a886c523a0e7)
>> [2016-01-29 11:11:33.749192] W [client-rpc-fops.c:2772:client3_3_lookup_cbk] 0-software-client-1: remote operation failed: No such file or directory. Path: <gfid:9aa6b79a-b18f-42f2-ae9a-a886c523a0e7> (9aa6b79a-b18f-42f2-ae9a-a886c523a0e7)
>>
>>
>> Ronny
>>
>> -- 
>> Ronny Adsetts
>> Technical Director
>> Amazing Internet Ltd, London
>> t: +44 20 8977 8943
>> w: www.amazinginternet.com
>>
>> Registered office: 85 Waldegrave Park, Twickenham, TW1 4TJ
>> Registered in England. Company No. 4042957
>>
>
>
>
>
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-users