[Gluster-users] Input/Output Error when deleting folder

Wed Jan 7 14:31:57 UTC 2015

Hmmm... different results on different nodes!
Below the output for each brick for each of the 5 nodes.

[root at gluster01-mi ~]# getfattr -m. -e hex -d /brick1/recorder/Rec218
getfattr: Removing leading '/' from absolute path names
# file: brick1/recorder/Rec218
trusted.ec.version=0x000000000000693a
trusted.gfid=0xa9d904af0d9e4018acb2881bd8b3c2e4
trusted.glusterfs.dht=0x000000010000000000000000ffffffff

[root at gluster01-mi ~]# getfattr -m. -e hex -d /brick2/recorder/Rec218
getfattr: Removing leading '/' from absolute path names
# file: brick2/recorder/Rec218
trusted.ec.version=0x000000000000693a
trusted.gfid=0xa9d904af0d9e4018acb2881bd8b3c2e4
trusted.glusterfs.dht=0x000000010000000000000000ffffffff

[root at gluster02-mi ~]# getfattr -m. -e hex -d /brick1/recorder/Rec218
getfattr: Removing leading '/' from absolute path names
# file: brick1/recorder/Rec218
trusted.ec.version=0x000000000000693a
trusted.gfid=0xa9d904af0d9e4018acb2881bd8b3c2e4
trusted.glusterfs.dht=0x000000010000000000000000ffffffff

[root at gluster02-mi ~]# getfattr -m. -e hex -d /brick2/recorder/Rec218
getfattr: Removing leading '/' from absolute path names
# file: brick2/recorder/Rec218
trusted.ec.version=0x000000000000693a
trusted.gfid=0xa9d904af0d9e4018acb2881bd8b3c2e4
trusted.glusterfs.dht=0x000000010000000000000000ffffffff

[root at gluster03-mi ~]# getfattr -m. -e hex -d /brick1/recorder/Rec218
getfattr: Removing leading '/' from absolute path names
# file: brick1/recorder/Rec218
trusted.gfid=0xbda849fca556469ead84ed074f2c1bcd

[root at gluster03-mi ~]# getfattr -m. -e hex -d /brick2/recorder/Rec218
getfattr: Removing leading '/' from absolute path names
# file: brick2/recorder/Rec218
trusted.gfid=0xbda849fca556469ead84ed074f2c1bcd

[root at gluster04-mi ~]# getfattr -m. -e hex -d /brick1/recorder/Rec218
getfattr: Removing leading '/' from absolute path names
# file: brick1/recorder/Rec218
trusted.ec.version=0x0000000000006939
trusted.gfid=0xa9d904af0d9e4018acb2881bd8b3c2e4
trusted.glusterfs.dht=0x000000010000000000000000ffffffff

[root at gluster04-mi ~]# getfattr -m. -e hex -d /brick2/recorder/Rec218
getfattr: Removing leading '/' from absolute path names
# file: brick2/recorder/Rec218
trusted.ec.version=0x0000000000006939
trusted.gfid=0xa9d904af0d9e4018acb2881bd8b3c2e4
trusted.glusterfs.dht=0x000000010000000000000000ffffffff

[root at gluster05-mi ~]# getfattr -m. -e hex -d /brick1/recorder/Rec218
getfattr: Removing leading '/' from absolute path names
# file: brick1/recorder/Rec218
trusted.ec.version=0x000000000000693a
trusted.gfid=0xa9d904af0d9e4018acb2881bd8b3c2e4
trusted.glusterfs.dht=0x000000010000000000000000ffffffff

[root at gluster05-mi ~]# getfattr -m. -e hex -d /brick2/recorder/Rec218
getfattr: Removing leading '/' from absolute path names
# file: brick2/recorder/Rec218
trusted.ec.version=0x000000000000693a
trusted.gfid=0xa9d904af0d9e4018acb2881bd8b3c2e4
trusted.glusterfs.dht=0x000000010000000000000000ffffffff

-----Original Message-----
From: Xavier Hernandez [mailto:xhernandez at datalab.es] 
Sent: mercoledì 7 gennaio 2015 15:22
To: RASTELLI Alessandro
Cc: gluster-users at gluster.org; CAZZANIGA Stefano; UBERTINI Gabriele; TECHNOLOGY - Supporto Sistemi OTT e Cloud; ORLANDO Luca
Subject: Re: [Gluster-users] Input/Output Error when deleting folder

Can you send me the result of the following command on all bricks ?

getfattr -m. -e hex -d <brick path>/Rec218

On 01/07/2015 03:15 PM, RASTELLI Alessandro wrote:
> Hi,
> we just did a clean reboot of the client.
>
> The config is as follows (2 bricks on each node):
>
> [root at gluster01-mi ~]# gluster volume info Volume Name: 
> storage-recorder
> Type: Disperse
> Volume ID: 97587d68-3834-43cc-8b95-11996e013bf2
> Status: Started
> Number of Bricks: 1 x (8 + 2) = 10
> Transport-type: tcp
> Bricks:
> Brick1: gluster01-mi:/brick1/recorder
> Brick2: gluster01-mi:/brick2/recorder
> Brick3: gluster02-mi:/brick1/recorder
> Brick4: gluster02-mi:/brick2/recorder
> Brick5: gluster03-mi:/brick1/recorder
> Brick6: gluster03-mi:/brick2/recorder
> Brick7: gluster04-mi:/brick1/recorder
> Brick8: gluster04-mi:/brick2/recorder
> Brick9: gluster05-mi:/brick1/recorder
> Brick10: gluster05-mi:/brick2/recorder
>
> A.
>
> -----Original Message-----
> From: Xavier Hernandez [mailto:xhernandez at datalab.es]
> Sent: mercoledì 7 gennaio 2015 15:11
> To: RASTELLI Alessandro
> Cc: gluster-users at gluster.org; CAZZANIGA Stefano; UBERTINI Gabriele; 
> TECHNOLOGY - Supporto Sistemi OTT e Cloud; ORLANDO Luca
> Subject: Re: [Gluster-users] Input/Output Error when deleting folder
>
> Did you stopped the client or unmounted the volume while the application was still writing to it ?
>
> And what is the volume configuration ? (gluster volume info)
>
> Xavi
>
> On 01/07/2015 02:21 PM, RASTELLI Alessandro wrote:
>> Hi Xavier,
>>
>> nothing special happened to volumes and bricks, we just restarted the 
>> client that writes into the folder Rec218.
>>
>> Before rebooting, the folder was OK.
>>
>> Thank you
>>
>> Alessandro
>>
>> *From:*Xavier Hernandez [mailto:xhernandez at datalab.es]
>> *Sent:* mercoledì 7 gennaio 2015 13:11
>> *To:* RASTELLI Alessandro
>> *Cc:* gluster-users at gluster.org; CAZZANIGA Stefano; UBERTINI 
>> Gabriele; TECHNOLOGY - Supporto Sistemi OTT e Cloud; ORLANDO Luca
>> *Subject:* Re: [Gluster-users] Input/Output Error when deleting 
>> folder
>>
>> Hi Alessandro,
>>
>> what is the volume configuration ?
>> did happen something special to the volume, bricks or that directory ?
>> did you directly removed or changed something in the bricks ?
>> did you replaced some brick ?
>>
>> Xavi
>>
>> On 01/07/2015 12:09 PM, RASTELLI Alessandro wrote:
>>
>>      Hi,
>>
>>      we encounter this error when I try to delete a folder on a
>>      Gluster3.6.1 environment (dispersed volume):
>>
>>      [2015-01-07 10:27:49.008999] W [fuse-bridge.c:483:fuse_entry_cbk]
>>      0-glusterfs-fuse: 10890: LOOKUP() /Rec218 => -1 (Input/output
>> error)
>>
>>      [2015-01-07 10:27:49.009542] W [ec-common.c:164:ec_check_status]
>>      0-storage-recorder-disperse-0: Operation failed on some subvolumes
>>      (up=3FF, mask=3FF, remaining=0, good=3CF, bad=30)
>>
>>      [2015-01-07 10:27:50.326991] E
>>      [ec-heal.c:656:ec_heal_prepare_others]
>>      0-storage-recorder-disperse-0: Don't know how to heal error 116
>>
>>      ls command gives the output:
>>
>>      ?????????? ? ?        ?               ?            ?  Rec218
>>
>>      I can't do any operation on the folder (move, rename, delete)
>>
>>      Can you please indicate how to fix?
>>
>>      Thanks
>>
>>      Alessandro
>>
>>
>>
>>
>>      _______________________________________________
>>
>>      Gluster-users mailing list
>>
>>      Gluster-users at gluster.org  <mailto:Gluster-users at gluster.org>
>>
>>      http://www.gluster.org/mailman/listinfo/gluster-users
>>