[Gluster-users] Unable to delete files but getfattr shows file is part of glusterfs
SINCOCK John
J.Sincock at fugro.com
Wed Jun 18 08:19:49 UTC 2014
Thanks Franco,
that's interesting, I've read some more emails to this list now, and yes this does look like the same issue that other people are seeing.
So I hope those patches will fix the problem.
I am not in any hurry to upgrade to 3.5 considering there still seem to be some other issues with 3.5 - but this seems to be quite a serious issue, if files copied to a gluster are not accessible if their parent directory did not end up on the longest-lived node!
I'll have to keep a close eye on this issue and upgrade as soon as the 3.5 looks stable
Thanks very much for the info.
Cheers and Regards,
John
-----Original Message-----
From: Franco Broi [mailto:franco.broi at iongeo.com]
Sent: Wednesday, 18 June 2014 3:15 PM
To: SINCOCK John
Cc: gluster-users at gluster.org
Subject: Re: [Gluster-users] Unable to delete files but getfattr shows file is part of glusterfs
Hi John
I got this yesterday, it was copied to this list.
Cheers,
On Tue, 2014-06-17 at 04:55 -0400, Susant Palai wrote:
Hi Franco:
> The following patches address the ENOTEMPTY issue.
>
> 1. http://review.gluster.org/#/c/7733/
> 2. http://review.gluster.org/#/c/7599/
>
> I think the above patches will be available in 3.5.1 which will be a
> minor upgrade.(Need ack from Niels de Vos.)
>
On Wed, 2014-06-18 at 13:26 +0930, SINCOCK John wrote:
> Thanks Pranith, I hope Susant can help!
>
>
>
> From: Pranith Kumar Karampuri [mailto:pkarampu at redhat.com]
> Sent: Wednesday, 18 June 2014 11:48 AM
> To: SINCOCK John; gluster-users at gluster.org
> Cc: Susant Palai
> Subject: Re: [Gluster-users] Unable to delete files but getfattr shows
> file is part of glusterfs
>
>
>
>
> I think Susant knows about this issue
>
> CCed him
>
> Pranith
>
> On 06/15/2014 07:16 PM, SINCOCK John wrote:
>
>
> Hi Everybody,
>
>
>
> We have recently setup a 2-node gluster, and are having some
> problems deleting some files!
>
> We haven’t accessed the gluster bricks directly, all access
> has been through the glusterfs mount
>
> The 1st node in our cluster also handles sharing the glusterfs
> volume via samba, to windows clients.
>
>
>
> Environment is:
>
> Kernel: 2.6.32-431.1.2.0.1.el6.x86_64 #1 SMP Fri Dec 13
> 13:06:13 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
>
> CentOS release 6.4 (Final)
>
>
>
> glusterfs-libs-3.4.1-3.el6.x86_64
>
> glusterfs-cli-3.4.1-3.el6.x86_64
>
> glusterfs-fuse-3.4.1-3.el6.x86_64
>
> glusterfs-server-3.4.1-3.el6.x86_64
>
> glusterfs-3.4.1-3.el6.x86_64
>
>
>
> 4 bricks on each node
>
> Gluster is setup without replication or
> striping
>
>
>
> Bricks on one of the two nodes (the other node
> is setup the same):
>
> /dev/sda1 9.1T 4.7T 4.5T
> 51% /mnt/glusterfs/bricks/1
>
> /dev/sda2 9.1T 3.7T 5.5T
> 40% /mnt/glusterfs/bricks/2
>
> /dev/sda3 9.1T 3.7T 5.5T
> 40% /mnt/glusterfs/bricks/3
>
> /dev/sda4 9.1T 4.1T 5.0T
> 46% /mnt/glusterfs/bricks/4
>
>
>
> Gluster volume mounted on 1st node:
>
> g-unit-1:/vol00 73T 31T 43T 42% /gluster/vol00
>
> Mount options:
>
> g-unit-1:/vol00 on /gluster/vol00 type fuse.glusterfs
>
> (rw,nosuid,nodev,noatime,default_permissions,allow_other,max_read=1310
> 72)
>
>
>
> So the problem is this:
>
> [root at g-unit-1 ~]# rm
> -Rf /gluster/vol00/data/CentOS3/CentOS_6.4_Final
>
> rm: cannot remove
> `/gluster/vol00/data/CentOS3/CentOS_6.4_Final/Packages':
> Directory not empty
>
> rm: cannot remove
> `/gluster/vol00/data/CentOS3/CentOS_6.4_Final/EFI/BOOT':
> Directory not empty
>
> [root at g-unit-1 ~]#
>
>
>
> And the delete leaves behind 2000 or so files:
>
> [root at g-unit-1 ~]#
> find /gluster/vol00/data/CentOS3/CentOS_6.4_Final/ -type f |
> wc -l
>
> 2185
>
> Almost all the left behind files are on the 2nd node,
> g-unit-2, but there are a few on g-unit-1
>
>
>
> From the 2nd node, g-unit-2:
>
> [root at g-unit-2 ~]# getfattr -m . -d -e
> hex
> /mnt/glusterfs/bricks/2/data/CentOS3/CentOS_6.4_Final/EFI/BOOT/BOOTX64
> .efi
>
> getfattr: Removing leading '/' from absolute path names
>
> # file:
>
> mnt/glusterfs/bricks/2/data/CentOS3/CentOS_6.4_Final/EFI/BOOT/BOOTX64.
> efi
>
> trusted.gfid=0xa9b0a66cf39b49c7890b2da3f631929f
>
> So the file is there, is tagged with some gluster info, and
> has definitely been copied onto the volume properly, via the
> glusterfs mount. It hasn’t been copied directly onto the
> brick.
>
>
>
> But from g-unit-1, accessing via the glusterfs mount, and
> trying to delete this file:
>
> [root at g-unit-1 ~]#
> rm
> /gluster/vol00/data/CentOS3/CentOS_6.4_Final/EFI/BOOT/BOOTX64.efi
>
> rm: remove regular file
>
> `/gluster/vol00/data/CentOS3/CentOS_6.4_Final/EFI/BOOT/BOOTX64.efi'? y
>
> rm: cannot remove
>
> `/gluster/vol00/data/CentOS3/CentOS_6.4_Final/EFI/BOOT/BOOTX64.efi':
> No such file or directory
>
>
>
> What the hell!?!!!!!
>
>
>
> Can someone please suggest the best course of action to
> resolve this and prevent it happening in the future?
>
>
>
> Thanks for any assistance.
>
>
>
> John
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-users
>
>
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-users
More information about the Gluster-users
mailing list