[Gluster-devel] rm -r problem on FreeBSD port

[Raghavendra Gowdappa]

----- Original Message -----
> From: "Rick Macklem" <rmacklem at uoguelph.ca>
> To: "Gluster Devel" <gluster-devel at gluster.org>
> Sent: Tuesday, January 19, 2016 4:48:21 AM
> Subject: [Gluster-devel] rm -r problem on FreeBSD port
> 
> Hi,
> 
> I have a simple gluster volume made up of 2 bricks using
> distribute (running on the FreeBSD port of 3.7.6).
> When I do a "rm -rf <dir>" for a fairly large tree, I
> sometimes get "Directory not empty" errors.
> When I look in the directory (even after an unmount, shutdown,
> restart, remount) the "ls -l" output for the fuse mounted
> volume and the 2 underlying bricks looks like...
> 
> # ls -l of fuse mounted gv0: directory
> 
> total 1
> ----------  1 root  wheel  345891 Jan 18 14:49 Makefile
> ----------  1 root  wheel  345891 Jan 18 14:49 Makefile
> ----------  1 root  wheel    1195 Jan 18 14:49 ata_if.c
> ----------  1 root  wheel    1195 Jan 18 14:49 ata_if.c
> ----------  1 root  wheel     576 Jan 18 14:49 fb_if.c
> ----------  1 root  wheel     576 Jan 18 14:49 fb_if.c
> ----------  1 root  wheel    1787 Jan 18 14:49 hdac_if.c
> ----------  1 root  wheel    1787 Jan 18 14:49 hdac_if.c
> ----------  1 root  wheel     753 Jan 18 14:49 mmcbus_if.c
> ----------  1 root  wheel     753 Jan 18 14:49 mmcbus_if.c
> drwxr-xr-x  3 root  wheel     512 Jan 14 12:34 modules
> ----------  1 root  wheel     602 Jan 18 14:49 power_if.c
> ----------  1 root  wheel     602 Jan 18 14:49 power_if.c
> ----------  1 root  wheel    1819 Jan 18 14:49 uart_if.c
> ----------  1 root  wheel    1819 Jan 18 14:49 uart_if.c
> 
> # ls -l of the directory in one of the 2 bricks
> 
> total 36
> ----------  2 root  wheel    0 Jan 18 14:49 Makefile
> ----------  2 root  wheel    0 Jan 18 14:49 ata_if.c
> ----------  2 root  wheel    0 Jan 18 14:49 fb_if.c
> ----------  2 root  wheel    0 Jan 18 14:49 hdac_if.c
> ----------  2 root  wheel    0 Jan 18 14:49 mmcbus_if.c
> drwxr-xr-x  3 root  wheel  512 Jan 14 17:34 modules
> ----------  2 root  wheel    0 Jan 18 14:49 power_if.c
> ----------  2 root  wheel    0 Jan 18 14:49 uart_if.c
> 
> # and ls -l of the underlying directory in the other brick
> 
> total 400
> -rw-r--r--  2 root  wheel  345891 Jan 14 11:50 Makefile
> -rw-r--r--  2 root  wheel    1195 Jan 14 13:19 ata_if.c
> -rw-r--r--  2 root  wheel     576 Jan 14 12:13 fb_if.c
> -rw-r--r--  2 root  wheel    1787 Jan 14 12:13 hdac_if.c
> -rw-r--r--  2 root  wheel     753 Jan 14 12:13 mmcbus_if.c
> drwxr-xr-x  3 root  wheel     512 Jan 14 12:34 modules
> -rw-r--r--  2 root  wheel     602 Jan 14 12:13 power_if.c
> -rw-r--r--  2 root  wheel    1819 Jan 14 12:13 uart_if.c
> 
> Anyone have an idea of what is causing this?

There is a similar known issue explained in bz:
https://bugzilla.redhat.com/show_bug.cgi?id=1245065

However as per our understanding, the contents of the directory whose rmdir failed with ENOTEMPTY should be _only_ empty directories. In your case there are non-directories. So, it might be a different issue.

Adding Sakshi, who is working on this class of bugs and might add something worthwhile.

> (I am thinking something like the FreeBSD fuse interface
>  is holding opens on the files when I am doing "rm -rf <dir>",
>  even though they shouldn't be open and the GlusterFS is doing
>  some trickery similar to NFS's silly rename to avoid the
>  files being removed before being closed?
>  But I have no idea if this theory holds any water.;-)
> 
> Thanks in advance for any hints, rick
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>