[Gluster-devel] dht and rename: how is it supposed to work?
Emmanuel Dreyfus
manu at netbsd.org
Fri Jul 29 05:15:10 UTC 2011
Hi everybody
Here is my latest bug: when the client does a rename(2) on a directory,
dht turns that into some weird dance involving link(2) on the server. Since
link(2) cannot operate on a directory, the rename(2) fails with EPERM.
How is rename supposed to work with dht? Is that bug NetBSD specific?
I just tried linking a directory on Linux, it fails as well. The
failure is SUSv2-compliant:
http://pubs.opengroup.org/onlinepubs/009695399/functions/link.html
Here are the details. I can reproduce the problem with this script:
client# cat test.sh
#!/bin/sh
rm -rf i386 machine inst.* *.core
mkdir i386
install -l s -r i386 machine
client# ./test.sh
install: machine: rename: Operation not permitted
Oddly, this fails only once. If I run the script twice, I get no errors.
I have to umount/mount the volume to reproduce the bug. Of course, the
script only fails with glusterfs. It runs fine on a native filesystem.
Here is what a kernel trace tells me about install(1) actions:
CALL __lstat30(0xbfbfc91c,0xbfbfc868)
NAMI "inst.04684a"
RET __lstat30 -1 errno 2 No such file or directory
CALL symlink(0xbfbff990,0xbfbfc91c)
MISC link-target: "i386"
NAMI "inst.04684a"
RET symlink 0
CALL __posix_rename(0xbfbfc91c,0xbfbff995)
NAMI "inst.04684a"
NAMI "machine"
RET __posix_rename -1 errno 1 Operation not permitted
Here is glusterfs client log:
[2011-07-29 06:45:18.283832] I [client3_1-fops.c:2056:client3_1_link_cbk]
0-gfs-client-0: remote operation failed: Operation not permitted
[2011-07-29 06:45:18.284401] I [client3_1-fops.c:2056:client3_1_link_cbk]
0-gfs-client-1: remote operation failed: Operation not permitted
[2011-07-29 06:45:18.290144] I [client3_1-fops.c:502:client3_1_unlink_cbk]
0-gfs-client-0: remote operation failed: No such file or directory
[2011-07-29 06:45:18.290699] I [client3_1-fops.c:502:client3_1_unlink_cbk]
0-gfs-client-1: remote operation failed: No such file or directory
[2011-07-29 06:45:18.290821] W [dht-rename.c:348:dht_rename_unlink_cbk]
0-gfs-dht: /rename/inst.04684a: unlink on gfs-replicate-0 failed
(No such file or director)
[2011-07-29 06:45:18.304395] W [fuse-bridge.c:1352:fuse_rename_cbk]
0-glusterfs-fuse: 50: /rename/inst.04684a -> /rename/machine => -1
(Operation not permitted)
Here is glusterfs server log:
[2011-07-29 06:45:18.283770] E [posix.c:2131:posix_link] 0-gfs-posix:
link /rename/inst.04684a to /rename/machine failed: Operation not permitted
[2011-07-29 06:45:18.283896] I [server3_1-fops.c:1050:server_link_cbk]
0-gfs-server: 60: LINK /rename/inst.04684a (49876739) ==> -1
(Operation not permitted)
[2011-07-29 06:45:18.290028] I [server3_1-fops.c:964:server_unlink_cbk]
0-gfs-server: 64: UNLINK /rename/machine (0) ==> -1
(No such file or directory)
And here is server kernel trace, which just confirms the logs above do
spot the right error:
CALL symlink(0xbb95d078,0xb65ff5c0)
MISC symlink-target: "i386"
NAMI "/export/wd3a/rename/inst.04684a"
RET symlink 0
(...)
CALL __lstat30(0xb6
CALL link(0xb65ff5c0,0xb65ff550)
NAMI "/export/wd3a/rename/inst.04684a"
NAMI "/export/wd3a/rename/machine"
RET link -1 errno 1 Operation not permitted
(...)
CALL unlink(0xb65ff550)
NAMI "/export/wd3a/rename/machine"
RET unlink -1 errno 2 No such file or directory
--
Emmanuel Dreyfus
manu at netbsd.org
More information about the Gluster-devel
mailing list