[Gluster-devel] atomic operations fails_

[Fredrik Widlund]

Hi,

I'll try to make myself clearer than in the earlier thread, since I need some help here. I'm not sure if I am missing something.

I am not able to use atomic operations using glusterfs 3.0.2, on a Arch Linux 2.6.32.7-1 x86_64 server.

I've stripped everything down to the most simple configuration
Server: a posix store with locks
Client: a tcp client

I mount the glusterfs on /gfs, and use a "writer" and a "reader" as below:
[writer]                root at frustration> while :; do date > /gfs/.file; mv /gfs/.file /gfs/file; done
[reader]               root at frustration> while :; do cat /gfs/file > /dev/null; done

Leaving this running, I get a flood of
cat: /gfs/file: No such file or directory

This should not happen on a filesystem that supports atomic operations?!

What this means is that I am unable to replace a file as an atomic operation, creating a race-condition where the file is sometimes not present. Furthermore this actually "desyncs" the server in a more complex setup, requiring a restart of the server part to make "desynced" files available again.

This reproduces using XFS as well as EXT2 as the underlying fs.

[glusterfs.vol]
volume media0
  type protocol/client
  option transport-type tcp
  option remote-host 127.0.0.1
  option remote-subvolume media0
end-volume

mounted with "glusterfs -f glusterfs.vol /gfs"

[glusterfsd.vol]
(se below debug log)

-x-

[2010-02-09 18:18:23] D [glusterfsd.c:424:_get_specfp] glusterfs: loading volume file glusterfsd.vol
[2010-02-09 18:18:23] D [xlator.c:739:xlator_set_type] xlator: dlsym(notify) on /usr/lib/glusterfs/3.0.2/xlator/features/locks.so: undefined symbol: notify -- neglecting
================================================================================
Version      : glusterfs 3.0.2 built on Feb  8 2010 15:23:34
git: v3.0.2
Starting Time: 2010-02-09 18:18:23
Command line : glusterfs -f glusterfsd.vol --debug
PID          : 19486
System name  : Linux
Nodename     : n100.p1.x0.local
Kernel Release : 2.6.32-ARCH
Hardware Identifier: x86_64

Given volfile:
+------------------------------------------------------------------------------+
  1: volume media0-store
  2:   type storage/posix
  3:   option directory /mnt
  4: end-volume
  5:
  6: volume media0
  7:   type features/locks
  8:   subvolumes media0-store
  9: end-volume
 10:
 11: volume server
 12:   type protocol/server
 13:   option transport-type tcp
 14:   option auth.addr.media0.allow *
 15:   subvolumes media0
 16: end-volume

+------------------------------------------------------------------------------+
[2010-02-09 18:18:23] D [glusterfsd.c:1370:main] glusterfs: running in pid 19486
[2010-02-09 18:18:23] D [transport.c:145:transport_load] transport: attempt to load file /usr/lib/glusterfs/3.0.2/transport/socket.so
[2010-02-09 18:18:23] N [glusterfsd.c:1396:main] glusterfs: Successfully started
[2010-02-09 18:18:43] D [addr.c:190:gf_auth] media0: allowed = "*", received addr = "127.0.0.1"
[2010-02-09 18:18:43] N [server-protocol.c:5812:mop_setvolume] server: accepted client from 127.0.0.1:1023
[2010-02-09 18:18:43] D [addr.c:190:gf_auth] media0: allowed = "*", received addr = "127.0.0.1"
[2010-02-09 18:18:43] N [server-protocol.c:5812:mop_setvolume] server: accepted client from 127.0.0.1:1022
[2010-02-09 18:18:58] D [server-resolve.c:238:resolve_path_deep] media0: RESOLVE RENAME() seeking deep resolution of /file
[2010-02-09 18:18:58] D [dict.c:303:dict_get] dict: @this=(nil) @key=0x7f2a38a9f79b
[2010-02-09 18:18:58] D [dict.c:303:dict_get] dict: @this=(nil) @key=0x7f2a38a9f7b3
[2010-02-09 18:18:58] D [dict.c:303:dict_get] dict: @this=(nil) @key=0x7f2a38a9f7cb
[2010-02-09 18:18:58] D [dict.c:303:dict_get] dict: @this=(nil) @key=0x7f2a38a9f79b
[2010-02-09 18:18:58] D [dict.c:303:dict_get] dict: @this=(nil) @key=0x7f2a38a9f7b3
[2010-02-09 18:18:58] D [dict.c:303:dict_get] dict: @this=(nil) @key=0x7f2a38a9f7cb
[2010-02-09 18:19:15] D [inode.c:916:inode_path] media0/inode: no dentry for non-root inode 502725
[2010-02-09 18:19:34] D [inode.c:916:inode_path] media0/inode: no dentry for non-root inode 502734
[2010-02-09 18:19:56] D [inode.c:916:inode_path] media0/inode: no dentry for non-root inode 502734
[...] repeated many times. Frequency of the "no dentry" entries depends on how aggressive the reader is.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-devel/attachments/20100209/d93f9123/attachment-0003.html>