[Gluster-users] GlusterFS behavior with 2 clients

[Matt Sorin]

Hello,

I recently discovered GlusterFS and followed the installation instructions and the relatively brief tutorials that describe setting up "test-volume" but the behavior I am seeing is odd.  I have 4 machines set up to test GlusterFS: 2 servers and 2 clients.

All the machines are running RHEL5U4 x86_64. The servers were set up as so:

192.168.56.101   server1
192.168.56.102   server2
192.168.56.103   client1
192.168.56.104   client2

glusterfs-core-3.1.0-1 and glusterfs-fuse-3.1.0-1 were installed on all 4 nodes. glusterd was started on all 4 nodes (though I expect only the bricks/peers need it).

The commands executed on server1 were:

gluster peer probe server2
    (no problems here)
gluster peer status
    (it returned the correct results but I don't have the output handy)
gluster volume create test-volume replica 2 transport tcp server1:/exp1 server2:/exp2
    (the command succeeded and it created an /exp1 directory on server1 and /exp2 on server2)
gluster volume start test-volume
    (again, no problems)


Now, on the clients...

Mount test-volume on the clients:
[root at client1 tmp]# mount -t glusterfs server1:/test-volume /mnt
[root at client2 ~]# mount -t glusterfs server1:/test-volume /mnt

Make a file on test-volume using client1:
[root at client1 tmp]# echo "1234" >/mnt/foo
[root at client1 tmp]# cat /mnt/foo
1234
[root at client1 tmp]# ls -al /mnt/foo
-rw-r--r-- 1 root root 5 Oct 28 19:34 /mnt/foo

Make sure the file is seen on client2:
[root at client2 ~]# cat /mnt/foo
1234
[root at client2 ~]# ls -al /mnt/foo
-rw-r--r-- 1 root root 5 Oct 28 19:34 /mnt/foo

Append to the file using client2:
[root at client2 ~]# echo "5678" >>/mnt/foo
[root at client2 ~]# ls -al /mnt/foo
-rw-r--r-- 1 root root 10 Oct 28 19:35 /mnt/foo
[root at client2 ~]# md5sum /mnt/foo
9053253e972cf40443a4083f452f24d4  /mnt/foo

Now check and see that client1 sees the changes:
[root at client1 tmp]# cat /mnt/foo
1234
[root at client1 tmp]# ls -al /mnt/foo
-rw-r--r-- 1 root root 10 Oct 28 19:35 /mnt/foo
[root at client1 tmp]# md5sum /mnt/foo
e7df7cd2ca07f4f1ab415d457a6e1c13  /mnt/foo

If I check server1:/exp1/foo and server2:/exp2/foo, the files are in agreement with what client2 thinks the file should look like.

client1 sees the new file size according to "ls" but "cat" and "md5sum" still only think the file in its original "1234" state is there.

If I unmount and remount test-volume on client1, it sees the modified file as it exists on server1 and server2.

I've tried this same test on both RHEL5U4 and Fedora Core 13 with the same results so I must be doing something wrong.

Can someone please tell me what I have done incorrectly?

Thank you for your patience with a newbie.

-
Matt