[Gluster-users] Replication deleting file contents

[Adrian Revill]

Hi
I have read through the docs and google and I think i am trying to do 
this right, but just wanted to be sure i have it correctly configured.

I have 2 servers factory1 and factory2, both clean installs of RHEL5.4 
on basic hardware.
I followed the instructions from 
http://gluster.com/community/documentation/index.php/GlusterFS_User_Guide, 
and have installed 3.0.0-1

On factory1 I ran the command
/usr/bin/glusterfs-volgen --name dataexp --raid 1 factory1:/data/export 
factory2:/data/export

I copied 
factory1-dataexp-export.vol to /etc/glusterfsd.vol on factory1
factory2-dataexp-export.vol to /etc/glusterfsd.vol on factory2

and coppied dataexp-tcp.vol to /etc/glusterfs.vol on both factory1 and 
factory2

Then on both servers I started the server with the init script and ran
mount -t glusterfs /etc/glusterfs/glusterfs.vol /mnt/export

Every thing works fine, i can copy files into either servers /mnt/export 
directory and they appear on the other server.

I then tried a failure scenario. With both servers and clients up i 
deleted all the files from /mnt/export, so far so good.

I then shut down the server on factory1, leaving the client up and 
copied three large files (715MB each) into /mnt/export on factory2
Both factory1 and factroy2 both show the files with the correct size and 
md5sum.

I then started the server on factory1.
This is where it goes wrong, as soon as factory1 starts, both clients 
show the file sizes of the files as 0, checking the backing store 
/data/export i see the files are also 0 size there too. The md5Sum is 
now wrong.

It seems that the replication has created the files on factory2, but 
before copying the contents, it has decided that the empty files on 
factory1 are newer than factory2 and decided to copy the zero size files 
back to factory1.

Any one got any ideas?




Here are the files that glusterfs-volgen made

factory1-dataexp-export.vol

## file auto generated by /usr/bin/glusterfs-volgen (export.vol)
# Cmd line:
# $ /usr/bin/glusterfs-volgen --name dataexp --raid 1 
factory1:/data/export factory2:/data/export

volume posix1
  type storage/posix
  option directory /data/export
end-volume

volume locks1
    type features/locks
    subvolumes posix1
end-volume

volume brick1
    type performance/io-threads
    option thread-count 8
    subvolumes locks1
end-volume

volume server-tcp
    type protocol/server
    option transport-type tcp
    option auth.addr.brick1.allow *
    option transport.socket.listen-port 6996
    option transport.socket.nodelay on
    subvolumes brick1
end-volume


factory2-dataexp-export.vol

## file auto generated by /usr/bin/glusterfs-volgen (export.vol)
# Cmd line:
# $ /usr/bin/glusterfs-volgen --name dataexp --raid 1 
factory1:/data/export factory2:/data/export

volume posix1
  type storage/posix
  option directory /data/export
end-volume

volume locks1
    type features/locks
    subvolumes posix1
end-volume

volume brick1
    type performance/io-threads
    option thread-count 8
    subvolumes locks1
end-volume

volume server-tcp
    type protocol/server
    option transport-type tcp
    option auth.addr.brick1.allow *
    option transport.socket.listen-port 6996
    option transport.socket.nodelay on
    subvolumes brick1
end-volume

dataexp-tcp.vol

## file auto generated by /usr/bin/glusterfs-volgen (mount.vol)
# Cmd line:
# $ /usr/bin/glusterfs-volgen --name dataexp --raid 1 
factory1:/data/export factory2:/data/export

# RAID 1
# TRANSPORT-TYPE tcp
volume factory2-1
    type protocol/client
    option transport-type tcp
    option remote-host factory2
    option transport.socket.nodelay on
    option transport.remote-port 6996
    option remote-subvolume brick1
end-volume

volume factory1-1
    type protocol/client
    option transport-type tcp
    option remote-host factory1
    option transport.socket.nodelay on
    option transport.remote-port 6996
    option remote-subvolume brick1
end-volume

volume mirror-0
    type cluster/replicate
    subvolumes factory1-1 factory2-1
end-volume

volume writebehind
    type performance/write-behind
    option cache-size 4MB
    subvolumes mirror-0
end-volume

volume readahead
    type performance/read-ahead
    option page-count 4
    subvolumes writebehind
end-volume

volume iocache
    type performance/io-cache
    option cache-size 1GB
    option cache-timeout 1
    subvolumes readahead
end-volume

volume quickread
    type performance/quick-read
    option cache-timeout 1
    option max-file-size 64kB
    subvolumes iocache
end-volume

volume statprefetch
    type performance/stat-prefetch
    subvolumes quickread
end-volume




______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email 
______________________________________________________________________