[Gluster-users] symbolic links changed to empty files

[Allan Latham]

Hi all

I'm running some initial sanity and performance checks on this:

root at h06 /root # glusterd -V
glusterfs 3.4.0beta4 built on Jul 10 2013 15:14:50

The setup is a two node replicating cluster (h06 and h65) the client is
on h06.

The network is limited to 100Mbits/second and rtt is 0.6ms so
performance is not spectacular but that is not the problem which
currently concerns me.

The test was to take a typical linux rootfs with 52K files of various
sizes totaling 1.9GByte and copy it to a gluster mount using tar:

root at h06 /root/mnt/h06 # sync;time (tar -c *|(cd /gluster/mnt/20 && tar
-x);sync)

real    20m24.836s
user    0m3.517s
sys     0m39.814s

Times are not brilliant but it will be OK for the usage scenario.

Extracts from 'mount':

/dev/mapper/vs-h06 on /root/mnt/h06 type ext4
(rw,relatime,barrier=1,data=ordered)

/var/lib/glusterd/vols/gl/gl-fuse.vol on /gluster/mnt type
fuse.glusterfs
(rw,nosuid,nodev,noatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)

Among other tests I wanted to see that the copy in /gluster/mnt/20 was
identical to the original in /root/mnt/h06

The original has 5261 symbolic links:

root at h06 /root/mnt/h06 # find -type l |wc
   5261    5261  182824

The copy has only 4089:

root at h65 /gluster/mnt/20 # find -type l |wc
   4089    4089  141644

Here is an example:

root at h06 /root # ls -ld ~/mnt/h06/bin/netcat
lrwxrwxrwx 1 root root 24 Jun 30 09:03 /root/mnt/h06/bin/netcat ->
/etc/alternatives/netcat

root at h06 /root # ls -ld /gluster/mnt/20/bin/netcat
---------- 1 root root 0 Jul 14 06:37 /gluster/mnt/20/bin/netcat

A scan with md5sum on the original and the copy in gluster shows only
these links as being different. All normal files checksum the same.

The mirrored gluster filesystem on h65 (no surprise) shows the identical
result - some symbolic links have been changed in empty files.

To the best of my knowledge the gluster filesystems are identical on the
two nodes but differ from the original.

To me it appears that the command:

(cd source && tar -c *)|(cd gluster && tar -x)

has changed some symbolic links in 'source' into empty files in 'gluster'.

I have used commands like the above tar everyday for years on other
filessytems and this is the first time I've come across something like this.

Any clues what I've done wrong or where I should look next?

All the best

Allan