[Gluster-devel] cache problem?

[Emmanuel Dreyfus]

Hi

Another weird bug. If I download a file through FTP to my glusterfs
volume, sometime the file gets out of sync between the client and
the backend. I get this:

client# md5 gnusrc.tgz 
MD5 (gnusrc.tgz) = 471a73c374ec2b5733571c01647a69d5

server# md5 /export/wd3a/tmp/gnusrc.tgz
MD5 (/export/wd3a/tmp/gnusrc.tgz) = cf03446a7f31713002ef3b74020b173f

If the client duplicates the file, it gets even more weird:

client# cp gnusrc.tgz gnusrc-cp.tgz
client# md5 gnusrc.tgz gnusrc-cp.tgz
MD5 (gnusrc-cp.tgz) = 471a73c374ec2b5733571c01647a69d5
MD5 (gnusrc.tgz) = 471a73c374ec2b5733571c01647a69d5

server# md5 /export/*/tmp/*.tgz     
MD5 (/export/wd3a/tmp/gnusrc-cp.tgz) = 471a73c374ec2b5733571c01647a69d5
MD5 (/export/wd3a/tmp/gnusrc.tgz) = cf03446a7f31713002ef3b74020b173f

At least now I can compare the files on the server.
server# mkdir -p /root/parts/gnusrc /root/parts/gnusrc-cp
server# cd /root/parts/gnusrc && split -b 1m /export/wd3a/tmp/gnusrc.tgz
server# cd /root/parts/gnusrc-cp && split -b 1m /export/wd3a/tmp/gnusrc-cp.tgz
server# diff -r /root/parts/gnusrc /root/parts/gnusrc-cp
Binary files /root/parts/gnusrc/xco and /root/parts/gnusrc-cp/xco differ
Binary files /root/parts/gnusrc/xcp and /root/parts/gnusrc-cp/xcp differ

Note that there are 80 chunks. Using hexdump -C the picture becomes
clearer. In xco, the data was replaced by zeros from offset 0x00045750
to offset 0x000fffff. In xcp, data was replaced by zeros from offert
0x00000000 to offset 0x00032fff. The two chuncks are neightbours, 
that means we have a single  hunk of data that was zered. Unfortunately,
neither the size nor the alignement look like page boundaries.

Since the glusterfs client has cached the right data, that means the bug
is somewhere within glusterfs, or at least its NetBSD port. I think NetBSD
FUSE elumation can be ruled out. Opinions?

-- 
Emmanuel Dreyfus
manu at netbsd.org