[Gluster-devel] Operation reordering and data corruption

[Emmanuel Dreyfus]

Hi

I experience a nasty data corruption bug that I can track down to a bad
interraction between how the NetBSD kernel resize files and an operation
reordering ability glusterfs seems to have.

Consider the exchanges below that I logged at the FUSE level:

>> SETATTR size = 12178 -> 32394
>> WRITE size = 12178
<< WRITE size = 12178 -> 32394
>> GETATTR size = 32394
<< GETATTR size = 32394 -> 12178
<< SETATTR size = 32394 -> 32394

SETATTR si sent before WRITE, but WRITE completes first. Then GETATTR
kicks in and it seems SETATTR is not yet completed, since the old size
is obtained.

This is a real problem, because the NetBSD kernel will detect a file
shrink in the GETATTR vnode method, and it will truncate the file to
match the new size. The result is that the WRITE that was just done is
discarded, and data is replaced by a chunk of zeroes.

In what conditions glusterfs performs operation reordering? Is it a bug
or a feature? Linux does not truncate in the same situation?

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu at netbsd.org