[Bugs] [Bug 1232391] New: Sharding - Use (f)xattrop (as opposed to (f)setxattr) to update shard size and block count

[bugzilla at redhat.com]

https://bugzilla.redhat.com/show_bug.cgi?id=1232391

            Bug ID: 1232391
           Summary: Sharding - Use (f)xattrop (as opposed to (f)setxattr)
                    to update shard size and block count
           Product: GlusterFS
           Version: mainline
         Component: sharding
          Keywords: Triaged
          Assignee: kdhananj at redhat.com
          Reporter: kdhananj at redhat.com
        QA Contact: bugs at gluster.org
                CC: bugs at gluster.org



Description of problem:

Running iozone on a sharded volume fails with EBADFD.
>From the strace output of iozone, it was found that the application ended up
reading fewer bytes than it was expecting to, and bailing out with EBADFD.
On closer inspection of the file on the backend, it was found that the 'size'
xattr was reflecting a smaller number than the actual size of the file.

Turns out this can happen when write-behind flushes the cached writes in one go
, causing them to hit the disk in an out-of-order fashion where the different
io-threads perform the writes in parallel, without any serialisation. For
example, when a write in the range [0-100] races with a write on the same file
in [101-200] range, it could so happen that the second write hits the disk
before the first. And this could cause the second write request to persist the
file size as 200 followed by the second write request to persist the size as
100 bytes, leading to incorrect file size accounting.

And then, this bug can also be hit in cases where the applications performing
I/O are multi-threaded in nature.

The solution involves using xattrop (adding/subtracting only the delta byte
count) as opposed to setxattr to update the size.

Note that even with this approach, things could go wrong if two or more threads
of an application, as part of writing off the end of a file, end up creating
holes in overlapping regions where the hole's contribution to the file size
could be counted more than once, leading to incorrect file size accounting. But
this would be tackled through restructuring of the /.shard backend, which is a
bigger change and will come in much later.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are on the CC list for the bug.