[Bugs] [Bug 1387878] New: Rebalance after add bricks corrupts files

[bugzilla at redhat.com]

https://bugzilla.redhat.com/show_bug.cgi?id=1387878

            Bug ID: 1387878
           Summary: Rebalance after add bricks corrupts files
           Product: GlusterFS
           Version: 3.8
         Component: core
          Severity: high
          Assignee: bugs at gluster.org
          Reporter: lindsay.mathieson at gmail.com
                CC: bugs at gluster.org



Created attachment 1213155
  --> https://bugzilla.redhat.com/attachment.cgi?id=1213155&action=edit
All the logs I could think of, include client logging from the VM

Description of problem:Starting a rebalance after adding three bricks to a rep
3 cluster causes VM image corruption


Version-Release number of selected component (if applicable): 3.8.4


How reproducible:


Steps to Reproduce:
1. Create a rep 3 sharded volume
2. Copy a KVM Image file (qcow2) to the volume
3. Start the VM (using gfapi)
4. Add 3 Bricks, converting the volume to  2 x 3 = 6
5. Start a Rebalance
6. Start Heavy IO on the VM (I used Crystal DiskMark in the windows VM)


Actual results: The VM rapidly crashes and becomes unbootable. "qemu-img check"
shows corruption of the image.


Expected results: Rebalance compete, VM's unaffected except for increased i/o
load.


Additional info:Heavy I/O after the rebalance seems to be the key, operation
will complete ok otherwise. 

Also I tried a diskmark before the rebalance, then after and it also completed
ok. But a diskmark only after the rebalance crashed within seconds. 

I believe that diskmark writes to the same areas each time, so maybe it is new
writes after the rebalance that is the problem.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
You are the assignee for the bug.