[Gluster-users] Strange file corruption

[Udo Giacomozzi]

Am 09.12.2015 um 14:39 schrieb Lindsay Mathieson:
>
> Udo, it occurs to me that if your VM's were running on #2 & #3 and you 
> live migrated them to #1 prior to rebooting #2/3, then you would 
> indeed rapidly get progressive VM corruption.
>
> However it wouldn't be due to the heal process, but rather the live 
> migration with "performance.stat-prefetch" on. This always leads to 
> qcow2 files becoming corrupted and unusable.

Nope. All VMs were running on #1, no exception.
Nodes #2 and #3 never had a VM running on them, so they were pratically 
idle since their installation.

Basically I set up node #1, including all VMs.
Then I've installed nodes #2 and #3, configured Proxmox and Gluster 
cluster and then waited quite some time until Gluster had synced up 
nodes #2 and #3 (healing).
 From then on, I've rebooted nodes 2 & 3, but in theory these nodes 
never had to do any writes to the Gluster volume at all.

If you're interested, you can read about my upgrade strategy in this 
Proxmox forum post: 
http://forum.proxmox.com/threads/24990-Upgrade-3-4-HA-cluster-to-4-0-via-reinstallation-with-minimal-downtime?p=125040#post125040

Also, It seems rather strange to me that pratically all ~15 VMs  (!) 
suffered from data corruption. It's like if Gluster considered node #2 
or #3 to be ahead and it "healed" in the wrong direction. I don't know..

BTW, once I understood what was going on, /with the problematic 
"healing" still in progress/, I was able to overwrite the bad images 
(still active on #1) by using standard Proxmox backup-restore and 
Gluster handled it correctly.


Anway, I really love the simplicity of Gluster (setting up and 
maintaining a cluster is extremely easy), but these healing issues are 
causing some headache to me... ;-)

Udo

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20151209/0d797b37/attachment.html>