[Gluster-users] VM fs becomes read only when one gluster node goes down

[Josh Boon]

Andre, 

I've not explored using a DNS solution to publish the gluster cluster addressing space but things you'll want to check out are network.ping-timeout and whether or not your VM goes read-only on filesystem error. If your network is consistent and robust tuning network.ping-timeout to a very low value such as three seconds will instruct the client to drop that client on failure. The default value for this is 42 seconds which will cause your VM to go read-only as you've seen. You could also choose to have your VM's mount their partitions errors=continue as well depending on the filesystem they run. Our setup has timeout at seven seconds and errors=continue and has survived both testing and storage node segfaults. No data integrity issues have presented yet but our data is mostly temporal so integrity hasn't been tested thoroughly. Also we're qemu 2.0 running gluster 3.6 on ubuntu 14.04 for those curious. 

Best, 
Josh 


From: "Roman" <romeo.r at gmail.com> 
To: "Krutika Dhananjay" <kdhananj at redhat.com> 
Cc: "gluster-users" <gluster-users at gluster.org>, gluster-devel at gluster.org 
Sent: Monday, October 26, 2015 1:33:57 PM 
Subject: Re: [Gluster-users] VM fs becomes read only when one gluster node goes down 

Hi, 
got backupvolfile-server=NODE2NAMEHERE in fstab ? :) 

2015-10-23 5:24 GMT+03:00 Krutika Dhananjay < kdhananj at redhat.com > : 



Could you share the output of 'gluster volume info', and also information as to which node went down on reboot? 

-Krutika 


BQ_BEGIN
From: "André Bauer" < abauer at magix.net > 
To: "gluster-users" < gluster-users at gluster.org > 
Cc: gluster-devel at gluster.org 
Sent: Friday, October 23, 2015 12:15:04 AM 
Subject: [Gluster-users] VM fs becomes read only when one gluster node goes down 

Hi, 

i have a 4 node Glusterfs 3.5.6 Cluster. 

My VM images are in an replicated distributed volume which is accessed 
from kvm/qemu via libgfapi. 

Mount is against storage.domain.local which has IPs for all 4 Gluster 
nodes set in DNS. 

When one of the Gluster nodes goes down (accidently reboot) a lot of the 
vms getting read only filesystem. Even when the node comes back up. 

How can i prevent this? 
I expect that the vm just uses the replicated file on the other node, 
without getting ro fs. 

Any hints? 

Thanks in advance. 

-- 
Regards 
André Bauer 

_______________________________________________ 
Gluster-users mailing list 
Gluster-users at gluster.org 
http://www.gluster.org/mailman/listinfo/gluster-users 



_______________________________________________ 
Gluster-users mailing list 
Gluster-users at gluster.org 
http://www.gluster.org/mailman/listinfo/gluster-users 

BQ_END




-- 
Best regards, 
Roman. 

_______________________________________________ 
Gluster-users mailing list 
Gluster-users at gluster.org 
http://www.gluster.org/mailman/listinfo/gluster-users 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20151026/5f7155a0/attachment.html>