[Gluster-users] GlusterFS 3.7

[Andreas Hollaus]

On 09/22/15 14:50, Gaurav Garg wrote:
> Hi Andreas,
>
>>> Are there any restrictions as to when I'm allowed to make changes to the GlusterFS
> volume (for instance: start/stop volume, add/remove brick or peer)?
>
> There are no restriction to make changes to the GlusterFS volume its all depend on your need whether you want to start/stop volume, add/remove brick or peer.
Sound promising, but as far as I can see it's a hard task to keep those configuration
files synced without any restrictions for the commands above. I guess that this is
just as hard as keeping data files in sync, but in that case you make use of those
extended attributes. Apparently they are not used for the GlusterFS configuration
files...
>
>>> How will it handle such changes when one of my two replicated servers is down? 
> When one of the replicated server is down then it will lookup to another server which is online, and if you do any file operation from mount point it will be reflected to the replica count which is currently online. When second replica come back online then you can start heal operation on glusterFS volume, it will update the 2nd replica which was offline. 
Heal operation is only applicable for the data stored on the GlusterFS volume, isn't
it? My question is about the GlusterFS volume configuration files.
>
>>> How will GlusterFS know which set of configuration files it can trust when the other server is connected
> again and the files will contain different information about the volume?
>
> When other node (server) come online then it will do handshake with the node which was online and request for configuration file and the offline node which come back online will update the configuration file based on the updated configuration file it received. But you should not flout glusterFS configuration by editing volfile manually to avoid obscure behavior.

Well, unfortunately I see another behaviour now that I made some tests on a couple of
VMs.

1. Suppose you have two VMs (VM1 & VM2), each hosting a brick for a GlusterFS volume
(gfsvol).
2. VM2 is disconnected from VM1, by taking down the network interface.
3. gfsvol is stopped and deleted by running commands on VM1.
4. A new volume (also) called gfsvol is created on VM1.
5. When VM2 is reconnected to the network, the volume configuration is automatically
copied from VM2 to VM1 by some internal stuff in GlusterFS!?!

This will result in a volume id mismatch on VM1, as one id is stored in the
configuration files and another is stored as extended attribute on the brick at VM1.
In this scenario it's obvious that the configuration is copied in the unwanted
direction, but in case it would have been copied in the other direction the mismatch
would have been on VM2 instead.

I wonder how GlusterFS would be able to determine which of the servers that has been
online and which has been offline? Both will continue to live throughout this
scenario, but they will just lose the connection to each other for a while. To me, it
seems that I've broken some restrictions to end up like this (or there's a bug in
GlusterFS)(?). Maybe the extended attribute should have been replaced as well as the
configuration files?

BTW: Yes, I rely on GlusterFS commands and avoid manual editing of files.


Regards
Andreas

> I hope that will answer your doubt.
>
> Thank you...
>
> Regards,
> Gaurav Garg
>  
>
> ----- Original Message -----
> From: "Andreas Hollaus" <Andreas.Hollaus at ericsson.com>
> To: gluster-users at gluster.org
> Sent: Tuesday, September 22, 2015 1:32:08 PM
> Subject: [Gluster-users] GlusterFS 3.7
>
> Hi,
>
> Are there any restrictions as to when I'm allowed to make changes to the GlusterFS
> volume (for instance: start/stop volume, add/remove brick or peer)? How will it
> handle such changes when one of my two replicated servers is down? How will GlusterFS
> know which set of configuration files it can trust when the other server is connected
> again and the files will contain different information about the volume? If these
> were data files on the GlusterFS volume that would have been handled by the extended
> file attributes, but how about the GlusterFS configuration itself?
>
> Regards
> Andreas
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users