[Gluster-users] Unnecessary healing in 3-node replication setup on reboot

[Vijay Bellur]

On Friday 16 October 2015 08:11 PM, Lindsay Mathieson wrote:
>
> On 17 October 2015 at 00:26, Udo Giacomozzi <udo.giacomozzi at indunet.it
> <mailto:udo.giacomozzi at indunet.it>> wrote:
>
>     To me this sounds like Gluster is not really suited for big files,
>     like as the main storage for VMs - since they are being modified
>     constantly.
>
>
> Depends :)
>
> Any replicated storage will have to heal its copies if they are written
> to when a node is down. So long as the files can still be read/written
> while being healed and the resource usage (CPU/Network) is not to high
> then it should be transparent - that's a major  whole pint of a
> replicated filesystem.
>
> I'm guessing that like me, you are running your gluster storage on your
> VM Hosts and you like me are a chronic tweaker, so tend to reboot the
> hosts more than you should. In that case you might want to consider
> moving your gluster storage to seperate dedicated nodes that you can
> leave up.
>
>     Or am I missing something? Perhaps Gluster can be configured to heal
>     only modified parts of the files?
>
>
>
> Not that I know of.
>

self-healing in gluster by default syncs only modified parts of the 
files from a source node. Gluster does a rolling checksum of a file 
needing self-heal to identify regions of the file which need to be 
synced over the network. This rolling checksum computation can sometimes 
be expensive and there are plans to have a lighter self-healing in 3.8 
with more granular changelogs that can do away with the need to do a 
rolling checksum.

You may also want to check sharding (currently in beta with 3.7) where 
large files are chunked to smaller fragments. With this scheme, 
self-healing (and rolling checksum computation thereby) happens only on 
those fragments that undergo changes when one of the nodes in a 
replicated set is offline. This has shown nice improvements in gluster's 
resource utilization during self-healing.

Regards,
Vijay