[Gluster-Maintainers] Upgrade issue when new mem type is added in libglusterfs

[Atin Mukherjee]

I still see the release notes for 3.8.1 & 3.7.13 not reflecting this
change.

Niels, Kaushal,

Shouldn't we highlight this as early as possible to the users given release
note is the best possible medium to capture all the known issues and the
work around?


~Atin


On Sat, Jul 9, 2016 at 10:02 PM, Atin Mukherjee <amukherj at redhat.com> wrote:

> We have hit a bug 1347250 in downstream (applicable upstream too) where it
> was seen that glusterd didnt regenerate the volfiles when it was interimly
> brought up with upgrade mode by yum. Log file captured that gsyncd
> --version failed to execute and hence glusterd init couldnt proceed till
> the volfile regeneration. Since the ret code is not handled here in spec
> file users wouldnt come to know about this and going forward this is going
> to cause major issues in healing and all and finally it exploits the
> possibility of split brains at its best.
>
> Further analysis by Kotresh & Raghavendra Talur reveals that gsyncd failed
> here because of the compatibility issue where gsyncd was still not upgraded
> where as glusterfs-server was and this failure was mainly because of change
> in the mem type enum. We have seen a similar issue for RDMA as well
> (probably a year back). So to be very generic this can happen in any
> upgrade path from one version to another where new mem type is introduced.
> We have seen this from 3.7.8 to 3.7.12 and 3.8. People upgrading from 3.6
> to 3.7/3.8 will also experience this issue.
>
> Till we work on this fix, I suggest all the release managers to highlight
> this in the release note of the latest releases with the following work
> around after yum update:
>
> 1. grep -irns "geo-replication module not working as desired" /var/log/glusterfs/etc-glusterfs-glusterd.vol.log | wc -l
>
>  If the output is non-zero, then go to step 2 else follow the rest of the steps as per the guide.
>
> 2.Check if glusterd instance is running or not by 'ps aux | grep glusterd', if it is, then stop the glusterd service.
>
>  3. glusterd --xlator-option *.upgrade=on -N
>
> and then proceed ahead with rest of the steps as per the guide.
>
> Thoughts?
>
> P.S : this email is limited to maintainers till we decide on the approach
> to highlight this issues to the users
>
>
> --
> Atin
> Sent from iPhone
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/maintainers/attachments/20160712/abce685d/attachment-0001.html>