[Gluster-users] Rolling upgrade from 3.6.3 to 3.10.5

Fri Aug 25 12:56:02 UTC 2017

Hi Diego,

Just to clarify, so did you do an offline upgrade with an existing cluster
(3.6.x => 3.10.x)?
Thanks.

On Fri, Aug 25, 2017 at 8:42 PM, Diego Remolina <dijuremo at gmail.com> wrote:

> I was never able to go from 3.6.x to 3.7.x without downtime. Then
> 3.7.x did not work well for me, so I stuck with 3.6.x until recently.
> I went from 3.6.x to 3.10.x but downtime was scheduled.
>
> Diego
>
> On Fri, Aug 25, 2017 at 8:25 AM, Yong Tseng <yongtw123 at gmail.com> wrote:
> > Hi Diego,
> >
> > Thanks for the information. I tried only setting 'allow-insecure on' but
> > nada.
> > The sentence "If you are using GlusterFS version 3.4.x or below, you can
> > upgrade it to following" in documentation is surely misleading.
> > So would you suggest creating a new 3.10 cluster from scratch then
> rsync(?)
> > the data from old cluster to the new?
> >
> >
> > On Fri, Aug 25, 2017 at 7:53 PM, Diego Remolina <dijuremo at gmail.com>
> wrote:
> >>
> >> You cannot do a rolling upgrade from 3.6.x to 3.10.x You will need
> >> downtime.
> >>
> >> Even 3.6 to 3.7 was not possible... see some references to it below:
> >>
> >> https://marc.info/?l=gluster-users&m=145136214452772&w=2
> >> https://gluster.readthedocs.io/en/latest/release-notes/3.7.1/
> >>
> >> # gluster volume set <volname> server.allow-insecure on Edit
> >> /etc/glusterfs/glusterd.vol to contain this line: option
> >> rpc-auth-allow-insecure on
> >>
> >> Post 1, restarting the volume would be necessary:
> >>
> >> # gluster volume stop <volname>
> >> # gluster volume start <volname>
> >>
> >>
> >> HTH,
> >>
> >> Diego
> >>
> >> On Fri, Aug 25, 2017 at 7:46 AM, Yong Tseng <yongtw123 at gmail.com>
> wrote:
> >> > Hi all,
> >> >
> >> > I'm currently in process of upgrading a replicated cluster (1 x 4)
> from
> >> > 3.6.3 to 3.10.5. The nodes run CentOS 6. However after upgrading the
> >> > first
> >> > node, the said node fails to connect to other peers (as seen via
> >> > 'gluster
> >> > peer status'), but somehow other non-upgraded peers can still see the
> >> > upgraded peer as connected.
> >> >
> >> > Writes to the Gluster volume via local mounts of non-upgraded peers
> are
> >> > replicated to the upgraded peer, but I can't write via the upgraded
> peer
> >> > as
> >> > the local mount seems forced to read-only.
> >> >
> >> > Launching heal operations from non-upgraded peers will output 'Commit
> >> > failed
> >> > on <upgraded peer IP>. Please check log for details'.
> >> >
> >> > In addition, during upgrade process there were warning messages about
> my
> >> > old
> >> > vol files renamed with .rpmsave extension. I tried starting Gluster
> with
> >> > my
> >> > old vol files but the problem persisted. I tried generating new vol
> >> > files
> >> > with 'glusterd --xlator-option "*.upgrade=on" -N', still no avail.
> >> >
> >> > Also I checked the brick log it had several messages about "failed to
> >> > get
> >> > client opversion". I don't know if this is pertinent. Could it be that
> >> > the
> >> > upgraded node cannot connect to older nodes but still can receive
> >> > instructions from them?
> >> >
> >> > Below are command outputs; some data are masked.
> >> > I'd provide more information if required.
> >> > Thanks in advance.
> >> >
> >> > ===> 'gluster volume status' ran on non-upgraded peers
> >> >
> >> > Status of volume: gsnfs
> >> > Gluster process                                         Port    Online
> >> > Pid
> >> >
> >> > ------------------------------------------------------------
> ------------------
> >> > Brick gs-nfs01:/ftpdata                                 49154   Y
> >> > 2931
> >> > Brick gs-nfs02:/ftpdata                                 49152   Y
> >> > 29875
> >> > Brick gs-nfs03:/ftpdata                                 49153   Y
> >> > 6987
> >> > Brick gs-nfs04:/ftpdata                                 49153   Y
> >> > 24768
> >> > Self-heal Daemon on localhost                           N/A     Y
> >> > 2938
> >> > Self-heal Daemon on gs-nfs04                            N/A     Y
> >> > 24788
> >> > Self-heal Daemon on gs-nfs03                            N/A     Y
> >> > 7007
> >> > Self-heal Daemon on <IP>                      N/A     Y       29866
> >> >
> >> > Task Status of Volume gsnfs
> >> >
> >> > ------------------------------------------------------------
> ------------------
> >> > There are no active volume tasks
> >> >
> >> >
> >> >
> >> > ===> 'gluster volume status' on upgraded peer
> >> >
> >> > Gluster process                             TCP Port  RDMA Port
> Online
> >> > Pid
> >> >
> >> > ------------------------------------------------------------
> ------------------
> >> > Brick gs-nfs02:/ftpdata                     49152     0          Y
> >> > 29875
> >> > Self-heal Daemon on localhost               N/A       N/A        Y
> >> > 29866
> >> >
> >> > Task Status of Volume gsnfs
> >> >
> >> > ------------------------------------------------------------
> ------------------
> >> > There are no active volume tasks
> >> >
> >> >
> >> >
> >> > ===> 'gluster peer status' on non-upgraded peer
> >> >
> >> > Number of Peers: 3
> >> >
> >> > Hostname: gs-nfs03
> >> > Uuid: 4c1544e6-550d-481a-95af-2a1da32d10ad
> >> > State: Peer in Cluster (Connected)
> >> >
> >> > Hostname: <IP>
> >> > Uuid: 17d554fd-9181-4b53-9521-55acf69ac35f
> >> > State: Peer in Cluster (Connected)
> >> > Other names:
> >> > gs-nfs02
> >> >
> >> > Hostname: gs-nfs04
> >> > Uuid: c6d165e6-d222-414c-b57a-97c64f06c5e9
> >> > State: Peer in Cluster (Connected)
> >> >
> >> >
> >> >
> >> > ===> 'gluster peer status' on upgraded peer
> >> >
> >> > Number of Peers: 3
> >> >
> >> > Hostname: gs-nfs03
> >> > Uuid: 4c1544e6-550d-481a-95af-2a1da32d10ad
> >> > State: Peer in Cluster (Disconnected)
> >> >
> >> > Hostname: gs-nfs01
> >> > Uuid: 90d3ed27-61ac-4ad3-93a9-3c2b68f41ecf
> >> > State: Peer in Cluster (Disconnected)
> >> > Other names:
> >> > <IP>
> >> >
> >> > Hostname: gs-nfs04
> >> > Uuid: c6d165e6-d222-414c-b57a-97c64f06c5e9
> >> > State: Peer in Cluster (Disconnected)
> >> >
> >> >
> >> > --
> >> > - Yong
> >> >
> >> > _______________________________________________
> >> > Gluster-users mailing list
> >> > Gluster-users at gluster.org
> >> > http://lists.gluster.org/mailman/listinfo/gluster-users
> >
> >
> >
> >
> > --
> > - Yong
>

-- 
- Yong
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20170825/5e7806e2/attachment.html>