[Bugs] [Bug 1544461] 3.8 -> 3.10 rolling upgrade fails (same for 3.12 or 3.13) on Ubuntu 14

bugzilla at redhat.com bugzilla at redhat.com
Wed Feb 14 09:43:48 UTC 2018


https://bugzilla.redhat.com/show_bug.cgi?id=1544461



--- Comment #11 from Marc <alexandrumarcu at gmail.com> ---
(In reply to Atin Mukherjee from comment #10)
> (In reply to Marc from comment #9)
> > (In reply to Atin Mukherjee from comment #8)
> > > Sorry to hear that the suggested work around didn't work :-/
> > > 
> > > Please try this out:
> > > 
> > > 1. bring down glusterd instance on 3.8.15 server
> > > 
> > > 2. from 3.10.10 server remove tier-enabled=0 entry but don't restart
> > > glusterd service.
> > > 
> > > 3. bring up glusterd instance on 3.10.10 server
> > > 
> > > 4. check 'gluster peer status' and 'gluster volume status' output on both
> > > the nodes
> > 
> > Hi Atin,
> > 
> > I didn't exactly understand the above steps, are you referring at step (1)
> > to stop the 3.10.10 instead of 3.8.15 server? Otherwise (2) step is done
> > already (tier-enabled=0 entry removed) and at step (3) ... i never stopped
> > the 3.10.10 instance so that i could bring up. The 'gluster peer status'
> > will show 'peer rejected( disconnected) if i stop the 3.8.15 server and i
> > run the command from 3.10.10.
> 
> I am requesting not to stop the glusterd instance running with 3.10.10. Just
> take out tier-enabled=0 entry from all the volume info files and then
> restart glusterd instance on 3.8.15. Please do let me know if you have any
> other confusions.

Ok, now i think i understood what you mean but i think i already tried that
with no improvement.

1. Restarted the only 3.10.10 server so the "tier-enabled=0" to be regenerated
back.
2. Removed "tier-enabled=0" without restarting GlusterFS service.
3. Restarted GlusterFS service on all other 4 x 3.8.15 GlusterFS servers.

On the only 3.10.10 server:

root at 2-gls-dus21-ci-efood-real-de:/var/lib/glusterd/vols/gluster_volume# 
gluster volume status
Status of volume: gluster_volume
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 2-gls-dus21-ci-efood-real-de.openstac
klocal:/export_vdb                          49152     0          Y       26586
Self-heal Daemon on localhost               N/A       N/A        Y       5776

Task Status of Volume gluster_volume
------------------------------------------------------------------------------
There are no active volume tasks


root at 2-gls-dus21-ci-efood-real-de:/var/lib/glusterd/vols/gluster_volume#
gluster peer status
Number of Peers: 4

Hostname: 3-gls-dus10-ci-efood-real-de.openstack.local
Uuid: 3d141235-9b93-4798-8e03-82a758216b0b
State: Peer Rejected (Connected)

Hostname: 1-gls-dus21-ci-efood-real-de.openstacklocal
Uuid: 7488286f-6bfa-46f8-bc50-9ee815e96c66
State: Peer Rejected (Connected)

Hostname: 1-gls-dus10-ci-efood-real-de.openstack.local
Uuid: 00839049-2ade-48f8-b5f3-66db0e2b9377
State: Peer Rejected (Connected)

Hostname: 2-gls-dus10-ci-efood-real-de.openstack.local
Uuid: 1617cd54-9b2a-439e-9aa6-30d4ecf303f8
State: Peer Rejected (Connected)

On the other 4 x 3.8.15 servers:


root at 2-gls-dus10-ci-efood-real-de:/home/ubuntu# gluster volume status
Status of volume: gluster_volume
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 2-gls-dus10-ci-efood-real-de.openstac
k.local:/export_vdb                         49153     0          Y       30521
Brick 1-gls-dus10-ci-efood-real-de.openstac
k.local:/export_vdb                         49152     0          Y       1663
Brick 1-gls-dus21-ci-efood-real-de:/export_
vdb                                         49152     0          Y       2322
Brick 3-gls-dus10-ci-efood-real-de.openstac
k.local:/export_vdb                         49153     0          Y       10854
Self-heal Daemon on localhost               N/A       N/A        Y       31225
Self-heal Daemon on 3-gls-dus10-ci-efood-re
al-de.openstack.local                       N/A       N/A        Y       10567
Self-heal Daemon on 1-gls-dus21-ci-efood-re
al-de.openstacklocal                        N/A       N/A        Y       31437
Self-heal Daemon on 1-gls-dus10-ci-efood-re
al-de.openstack.local                       N/A       N/A        Y       15453

Task Status of Volume gluster_volume
------------------------------------------------------------------------------
There are no active volume tasks

root at 2-gls-dus10-ci-efood-real-de:/home/ubuntu# gluster peer status
Number of Peers: 4

Hostname: 3-gls-dus10-ci-efood-real-de.openstack.local
Uuid: 3d141235-9b93-4798-8e03-82a758216b0b
State: Peer in Cluster (Connected)

Hostname: 1-gls-dus21-ci-efood-real-de.openstacklocal
Uuid: 7488286f-6bfa-46f8-bc50-9ee815e96c66
State: Peer in Cluster (Connected)

Hostname: 1-gls-dus10-ci-efood-real-de.openstack.local
Uuid: 00839049-2ade-48f8-b5f3-66db0e2b9377
State: Peer in Cluster (Connected)

Hostname: 2-gls-dus21-ci-efood-real-de.openstacklocal
Uuid: 0c698b11-9078-441a-9e7f-442befeef7a9
State: Peer Rejected (Connected)


PS: I only removed the "tier-enabled=0" from here 
/var/lib/glusterd/vols/gluster_volume/info . If is there any place where i need
to edit please let me know.
PSS: This is not urgent for me, this is a test environment and i can live with
4xgluster until an official fix is released (unless you need me to test
different scenarios for testing). Can you tell what is the usual timeframe
before a fix like this gets released?

Thank you

-- 
You are receiving this mail because:
You are on the CC list for the bug.
Unsubscribe from this bug https://bugzilla.redhat.com/token.cgi?t=v31tkMDtSV&a=cc_unsubscribe


More information about the Bugs mailing list