<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>I had replied (reply-to-all) to this email y'day but it I don't
see it on the list. Anyway pasting it again:<br>
</p>
<br>
<tt></tt><tt>
</tt>
<div class="moz-cite-prefix"><tt>On 09/21/2017 10:03 AM, Ravishankar
N wrote:</tt><tt><br>
</tt>
</div>
<tt>
</tt>
<blockquote type="cite"
cite="mid:8a40a601-f325-551d-f8cd-fae71b70bf4d@redhat.com"><tt><br>
</tt>
<div class="moz-cite-prefix"><tt>On 09/20/2017 01:45 PM, Martin
Toth wrote:</tt><tt><br>
</tt> </div>
<blockquote type="cite"
cite="mid:65389456-6A62-462A-82F9-DCFF93B8975D@gmail.com">
<div class=""><tt><b class="">My Questions</b></tt></div>
<div class=""><tt>- is heal and rebalance necessary in order to
upgrade replica 2 to replica 3 ?</tt></div>
</blockquote>
<tt> No, `gluster volume add-brick voname replica 3
node3.san:/tank/gluster/brick1' should automatically trigger
healing in gluster 3.12 (actually earlier than 3.12 but I don't
remember which release the automatic healing was added on
add-brick command). Ensure that `gluster volume heal volname
info` eventually becomes zero.</tt><tt><br>
</tt>
<blockquote type="cite"
cite="mid:65389456-6A62-462A-82F9-DCFF93B8975D@gmail.com">
<div class=""><tt>- is this upgrade procedure OK ? What
more/else should I do in order to do this upgrade correctly
?</tt></div>
<div class=""><tt><br class="">
</tt> </div>
</blockquote>
<tt> Looks okay in theory (I haven't tried out a 3.7 to 3.12
upgrade). It might be good to check there are no pending heals
before you stop the old nodes for the upgrade.</tt><tt><br>
</tt><tt> -Ravi</tt></blockquote>
<br>
-Ravi<br>
<br>
<div class="moz-cite-prefix">On 09/21/2017 09:50 PM, Amye Scavarda
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CACDUr8VRy6f+_GCwJ7h4yrRhHj8-DU8nfnhr_B+m=DEvEP=p1Q@mail.gmail.com">
<pre wrap="">Just making sure this gets through.
---------- Forwarded message ----------
From: Martin Toth <a class="moz-txt-link-rfc2396E" href="mailto:snowmailer@gmail.com"><snowmailer@gmail.com></a>
Date: Thu, Sep 21, 2017 at 9:17 AM
Subject: Upgrade Gluster 3.7 to 3.12 and add 3rd replica [howto/help]
To: <a class="moz-txt-link-abbreviated" href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>
Cc: Marek Toth <a class="moz-txt-link-rfc2396E" href="mailto:scorpion909@gmail.com"><scorpion909@gmail.com></a>, <a class="moz-txt-link-abbreviated" href="mailto:amye@redhat.com">amye@redhat.com</a>
Hello all fellow GlusterFriends,
I would like you to comment / correct my upgrade procedure steps on
replica 2 volume of 3.7.x gluster.
Than I would like to change replica 2 to replica 3 in order to correct
quorum issue that Infrastructure currently has.
Infrastructure setup:
- all clients running on same nodes as servers (FUSE mounts)
- under gluster there is ZFS pool running as raidz2 with SSD ZLOG/ZIL cache
- all two hypervisor running as GlusterFS nodes and also Qemu compute
nodes (Ubuntu 16.04 LTS)
- we are running Qemu VMs that accesses VMs disks via gfapi (Opennebula)
- we currently run : 1x2 , Type: Replicate volume
Current Versions :
glusterfs-* [package] 3.7.6-1ubuntu1
qemu-* [package] 2.5+dfsg-5ubuntu10.2glusterfs3.7.14xenial1
What we need : (New versions)
- upgrade GlusterFS to 3.12 LTM version (Ubuntu 16.06 LTS packages are
EOL - see <a class="moz-txt-link-freetext" href="https://www.gluster.org/community/release-schedule/">https://www.gluster.org/community/release-schedule/</a>)
- I want to use
<a class="moz-txt-link-freetext" href="https://launchpad.net/~gluster/+archive/ubuntu/glusterfs-3.12">https://launchpad.net/~gluster/+archive/ubuntu/glusterfs-3.12</a> as
package repository for 3.12
- upgrade Qemu (with build-in support for libgfapi) -
<a class="moz-txt-link-freetext" href="https://launchpad.net/~monotek/+archive/ubuntu/qemu-glusterfs-3.12">https://launchpad.net/~monotek/+archive/ubuntu/qemu-glusterfs-3.12</a>
- (sadly Ubuntu has packages build without libgfapi support)
- add third node to replica setup of volume (this is probably most
dangerous operation)
Backup Phase
- backup "NFS storage” - raw DATA that runs on VMs
- stop all running VMs
- backup all running VMs (Qcow2 images) outside of gluster
Upgrading Gluster Phase
- killall glusterfs glusterfsd glusterd (on every server)
(this should stop all gluster services - server and client as it runs
on same nodes)
- install new Gluster Server and Client packages from repository
mentioned upper (on every server)
- install new Monotek's qemu glusterfs package with gfapi enabled
support (on every server)
- /etc/init.d/glusterfs-server start (on every server)
- /etc/init.d/glusterfs-server status - verify that all runs ok (on
every server)
- check :
- gluster volume info
- gluster volume status
- check gluster FUSE clients, if mounts working as expected
- test if various VMs are able tu boot and run as expected (if
libgfapi works in Qemu)
- reboot all nodes - do system upgrade of packages
- test and check again
Adding third node to replica 2 setup (replica 2 => replica 3)
(volumes will be mounted and up after upgrade and we tested VMs are
able to be served with libgfapi = upgrade of gluster sucessfuly
completed)
(next we extend replica 2 to replica 3 while volumes are mounted but
no data is touched = no running VMs, only glusterfs servers and
clients on nodes)
- issue command : gluster volume add-brick volume replica 3
node3.san:/tank/gluster/brick1 (on new single node - node3)
so we change :
Bricks:
Brick1: node1.san:/tank/gluster/brick1
Brick2: node2.san:/tank/gluster/brick1
to :
Bricks:
Brick1: node1.san:/tank/gluster/brick1
Brick2: node2.san:/tank/gluster/brick1
Brick3: node3.san:/tank/gluster/brick1
- check gluster status
- (is rebalance / heal required here ?)
- start all VMs and start celebration :)
My Questions
- is heal and rebalance necessary in order to upgrade replica 2 to replica 3 ?
- is this upgrade procedure OK ? What more/else should I do in order
to do this upgrade correctly ?
Many thanks to all for support. Hope my little preparation howto will
help others to solve same situation.
Best Regards,
Martin
</pre>
</blockquote>
<br>
</body>
</html>