<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><br class=""></div><div class="">Hi Sunil,</div><div class=""><br class=""></div><div class="">many thanks to you too.</div><div class="">I will follow your suggestions and the guide for upgrading to 3.12</div><div class=""><br class=""></div><div class="">Crossing fingers :-)</div><div class="">Regards,</div><div class="">Mauro</div><br class=""><div><blockquote type="cite" class=""><div class="">Il giorno 15 set 2018, alle ore 11:57, Sunil Kumar Heggodu Gopala Acharya <<a href="mailto:sheggodu@redhat.com" class="">sheggodu@redhat.com</a>> ha scritto:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div dir="ltr" class=""><div dir="ltr" class=""><div class="">Hi Mauro,<br class=""><br class=""></div>As Nithya highlighted FALLOCATE support for EC volumes went in 3.11 as part of <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1454686" class="">https://bugzilla.redhat.com/show_bug.cgi?id=1454686</a>. Hence, upgrading to 3.12 as suggested before would be a right move.<br class=""><br class=""></div><div class="">Here is the documentation for upgrading to 3.12: <a href="https://docs.gluster.org/en/latest/Upgrade-Guide/upgrade_to_3.12/" class="">https://docs.gluster.org/en/latest/Upgrade-Guide/upgrade_to_3.12/</a><br class=""></div></div></div><div class="gmail_extra"><br clear="all" class=""><div class=""><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr" class="">Regards,<br class=""><div style="font-weight: bold; margin: 0px; padding: 0px; font-size: 14px; text-transform: uppercase;" class=""><span class="">Sunil kumar</span> <span class="">Acharya</span></div><p style="font-weight:normal;font-size:10px;margin:0px 0px 4px;text-transform:uppercase" class=""><span class="">Senior Software Engineer</span></p><div style="font-weight: normal; margin: 0px; font-size: 10px; color: rgb(153, 153, 153);" class=""><a style="color:#0088ce;font-size:10px;margin:0;text-decoration:none;font-family:'overpass',sans-serif" href="https://www.redhat.com/" target="_blank" class="">Red Hat <span class=""><br class=""><br class=""></span></a></div><p style="font-weight:normal;margin:0px 0px 6px;font-size:10px;color:rgb(153,153,153)" class="">
<span href="tel:+91-8067935170" class="">T: <a href="http://redhatemailsignature-marketing.itos.redhat.com/" style="color:#0088ce;font-size:10px;margin:0;text-decoration:none;font-family:'overpass',sans-serif" target="_blank" class="">+91-8067935170</a> </span>
</p>
<a href="https://red.ht/sig" target="_blank" class=""> <img src="https://www.redhat.com/profiles/rh/themes/redhatdotcom/img/logo-red-hat-black.png" width="90" height="auto" class=""></a>
<div class=""><a href="https://redhat.com/trusted" style="text-decoration:none;color:#c00;font-weight:bold" target="_blank" class="">TRIED. TESTED. TRUSTED.</a></div><br class=""></div></div></div>
<br class=""><div class="gmail_quote">On Sat, Sep 15, 2018 at 3:42 AM, Mauro Tridici <span dir="ltr" class=""><<a href="mailto:mauro.tridici@cmcc.it" target="_blank" class="">mauro.tridici@cmcc.it</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><div class=""><br class=""></div><div class="">Hi Nithya,</div><div class=""><br class=""></div><div class="">thank you very much for your answer.</div><div class="">I will wait for @Sunil opinion too before starting the upgrade procedure.</div><div class=""><br class=""></div><div class="">Since it will be the first upgrade of our Gluster cluster, I would like to know if it could be a “virtually dangerous" procedure and if it will be the risk of losing data :-) </div><div class="">Unfortunately, I can’t do a preventive copy of the volume data in another location.</div><div class="">If it is possible, could you please illustrate the right steps needed to complete the upgrade procedure from the 3.10.5 to the 3.12 version?</div><div class=""><br class=""></div><div class="">Thank you again, Nithya.</div><div class="">Thank you to all of you for the help!</div><div class=""><br class=""></div><div class="">Regards,</div><div class="">Mauro</div><div class=""><div class="h5"><br class=""><div class=""><blockquote type="cite" class=""><div class="">Il giorno 14 set 2018, alle ore 16:59, Nithya Balachandran <<a href="mailto:nbalacha@redhat.com" target="_blank" class="">nbalacha@redhat.com</a>> ha scritto:</div><br class="m_-7509393663119327190Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div dir="ltr" class="">Hi Mauro,<div class=""><br class=""></div><div class=""><br class=""></div><div class="">The rebalance code started using fallocate in 3.10.5 (<a href="https://bugzilla.redhat.com/show_bug.cgi?id=1473132" target="_blank" class="">https://bugzilla.redhat.com/<wbr class="">show_bug.cgi?id=1473132</a>) which works fine on replicated volumes. However, we neglected to test this with EC volumes on 3.10. Once we discovered the issue, the EC fallocate implementation was made available in 3.11.</div><div class=""><br class=""></div><div class="">At this point, I'm afraid the only option I see is to upgrade to at least 3.12.</div><div class=""><br class=""></div><div class="">@Sunil, do you have anything to add?</div><div class=""><br class=""></div><div class="">Regards,</div><div class="">Nithya</div></div></div><div class="gmail_extra"><br class=""><div class="gmail_quote">On 13 September 2018 at 18:34, Mauro Tridici <span dir="ltr" class=""><<a href="mailto:mauro.tridici@cmcc.it" target="_blank" class="">mauro.tridici@cmcc.it</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><div class=""><br class=""></div><div class="">Hi Nithya,</div><div class=""><br class=""></div><div class="">thank you for involving EC group.</div><div class="">I will wait for your suggestions.</div><div class=""><br class=""></div><div class="">Regards,</div><div class="">Mauro</div><div class=""><div class="m_-7509393663119327190h5"><br class=""><div class=""><blockquote type="cite" class=""><div class="">Il giorno 13 set 2018, alle ore 13:38, Nithya Balachandran <<a href="mailto:nbalacha@redhat.com" target="_blank" class="">nbalacha@redhat.com</a>> ha scritto:</div><br class="m_-7509393663119327190m_821703117890918883Apple-interchange-newline"><div class=""><div dir="ltr" class="">This looks like an issue because rebalance switched to using fallocate which EC did not have implemented at that point.<div class=""><br class=""></div><div class="">@Pranith, @Ashish, which version of gluster had support for fallocate in EC?</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">Regards,</div><div class="">Nithya</div></div><div class="gmail_extra"><br class=""><div class="gmail_quote">On 12 September 2018 at 19:24, Mauro Tridici <span dir="ltr" class=""><<a href="mailto:mauro.tridici@cmcc.it" target="_blank" class="">mauro.tridici@cmcc.it</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><div class=""><div class="">Dear All,</div><div class=""><br class=""></div><div class="">I recently added 3 servers (each one with 12 bricks) to an existing Gluster Distributed Disperse Volume.</div><div class="">Volume extension has been completed without error and I already executed the rebalance procedure with fix-layout option with no problem.</div><div class="">I just launched the rebalance procedure without fix-layout option, but, as you can see in the output below, I noticed that some failures have been detected.</div><div class=""><br class=""></div><div class=""><font size="2" face="Courier" class="">[root@s01 glusterfs]# gluster v rebalance tier2 status</font></div><div class=""><font size="2" face="Courier" class=""> Node Rebalanced-files size scanned failures skipped status run time in h:m:s</font></div><div class=""><font size="2" face="Courier" class=""> --------- ----------- ----------- ----------- ----------- ----------- ------------ --------------</font></div><div class=""><font size="2" face="Courier" class=""> localhost 71176 3.2MB 2137557 1530391 8128 in progress 13:59:05</font></div><div class=""><font size="2" face="Courier" class=""> s02-stg 0 0Bytes 0 0 0 completed 11:53:28</font></div><div class=""><font size="2" face="Courier" class=""> s03-stg 0 0Bytes 0 0 0 completed 11:53:32</font></div><div class=""><font size="2" face="Courier" class=""> s04-stg 0 0Bytes 0 0 0 completed 0:00:06</font></div><div class=""><font size="2" face="Courier" class=""> s05-stg 15 0Bytes 17055 0 18 completed 10:48:01</font></div><div class=""><font size="2" face="Courier" class=""> s06-stg 0 0Bytes 0 0 0 completed 0:00:06</font></div><div class=""><font size="2" face="Courier" class="">Estimated time left for rebalance to complete : 0:46:53</font></div><div class=""><font size="2" face="Courier" class="">volume rebalance: tier2: success</font></div><div class=""><br class=""></div><div class="">In the volume rebalance log file, I detected a lot of error messages similar to the following ones:</div><div class=""><br class=""></div><div class=""><font size="2" face="Courier" class="">[2018-09-12 13:15:50.756703] E [MSGID: 0] [dht-rebalance.c:1696:dht_migr<wbr class="">ate_file] 0-tier2-dht: Create dst failed on - tier2-disperse-6 for file - /CSP/sp1/CESM/archive/sps_2005<wbr class="">08_003/atm/hist/postproc/<a href="http://sps_200508_003.cam.h0.2005-12_grid.nc/" target="_blank" class="">sps_2<wbr class="">00508_003.cam.h0.2005-12_grid.<wbr class="">nc</a></font></div><div class=""><font size="2" face="Courier" class="">[2018-09-12 13:15:50.757025] E [MSGID: 109023] [dht-rebalance.c:2733:gf_defra<wbr class="">g_migrate_single_file] 0-tier2-dht: migrate-data failed for /CSP/sp1/CESM/archive/sps_2005<wbr class="">08_003/atm/hist/postproc/<a href="http://sps_200508_003.cam.h0.2005-12_grid.nc/" target="_blank" class="">sps_2<wbr class="">00508_003.cam.h0.2005-12_grid.<wbr class="">nc</a></font></div><div class=""><font size="2" face="Courier" class="">[2018-09-12 13:15:50.759183] E [MSGID: 109023] [dht-rebalance.c:844:__dht_reb<wbr class="">alance_create_dst_file] 0-tier2-dht: fallocate failed for /CSP/sp1/CESM/archive/sps_2005<wbr class="">08_003/atm/hist/postproc/<a href="http://sps_200508_003.cam.h0.2005-09_grid.nc/" target="_blank" class="">sps_2<wbr class="">00508_003.cam.h0.2005-09_grid.<wbr class="">nc</a> on tier2-disperse-9 (Operation not supported)</font></div><div class=""><font size="2" face="Courier" class="">[2018-09-12 13:15:50.759206] E [MSGID: 0] [dht-rebalance.c:1696:dht_migr<wbr class="">ate_file] 0-tier2-dht: Create dst failed on - tier2-disperse-9 for file - /CSP/sp1/CESM/archive/sps_2005<wbr class="">08_003/atm/hist/postproc/<a href="http://sps_200508_003.cam.h0.2005-09_grid.nc/" target="_blank" class="">sps_2<wbr class="">00508_003.cam.h0.2005-09_grid.<wbr class="">nc</a></font></div><div class=""><font size="2" face="Courier" class="">[2018-09-12 13:15:50.759536] E [MSGID: 109023] [dht-rebalance.c:2733:gf_defra<wbr class="">g_migrate_single_file] 0-tier2-dht: migrate-data failed for /CSP/sp1/CESM/archive/sps_2005<wbr class="">08_003/atm/hist/postproc/<a href="http://sps_200508_003.cam.h0.2005-09_grid.nc/" target="_blank" class="">sps_2<wbr class="">00508_003.cam.h0.2005-09_grid.<wbr class="">nc</a></font></div><div class=""><font size="2" face="Courier" class="">[2018-09-12 13:15:50.777219] E [MSGID: 109023] [dht-rebalance.c:844:__dht_reb<wbr class="">alance_create_dst_file] 0-tier2-dht: fallocate failed for /CSP/sp1/CESM/archive/sps_2005<wbr class="">08_003/atm/hist/postproc/<a href="http://sps_200508_003.cam.h0.2006-01_grid.nc/" target="_blank" class="">sps_2<wbr class="">00508_003.cam.h0.2006-01_grid.<wbr class="">nc</a> on tier2-disperse-10 (Operation not supported)</font></div><div class=""><font size="2" face="Courier" class="">[2018-09-12 13:15:50.777241] E [MSGID: 0] [dht-rebalance.c:1696:dht_migr<wbr class="">ate_file] 0-tier2-dht: Create dst failed on - tier2-disperse-10 for file - /CSP/sp1/CESM/archive/sps_2005<wbr class="">08_003/atm/hist/postproc/<a href="http://sps_200508_003.cam.h0.2006-01_grid.nc/" target="_blank" class="">sps_2<wbr class="">00508_003.cam.h0.2006-01_grid.<wbr class="">nc</a></font></div><div class=""><font size="2" face="Courier" class="">[2018-09-12 13:15:50.777676] E [MSGID: 109023] [dht-rebalance.c:2733:gf_defra<wbr class="">g_migrate_single_file] 0-tier2-dht: migrate-data failed for /CSP/sp1/CESM/archive/sps_2005<wbr class="">08_003/atm/hist/postproc/<a href="http://sps_200508_003.cam.h0.2006-01_grid.nc/" target="_blank" class="">sps_2<wbr class="">00508_003.cam.h0.2006-01_grid.<wbr class="">nc</a></font></div><div class=""><br class=""></div><div class="">Could you please help me to understand what is happening and how to solve it?</div><div class=""><br class=""></div><div class="">Our Gluster implementation is based on Gluster v.3.10.5</div><div class=""><br class=""></div><div class="">Thank you in advance,</div><div class="">Mauro</div>
</div>
<br class=""></div><br class="">______________________________<wbr class="">_________________<br class="">
Gluster-users mailing list<br class="">
<a href="mailto:Gluster-users@gluster.org" target="_blank" class="">Gluster-users@gluster.org</a><br class="">
<a href="https://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank" class="">https://lists.gluster.org/mail<wbr class="">man/listinfo/gluster-users</a><br class=""></blockquote></div><br class=""></div>
</div></blockquote></div><br class=""></div></div><div class="">
<span class="m_-7509393663119327190m_821703117890918883Apple-style-span" style="border-collapse:separate;font-family:Helvetica;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:normal;font-variant-alternates:normal;font-variant-east-asian:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:-webkit-auto;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><span class="m_-7509393663119327190m_821703117890918883Apple-style-span" style="border-collapse:separate;font-family:Helvetica;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:normal;font-variant-alternates:normal;font-variant-east-asian:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:-webkit-auto;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><div class=""><br class="m_-7509393663119327190m_821703117890918883Apple-interchange-newline">-------------------------</div><div class="">Mauro Tridici</div><div class=""><br class=""></div><div class="">Fondazione CMCC</div><div class="">CMCC Supercomputing Center</div><div class="">presso Complesso Ecotekne - Università del Salento -</div><div class="">Strada Prov.le Lecce - Monteroni sn</div><div class="">73100 Lecce IT</div><div class=""><a href="http://www.cmcc.it/" target="_blank" class="">http://www.cmcc.it</a></div><div class=""><br class=""></div><div class="">mobile: (+39) 327 5630841</div><div class="">email: <a href="mailto:mauro.tridici@cmcc.it" target="_blank" class="">mauro.tridici@cmcc.it</a></div></span></span>
</div>
<br class=""></div></blockquote></div><br class=""></div>
</div></blockquote></div><br class=""><div class="">
<span class="m_-7509393663119327190Apple-style-span" style="border-collapse: separate; font-family: Helvetica; font-style: normal; font-variant-ligatures: normal; font-variant-position: normal; font-variant-caps: normal; font-variant-numeric: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;"><span class="m_-7509393663119327190Apple-style-span" style="border-collapse: separate; font-family: Helvetica; font-style: normal; font-variant-ligatures: normal; font-variant-position: normal; font-variant-caps: normal; font-variant-numeric: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;"><div class=""><br class="m_-7509393663119327190Apple-interchange-newline">-------------------------</div><div class="">Mauro Tridici</div><div class=""><br class=""></div><div class="">Fondazione CMCC</div><div class="">CMCC Supercomputing Center</div><div class="">presso Complesso Ecotekne - Università del Salento -</div><div class="">Strada Prov.le Lecce - Monteroni sn</div><div class="">73100 Lecce IT</div><div class=""><a href="http://www.cmcc.it/" target="_blank" class="">http://www.cmcc.it</a></div><div class=""><br class=""></div><div class="">mobile: (+39) 327 5630841</div><div class="">email: <a href="mailto:mauro.tridici@cmcc.it" target="_blank" class="">mauro.tridici@cmcc.it</a></div></span></span>
</div>
<br class=""></div></div></div></blockquote></div><br class=""></div>
</div></blockquote></div><br class=""><div class="">
<span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div class=""><br class="Apple-interchange-newline">-------------------------</div><div class="">Mauro Tridici</div><div class=""><br class=""></div><div class="">Fondazione CMCC</div><div class="">CMCC Supercomputing Center</div><div class="">presso Complesso Ecotekne - Università del Salento -</div><div class="">Strada Prov.le Lecce - Monteroni sn</div><div class="">73100 Lecce IT</div><div class=""><a href="http://www.cmcc.it" class="">http://www.cmcc.it</a></div><div class=""><br class=""></div><div class="">mobile: (+39) 327 5630841</div><div class="">email: <a href="mailto:mauro.tridici@cmcc.it" class="">mauro.tridici@cmcc.it</a></div></span></span>
</div>
<br class=""></body></html>