<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Feb 5, 2019 at 8:43 PM Nithya Balachandran <<a href="mailto:nbalacha@redhat.com">nbalacha@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, 5 Feb 2019 at 17:26, deepu srinivasan <<a href="mailto:sdeepugd@gmail.com" target="_blank">sdeepugd@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">HI Nithya<div>We have a test gluster setup.We are testing the rebalancing option of gluster. So we started the volume which have 1x3 brick with some data on it .<br>command : <span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0);font-family:Menlo;font-size:13px">gluster volume create test-volume replica 3 192.168.xxx.xx1:/home/data/repl 192.168.xxx.xx2:/home/data/repl 192.168.xxx.xx3:/home/data/repl.</span></div><div><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0);font-family:Menlo;font-size:13px"><br></span></div><div><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0);font-family:Menlo;font-size:13px">Now we tried to expand the cluster storage by adding three more bricks. </span></div><div><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0);font-family:Menlo;font-size:13px">command : </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0);font-family:Menlo;font-size:13px">gluster volume add-brick test-volume 192.168.</span><span style="color:rgb(0,0,0);font-family:Menlo;font-size:13px;font-variant-ligatures:no-common-ligatures">xxx.xx4</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0);font-family:Menlo;font-size:13px">:/home/data/repl 192.168.</span><span style="color:rgb(0,0,0);font-family:Menlo;font-size:13px;font-variant-ligatures:no-common-ligatures">xxx.xx5</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0);font-family:Menlo;font-size:13px">:/home/data/repl 192.168.</span><span style="color:rgb(0,0,0);font-family:Menlo;font-size:13px;font-variant-ligatures:no-common-ligatures">xxx.xx6</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0);font-family:Menlo;font-size:13px">:/home/data/repl</span></div><div><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0);font-family:Menlo;font-size:13px"><br></span></div><div><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0);font-family:Menlo;font-size:13px">So after the brick addition we tried to rebalance the layout and the data.</span></div><div><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0);font-family:Menlo;font-size:13px">command : </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0);font-family:Menlo;font-size:13px">gluster volume rebalance test-volume fix-layout start.</span></div><div><font face="Menlo" color="#000000"><span style="font-variant-ligatures:no-common-ligatures">The command exited with status "</span></font><span style="color:rgb(80,0,80)">Error : Request timed out</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0);font-family:Menlo">".<br></span></div></div></div></blockquote><div><br></div><div>This sounds like an error in the cli or glusterd. Can you send the glusterd.log from the node on which you ran the command?</div></div></div></blockquote><div><br></div><div>It seems to me that glusterd took more than 120 seconds to process the command and hence cli timed out. We can confirm the same by checking the status of the rebalance below which indicates rebalance did kick in and eventually completed. We need to understand why did it take such longer, so please pass on the cli and glusterd log from all the nodes as Nithya requested for.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div><br></div><div>regards,</div><div>Nithya </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0);font-family:Menlo"><br>After the failure of the command, we tried to view the status of the command and it is something like this :</span></div>
<div>
<p class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-s1" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>Node Rebalanced-files<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>size <span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>scanned<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>failures <span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>skipped <span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>status<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>run time in h:m:s</span></p>
<p class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-s1" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>---------<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>----------- <span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>----------- <span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>----------- <span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>----------- <span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>----------- <span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>------------ <span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>--------------</span></p>
<p class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-s1" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>localhost <span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>41<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>41.0MB<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>8200 <span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>0 <span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>0<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>completed<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>0:00:09</span></p>
<p class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-s1" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>192.168.xxx.xx4 <span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>79<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>79.0MB<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>8231 <span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>0 <span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>0<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>completed<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>0:00:12</span></p>
<p class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-s1" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>192.168.xxx.xx6 <span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>58<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>58.0MB<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>8281 <span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>0 <span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>0<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>completed<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>0:00:10</span></p>
<p class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-s1" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>192.168.xxx.xx2<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>136 <span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>136.0MB<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>8566 <span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>0 <span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>136<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>completed<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>0:00:07</span></p>
<p class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-s1" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>192.168.xxx.xx4<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>129 <span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>129.0MB<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>8566 <span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>0 <span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>129<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>completed<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>0:00:07</span></p>
<p class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-s1" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>192.168.xxx.xx6<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>201 <span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>201.0MB<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>8566 <span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>0 <span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>201<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>completed<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Apple-converted-space"> </span>0:00:08</span></p></div>
</div><div><br></div>Is the rebalancing option working fine? Why did gluster throw the error saying that "<span class="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-gr_ gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-gr_884 gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-gr-alert gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-gr_gramm gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-gr_inline_cards gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-gr_run_anim gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-Style gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-multiReplace" id="gmail-m_6299188646181137345gmail-m_5294528657150404699gmail-884" style="display:inline;color:rgb(80,0,80);border-bottom:2px solid transparent;background-repeat:no-repeat">Error :</span><span style="color:rgb(80,0,80)"> Request timed out"?</span><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">.On Tue, Feb 5, 2019 at 4:23 PM Nithya Balachandran <<a href="mailto:nbalacha@redhat.com" target="_blank">nbalacha@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi,<div>Please provide the exact step at which you are seeing the error. It would be ideal if you could copy-paste the command and the error.</div><div><br></div><div>Regards,</div><div>Nithya</div><div><br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, 5 Feb 2019 at 15:24, deepu srinivasan <<a href="mailto:sdeepugd@gmail.com" target="_blank">sdeepugd@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">HI everyone. I am getting "Error : Request timed out " while doing rebalance . I have aded new bricks to my replicated volume.i.e. First it was 1x3 volume and added three more bricks to make it distributed-replicated volume(2x3) . What should i do for the timeout error ?<br></div></div>_______________________________________________<br>Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="https://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-users</a></blockquote></div>
</blockquote></div></div>
</blockquote></div></div>
_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="https://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-users</a></blockquote></div></div>