<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jan 24, 2019 at 12:08 PM Shaik Salam <<a href="mailto:shaik.salam@tcs.com">shaik.salam@tcs.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><font size="2" face="sans-serif">Hi All,</font>
<br>
<br><font size="2" face="sans-serif">Could you please help us to resolve
issue (atleast workaround).</font>
<br><font size="2" face="sans-serif">429 volumes are not requested at all
in cluster. I am trying to create only one volume at a time.</font>
<br>
<br><font size="2" face="sans-serif">BR</font>
<br><font size="2" face="sans-serif">Salam</font>
<br>
<br>
<br>
<br><font size="1" color="#5f5f5f" face="sans-serif">From:
</font><font size="1" face="sans-serif">"Amar Tumballi
Suryanarayan" <<a href="mailto:atumball@redhat.com" target="_blank">atumball@redhat.com</a>></font>
<br><font size="1" color="#5f5f5f" face="sans-serif">To:
</font><font size="1" face="sans-serif">"Shaik Salam"
<<a href="mailto:shaik.salam@tcs.com" target="_blank">shaik.salam@tcs.com</a>></font>
<br><font size="1" color="#5f5f5f" face="sans-serif">Cc:
</font><font size="1" face="sans-serif">"<a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>
List" <<a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>>, "Madhu Rajanna"
<<a href="mailto:mrajanna@redhat.com" target="_blank">mrajanna@redhat.com</a>>, "Raghavendra Talur" <<a href="mailto:rtalur@redhat.com" target="_blank">rtalur@redhat.com</a>>,
"Michael Adam" <<a href="mailto:madam@redhat.com" target="_blank">madam@redhat.com</a>></font>
<br><font size="1" color="#5f5f5f" face="sans-serif">Date:
</font><font size="1" face="sans-serif">01/24/2019 11:59 AM</font>
<br><font size="1" color="#5f5f5f" face="sans-serif">Subject:
</font><font size="1" face="sans-serif">Re: Failed to
provision volume with StorageClass "glusterfs-storage": glusterfs:
create volume err: error creating volume</font>
<br>
<hr noshade>
<br>
<br>
<br><font size="2" color="#ff8141"><b>"External email. Open with Caution"</b></font>
<br><font size="3">Looks like the requests to create volume has increased
significantly. Heketi can handle 8 parallel volume create requests, and
looks like there are 429 volume create requests pending.</font>
<br>
<br></blockquote><div>I was wrong. There seems to be just 8 requests pending. Not sure why its not proceeding though.</div><div><br></div><div>-Amar</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><font size="3">I am not an expert in this. Added few more people in CC
to help when they get to see this.</font>
<br>
<br><font size="3">-Amar</font>
<br>
<br><font size="3">On Thu, Jan 24, 2019 at 11:28 AM Shaik Salam <</font><a href="mailto:shaik.salam@tcs.com" target="_blank"><font size="3" color="blue"><u>shaik.salam@tcs.com</u></font></a><font size="3">>
wrote:</font>
<br><font size="2" face="sans-serif">Hi Surya,</font><font size="3"> <br>
</font><font size="2" face="sans-serif"><br>
Could you please help us to resolve below issue (at lease workaround for
creating volume)</font><font size="3"> </font><font size="2" face="sans-serif"><br>
Attached db dump and log. Please let me know any other things need to check.</font><font size="3">
</font><font size="2" face="sans-serif"><br>
Please guide us.</font><font size="3"> <br>
</font><font size="2" face="sans-serif"><br>
BR</font><font size="3"> </font><font size="2" face="sans-serif"><br>
Salam</font><font size="3"> <br>
<br>
<br>
</font><font size="1" color="#5f5f5f" face="sans-serif"><br>
From: </font><font size="1" face="sans-serif">Shaik
Salam/HYD/TCS</font><font size="3"> </font><font size="1" color="#5f5f5f" face="sans-serif"><br>
To: </font><font size="1" face="sans-serif">"Amar
Tumballi Suryanarayan" <</font><a href="mailto:atumball@redhat.com" target="_blank"><font size="1" color="blue" face="sans-serif"><u>atumball@redhat.com</u></font></a><font size="1" face="sans-serif">>,
</font><a href="mailto:bugs@gluster.org" target="_blank"><font size="1" color="blue" face="sans-serif"><u>bugs@gluster.org</u></font></a><font size="1" face="sans-serif">,
"</font><a href="mailto:gluster-users@gluster.org" target="_blank"><font size="1" color="blue" face="sans-serif"><u>gluster-users@gluster.org</u></font></a><font size="1" face="sans-serif">
List" <</font><a href="mailto:gluster-users@gluster.org" target="_blank"><font size="1" color="blue" face="sans-serif"><u>gluster-users@gluster.org</u></font></a><font size="1" face="sans-serif">></font><font size="3">
</font><font size="1" color="#5f5f5f" face="sans-serif"><br>
Cc: </font><font size="1" face="sans-serif">"Murali
Kottakota" <</font><a href="mailto:murali.kottakota@tcs.com" target="_blank"><font size="1" color="blue" face="sans-serif"><u>murali.kottakota@tcs.com</u></font></a><font size="1" face="sans-serif">>,
"Sanju Rakonde" <</font><a href="mailto:srakonde@redhat.com" target="_blank"><font size="1" color="blue" face="sans-serif"><u>srakonde@redhat.com</u></font></a><font size="1" face="sans-serif">></font><font size="3">
</font><font size="1" color="#5f5f5f" face="sans-serif"><br>
Date: </font><font size="1" face="sans-serif">01/23/2019
06:19 PM</font><font size="3"> </font><font size="1" color="#5f5f5f" face="sans-serif"><br>
Subject: </font><font size="1" face="sans-serif">
Failed to provision volume with StorageClass "glusterfs-storage":
glusterfs: create volume err: error creating volume </font><font size="3"><br>
</font>
<hr noshade><font size="3"><br>
<br>
<br>
<br>
</font><font size="2" color="#2f2f2f" face="Segoe UI"><br>
Hi,</font><font size="3"> <br>
</font><font size="2" color="#2f2f2f" face="Segoe UI"><br>
We are facing also following issue on openshift origin while we are creating
pvc for pods.</font><font size="3"> <br>
</font><font size="2" color="#2f2f2f" face="Segoe UI"><br>
Failed to provision volume with StorageClass "glusterfs-storage":
glusterfs: create volume err: error creating volume<br>
Failed to provision volume with StorageClass "glusterfs-storage":
glusterfs: create volume err: error creating volume Server busy. Retry
operation later..</font><font size="3"> <br>
</font><font size="2" color="#2f2f2f" face="Segoe UI"><br>
Please find heketidb dump and log</font><font size="3"> <br>
</font><font size="2" face="sans-serif"><br>
[negroni] Completed 429 Too Many Requests in 250.763µs</font><font size="3">
</font><font size="2" face="sans-serif"><br>
[negroni] Started POST /volumes</font><font size="3"> </font><font size="2" face="sans-serif"><br>
[heketi] WARNING 2019/01/23 12:07:49 operations in-flight (8) exceeds limit
(8)</font><font size="3"> </font><font size="2" face="sans-serif"><br>
[negroni] Completed 429 Too Many Requests in 169.08µs</font><font size="3">
</font><font size="2" face="sans-serif"><br>
[negroni] Started DELETE /volumes/520bc5f4e1bfd029855a72f9ca7ebf6c</font><font size="3">
</font><font size="2" face="sans-serif"><br>
[negroni] Completed 404 Not Found in 148.125µs</font><font size="3"> </font><font size="2" face="sans-serif"><br>
[negroni] Started POST /volumes</font><font size="3"> </font><font size="2" face="sans-serif"><br>
[heketi] WARNING 2019/01/23 12:08:04 operations in-flight (8) exceeds limit
(8)</font><font size="3"> </font><font size="2" face="sans-serif"><br>
[negroni] Completed 429 Too Many Requests in 496.624µs</font><font size="3">
</font><font size="2" face="sans-serif"><br>
[negroni] Started POST /volumes</font><font size="3"> </font><font size="2" face="sans-serif"><br>
[heketi] WARNING 2019/01/23 12:08:04 operations in-flight (8) exceeds limit
(8)</font><font size="3"> </font><font size="2" face="sans-serif"><br>
[negroni] Completed 429 Too Many Requests in 101.673µs</font><font size="3">
</font><font size="2" face="sans-serif"><br>
[negroni] Started POST /volumes</font><font size="3"> </font><font size="2" face="sans-serif"><br>
[heketi] WARNING 2019/01/23 12:08:19 operations in-flight (8) exceeds limit
(8)</font><font size="3"> </font><font size="2" face="sans-serif"><br>
[negroni] Completed 429 Too Many Requests in 209.681µs</font><font size="3">
</font><font size="2" face="sans-serif"><br>
[negroni] Started POST /volumes</font><font size="3"> </font><font size="2" face="sans-serif"><br>
[heketi] WARNING 2019/01/23 12:08:19 operations in-flight (8) exceeds limit
(8)</font><font size="3"> </font><font size="2" face="sans-serif"><br>
[negroni] Completed 429 Too Many Requests in 103.595µs</font><font size="3">
</font><font size="2" face="sans-serif"><br>
[negroni] Started POST /volumes</font><font size="3"> </font><font size="2" face="sans-serif"><br>
[heketi] WARNING 2019/01/23 12:08:34 operations in-flight (8) exceeds limit
(8)</font><font size="3"> </font><font size="2" face="sans-serif"><br>
[negroni] Completed 429 Too Many Requests in 297.594µs</font><font size="3">
</font><font size="2" face="sans-serif"><br>
[negroni] Started POST /volumes</font><font size="3"> </font><font size="2" face="sans-serif"><br>
[heketi] WARNING 2019/01/23 12:08:34 operations in-flight (8) exceeds limit
(8)</font><font size="3"> </font><font size="2" face="sans-serif"><br>
[negroni] Completed 429 Too Many Requests in 96.75µs</font><font size="3">
</font><font size="2" face="sans-serif"><br>
[negroni] Started POST /volumes</font><font size="3"> </font><font size="2" face="sans-serif"><br>
[negroni] Started POST /volumes</font><font size="3"> </font><font size="2" face="sans-serif"><br>
[heketi] WARNING 2019/01/23 12:08:49 operations in-flight (8) exceeds limit
(8)</font><font size="3"> </font><font size="2" face="sans-serif"><br>
[negroni] Completed 429 Too Many Requests in 477.007µs</font><font size="3">
</font><font size="2" face="sans-serif"><br>
[heketi] WARNING 2019/01/23 12:08:49 operations in-flight (8) exceeds limit
(8)</font><font size="3"> </font><font size="2" face="sans-serif"><br>
[negroni] Completed 429 Too Many Requests in 165.38µs</font><font size="3">
</font><font size="2" face="sans-serif"><br>
[negroni] Started POST /volumes</font><font size="3"> </font><font size="2" face="sans-serif"><br>
[heketi] WARNING 2019/01/23 12:09:04 operations in-flight (8) exceeds limit
(8)</font><font size="3"> </font><font size="2" face="sans-serif"><br>
[negroni] Completed 429 Too Many Requests in 488.253µs</font><font size="3">
</font><font size="2" face="sans-serif"><br>
[negroni] Started POST /volumes</font><font size="3"> </font><font size="2" face="sans-serif"><br>
[heketi] WARNING 2019/01/23 12:09:04 operations in-flight (8) exceeds limit
(8)</font><font size="3"> </font><font size="2" face="sans-serif"><br>
[negroni] Completed 429 Too Many Requests in 171.836µs</font><font size="3">
</font><font size="2" face="sans-serif"><br>
[negroni] Started POST /volumes</font><font size="3"> </font><font size="2" face="sans-serif"><br>
[heketi] WARNING 2019/01/23 12:09:19 operations in-flight (8) exceeds limit
(8)</font><font size="3"> </font><font size="2" face="sans-serif"><br>
[negroni] Completed 429 Too Many Requests in 208.59µs</font><font size="3">
</font><font size="2" face="sans-serif"><br>
[negroni] Started POST /volumes</font><font size="3"> </font><font size="2" face="sans-serif"><br>
[heketi] WARNING 2019/01/23 12:09:19 operations in-flight (8) exceeds limit
(8)</font><font size="3"> </font><font size="2" face="sans-serif"><br>
[negroni] Completed 429 Too Many Requests in 125.141µs</font><font size="3">
</font><font size="2" face="sans-serif"><br>
[negroni] Started DELETE /volumes/99e87ecd0a816ac34ae5a04eabc1d606</font><font size="3">
</font><font size="2" face="sans-serif"><br>
[negroni] Completed 404 Not Found in 138.687µs</font><font size="3"> </font><font size="2" face="sans-serif"><br>
[negroni] Started POST /volumes</font><font size="3"> <br>
<br>
</font><font size="2" face="sans-serif"><br>
BR</font><font size="3"> </font><font size="2" face="sans-serif"><br>
Salam</font><font size="3"> </font>
<br><font size="3">=====-----=====-----=====<br>
Notice: The information contained in this e-mail<br>
message and/or attachments to it may contain <br>
confidential or privileged information. If you are <br>
not the intended recipient, any dissemination, use, <br>
review, distribution, printing or copying of the <br>
information contained in this e-mail message <br>
and/or attachments to it are strictly prohibited. If <br>
you have received this communication in error, <br>
please notify us by reply e-mail or telephone and <br>
immediately and permanently delete the message <br>
and any attachments. Thank you</font>
<p>
<br>
<br><font size="3">-- </font>
<br><font size="3">Amar Tumballi (amarts)</font>
<br></p></blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div>Amar Tumballi (amarts)<br></div></div></div></div></div></div>