[Gluster-users] Failed to provision volume with StorageClass "glusterfs-storage": glusterfs: create volume err: error creating volume

[Amar Tumballi Suryanarayan]

Looks like the requests to create volume has increased significantly.
Heketi can handle 8 parallel volume create requests, and looks like there
are  429 volume create requests pending.

I am not an expert in this. Added few more people in CC to help when they
get to see this.

-Amar

On Thu, Jan 24, 2019 at 11:28 AM Shaik Salam <shaik.salam at tcs.com> wrote:

> Hi Surya,
>
> Could you please help us to resolve below issue (at lease workaround for
> creating volume)
> Attached db dump and log. Please let me know any other things need to
> check.
> Please guide us.
>
> BR
> Salam
>
>
>
> From:        Shaik Salam/HYD/TCS
> To:        "Amar Tumballi Suryanarayan" <atumball at redhat.com>,
> bugs at gluster.org, "gluster-users at gluster.org List" <
> gluster-users at gluster.org>
> Cc:        "Murali Kottakota" <murali.kottakota at tcs.com>, "Sanju Rakonde"
> <srakonde at redhat.com>
> Date:        01/23/2019 06:19 PM
> Subject:         Failed to provision volume with StorageClass
> "glusterfs-storage": glusterfs: create volume err: error creating volume
> ------------------------------
>
>
>
>
>
> Hi,
>
> We are facing also following issue on openshift origin while we are
> creating pvc for pods.
>
> Failed to provision volume with StorageClass "glusterfs-storage":
> glusterfs: create volume err: error creating volume
> Failed to provision volume with StorageClass "glusterfs-storage":
> glusterfs: create volume err: error creating volume Server busy. Retry
> operation later..
>
> Please find heketidb dump and log
>
> [negroni] Completed 429 Too Many Requests in 250.763µs
> [negroni] Started POST /volumes
> [heketi] WARNING 2019/01/23 12:07:49 operations in-flight (8) exceeds
> limit (8)
> [negroni] Completed 429 Too Many Requests in 169.08µs
> [negroni] Started DELETE /volumes/520bc5f4e1bfd029855a72f9ca7ebf6c
> [negroni] Completed 404 Not Found in 148.125µs
> [negroni] Started POST /volumes
> [heketi] WARNING 2019/01/23 12:08:04 operations in-flight (8) exceeds
> limit (8)
> [negroni] Completed 429 Too Many Requests in 496.624µs
> [negroni] Started POST /volumes
> [heketi] WARNING 2019/01/23 12:08:04 operations in-flight (8) exceeds
> limit (8)
> [negroni] Completed 429 Too Many Requests in 101.673µs
> [negroni] Started POST /volumes
> [heketi] WARNING 2019/01/23 12:08:19 operations in-flight (8) exceeds
> limit (8)
> [negroni] Completed 429 Too Many Requests in 209.681µs
> [negroni] Started POST /volumes
> [heketi] WARNING 2019/01/23 12:08:19 operations in-flight (8) exceeds
> limit (8)
> [negroni] Completed 429 Too Many Requests in 103.595µs
> [negroni] Started POST /volumes
> [heketi] WARNING 2019/01/23 12:08:34 operations in-flight (8) exceeds
> limit (8)
> [negroni] Completed 429 Too Many Requests in 297.594µs
> [negroni] Started POST /volumes
> [heketi] WARNING 2019/01/23 12:08:34 operations in-flight (8) exceeds
> limit (8)
> [negroni] Completed 429 Too Many Requests in 96.75µs
> [negroni] Started POST /volumes
> [negroni] Started POST /volumes
> [heketi] WARNING 2019/01/23 12:08:49 operations in-flight (8) exceeds
> limit (8)
> [negroni] Completed 429 Too Many Requests in 477.007µs
> [heketi] WARNING 2019/01/23 12:08:49 operations in-flight (8) exceeds
> limit (8)
> [negroni] Completed 429 Too Many Requests in 165.38µs
> [negroni] Started POST /volumes
> [heketi] WARNING 2019/01/23 12:09:04 operations in-flight (8) exceeds
> limit (8)
> [negroni] Completed 429 Too Many Requests in 488.253µs
> [negroni] Started POST /volumes
> [heketi] WARNING 2019/01/23 12:09:04 operations in-flight (8) exceeds
> limit (8)
> [negroni] Completed 429 Too Many Requests in 171.836µs
> [negroni] Started POST /volumes
> [heketi] WARNING 2019/01/23 12:09:19 operations in-flight (8) exceeds
> limit (8)
> [negroni] Completed 429 Too Many Requests in 208.59µs
> [negroni] Started POST /volumes
> [heketi] WARNING 2019/01/23 12:09:19 operations in-flight (8) exceeds
> limit (8)
> [negroni] Completed 429 Too Many Requests in 125.141µs
> [negroni] Started DELETE /volumes/99e87ecd0a816ac34ae5a04eabc1d606
> [negroni] Completed 404 Not Found in 138.687µs
> [negroni] Started POST /volumes
>
>
> BR
> Salam
>
> =====-----=====-----=====
> Notice: The information contained in this e-mail
> message and/or attachments to it may contain
> confidential or privileged information. If you are
> not the intended recipient, any dissemination, use,
> review, distribution, printing or copying of the
> information contained in this e-mail message
> and/or attachments to it are strictly prohibited. If
> you have received this communication in error,
> please notify us by reply e-mail or telephone and
> immediately and permanently delete the message
> and any attachments. Thank you
>
>

-- 
Amar Tumballi (amarts)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20190124/6d9f646b/attachment.html>