[Gluster-devel] Gluster builder being hit by too much process
mscherer at redhat.com
Fri Oct 6 09:01:19 UTC 2017
so on the last 2 days, I have been contacted by people because some
builders were failling. Upon investigation ( https://bugzilla.redhat.co
m/show_bug.cgi?id=1498390 ), the main issue seems to be the following:
Each build failed had a set of glusterd process (around 300) that where
started by jenkins, to test regression for this change:
(found due to environment variable of the process)
But upon closer inspection of the patch, it doesn't seems buggy, so my
suspicion are on the test case, who is also quite simple (and likely
bug free), but who also seems to start a ton of of volume (around 1000
) if I am not wrong, and this do seems to result into a large number of
process being created.
Could it be that the test case do uncover a bug in the test suite, or a
bug in gluster ?
Looking at the test suite, I see that the cleanup function is
conveniently ignoring a ton of error:
which do not help to see what is going wrong.
I also do not see out of memory errors, but munin graph seems to stop
right at the same time, so maybe that's just a ressource issue.
So my questions are:
- is gluster supposed to scale gracefully with 1000 volumes on 1 single
- how much ressources should we plan for that ? (right now, we have 2G
VM, we can't increase much without reinstalling the whole set of
If you see any builders not working, please ping me on irc.
Sysadmin, Community Infrastructure and Platform, OSAS
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 836 bytes
Desc: This is a digitally signed message part
More information about the Gluster-devel