[Gluster-devel] glusterd crashing

Sat Mar 5 16:36:37 UTC 2016

Sure Atin.  I am itching to contribute code. But worried due to lack of
experience in sending patches. Can somebody please send me across how to do
this? Consider me a total newbie and please be as descriptive as possible
:).

-Ajil

On Sat, Mar 5, 2016 at 12:46 PM, Atin Mukherjee <atin.mukherjee83 at gmail.com>
wrote:

> -Atin
> Sent from one plus one
> On 05-Mar-2016 11:46 am, "Ajil Abraham" <ajil95.abraham at gmail.com> wrote:
> >
> > Thanks for all the support.  After handling the input validation in my
> code, Glusterd no longer crashes.  I am still waiting for clearance from my
> superior to pass on all the details. Expecting him to revert by this Sunday.
> Great to know that and we appreciate your contribution, if you happen to
> find any issues feel free to send patches :)
> >
> > - Ajil
> >
> > On Fri, Mar 4, 2016 at 10:20 AM, Joseph Fernandes <josferna at redhat.com>
> wrote:
> >>
> >> Well that may not be completely correct !
> >>
> >> Its  "gluster volume status all", unlike volume maintenance operation
> which are rare.
> >>
> >> Status can be issued multiple times in a day or might be put in a
> script/cron-job to check the health of the
> >> cluster.
> >> But anyways the fix is ready as the bug says.
> >>
> >> Crash is what we need to worry about.
> >>
> >> ~Joe
> >>
> >> ----- Original Message -----
> >> > From: "Atin Mukherjee" <amukherj at redhat.com>
> >> > To: "Joseph Fernandes" <josferna at redhat.com>, "Atin Mukherjee" <
> atin.mukherjee83 at gmail.com>
> >> > Cc: "Gluster Devel" <gluster-devel at gluster.org>, "Ajil Abraham" <
> ajil95.abraham at gmail.com>
> >> > Sent: Friday, March 4, 2016 9:37:43 AM
> >> > Subject: Re: [Gluster-devel] glusterd crashing
> >> >
> >> >
> >> >
> >> > On 03/04/2016 07:10 AM, Joseph Fernandes wrote:
> >> > > Might be this bug can give some context on the mem-leak (fix
> recently
> >> > > merged on master but not on 3.7.x)
> >> > >
> >> > > https://bugzilla.redhat.com/show_bug.cgi?id=1287517
> >> > Yes, this is what we'd be fixing in 3.7.x too, but if you refer to [1]
> >> > the hike is seen when a command is run in a loop which is typically
> not
> >> > a use case in any production setup.
> >> >
> >> > [1] https://bugzilla.redhat.com/show_bug.cgi?id=1287517#c15
> >> > >
> >> > > ~Joe
> >> > >
> >> > >
> >> > > ----- Original Message -----
> >> > >> From: "Atin Mukherjee" <atin.mukherjee83 at gmail.com>
> >> > >> To: "Joseph Fernandes" <josferna at redhat.com>
> >> > >> Cc: "Gluster Devel" <gluster-devel at gluster.org>, "Ajil Abraham"
> >> > >> <ajil95.abraham at gmail.com>
> >> > >> Sent: Friday, March 4, 2016 7:01:54 AM
> >> > >> Subject: Re: [Gluster-devel] glusterd crashing
> >> > >>
> >> > >> -Atin
> >> > >> Sent from one plus one
> >> > >> On 04-Mar-2016 6:12 am, "Joseph Fernandes" <josferna at redhat.com>
> wrote:
> >> > >>>
> >> > >>> Hi Ajil,
> >> > >>>
> >> > >>> Well few things,
> >> > >>>
> >> > >>> 1. Whenever you see a crash its better to send across the
> Backtrace(BT)
> >> > >> using gdb and attach the log files (or share it via some cloud
> drive)
> >> > >>>
> >> > >>> 2. About the memory leak, What kind of tools are you using for
> profiling
> >> > >> memory, valgrind ? if so please attach the valgrind reports.
> >> > >>>    $> glusterd --xlator-option *.run-with-valgrind=yes
> >> > >>>
> >> > >>> 3. Well I am not sure if glusterd uses any of the mempools as we
> do in
> >> > >> client and brick processes, Atin can shed some light on this.
> >> > >>>    Well In that case you can used the statedump mechanism check
> for
> >> > >> mem-leaks check the glusterfs/doc/debugging/statedump.md
> >> > >> GlusterD does use mempool and it has infra for capturing statedump
> as
> >> > >> well.
> >> > >> I am aware of few bytes of memory leaks in few paths which is
> really not a
> >> > >> huge concern but it shouldn't crash.
> >> > >>>
> >> > >>> Hope this helps
> >> > >>>
> >> > >>> ~Joe
> >> > >>>
> >> > >>>
> >> > >>> ----- Original Message -----
> >> > >>>> From: "Ajil Abraham" <ajil95.abraham at gmail.com>
> >> > >>>> To: "Atin Mukherjee" <atin.mukherjee83 at gmail.com>
> >> > >>>> Cc: "Gluster Devel" <gluster-devel at gluster.org>
> >> > >>>> Sent: Thursday, March 3, 2016 10:48:56 PM
> >> > >>>> Subject: Re: [Gluster-devel] glusterd crashing
> >> > >>>>
> >> > >>>> Hi Atin,
> >> > >>>>
> >> > >>>> The inputs I use are as per the requirements of a project I am
> working
> >> > >> on for
> >> > >>>> one of the large finance institutions in Dubai. I will try to
> handle the
> >> > >>>> input validation within my code. I uncovered some of the issues
> while
> >> > >> doing
> >> > >>>> a thorough testing of my code.
> >> > >>>>
> >> > >>>> I tried with 3.7.6 and also my own build from master branch. I
> will
> >> > >> check
> >> > >>>> with my superiors before sending you backtrace and other
> details. So
> >> > >> far, I
> >> > >>>> have seen memory leak in 100s of KBs.
> >> > >>>>
> >> > >>>> -Ajil
> >> > >>>>
> >> > >>>>
> >> > >>>> On Thu, Mar 3, 2016 at 10:17 PM, Atin Mukherjee <
> >> > >> atin.mukherjee83 at gmail.com
> >> > >>>>> wrote:
> >> > >>>>
> >> > >>>>
> >> > >>>>
> >> > >>>>
> >> > >>>> Hi Ajil,
> >> > >>>>
> >> > >>>> Its good to see that you are doing a thorough testing gluster.
> From
> >> > >> your mail
> >> > >>>> it looks like your automation focuses on mostly negative tests.
> I need
> >> > >> few
> >> > >>>> additional details to get to know whether they are known:
> >> > >>>>
> >> > >>>> 1. Version of gluster
> >> > >>>> 2. Backtrace of the crash along with reproducer
> >> > >>>> 3. Amount of memory leak in terms of bytes/KB/MB?? Have you
> already
> >> > >>>> identified them?
> >> > >>>>
> >> > >>>> -Atin
> >> > >>>> Sent from one plus one
> >> > >>>> On 03-Mar-2016 10:01 pm, "Ajil Abraham" <
> ajil95.abraham at gmail.com >
> >> > >> wrote:
> >> > >>>>
> >> > >>>>
> >> > >>>>
> >> > >>>> For my project, I am trying to do some automation using
> glusterd. It is
> >> > >> very
> >> > >>>> frustrating to see it crashing frequently. Looks like input
> validation
> >> > >> is
> >> > >>>> the culprit. I also see lot of buffer overflow and memory leak
> issues.
> >> > >>>> Making a note of these and will try to fix them. Surprised to
> see such
> >> > >> basic
> >> > >>>> issues still existing in Gluster.
> >> > >>>>
> >> > >>>> -Ajil
> >> > >>>>
> >> > >>>> _______________________________________________
> >> > >>>> Gluster-devel mailing list
> >> > >>>> Gluster-devel at gluster.org
> >> > >>>> http://www.gluster.org/mailman/listinfo/gluster-devel
> >> > >>>>
> >> > >>>>
> >> > >>>> _______________________________________________
> >> > >>>> Gluster-devel mailing list
> >> > >>>> Gluster-devel at gluster.org
> >> > >>>> http://www.gluster.org/mailman/listinfo/gluster-devel
> >> > >>
> >> > > _______________________________________________
> >> > > Gluster-devel mailing list
> >> > > Gluster-devel at gluster.org
> >> > > http://www.gluster.org/mailman/listinfo/gluster-devel
> >> > >
> >> >
> >
> >
> >
> > _______________________________________________
> > Gluster-devel mailing list
> > Gluster-devel at gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20160305/29bc9bee/attachment.html>