[Gluster-infra] tests/basic/tier/tier-file-create.t dumping core on Linux

Dan Lambright dlambrig at redhat.com
Tue Mar 8 18:05:05 UTC 2016



----- Original Message -----
> From: "Niels de Vos" <ndevos at redhat.com>
> To: "Dan Lambright" <dlambrig at redhat.com>
> Cc: "Krutika Dhananjay" <kdhananj at redhat.com>, "Vijay Bellur" <vbellur at redhat.com>, "Gluster Devel"
> <gluster-devel at gluster.org>, "Pranith Karampuri" <pkarampu at redhat.com>, "gluster-infra" <gluster-infra at gluster.org>,
> "RHGS tiering mailing list" <rhgs-tiering at redhat.com>
> Sent: Tuesday, March 8, 2016 12:36:58 PM
> Subject: Re: [Gluster-infra] tests/basic/tier/tier-file-create.t dumping core on Linux
> 
> On Tue, Mar 08, 2016 at 08:45:12AM -0500, Dan Lambright wrote:
> > 
> > 
> > ----- Original Message -----
> > > From: "Niels de Vos" <ndevos at redhat.com>
> > > To: "Krutika Dhananjay" <kdhananj at redhat.com>, "Vijay Bellur"
> > > <vbellur at redhat.com>
> > > Cc: "Dan Lambright" <dlambrig at redhat.com>, "Gluster Devel"
> > > <gluster-devel at gluster.org>, "Pranith Karampuri"
> > > <pkarampu at redhat.com>, "gluster-infra" <gluster-infra at gluster.org>, "RHGS
> > > tiering mailing list"
> > > <rhgs-tiering at redhat.com>
> > > Sent: Tuesday, March 8, 2016 8:37:14 AM
> > > Subject: Re: [Gluster-infra] tests/basic/tier/tier-file-create.t dumping
> > > core on Linux
> > > 
> > > On Tue, Mar 08, 2016 at 07:00:07PM +0530, Krutika Dhananjay wrote:
> > > > I did talk to the author and he is going to look into the issue.
> > > > 3.7.9 is round the corner and we certainly don't want bad tests to
> > > > block
> > > > patches that need to go in.
> > > 
> > > 3.7.9 has been delayed already. There is hardly a good reason for
> > > changes to delay the relase even more. 3.7.10 will be done in a few
> > > weeks at the end of March, anything non-regression should probably not
> > > be included at this point anymore.
> > > 
> > > Vijay is the release manager for 3.7.9 and .10, you'll need to come with
> > > extreme strong points for getting more patches included.
> > 
> > I've no desire to disrupt downstream releases and if we are indeed out
> > of time for 3.7.9, then thats life.
> 
> 3.7.x releases are planned for each 30th of the month, 3.7.9 was
> supposed to be tagged for relaese the end of February. We are trying to
> get back to this schdule, and are failing hard :-(
> 
> Upstream has a very strict schedule (that we fail to follow because the
> release managers are occupied elsewhere), downstream (RHGS) should be
> aware of that as is it documented on
> https://www.gluster.org/community/release-schedule/ . Please pass that
> link on to anyone that expects changes to get merged in certain upstream
> releases. In general a release needs a few days of preparation, so
> patches for a release need to be ready at least ~5 working days in
> advance.
> 
> > That said, obviously a test failure is a failure, and once its masked
> > it tends to be forgotton.
> 
> When a test is marked as bad_test, we require a bug to be filed for that
> test. It should not be forgotten, the component maintainers are expected
> to track (and eventually fix) the open bugs for their components.
> 
> > I suggest we do not mask this test upstream (does it happen there?)
> > and give some urgency to chasing it down.
> 
> I assumed this is all for upstream. Not sure why an internal Red Hat
> list (rhgs-tiering) was put on CC.

-1

> 
> HTH,
> Niels
> 
> 
> > 
> > 
> > > 
> > > Niels
> > > 
> > > 
> > > > 
> > > > -Krutika
> > > > 
> > > > On Tue, Mar 8, 2016 at 6:50 PM, Dan Lambright <dlambrig at redhat.com>
> > > > wrote:
> > > > 
> > > > >
> > > > >
> > > > > ----- Original Message -----
> > > > > > From: "Krutika Dhananjay" <kdhananj at redhat.com>
> > > > > > To: "Pranith Karampuri" <pkarampu at redhat.com>
> > > > > > Cc: "gluster-infra" <gluster-infra at gluster.org>, "Gluster Devel" <
> > > > > gluster-devel at gluster.org>, "RHGS tiering mailing
> > > > > > list" <rhgs-tiering at redhat.com>, "Dan Lambright"
> > > > > > <dlambrig at redhat.com>
> > > > > > Sent: Tuesday, March 8, 2016 12:15:20 AM
> > > > > > Subject: Re: [Gluster-infra] tests/basic/tier/tier-file-create.t
> > > > > > dumping
> > > > > core on Linux
> > > > > >
> > > > > > It has been failing rather frequently.
> > > > > > Have reported a bug at
> > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1315560
> > > > > > For now, have moved it to bad tests here:
> > > > > > http://review.gluster.org/#/c/13632/1
> > > > > >
> > > > >
> > > > >
> > > > > Masking tests is a bad habit. It would be better to fix the problem,
> > > > > and
> > > > > it looks like a real bug.
> > > > > The author of the test should help chase this down.
> > > > >
> > > > > > -Krutika
> > > > > >
> > > > > > On Mon, Mar 7, 2016 at 4:17 PM, Krutika Dhananjay
> > > > > > <kdhananj at redhat.com>
> > > > > > wrote:
> > > > > >
> > > > > > > +Pranith
> > > > > > >
> > > > > > > -Krutika
> > > > > > >
> > > > > > >
> > > > > > > On Sat, Mar 5, 2016 at 11:34 PM, Dan Lambright
> > > > > > > <dlambrig at redhat.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > >>
> > > > > > >>
> > > > > > >> ----- Original Message -----
> > > > > > >> > From: "Dan Lambright" <dlambrig at redhat.com>
> > > > > > >> > To: "Shyam" <srangana at redhat.com>
> > > > > > >> > Cc: "Krutika Dhananjay" <kdhananj at redhat.com>, "Gluster Devel"
> > > > > > >> > <
> > > > > > >> gluster-devel at gluster.org>, "Rafi Kavungal Chundattu
> > > > > > >> > Parambil" <rkavunga at redhat.com>, "Nithya Balachandran" <
> > > > > > >> nbalacha at redhat.com>, "Joseph Fernandes"
> > > > > > >> > <josferna at redhat.com>, "gluster-infra"
> > > > > > >> > <gluster-infra at gluster.org>
> > > > > > >> > Sent: Friday, March 4, 2016 9:51:18 AM
> > > > > > >> > Subject: Re: [Gluster-infra]
> > > > > > >> > tests/basic/tier/tier-file-create.t
> > > > > > >> dumping core on Linux
> > > > > > >> >
> > > > > > >> >
> > > > > > >> >
> > > > > > >> > ----- Original Message -----
> > > > > > >> > > From: "Shyam" <srangana at redhat.com>
> > > > > > >> > > To: "Krutika Dhananjay" <kdhananj at redhat.com>, "Gluster
> > > > > > >> > > Devel"
> > > > > > >> > > <gluster-devel at gluster.org>, "Rafi Kavungal Chundattu
> > > > > > >> > > Parambil" <rkavunga at redhat.com>, "Nithya Balachandran"
> > > > > > >> > > <nbalacha at redhat.com>, "Joseph Fernandes"
> > > > > > >> > > <josferna at redhat.com>, "Dan Lambright" <dlambrig at redhat.com>
> > > > > > >> > > Cc: "gluster-infra" <gluster-infra at gluster.org>
> > > > > > >> > > Sent: Friday, March 4, 2016 9:45:17 AM
> > > > > > >> > > Subject: Re: [Gluster-infra]
> > > > > > >> > > tests/basic/tier/tier-file-create.t
> > > > > > >> dumping
> > > > > > >> > > core on Linux
> > > > > > >> > >
> > > > > > >> > > Facing the same problem in the following runs as well,
> > > > > > >> > >
> > > > > > >> > > 1)
> > > > > > >> > >
> > > > > > >>
> > > > > https://build.gluster.org/job/rackspace-regression-2GB-triggered/18767/console
> > > > > > >> > > 2)
> > > > > https://build.gluster.org/job/regression-test-burn-in/546/console
> > > > > > >> > > 3)
> > > > > https://build.gluster.org/job/regression-test-burn-in/547/console
> > > > > > >> > > 4)
> > > > > https://build.gluster.org/job/regression-test-burn-in/549/console
> > > > > > >> > >
> > > > > > >> > > Last successful burn-in was: 545 (but do not see the test
> > > > > > >> > > having
> > > > > been
> > > > > > >> > > run here, so this is inconclusive)
> > > > > > >> > >
> > > > > > >> > > burn-in test 544 is hung on the same test here,
> > > > > > >> > > https://build.gluster.org/job/regression-test-burn-in/544/console
> > > > > > >> > >
> > > > > > >> > > (and at this point I am stopping the hunt for when this last
> > > > > > >> succeeded :) )
> > > > > > >> > >
> > > > > > >> > > Let's know if anyone is taking a peek at the cores.
> > > > > > >> >
> > > > > > >> > hm. Not familiar with this test. Written by Pranith? I'll
> > > > > > >> > look.
> > > > > > >>
> > > > > > >> We are doing lookup everywhere, and building up a dict of the
> > > > > > >> extended
> > > > > > >> attributes of a file as we traverse each sub volume across the
> > > > > > >> hot
> > > > > > >> and
> > > > > > >> cold
> > > > > > >> tiers. The length field of one of the EC keys is corrupted.
> > > > > > >>
> > > > > > >> Not clear why this is happening.. I see no tiering relationship
> > > > > > >> as
> > > > > > >> of
> > > > > > >> yet, its possible the file is being demoted in parallel to the
> > > > > foreground
> > > > > > >> script operation.
> > > > > > >>
> > > > > > >> The test runs fine on my machines.  Does this reproduce
> > > > > > >> consistently
> > > > > on
> > > > > > >> one of the Jenkins machines? If so, getting onto it would be the
> > > > > > >> next
> > > > > > >> step.
> > > > > > >> I think that would be preferable to masking this test case.
> > > > > > >>
> > > > > > >>
> > > > > > >> >
> > > > > > >> > >
> > > > > > >> > > Thanks,
> > > > > > >> > > Shyam
> > > > > > >> > >
> > > > > > >> > >
> > > > > > >> > >
> > > > > > >> > > On 03/04/2016 07:40 AM, Krutika Dhananjay wrote:
> > > > > > >> > > > Could someone from tiering dev team please take a look?
> > > > > > >> > > >
> > > > > > >> > > >
> > > > > > >>
> > > > > https://build.gluster.org/job/rackspace-regression-2GB-triggered/18793/console
> > > > > > >> > > >
> > > > > > >> > > > -Krutika
> > > > > > >> > > >
> > > > > > >> > > >
> > > > > > >> > > > _______________________________________________
> > > > > > >> > > > Gluster-infra mailing list
> > > > > > >> > > > Gluster-infra at gluster.org
> > > > > > >> > > > http://www.gluster.org/mailman/listinfo/gluster-infra
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > 
> > > > _______________________________________________
> > > > Gluster-infra mailing list
> > > > Gluster-infra at gluster.org
> > > > http://www.gluster.org/mailman/listinfo/gluster-infra
> > > 
> > > 
> 


More information about the Gluster-infra mailing list