[Gluster-devel] New project on the Forge - gstatus

[Anand Avati]

KP, Vipul,

It will be awesome to get io-stats like instrumentation on the client side.
Here are some further thoughts on how to implement that. If you have a
recent git HEAD build, I would suggest that you explore the latency stats
on the client side exposed through meta at
$MNT/.meta/graphs/active/$xlator/profile. You can enable latency
measurement with "echo 1 > $MNT/.meta/measure_latency". I would suggest
extending these stats with the extra ones io-stats has, and make
glusterfsiostats expose these stats.

If you can compare libglusterfs/src/latency.c:gf_latency_begin(),
gf_latency_end() and gf_latency_udpate() with the macros in io-stats.c
UPDATE_PROFILE_STATS()
and START_FOP_LATENCY(), you will quickly realize how a lot of logic is
duplicated between io-stats and latency.c. If you can enhance latency.c and
make it capture the remaining stats what io-stats is capturing, the
benefits of this approach would be:

- stats are already getting captured at all xlator levels, and not just at
the position where io-stats is inserted
- file like interface makes the stats more easily inspectable and
consumable, and updated on the fly
- conforms with the way rest of the internals are exposed through $MNT/.meta

In order to this, you might want to look into:

- latency.c as of today captures fop count, mean latency, total time,
whereas io-stats measures these along with min-time, max-time and
block-size histogram.
- extend gf_proc_dump_latency_info() to dump the new stats
- either prettify that output like 'volume profile info' output, or JSONify
it like xlators/meta/src/frames-file.c
- add support for cumulative vs interval stats (store an extra copy of
this->latencies[])

etc..

Thanks!


On Fri, Apr 25, 2014 at 9:09 PM, Krishnan Parthasarathi <kparthas at redhat.com
> wrote:

> [Resending due to gluster-devel mailing list issue]
>
> Apologies for the late reply.
>
> glusterd uses its socket connection with brick processes (where io-stats
> xlator is loaded) to
> gather information from io-stats via an RPC request. This facility is
> restricted to brick processes
> as it stands today.
>
> Some background ...
> io-stats xlator is loaded, both in GlusterFS mounts and brick processes.
> So, we have the capabilities
> to monitor I/O statistics on both sides. To collect I/O statistics at the
> server side, we have
>
> # gluster volume profile VOLNAME [start | info | stop]
> AND
> #gluster volume top VOLNAME info [and other options]
>
> We don't have a usable way of gathering I/O statistics (not monitoring,
> though the counters could be enhanced)
> at the client-side, ie. for a given mount point. This is the gap
> glusterfsiostat aims to fill. We need to remember
> that the machines hosting GlusterFS mounts may not have glusterd installed
> on them.
>
> We are considering rrdtool as a possible statistics database because it
> seems like a natural choice for storing time-series
> data. rrdtool is capable of answering high-level statistical queries on
> statistics that were logged in it by io-stats xlator
> over and above printing running counters periodically.
>
> Hope this gives some more clarity on what we are thinking.
>
> thanks,
> Krish
> ----- Original Message -----
>
> > Probably me not understanding.
>
> > the comment "iostats making data available to glusterd over RPC" - is
> what I
> > latched on to. I wondered whether this meant that a socket could be
> opened
> > that way to get at the iostats data flow.
>
> > Cheers,
>
> > PC
>
> > ----- Original Message -----
>
> > > From: "Vipul Nayyar" <nayyar_vipul at yahoo.com>
> >
> > > To: "Paul Cuzner" <pcuzner at redhat.com>, "Krishnan Parthasarathi"
> > > <kparthas at redhat.com>
> >
> > > Cc: "Vijay Bellur" <vbellur at redhat.com>, "gluster-devel"
> > > <gluster-devel at nongnu.org>
> >
> > > Sent: Thursday, 20 February, 2014 5:06:27 AM
> >
> > > Subject: Re: [Gluster-devel] New project on the Forge - gstatus
> >
>
> > > Hi Paul,
> >
>
> > > I'm really not sure, if this can be done in python(at least
> comfortably).
> > > Maybe we can tread on the same path as Justin's glusterflow in python.
> But
> > > I
> > > don't think, all the io-stats counters will be available with the way
> how
> > > Justin's used Jeff Darcy's previous work to build his tool. I can be
> wrong.
> > > My knowledge is a bit incomplete and based on very less experience as a
> > > user
> > > and an amateur Gluster developer. Please do correct me, if I can be.
> >
>
> > > Regards
> >
> > > Vipul Nayyar
> >
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-devel/attachments/20140516/0bbd40a7/attachment-0002.html>