[Gluster-devel] new glusterfs logging framework

[Balamurugan Arumugam]

----- Original Message -----
> From: "Niels de Vos" <ndevos at redhat.com>
> To: "Balamurugan Arumugam" <bala at gluster.com>
> Cc: gluster-devel at nongnu.org, gluster-users at gluster.org
> Sent: Tuesday, August 6, 2013 8:48:58 PM
> Subject: Re: [Gluster-devel] new glusterfs logging framework
> 
> On Tue, Jul 30, 2013 at 05:29:43AM -0400, Balamurugan Arumugam wrote:
> > 
> > Hi All,
> > 
> > Recently new logging framework was introduced [1][2][3] in glusterfs
> > master branch.  You could read more about this on doc/logging.txt.  In
> > brief, current log target is moved to syslog and user has an option to
> > this new logging at compile time (passing '--disable-syslog' to
> > ./configure or '--without syslog' to rpmbuild) and run time (having
> > a file /var/log/glusterd/logger.conf and restarting gluster services).
> > 
> > As rsyslog is used as syslog server in Fedora and CentOS/RHEL and
> > default configuration of rsyslog does not have any rule specific to
> > gluster logs, you see all logs are in /var/log/messages in JSON
> > format.
> 
> As the below does not function on RHEL-6 with its own rsyslog (no
> rsyslog-mmjsonparse available), I am creating an empty
> /var/lib/glusterd/logger.conf which (after a restart) makes glusterd and
> friends to log to their standard files again.
> 
> Do you have some progress on the rsyslog-mmjsonparse for rhel5 and
> rhel6?
> 

mmcount is a rsyslog plugin (developed by me) available rsyslog upstream which tracks each gluster log using error codes.  The error code is the entity enabled by gluster logging framework.  This module is back ported to rhel6 which does tracking and json parsing.  I am in the process of submitting this to rhel 6 downstream.  I haven't tried rhel5 yet and I will do that too.


> Also, it would really be nice to include that
> /etc/rsyslog.d/gluster.conf in the rpms and add a dependency on the
> rsyslog-mmjsonparse package. Or, provide a /var/lib/glusterd/logger.conf
> for the distributions that do not have rsyslog-mmjsonparse.
> 

Yes.  I had similar thought on this to do so.  I will send the fix soon.

Thanks,

Regards,
Bala


> Thanks,
> Niels
> 
> > 
> > Below is the way to make them neat and clean.
> > 
> > For fedora users:
> > 1. It requires to install rsyslog-mmjsonparse rpm (yum -y install
> > rsyslog-mmjsonparse)
> > 2. Place below configuration under /etc/rsyslog.d/gluster.conf file.
> > 
> > #$RepeatedMsgReduction on
> > 
> > $ModLoad mmjsonparse
> > *.* :mmjsonparse:
> > 
> > template (name="GlusterLogFile" type="string"
> > string="/var/log/gluster/%app-name%.log")
> > template (name="GlusterPidLogFile" type="string"
> > string="/var/log/gluster/%app-name%-%procid%.log")
> > 
> > template(name="GLFS_template" type="list") {
> >    property(name="$!mmcount")
> >    constant(value="/")
> >    property(name="syslogfacility-text" caseConversion="upper")
> >    constant(value="/")
> >    property(name="syslogseverity-text" caseConversion="upper")
> >    constant(value=" ")
> >    constant(value="[")
> >    property(name="timereported" dateFormat="rfc3339")
> >    constant(value="] ")
> >    constant(value="[")
> >    property(name="$!gf_code")
> >    constant(value="] ")
> >    constant(value="[")
> >    property(name="$!gf_message")
> >    constant(value="] ")
> >    property(name="$!msg")
> >    constant(value="\n")
> > }
> > 
> > if $app-name == 'gluster' or $app-name == 'glusterd' then {
> >    action(type="omfile"
> >           DynaFile="GlusterLogFile"
> >           Template="GLFS_template")
> >    stop
> > }
> > 
> > if $app-name contains 'gluster' then {
> >    action(type="omfile"
> >           DynaFile="GlusterPidLogFile"
> >           Template="GLFS_template")
> >    stop
> > }
> > 
> > 
> > 3. Restart rsyslog (service rsyslog restart)
> > 4. Done. All gluster process specific logs are separated into
> > /var/log/gluster/ directory
> > 
> > 
> > Note: Fedora 19 users
> > There is a bug in rsyslog of fedora 19 [4], so its required to recompile
> > rsyslog source rpm downloaded from fedora repository ('rpmbuild --rebuild
> > rsyslog-7.2.6-1.fc19.src.rpm' works fine) and use generated rsyslog and
> > rsyslog-mmjsonparse binary rpms
> > 
> > For CentOS/RHEL users:
> > Current rsyslog available in CentOS/RHEL does not have json support.  I
> > have added the support which requires some testing.  I will update once
> > done.
> > 
> > 
> > TODO:
> > 1. need to add volume:brick specific tag to logging so that those logs can
> > be separated out than pid.
> > 2. enable gfapi to use this logging framework
> > 
> > 
> > I would like to get feedback/suggestion about this logging framework
> > 
> > 
> > Regards,
> > Bala
> > 
> > 
> > [1] http://review.gluster.org/4977
> > [2] http://review.gluster.org/5002
> > [3] http://review.gluster.org/4915
> > [4] https://bugzilla.redhat.com/show_bug.cgi?id=989886
> > 
> > _______________________________________________
> > Gluster-devel mailing list
> > Gluster-devel at nongnu.org
> > https://lists.nongnu.org/mailman/listinfo/gluster-devel
> 
> --
> Niels de Vos
> Sr. Software Maintenance Engineer
> Support Engineering Group
> Red Hat Global Support Services
>