[Gluster-devel] [Gluster-users] new glusterfs logging framework

[Balamurugan Arumugam]

----- Original Message -----
> From: "Joe Julian" <joe at julianfamily.org>
> To: "Pablo" <paa.listas at gmail.com>, "Balamurugan Arumugam" <bala at gluster.com>
> Cc: gluster-users at gluster.org, gluster-devel at nongnu.org
> Sent: Tuesday, July 30, 2013 9:26:55 PM
> Subject: Re: [Gluster-users] new glusterfs logging framework
> 
> Configuration files should be under /etc per FSH standards. Move the
> logger.conf to /etc/glusterfs.
> 

This will be done.


> I, personally, like json logs since I'm shipping to logstash. :-) My one
> suggestion would be to ensure the timestamps are in rfc3164.
> 

rsyslog supports rfc3339 (a profile of ISO8601) and we use this.  Let me know your thoughts on continue using it.


> Yes, those are complex steps, but the rpm/deb packaging should take care of
> dependencies and setting up logical defaults.
> 

Yes. I am planning to add rsyslog configuration for gluster at install time.


> IMHO, since this is a departure from the way it's been before now, the config
> file should enable this new behavior, not disable it, to avoid breaking
> existing monitoring installations.
> 

Do you mean to continue current logging in addition to syslog way?

Regards,
Bala


> Pablo <paa.listas at gmail.com> wrote:
> >I think that adding all that 'rsyslog' configuration only to see logs
> >is
> >too much. (I admit it, I don't know how to configure rsyslog at that
> >level so that may influence my opinion)
> >
> >Regards,
> >
> >
> >El 30/07/2013 06:29 a.m., Balamurugan Arumugam escribió:
> >> Hi All,
> >>
> >> Recently new logging framework was introduced [1][2][3] in glusterfs
> >master branch.  You could read more about this on doc/logging.txt.  In
> >brief, current log target is moved to syslog and user has an option to
> >this new logging at compile time (passing '--disable-syslog' to
> >./configure or '--without syslog' to rpmbuild) and run time (having a
> >file /var/log/glusterd/logger.conf and restarting gluster services).
> >>
> >> As rsyslog is used as syslog server in Fedora and CentOS/RHEL and
> >default configuration of rsyslog does not have any rule specific to
> >gluster logs, you see all logs are in /var/log/messages in JSON format.
> >>
> >> Below is the way to make them neat and clean.
> >>
> >> For fedora users:
> >> 1. It requires to install rsyslog-mmjsonparse rpm (yum -y install
> >rsyslog-mmjsonparse)
> >> 2. Place below configuration under /etc/rsyslog.d/gluster.conf file.
> >>
> >> #$RepeatedMsgReduction on
> >>
> >> $ModLoad mmjsonparse
> >> *.* :mmjsonparse:
> >>
> >> template (name="GlusterLogFile" type="string"
> >string="/var/log/gluster/%app-name%.log")
> >> template (name="GlusterPidLogFile" type="string"
> >string="/var/log/gluster/%app-name%-%procid%.log")
> >>
> >> template(name="GLFS_template" type="list") {
> >>     property(name="$!mmcount")
> >>     constant(value="/")
> >>     property(name="syslogfacility-text" caseConversion="upper")
> >>     constant(value="/")
> >>     property(name="syslogseverity-text" caseConversion="upper")
> >>     constant(value=" ")
> >>     constant(value="[")
> >>     property(name="timereported" dateFormat="rfc3339")
> >>     constant(value="] ")
> >>     constant(value="[")
> >>     property(name="$!gf_code")
> >>     constant(value="] ")
> >>     constant(value="[")
> >>     property(name="$!gf_message")
> >>     constant(value="] ")
> >>     property(name="$!msg")
> >>     constant(value="\n")
> >> }
> >>
> >> if $app-name == 'gluster' or $app-name == 'glusterd' then {
> >>     action(type="omfile"
> >>            DynaFile="GlusterLogFile"
> >>            Template="GLFS_template")
> >>     stop
> >> }
> >>
> >> if $app-name contains 'gluster' then {
> >>     action(type="omfile"
> >>            DynaFile="GlusterPidLogFile"
> >>            Template="GLFS_template")
> >>     stop
> >> }
> >>
> >>
> >> 3. Restart rsyslog (service rsyslog restart)
> >> 4. Done. All gluster process specific logs are separated into
> >/var/log/gluster/ directory
> >>
> >>
> >> Note: Fedora 19 users
> >> There is a bug in rsyslog of fedora 19 [4], so its required to
> >recompile rsyslog source rpm downloaded from fedora repository
> >('rpmbuild --rebuild rsyslog-7.2.6-1.fc19.src.rpm' works fine) and use
> >generated rsyslog and rsyslog-mmjsonparse binary rpms
> >>
> >> For CentOS/RHEL users:
> >> Current rsyslog available in CentOS/RHEL does not have json support.
> >I have added the support which requires some testing.  I will update
> >once done.
> >>
> >>
> >> TODO:
> >> 1. need to add volume:brick specific tag to logging so that those
> >logs can be separated out than pid.
> >> 2. enable gfapi to use this logging framework
> >>
> >>
> >> I would like to get feedback/suggestion about this logging framework
> >>
> >>
> >> Regards,
> >> Bala
> >>
> >>
> >> [1] http://review.gluster.org/4977
> >> [2] http://review.gluster.org/5002
> >> [3] http://review.gluster.org/4915
> >> [4] https://bugzilla.redhat.com/show_bug.cgi?id=989886
> >> _______________________________________________
> >> Gluster-users mailing list
> >> Gluster-users at gluster.org
> >> http://supercolony.gluster.org/mailman/listinfo/gluster-users
> >>
> >
> >_______________________________________________
> >Gluster-users mailing list
> >Gluster-users at gluster.org
> >http://supercolony.gluster.org/mailman/listinfo/gluster-users
>