[Gluster-devel] [Gluster-users] new glusterfs logging framework

Balamurugan Arumugam barumuga at redhat.com
Wed Jul 31 09:52:20 UTC 2013 


----- Original Message -----
> From: "Anand Avati" <anand.avati at gmail.com>
> To: "Balamurugan Arumugam" <barumuga at redhat.com>
> Cc: "gluster-users" <gluster-users at gluster.org>, "Pablo" <paa.listas at gmail.com>, "Gluster Devel"
> <gluster-devel at nongnu.org>
> Sent: Wednesday, July 31, 2013 12:27:57 PM
> Subject: Re: [Gluster-devel] [Gluster-users] new glusterfs logging framework
> 
> 
> On Tue, Jul 30, 2013 at 11:39 PM, Balamurugan Arumugam < barumuga at redhat.com
> > wrote:
> 
> 
> 
> 
> 
> ----- Original Message -----
> > From: "Joe Julian" < joe at julianfamily.org >
> > To: "Pablo" < paa.listas at gmail.com >, "Balamurugan Arumugam" <
> > bala at gluster.com >
> > Cc: gluster-users at gluster.org , gluster-devel at nongnu.org
> > Sent: Tuesday, July 30, 2013 9:26:55 PM
> > Subject: Re: [Gluster-users] new glusterfs logging framework
> > 
> > Configuration files should be under /etc per FSH standards. Move the
> > logger.conf to /etc/glusterfs.
> > 
> 
> This will be done.
> 
> 
> > I, personally, like json logs since I'm shipping to logstash. :-) My one
> > suggestion would be to ensure the timestamps are in rfc3164.
> > 
> 
> rsyslog supports rfc3339 (a profile of ISO8601) and we use this. Let me know
> your thoughts on continue using it.
> 
> 
> > Yes, those are complex steps, but the rpm/deb packaging should take care of
> > dependencies and setting up logical defaults.
> > 
> 
> Yes. I am planning to add rsyslog configuration for gluster at install time.
> 
> 
> > IMHO, since this is a departure from the way it's been before now, the
> > config
> > file should enable this new behavior, not disable it, to avoid breaking
> > existing monitoring installations.
> > 
> 
> Do you mean to continue current logging in addition to syslog way?
> 
> 
> This means unless explicitly configured with syslog, by default we should be
> logging to gluster logs as before.
> 

Ok.  This will be done.


Regards,
Bala


> Avati
> 
> 
> 
> Regards,
> Bala
> 
> 
> > Pablo < paa.listas at gmail.com > wrote:
> > >I think that adding all that 'rsyslog' configuration only to see logs
> > >is
> > >too much. (I admit it, I don't know how to configure rsyslog at that
> > >level so that may influence my opinion)
> > > 
> > >Regards,
> > > 
> > > 
> > >El 30/07/2013 06:29 a.m., Balamurugan Arumugam escribió:
> > >> Hi All,
> > >> 
> > >> Recently new logging framework was introduced [1][2][3] in glusterfs
> > >master branch. You could read more about this on doc/logging.txt. In
> > >brief, current log target is moved to syslog and user has an option to
> > >this new logging at compile time (passing '--disable-syslog' to
> > >./configure or '--without syslog' to rpmbuild) and run time (having a
> > >file /var/log/glusterd/logger.conf and restarting gluster services).
> > >> 
> > >> As rsyslog is used as syslog server in Fedora and CentOS/RHEL and
> > >default configuration of rsyslog does not have any rule specific to
> > >gluster logs, you see all logs are in /var/log/messages in JSON format.
> > >> 
> > >> Below is the way to make them neat and clean.
> > >> 
> > >> For fedora users:
> > >> 1. It requires to install rsyslog-mmjsonparse rpm (yum -y install
> > >rsyslog-mmjsonparse)
> > >> 2. Place below configuration under /etc/rsyslog.d/gluster.conf file.
> > >> 
> > >> #$RepeatedMsgReduction on
> > >> 
> > >> $ModLoad mmjsonparse
> > >> *.* :mmjsonparse:
> > >> 
> > >> template (name="GlusterLogFile" type="string"
> > >string="/var/log/gluster/%app-name%.log")
> > >> template (name="GlusterPidLogFile" type="string"
> > >string="/var/log/gluster/%app-name%-%procid%.log")
> > >> 
> > >> template(name="GLFS_template" type="list") {
> > >> property(name="$!mmcount")
> > >> constant(value="/")
> > >> property(name="syslogfacility-text" caseConversion="upper")
> > >> constant(value="/")
> > >> property(name="syslogseverity-text" caseConversion="upper")
> > >> constant(value=" ")
> > >> constant(value="[")
> > >> property(name="timereported" dateFormat="rfc3339")
> > >> constant(value="] ")
> > >> constant(value="[")
> > >> property(name="$!gf_code")
> > >> constant(value="] ")
> > >> constant(value="[")
> > >> property(name="$!gf_message")
> > >> constant(value="] ")
> > >> property(name="$!msg")
> > >> constant(value="\n")
> > >> }
> > >> 
> > >> if $app-name == 'gluster' or $app-name == 'glusterd' then {
> > >> action(type="omfile"
> > >> DynaFile="GlusterLogFile"
> > >> Template="GLFS_template")
> > >> stop
> > >> }
> > >> 
> > >> if $app-name contains 'gluster' then {
> > >> action(type="omfile"
> > >> DynaFile="GlusterPidLogFile"
> > >> Template="GLFS_template")
> > >> stop
> > >> }
> > >> 
> > >> 
> > >> 3. Restart rsyslog (service rsyslog restart)
> > >> 4. Done. All gluster process specific logs are separated into
> > >/var/log/gluster/ directory
> > >> 
> > >> 
> > >> Note: Fedora 19 users
> > >> There is a bug in rsyslog of fedora 19 [4], so its required to
> > >recompile rsyslog source rpm downloaded from fedora repository
> > >('rpmbuild --rebuild rsyslog-7.2.6-1.fc19.src.rpm' works fine) and use
> > >generated rsyslog and rsyslog-mmjsonparse binary rpms
> > >> 
> > >> For CentOS/RHEL users:
> > >> Current rsyslog available in CentOS/RHEL does not have json support.
> > >I have added the support which requires some testing. I will update
> > >once done.
> > >> 
> > >> 
> > >> TODO:
> > >> 1. need to add volume:brick specific tag to logging so that those
> > >logs can be separated out than pid.
> > >> 2. enable gfapi to use this logging framework
> > >> 
> > >> 
> > >> I would like to get feedback/suggestion about this logging framework
> > >> 
> > >> 
> > >> Regards,
> > >> Bala
> > >> 
> > >> 
> > >> [1] http://review.gluster.org/4977
> > >> [2] http://review.gluster.org/5002
> > >> [3] http://review.gluster.org/4915
> > >> [4] https://bugzilla.redhat.com/show_bug.cgi?id=989886
> > >> _______________________________________________
> > >> Gluster-users mailing list
> > >> Gluster-users at gluster.org
> > >> http://supercolony.gluster.org/mailman/listinfo/gluster-users
> > >> 
> > > 
> > >_______________________________________________
> > >Gluster-users mailing list
> > > Gluster-users at gluster.org
> > > http://supercolony.gluster.org/mailman/listinfo/gluster-users
> > 
> 
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> https://lists.nongnu.org/mailman/listinfo/gluster-devel
> 
> 
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> https://lists.nongnu.org/mailman/listinfo/gluster-devel
>

More information about the Gluster-devel mailing list