[Gluster-devel] new glusterfs logging framework

[Balamurugan Arumugam]

Hi All,

Recently new logging framework was introduced [1][2][3] in glusterfs master branch.  You could read more about this on doc/logging.txt.  In brief, current log target is moved to syslog and user has an option to this new logging at compile time (passing '--disable-syslog' to ./configure or '--without syslog' to rpmbuild) and run time (having a file /var/log/glusterd/logger.conf and restarting gluster services).

As rsyslog is used as syslog server in Fedora and CentOS/RHEL and default configuration of rsyslog does not have any rule specific to gluster logs, you see all logs are in /var/log/messages in JSON format.

Below is the way to make them neat and clean.

For fedora users:
1. It requires to install rsyslog-mmjsonparse rpm (yum -y install rsyslog-mmjsonparse)
2. Place below configuration under /etc/rsyslog.d/gluster.conf file.

#$RepeatedMsgReduction on

$ModLoad mmjsonparse
*.* :mmjsonparse:

template (name="GlusterLogFile" type="string" string="/var/log/gluster/%app-name%.log")
template (name="GlusterPidLogFile" type="string" string="/var/log/gluster/%app-name%-%procid%.log")

template(name="GLFS_template" type="list") {
   property(name="$!mmcount")
   constant(value="/")
   property(name="syslogfacility-text" caseConversion="upper")
   constant(value="/")
   property(name="syslogseverity-text" caseConversion="upper")
   constant(value=" ")
   constant(value="[")
   property(name="timereported" dateFormat="rfc3339")
   constant(value="] ")
   constant(value="[")
   property(name="$!gf_code")
   constant(value="] ")
   constant(value="[")
   property(name="$!gf_message")
   constant(value="] ")
   property(name="$!msg")
   constant(value="\n")
}

if $app-name == 'gluster' or $app-name == 'glusterd' then {
   action(type="omfile"
          DynaFile="GlusterLogFile"
          Template="GLFS_template")
   stop
}

if $app-name contains 'gluster' then {
   action(type="omfile"
          DynaFile="GlusterPidLogFile"
          Template="GLFS_template")
   stop
}


3. Restart rsyslog (service rsyslog restart)
4. Done. All gluster process specific logs are separated into /var/log/gluster/ directory


Note: Fedora 19 users
There is a bug in rsyslog of fedora 19 [4], so its required to recompile rsyslog source rpm downloaded from fedora repository ('rpmbuild --rebuild rsyslog-7.2.6-1.fc19.src.rpm' works fine) and use generated rsyslog and rsyslog-mmjsonparse binary rpms

For CentOS/RHEL users:
Current rsyslog available in CentOS/RHEL does not have json support.  I have added the support which requires some testing.  I will update once done.


TODO:
1. need to add volume:brick specific tag to logging so that those logs can be separated out than pid.
2. enable gfapi to use this logging framework


I would like to get feedback/suggestion about this logging framework


Regards,
Bala


[1] http://review.gluster.org/4977
[2] http://review.gluster.org/5002
[3] http://review.gluster.org/4915
[4] https://bugzilla.redhat.com/show_bug.cgi?id=989886