[Gluster-devel] new glusterfs logging framework

[Niels de Vos]

On Tue, Jul 30, 2013 at 05:29:43AM -0400, Balamurugan Arumugam wrote:
> 
> Hi All,
> 
> Recently new logging framework was introduced [1][2][3] in glusterfs 
> master branch.  You could read more about this on doc/logging.txt.  In 
> brief, current log target is moved to syslog and user has an option to 
> this new logging at compile time (passing '--disable-syslog' to 
> ./configure or '--without syslog' to rpmbuild) and run time (having 
> a file /var/log/glusterd/logger.conf and restarting gluster services).
> 
> As rsyslog is used as syslog server in Fedora and CentOS/RHEL and 
> default configuration of rsyslog does not have any rule specific to 
> gluster logs, you see all logs are in /var/log/messages in JSON 
> format.

As the below does not function on RHEL-6 with its own rsyslog (no 
rsyslog-mmjsonparse available), I am creating an empty 
/var/lib/glusterd/logger.conf which (after a restart) makes glusterd and 
friends to log to their standard files again.

Do you have some progress on the rsyslog-mmjsonparse for rhel5 and 
rhel6?

Also, it would really be nice to include that 
/etc/rsyslog.d/gluster.conf in the rpms and add a dependency on the 
rsyslog-mmjsonparse package. Or, provide a /var/lib/glusterd/logger.conf 
for the distributions that do not have rsyslog-mmjsonparse.

Thanks,
Niels

> 
> Below is the way to make them neat and clean.
> 
> For fedora users:
> 1. It requires to install rsyslog-mmjsonparse rpm (yum -y install rsyslog-mmjsonparse)
> 2. Place below configuration under /etc/rsyslog.d/gluster.conf file.
> 
> #$RepeatedMsgReduction on
> 
> $ModLoad mmjsonparse
> *.* :mmjsonparse:
> 
> template (name="GlusterLogFile" type="string" string="/var/log/gluster/%app-name%.log")
> template (name="GlusterPidLogFile" type="string" string="/var/log/gluster/%app-name%-%procid%.log")
> 
> template(name="GLFS_template" type="list") {
>    property(name="$!mmcount")
>    constant(value="/")
>    property(name="syslogfacility-text" caseConversion="upper")
>    constant(value="/")
>    property(name="syslogseverity-text" caseConversion="upper")
>    constant(value=" ")
>    constant(value="[")
>    property(name="timereported" dateFormat="rfc3339")
>    constant(value="] ")
>    constant(value="[")
>    property(name="$!gf_code")
>    constant(value="] ")
>    constant(value="[")
>    property(name="$!gf_message")
>    constant(value="] ")
>    property(name="$!msg")
>    constant(value="\n")
> }
> 
> if $app-name == 'gluster' or $app-name == 'glusterd' then {
>    action(type="omfile"
>           DynaFile="GlusterLogFile"
>           Template="GLFS_template")
>    stop
> }
> 
> if $app-name contains 'gluster' then {
>    action(type="omfile"
>           DynaFile="GlusterPidLogFile"
>           Template="GLFS_template")
>    stop
> }
> 
> 
> 3. Restart rsyslog (service rsyslog restart)
> 4. Done. All gluster process specific logs are separated into /var/log/gluster/ directory
> 
> 
> Note: Fedora 19 users
> There is a bug in rsyslog of fedora 19 [4], so its required to recompile rsyslog source rpm downloaded from fedora repository ('rpmbuild --rebuild rsyslog-7.2.6-1.fc19.src.rpm' works fine) and use generated rsyslog and rsyslog-mmjsonparse binary rpms
> 
> For CentOS/RHEL users:
> Current rsyslog available in CentOS/RHEL does not have json support.  I have added the support which requires some testing.  I will update once done.
> 
> 
> TODO:
> 1. need to add volume:brick specific tag to logging so that those logs can be separated out than pid.
> 2. enable gfapi to use this logging framework
> 
> 
> I would like to get feedback/suggestion about this logging framework
> 
> 
> Regards,
> Bala
> 
> 
> [1] http://review.gluster.org/4977
> [2] http://review.gluster.org/5002
> [3] http://review.gluster.org/4915
> [4] https://bugzilla.redhat.com/show_bug.cgi?id=989886
> 
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> https://lists.nongnu.org/mailman/listinfo/gluster-devel

-- 
Niels de Vos
Sr. Software Maintenance Engineer
Support Engineering Group
Red Hat Global Support Services