[Gluster-users] [Gluster-devel] Modifying gluster's logging mechanism

[Yaniv Kaul]

On Fri, Nov 22, 2019 at 11:45 AM Barak Sason Rofman <bsasonro at redhat.com>
wrote:

> Thank you for your input Atin and Xie Changlong.
>
> Regarding log ordering - my initial thought was to do it offline using a
> dedicated too. Should be straight forward, as the logs have time stamp
> composed of seconds and microseconds, so ordering them using this value is
> definitely possible.
> This is actually one of the main reasons I wanted to bring this up for
> discussion - will it be fine with the community to run a dedicated tool to
> reorder the logs offline?
> Reordering the logs offline will allow us to gain the most performance
> improvement, as keeping the logs order online will have some cost (probably
> through stronger synchronization).
> Moreover, we can take log viewing one step further and maybe create some
> GUI system (JAVA based?) to view and handle logs (e.g. one window to show
> the combined order logs, other windows to show logs per thread etc').
>

If you need an external tool (please not Java - let's not add another
language to the project), you might as well move to binary logging.
I believe we need to be able to do this sort using Linux command line tools
only.

>
> Regarding the test method - my initial testing was done by removing all
> logging from regression. Modifying the method "skip_logging" to return
> 'true' in all cases seems to remove most of the logs (though not all, "to
> be on the safe side", really removing all logging related methods is
> probably even better).
>

This is not a fair comparison:
1. The regression tests are running with debug log
2. Not logging at all != replacing the logging framework - the new one will
have its own overhead as well.
3. I believe there's a lot of code that is not real life scenarios there -
such as dumping a lot of data there (which causes a lot of calls to
inode_find() and friends - for example).
Y.

As regression tests use mostly single-node tests, some additional testing
> was needed. I've written a couple of very basic scripts to create large
> number of files / big files, read / write to / from them, move them around
> and perform some other basic functionality.
> I'd actually be glad to test this in some 'real world' use cases - if you
> have specific use cases that you use frequently, we can model them and
> benchmark against - this will likely offer an even more accurate benchmark.
>
> On Fri, Nov 22, 2019 at 7:27 AM Xie Changlong <zgrep at 139.com> wrote:
>
> >
> > 在 2019/11/21 21:04, Barak Sason Rofman 写道:
> >
> > I see two design / implementation problems with that mechanism:
> >
> >    1.
> >
> >    The mutex that guards the log file is likely under constant contention
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20191122/b47524ab/attachment.html>