<div dir="ltr"><div>Thank you all for participating in this discussion.<br><br></div>Regarding Yaniv's comments:<br><span class="gmail-im"></span><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="font-family:arial,helvetica,sans-serif" class="gmail_default">If
you need an external tool (please not Java - let's not add another
language to the project), you might as well move to binary logging.</div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default">I believe we need to be able to do this sort using Linux command line tools only.</div></blockquote><div>My intention was not at all to perform ordering using Java.<br></div><div>Ordering of the logs can be done easily using Python or even C (these are the tools that I know). <br></div><div>I'd be happy to know how this can be done with just Linux CLI, please share your insight.<br></div><div>Regarding Java - I only meant to use it as a GUI tool (Java Swift).<br></div><div>A GUI tool that presents ordered logs in a couple of ways (e.g. a tab that shows the sorted logs for all threads, separate tabs per threads etc') may add some value.<br></div><div>Regarding the binary logging, actually the system I proposed is already "semi binary", as it logs time-stamp as raw hex. A switch to full binary is fairly simple and I do see advantages with that proposal. <br></div><div><br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="font-family:arial,helvetica,sans-serif" class="gmail_default">This is not a fair comparison:</div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default">1. The regression tests are running with debug log<br></div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default">2. Not logging at all != replacing the logging framework - the new one will have its own overhead as well.</div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default">3.
I believe there's a lot of code that is not real life scenarios there -
such as dumping a lot of data there (which causes a lot of calls to
inode_find() and friends - for example).<br></div></blockquote><div>1 - Actually I'm not sure about this. need to verify. <br></div><div>2 - I haven't claimed that with a new system we'd suddenly have a 20% performance increase. I have pointed out a potential problematic influence of the current mechanism that users (and developers) may be unaware of (as Strahil's comment suggests).<br></div><div>3 - That's the power of a community. I encourage users and developers to perform further tests on the matter, with real life scenarios, so we'd have a better understanding of the impact.<br><br></div><div>Regarding Ravi's comments:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">maintaining causality of messages and working <br>
with command line text editors and tools on log files is important IMO.<br></blockquote><div>Will running a tool in the form of "# logOrderer /someDireWithLogs" and having logs sorted in the way they are currently sorted will be so bad?<br></div><div>Furthermore, The system I proposed can easily produce ordered logs if no threads are registered for a private buffer (As I mentioned, and as the project documentation mentions, if a thread doesn;t have a private buffer, he automatically falls down to "level 2" writing, which is writing to a shared buffer - basic async logging. Level 2 and 3 maintain log ordering).<br><br></div><div>I think at this point we can focus the discussion at 2 points:<br></div><div>1 - Do we want to change the current system?<br></div><div>The current mechanism is "synced" logging, which definitely hurts performance. Are we OK with taking that hit or do we want to improve?<br></div><div>"Async logging" is not a new concept it certainly has it's advantages over "synced" logging.<br></div><div>2 - Given that the answer for question 1 is "yes", what do we require from the new logging system?<br></div><div>I proposed a system I've been developing as a side project for the past couple of weeks and I'd appreciate looking at the proposed mechanism if comments are made specifically on it (and remember that the project is still a work in progress).<br></div><div>Lastly, there are a lot of logging alternatives out there (e.g. Log4c) which are definitely worth consideration.<br><br></div><div>Lastly I want to adress Strahil's comment:</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div>As an end user, I think that performance improvents must be of outmost priority and this 'async logging' approach makes sense.<br></div><div>Actually, you make me think If I really need such detailed
logs (I'm running an oVirt lab) , as I can benefit from logless
gluster's performance.</div></blockquote><div>Obviously there are many different users with many different use cases out there and I believe we should be flexible enough to provide them with a suitable solution for their needs. I'd hate to see users turn off logging just because it hurts their performance as it would hurt our ability as developers to provide support when needed.<br></div><div><br></div><div>Again, thank you for participating and looking forward for more comments and input, <br></div>
</div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Nov 22, 2019 at 12:19 PM Ravishankar N <<a href="mailto:ravishankar@redhat.com">ravishankar@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>
On 22/11/19 3:13 pm, Barak Sason Rofman wrote:<br>
> This is actually one of the main reasons I wanted to bring this up for <br>
> discussion - will it be fine with the community to run a dedicated <br>
> tool to reorder the logs offline?<br>
<br>
I think it is a bad idea to log without ordering and later relying on an <br>
external tool to sort it. This is definitely not something I would want <br>
to do while doing test and development or debugging field issues. <br>
Structured logging is definitely useful for gathering statistics and <br>
post-processing to make reports and charts and what not, but from a <br>
debugging point of view, maintaining causality of messages and working <br>
with command line text editors and tools on log files is important IMO.<br>
<br>
I had a similar concerns when brick multiplexing feature was developed <br>
where a single log file was used for logging all multiplexed bricks' <br>
logs. So much extra work to weed out messages of 99 processes to read <br>
the log of the 1 process you are interested in.<br>
<br>
Regards,<br>
Ravi<br>
<br>
</blockquote></div><br clear="all"><br>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div style="margin-left:44px"><div style="margin-bottom:0px;margin-left:0px;padding-bottom:5px"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><font face="RedHatText, sans-serif"><span style="font-size:14px;text-transform:capitalize"><b>Barak Sason Rofman</b></span></font><br></div></div></div></div></div></div></div></div></div></div><div style="margin-left:44px"><div style="margin-bottom:0px;margin-left:0px;padding-bottom:5px"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><p style="margin:0px;font-size:14px;color:rgb(0,0,0);font-family:overpass,sans-serif;font-weight:bold;padding:0px;text-transform:uppercase"><span style="font-family:RedHatText,sans-serif;font-size:12px;text-transform:capitalize;font-weight:normal;color:rgb(34,34,34)">Gluster Storage </span><span style="font-family:RedHatText,sans-serif;font-size:12px;text-transform:capitalize;font-weight:normal;color:rgb(34,34,34)">Development</span></p></div></div></div></div></div></div></div></div></div></div><div style="margin-left:44px"><div style="margin-bottom:0px;margin-left:0px;padding-bottom:5px"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><p style="margin:0px 0px 4px;font-size:12px;font-family:RedHatText,sans-serif"><a href="https://www.redhat.com/" style="color:rgb(0,136,206);margin:0px" target="_blank">Red Hat Israel</a></p><div style="margin-bottom:4px"><p style="margin:0px;font-size:12px;font-family:RedHatText,sans-serif">34 Jerusalem rd. Ra'anana, 43501</p></div><p style="margin:0px;font-size:12px;font-family:RedHatText,sans-serif"><span style="margin:0px;padding:0px">bsasonro<a href="mailto:adi@redhat.com" style="color:rgb(0,0,0);margin:0px" target="_blank">@redhat.com</a> </span> T: <u>+972-9-7692304</u><br>M: <u>+972-52-4326355</u></p><div style="margin-top:12px"><table border="0"><tbody><tr><td width="100px"><a href="https://red.ht/sig" style="color:rgb(17,85,204)" target="_blank"><img src="https://static.redhat.com/libs/redhat/brand-assets/latest/corp/logo.png" width="90" height="auto"></a></td></tr></tbody></table></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div>