<div dir="ltr"><div dir="ltr"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Nov 22, 2019 at 11:45 AM Barak Sason Rofman <<a href="mailto:bsasonro@redhat.com">bsasonro@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Thank you for your input Atin and Xie Changlong.<br>
<br>
Regarding log ordering - my initial thought was to do it offline using a<br>
dedicated too. Should be straight forward, as the logs have time stamp<br>
composed of seconds and microseconds, so ordering them using this value is<br>
definitely possible.<br>
This is actually one of the main reasons I wanted to bring this up for<br>
discussion - will it be fine with the community to run a dedicated tool to<br>
reorder the logs offline?<br>
Reordering the logs offline will allow us to gain the most performance<br>
improvement, as keeping the logs order online will have some cost (probably<br>
through stronger synchronization).<br>
Moreover, we can take log viewing one step further and maybe create some<br>
GUI system (JAVA based?) to view and handle logs (e.g. one window to show<br>
the combined order logs, other windows to show logs per thread etc').<br></blockquote><div><br></div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default">If you need an external tool (please not Java - let's not add another language to the project), you might as well move to binary logging.</div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default">I believe we need to be able to do this sort using Linux command line tools only.</div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default"></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Regarding the test method - my initial testing was done by removing all<br>
logging from regression. Modifying the method "skip_logging" to return<br>
'true' in all cases seems to remove most of the logs (though not all, "to<br>
be on the safe side", really removing all logging related methods is<br>
probably even better).<br></blockquote><div><br></div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default">This is not a fair comparison:</div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default">1. The regression tests are running with debug log<br></div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default">2. Not logging at all != replacing the logging framework - the new one will have its own overhead as well.</div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default">3. I believe there's a lot of code that is not real life scenarios there - such as dumping a lot of data there (which causes a lot of calls to inode_find() and friends - for example).</div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default">Y.</div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default"><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
As regression tests use mostly single-node tests, some additional testing<br>
was needed. I've written a couple of very basic scripts to create large<br>
number of files / big files, read / write to / from them, move them around<br>
and perform some other basic functionality.<br>
I'd actually be glad to test this in some 'real world' use cases - if you<br>
have specific use cases that you use frequently, we can model them and<br>
benchmark against - this will likely offer an even more accurate benchmark.<br>
<br>
On Fri, Nov 22, 2019 at 7:27 AM Xie Changlong <<a href="mailto:zgrep@139.com" target="_blank">zgrep@139.com</a>> wrote:<br>
<br>
><br>
> 在 2019/11/21 21:04, Barak Sason Rofman 写道:<br>
><br>
> I see two design / implementation problems with that mechanism:<br>
><br>
> 1.<br>
><br>
> The mutex that guards the log file is likely under constant contention</blockquote></div></div>