<html><head></head><body>The reason for the long (42 second) ping-timeout is because re-establishing fd's and locks can be a very expensive operation. With an average MTBF of 45000 hours for a server, even just a replica 2 would result in a 42 second MTTR every 2.6 years, or 6 nines of uptime.<br><br><div class="gmail_quote">On December 27, 2017 3:17:01 AM PST, Omar Kohl <omar.kohl@iternity.com> wrote:<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<pre class="k9mail">Hi,<br /><br /><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;"> If you set it to 10 seconds, and a node goes down, you'll see a 10 seconds freez in all I/O for the volume.<br /></blockquote><br />Exactly! ONLY 10 seconds instead of the default 42 seconds :-)<br /><br />As I said before the problem with the 42 seconds is that a Windows Samba Client will disconnect (and therefore interrupt any read/write operation) after waiting for about 25 seconds. So 42 seconds is too high. In this case it would therefore make more sense to reduce the ping-timeout, right?<br /><br />Has anyone done any performance measurements on what the implications of a low ping-timeout are? What are the costs of "triggering heals all the time"?<br /><br />On a related note I found the extras/hook-scripts/start/post/<a href="http://S29CTDBsetup.sh">S29CTDBsetup.sh</a> script that mounts a CTDB (Samba) share and explicitly sets the ping-timeout to 10 seconds. There is a comment saying: "Make sure ping-timeout is not default for CTDB volume". Unfortunately there is no explanation in the script, in the commit or in the Gerrit review history (<a href="https://review.gluster.org/#/c/7569">https://review.gluster.org/#/c/7569</a>/, <a href="https://review.gluster.org/#/c/8007">https://review.gluster.org/#/c/8007</a>/) for WHY you make sure ping-timeout is not default. Can anyone tell me the reason?<br /><br />Kind regards,<br />Omar<br /><br />-----Ursprüngliche Nachricht-----<br />Von: gluster-users-bounces@gluster.org [mailto:gluster-users-bounces@gluster.org] Im Auftrag von lemonnierk@ulrar.net<br />Gesendet: Dienstag, 26. Dezember 2017 22:05<br />An: gluster-users@gluster.org<br />Betreff: Re: [Gluster-users] Exact purpose of <a href="http://network.ping">network.ping</a>-timeout<br /><br />Hi,<br /><br />It's just the delay for which a node can stop responding before being marked as down.<br />Basically that's how long a node can go down before a heal becomes necessary to bring it back.<br /><br />If you set it to 10 seconds, and a node goes down, you'll see a 10 seconds freez in all I/O for the volume. That's why you don't want it too high (having a 2 minutes freez on I/O for example would be pretty bad, depending on what you host), but you don't want it too low either (to avoid triggering heals all the time).<br /><br />You can configure it because it depends on what you host. You might be okay with a few minutes freez to avoid a heal, or you might not care about heals at all and prefer a very low value to avoid feezes.<br />The default value should work pretty well for most things though<br /><br />On Tue, Dec 26, 2017 at 01:11:48PM +0000, Omar Kohl wrote:<br /><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;"> Hi,<br /> <br /> I have a question regarding the "ping-timeout" option. I have been researching its purpose for a few days and it is not completely clear to me. Especially that it is apparently strongly encouraged by the Gluster community not to change or at least decrease this value!<br /> <br /> Assuming that I set ping-timeout to 10 seconds (instead of the default 42) this would mean that if I have a network outage of 11 seconds then Gluster internally would have to re-allocate some resources that it freed after the 10 seconds, correct? But apart from that there are no negative implications, are there? For instance if I'm copying files during the network outage then those files will continue copying after those 11 seconds.<br /> <br /> This means that the only purpose of ping-timeout is to save those extra resources that are used by "short" network outages. Is that correct?<br /> <br /> If I am confident that my network will not have many 11 second outages and if they do occur I am willing to incur those extra costs due to resource allocation is there any reason not to set ping-timeout to 10 seconds?<br /> <br /> The problem I have with a long ping-timeout is that the Windows Samba Client disconnects after 25 seconds. So if one of the nodes of a Gluster cluster shuts down ungracefully then the Samba Client disconnects and the file that was being copied is incomplete on the server. These "costs" seem to be much higher than the potential costs of those Gluster resource re-allocations. But it is hard to estimate because there is not clear documentation what exactly those Gluster costs are.<br /> <br /> In general I would be very interested in a comprehensive explanation of ping-timeout and the up- and downsides of setting high or low values for it.<br /> <br /> Kinds regards,<br /> Omar<br /><hr /><br /> Gluster-users mailing list<br /> Gluster-users@gluster.org<br /> <a href="http://lists.gluster.org/mailman/listinfo/gluster-users">http://lists.gluster.org/mailman/listinfo/gluster-users</a><br /></blockquote><hr /><br />Gluster-users mailing list<br />Gluster-users@gluster.org<br /><a href="http://lists.gluster.org/mailman/listinfo/gluster-users">http://lists.gluster.org/mailman/listinfo/gluster-users</a><br /></pre></blockquote></div><br>
-- <br>
Sent from my Android device with K-9 Mail. Please excuse my brevity.</body></html>