<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"><html>
<head>
<meta name="Generator" content="Zarafa WebAccess v7.1.14-51822">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>RE: [Gluster-users] Gluster native mount is really slow compared to nfs</title>
<style type="text/css">
body
{
font-family: Arial, Verdana, Sans-Serif ! important;
font-size: 12px;
padding: 5px 5px 5px 5px;
margin: 0px;
border-style: none;
background-color: #ffffff;
}
p, ul, li
{
margin-top: 0px;
margin-bottom: 0px;
}
</style>
</head>
<body>
<p>PS: I just tested between these 2:</p><p> </p><div>mount -t glusterfs -o <b>negative-timeout=1</b>,use-readdirp=no,log-level=WARNING,log-file=/var/log/glusterxxx.log 192.168.140.41:/www /var/www</div><div> </div><div>mount -t glusterfs -o use-readdirp=no,log-level=WARNING,log-file=/var/log/glusterxxx.log 192.168.140.41:/www /var/www</div><div> </div><div>So it means only 1 second negative timeout...</div><div> </div><p>In this particular test: ./smallfile_cli.py --top /var/www/test --host-set 192.168.140.41 --threads 8 --files 50000 --file-size 64 --record-size 64</p><div> </div><div> </div><div>The result is about 4 seconds with the negative timeout of 1 second defined and <u>many many minutes without the negative timeout</u> (I quit after 15 minutes of waiting)</div><div> </div><div>I will go over to some real world tests now to see how it performs there.</div><div> </div><div> </div><div>Regards</div><div>Jo</div><p> </p><p> </p><p><br /> </p><blockquote style="border-left: 2px solid #325FBA; padding-left: 5px;margin-left:5px;">-----Original message-----<br /><strong>From:</strong> Jo Goossens <jo.goossens@hosted-power.com><br /><strong>Sent:</strong> Tue 11-07-2017 18:23<br /><strong>Subject:</strong> Re: [Gluster-users] Gluster native mount is really slow compared to nfs<br /><strong>To:</strong> Vijay Bellur <vbellur@redhat.com>; <br /><strong>CC:</strong> gluster-users@gluster.org; <br /><style type="text/css">body { font-family: monospace; }</style> <style type="text/css"> .bodyclass { font-family: Arial, Verdana, Sans-Serif ! important; font-size: 12px; padding: 5px 5px 5px 5px; margin: 0px; border-style: none; background-color: #ffffff; } p, ul, li { margin-top: 0px; margin-bottom: 0px; } </style> <div><p>Hello Vijay,</p><p> </p><p> </p><p>What do you mean exactly? What info is missing?</p><p> </p><p>PS: I already found out that for this particular test all the difference is made by : negative-timeout=600 , when removing it, it's much much slower again.</p><p> </p><p> </p><p>Regards</p><p>Jo<br /> </p><blockquote style="border-left: 2px solid #325FBA; padding-left: 5px;margin-left:5px;">-----Original message-----<br /><strong>From:</strong> Vijay Bellur <vbellur@redhat.com><br /><strong>Sent:</strong> Tue 11-07-2017 18:16<br /><strong>Subject:</strong> Re: [Gluster-users] Gluster native mount is really slow compared to nfs<br /><strong>To:</strong> Jo Goossens <jo.goossens@hosted-power.com>; <br /><strong>CC:</strong> gluster-users@gluster.org; Joe Julian <joe@julianfamily.org>; <br /><style type="text/css">.bodyclass { font-family: monospace; }</style> <div dir="ltr"><br /><div><br /><div>On Tue, Jul 11, 2017 at 11:39 AM, Jo Goossens <span dir="ltr"><<a href="mailto:jo.goossens@hosted-power.com" onclick="parent.webclient.openWindow(this, 'createmail', 'index.php?load=dialog&task=createmail_standard&to=jo.goossens@hosted-power.com'); return false;" title="This external link will open in a new window" target="_blank">jo.goossens@hosted-power.com</a>></span> wrote:<br /><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div><p>Hello Joe,</p><p> </p><p> </p><p>I just did a mount like this (added the bold):</p><p> </p><div>mount -t glusterfs -o <b>attribute-timeout=600,entry-timeout=600,negative-timeout=600,fopen-keep-cache</b>,use-readdirp=no,log-level=WARNING,log-file=/var/log/glusterxxx.log 192.168.140.41:/www /var/www</div><div> </div><p>Results:</p><p> </p><div>root@app1:~/smallfile-master# ./smallfile_cli.py --top /var/www/test --host-set 192.168.140.41 --threads 8 --files 5000 --file-size 64 --record-size 64</div><div><div><div>smallfile version 3.0</div><div> hosts in test : ['192.168.140.41']</div><div> top test directory(s) : ['/var/www/test']</div><div> operation : cleanup</div><div> files/thread : 5000</div><div> threads : 8</div><div> record size (KB, 0 = maximum) : 64</div><div> file size (KB) : 64</div><div> file size distribution : fixed</div><div> files per dir : 100</div><div> dirs per dir : 10</div><div> threads share directories? : N</div><div> filename prefix :</div><div> filename suffix :</div><div> hash file number into dir.? : N</div><div> fsync after modify? : N</div><div> pause between files (microsec) : 0</div><div> finish all requests? : Y</div><div> stonewall? : Y</div><div> measure response times? : N</div><div> verify read? : Y</div><div> verbose? : False</div><div> log to stderr? : False</div><div> ext.attr.size : 0</div><div> ext.attr.count : 0</div><div> permute host directories? : N</div><div> remote program directory : /root/smallfile-master</div><div> network thread sync. dir. : /var/www/test/network_shared</div><div>starting all threads by creating starting gate file /var/www/test/network_shared/starting_gate.tmp</div></div></div><div>host = 192.168.140.41,thr = 00,elapsed = 1.232004,files = 5000,records = 0,status = ok</div><div>host = 192.168.140.41,thr = 01,elapsed = 1.148738,files = 5000,records = 0,status = ok</div><div>host = 192.168.140.41,thr = 02,elapsed = 1.130913,files = 5000,records = 0,status = ok</div><div>host = 192.168.140.41,thr = 03,elapsed = 1.183088,files = 5000,records = 0,status = ok</div><div>host = 192.168.140.41,thr = 04,elapsed = 1.220752,files = 5000,records = 0,status = ok</div><div>host = 192.168.140.41,thr = 05,elapsed = 1.228039,files = 5000,records = 0,status = ok</div><div>host = 192.168.140.41,thr = 06,elapsed = 1.216787,files = 5000,records = 0,status = ok</div><div>host = 192.168.140.41,thr = 07,elapsed = 1.229036,files = 5000,records = 0,status = ok</div><div>total threads = 8</div><div>total files = 40000</div><div>100.00% of requested files processed, minimum is 70.00</div><div>1.232004 sec elapsed time</div><div>32467.428972 files/sec</div><div> </div><p> </p><div>root@app1:~/smallfile-master# ./smallfile_cli.py --top /var/www/test --host-set 192.168.140.41 --threads 8 --files 50000 --file-size 64 --record-size 64</div><div>smallfile version 3.0</div><div> hosts in test : ['192.168.140.41']</div><div> top test directory(s) : ['/var/www/test']</div><div> operation : cleanup</div><div> files/thread : 50000</div><div> threads : 8</div><div> record size (KB, 0 = maximum) : 64</div><div> file size (KB) : 64</div><div> file size distribution : fixed</div><div> files per dir : 100</div><div> dirs per dir : 10</div><div> threads share directories? : N</div><div> filename prefix :</div><div> filename suffix :</div><div> hash file number into dir.? : N</div><div> fsync after modify? : N</div><div> pause between files (microsec) : 0</div><div> finish all requests? : Y</div><div> stonewall? : Y</div><div> measure response times? : N</div><div> verify read? : Y</div><div> verbose? : False</div><div> log to stderr? : False</div><div> ext.attr.size : 0</div><div> ext.attr.count : 0</div><div> permute host directories? : N</div><div> remote program directory : /root/smallfile-master</div><div> network thread sync. dir. : /var/www/test/network_shared</div><div>starting all threads by creating starting gate file /var/www/test/network_shared/starting_gate.tmp</div><div>host = 192.168.140.41,thr = 00,elapsed = 4.242312,files = 50000,records = 0,status = ok</div><div>host = 192.168.140.41,thr = 01,elapsed = 4.250831,files = 50000,records = 0,status = ok</div><div>host = 192.168.140.41,thr = 02,elapsed = 3.771269,files = 50000,records = 0,status = ok</div><div>host = 192.168.140.41,thr = 03,elapsed = 4.060653,files = 50000,records = 0,status = ok</div><div>host = 192.168.140.41,thr = 04,elapsed = 3.880653,files = 50000,records = 0,status = ok</div><div>host = 192.168.140.41,thr = 05,elapsed = 3.847107,files = 50000,records = 0,status = ok</div><div>host = 192.168.140.41,thr = 06,elapsed = 3.895537,files = 50000,records = 0,status = ok</div><div>host = 192.168.140.41,thr = 07,elapsed = 3.966394,files = 50000,records = 0,status = ok</div><div>total threads = 8</div><div>total files = 400000</div><div>100.00% of requested files processed, minimum is 70.00</div><div>4.250831 sec elapsed time</div><div>94099.245073 files/sec</div><div>root@app1:~/smallfile-master#</div><div> </div><p> </p><p>As you can see it's now crazy fast, I think close to or faster than nfs !! What the hell!??!</p><p> </p><p>I'm so exited I already post. Any suggestions for those parameters? I will do additional testing over here , because this is ridiculous. That woud mean defaults or no good at all...</p><p> </p></div></blockquote><div> </div><div>Would it be possible to profile the client [1] with defaults and the set of options used now? That could help in understanding the performance delta better.</div><div> </div><div>Thanks,</div><div>Vijay</div><div> </div><div>[1] <a href="https://gluster.readthedocs.io/en/latest/Administrator%20Guide/Performance%20Testing/#client-side-profiling" title="This external link will open in a new window" target="_blank">https://gluster.readthedocs.io/en/latest/Administrator%20Guide/Performance%20Testing/#client-side-profiling</a> </div><div> </div></div></div></div> </blockquote></div> <pre>
_______________________________________________<br /> Gluster-users mailing list<br /> Gluster-users@gluster.org<br /> http://lists.gluster.org/mailman/listinfo/gluster-users</pre> </blockquote>
</body>
</html>