<div dir="ltr">Hi,<div><br></div><div>I have been conducting performance tests over the past day on our new HW where we plan to deploy a scalable file-system solution. I hope the results can be helpful to someone.</div><div>I hope to receive some feedback regarding optimizations and volume xlator setup.</div><div>If necessary volume profiling has been collected.</div><div><br></div><div>Regards</div><div>Davide </div><div><br></div><div><span id="gmail-docs-internal-guid-7645d85e-7fff-e5e2-646d-d9a562c629c1"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">The Clients</span></p><div dir="ltr" style="margin-left:0pt"><table style="border:none;border-collapse:collapse"><colgroup><col width="91"><col width="165"></colgroup><tbody><tr style="height:15pt"><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">nr of clients</span></p></td><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;text-align:right"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">6</span></p></td></tr><tr style="height:15pt"><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Network</span></p></td><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;text-align:right"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">10Gb</span></p></td></tr><tr style="height:15pt"><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Clients Mem</span></p></td><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;text-align:right"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">128GB</span></p></td></tr><tr style="height:26pt"><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Clients Cores</span></p></td><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;text-align:right"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">22</span></p></td></tr><tr style="height:15pt"><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Centos</span></p></td><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;text-align:right"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">7.5.1804</span></p></td></tr><tr style="height:26pt"><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Kernel</span></p></td><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;text-align:right"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">3.10.0-862.14.4.el7.x86_64</span></p></td></tr></tbody></table></div><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">The Servers</span></p><div dir="ltr" style="margin-left:0pt"><table style="border:none;border-collapse:collapse"><colgroup><col width="234"><col width="165"><col width="100"></colgroup><tbody><tr style="height:15pt"><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">nr of servers</span></p></td><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;text-align:right"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">3</span></p></td><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><br></td></tr><tr style="height:15pt"><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Network</span></p></td><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;text-align:right"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">100Gb</span></p></td><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204) rgb(0,0,0) rgb(204,204,204) rgb(204,204,204);vertical-align:bottom;padding:2pt 0pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">*node to node is 100Gb, to clients 10Gb</span></p></td></tr><tr style="height:15pt"><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Server Mem</span></p></td><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;text-align:right"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">377GB</span></p></td><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><br></td></tr><tr style="height:15pt"><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Server Cores</span></p></td><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;text-align:right"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">56</span></p></td><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt 0pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">*Intel 5120 CPU</span></p></td></tr><tr style="height:15pt"><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Storage</span></p></td><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;text-align:right"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">4x8TB NVME</span></p></td><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><br></td></tr><tr style="height:15pt"><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Centos</span></p></td><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;text-align:right"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">7.5.1804</span></p></td><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><br></td></tr><tr style="height:26pt"><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Kernel</span></p></td><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;text-align:right"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">3.10.0-862.14.4.el7.x86_64</span></p></td><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><br></td></tr></tbody></table></div><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Gluster version</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Both clients and servers running glusterfs 4.1.5 (glusterd not glusterd2)</span></p><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Brick Setup</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">The Bricks have been automatically configured by heketi at the volume creation resulting in:</span></p><ul style="margin-top:0pt;margin-bottom:0pt"><li dir="ltr" style="list-style-type:disc;font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">1 VG per NVME disk</span></p></li><li dir="ltr" style="list-style-type:disc;font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">1 thinpool with one LV</span></p></li><li dir="ltr" style="list-style-type:disc;font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">1 LV mapped to one brick</span></p></li><li dir="ltr" style="list-style-type:disc;font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">1 x 3 = 3</span></p></li></ul><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">The tests</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">The tests have been carried out using smallfile utility:</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><a href="https://github.com/distributed-system-analysis/smallfile" style="text-decoration-line:none"><span style="font-size:11pt;font-family:Arial;background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;text-decoration-line:underline;vertical-align:baseline;white-space:pre-wrap">https://github.com/distributed-system-analysis/smallfile</span></a></p><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">A set of comparative tests have been carried out between the following platforms, these tests include gluster volume profiling:</span></p><ul style="margin-top:0pt;margin-bottom:0pt"><li dir="ltr" style="list-style-type:disc;font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">proprietary appliance NVME over iSCSI, top of the range (1 client only)</span></p></li><li dir="ltr" style="list-style-type:disc;font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Proprietary appliance SSD service NFS, top of the range</span></p></li><li dir="ltr" style="list-style-type:disc;font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Gluster 3 nodes cluster specs above</span></p></li></ul><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">A set of longer running  and resilience tests for only Glusterfs, in these tests the system graph metrics are available and physically unplugging drives/nodes has been done </span></p><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Volume profiling:</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Has shown by the 8threads 4K test below, gluster volume profiling did not incur in any performance degradation so has been left on for all the 5K files tests. Volume profiling results are enclosed.</span></p><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Gluster volume options</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">The following volume options have been configured based on previous experiences. Some of these options have been tested default VS custom as shown by the 4threads test.</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Other options haven’t been explicitly set since already enabled at the default values. </span></p><div dir="ltr" style="margin-left:0pt"><table style="border:none;border-collapse:collapse"><colgroup><col width="234"></colgroup><tbody><tr style="height:15pt"><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Options Reconfigured:</span></p></td></tr><tr style="height:15pt"><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">client.event-threads 3</span></p></td></tr><tr style="height:15pt"><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">performance.cache-size 8GB</span></p></td></tr><tr style="height:15pt"><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">performance.io-thread-count 24</span></p></td></tr><tr style="height:15pt"><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">network.inode-lru-limit 1048576</span></p></td></tr><tr style="height:15pt"><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">performance.parallel-readdir on</span></p></td></tr><tr style="height:15pt"><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">performance.cache-invalidation on</span></p></td></tr><tr style="height:15pt"><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">performance.md-cache-timeout 600</span></p></td></tr><tr style="height:15pt"><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">features.cache-invalidation on</span></p></td></tr><tr style="height:26pt"><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">features.cache-invalidation-timeout 600</span></p></td></tr><tr style="height:15pt"><td style="border-width:0.75pt;border-style:solid;border-color:rgb(204,204,204);vertical-align:bottom;padding:2pt"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">performance.client-io-threads on</span></p></td></tr></tbody></table></div><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">The Cache</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">I did not noticed any difference in re-mounting the share or dropping the cache in /proc and in real world i want to use any cache available as much as possible so none of the tests have been clearing caches or re-mounting. To note however that performing two subsequent stat operations resulted in aprox 10x faster FOPS, this test result is not recorded.</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><br></span></p><p style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><b>The results</b></span></p><p style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Attached</span></p></span><br class="gmail-Apple-interchange-newline"></div></div>