<html><head></head><body><div style="color:#000; background-color:#fff; font-family:Courier New, courier, monaco, monospace, sans-serif;font-size:16px"><div id="yui_3_16_0_ym19_1_1548325460022_7272"><span>Dear Amar, Community,</span></div><div id="yui_3_16_0_ym19_1_1548325460022_7274"><span><br></span></div><div id="yui_3_16_0_ym19_1_1548325460022_7275" dir="ltr"><span id="yui_3_16_0_ym19_1_1548325460022_7299">it seems the issue is in the fuse client itself</span>.</div><div dir="ltr" id="yui_3_16_0_ym19_1_1548325460022_7384"><br></div><div dir="ltr" id="yui_3_16_0_ym19_1_1548325460022_7383">Here is the latest update:</div><div dir="ltr" id="yui_3_16_0_ym19_1_1548325460022_7300">1. I have added the following:</div><div dir="ltr" id="yui_3_16_0_ym19_1_1548325460022_7385">server.event-threads: 4<br id="yui_3_16_0_ym19_1_1548325460022_7416">client.event-threads: 4<br id="yui_3_16_0_ym19_1_1548325460022_7417">performance.stat-prefetch: on</div><div dir="ltr" id="yui_3_16_0_ym19_1_1548325460022_7610">performance.strict-o-direct: off<br id="yui_3_16_0_ym19_1_1548325460022_7418"></div><div dir="ltr" id="yui_3_16_0_ym19_1_1548325460022_7608">Results: no change</div><div dir="ltr" id="yui_3_16_0_ym19_1_1548325460022_7645"><br></div><div dir="ltr" id="yui_3_16_0_ym19_1_1548325460022_7662">2. Allowed nfs and connected ovirt1 to the gluster volume:</div><div dir="ltr" id="yui_3_16_0_ym19_1_1548325460022_7661">nfs.disable: off<br id="yui_3_16_0_ym19_1_1548325460022_7665"></div><div dir="ltr">Results: Drastic improvement in performance as follows:</div><div dir="ltr"><br></div><div dir="ltr" id="yui_3_16_0_ym19_1_1548325460022_7788">[root@ovirt1 data]# dd if=/dev/zero of=largeio bs=1M count=5000 status=progress<br id="yui_3_16_0_ym19_1_1548325460022_7800">5000+0 records in<br id="yui_3_16_0_ym19_1_1548325460022_7801">5000+0 records out<br id="yui_3_16_0_ym19_1_1548325460022_7802">5242880000 bytes (5.2 GB) copied, 53.0443 s, 98.8 MB/s</div><div dir="ltr" id="yui_3_16_0_ym19_1_1548325460022_7841"><br></div><div dir="ltr" id="yui_3_16_0_ym19_1_1548325460022_7961">So I would be happy if anyone guide me in order to fix the situation as the fuse client is the best way to use glusterfs, and it seems the glusterfs-server is not the guilty one.</div><div dir="ltr" id="yui_3_16_0_ym19_1_1548325460022_7962"><br></div><div dir="ltr">Thanks in advance for your guidance.I have learned so much.</div><div dir="ltr"><br></div><div dir="ltr">Best Regards,</div><div dir="ltr">Strahil Nikolov<br></div><div class="qtdSeparateBR"><br><br></div><div class="yahoo_quoted" id="yui_3_16_0_ym19_1_1548325460022_7305" style="display: block;"> <div style="font-family: Courier New, courier, monaco, monospace, sans-serif; font-size: 16px;" id="yui_3_16_0_ym19_1_1548325460022_7304"> <div style="font-family: HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif; font-size: 16px;" id="yui_3_16_0_ym19_1_1548325460022_7303"> <div dir="ltr" id="yui_3_16_0_ym19_1_1548325460022_7302"> <font id="yui_3_16_0_ym19_1_1548325460022_7301" size="2" face="Arial"> <hr size="1"> <b><span style="font-weight:bold;">От:</span></b> Strahil <hunter86_bg@yahoo.com><br> <b><span style="font-weight: bold;">До:</span></b> Amar Tumballi Suryanarayan <atumball@redhat.com> <br><b><span style="font-weight: bold;">Копие:</span></b> Gluster-users <gluster-users@gluster.org><br> <b><span style="font-weight: bold;">Изпратен:</span></b> сряда, 23 януари 2019 г. 18:44<br> <b><span style="font-weight: bold;">Тема:</span></b> Re: [Gluster-users] Gluster performance issues - need advise<br> </font> </div> <div class="y_msg_container" id="yui_3_16_0_ym19_1_1548325460022_7382"><br><div id="yiv6089672943"><div id="yui_3_16_0_ym19_1_1548325460022_7381"><div id="yui_3_16_0_ym19_1_1548325460022_7380"><div id="yui_3_16_0_ym19_1_1548325460022_7379">Dear Amar,<div><br clear="none"></div><div id="yui_3_16_0_ym19_1_1548325460022_7378">Thanks for your email.</div><div><br clear="none"></div><div>Actually my concerns were on both topics.</div><div>Would you recommend any perf options that will be suitable ?</div><div><br clear="none"></div><div>After mentioning the network usage, I just checked it and it seems duringthe test session, ovirt1 (both client and host) is using no more than 455Mbit/s which is half the network bandwidth.</div><div><br clear="none"></div><div>I'm still in the middle of nowhere, so any ideas are welcome.</div><div><br clear="none"></div><div>Best Regards,</div><div>Strahil Nikolov</div></div><div class="yiv6089672943yqt8450526150" id="yiv6089672943yqt03107"><div><br clear="none"><div class="yiv6089672943elided-text">On Jan 23, 2019 17:49, Amar Tumballi Suryanarayan <atumball@redhat.com> wrote:<br clear="none"><blockquote style="margin:0 0 0 0.8ex;border-left:1px #ccc solid;padding-left:1ex;"><div dir="ltr">I didn't understand the issue properly. Mostly I missed something.<div><br clear="none"></div><div>Are you concerned the performance is 49MB/s with and without perf options? or are you expecting it to be 123MB/s as over the n/w you get that speed?</div><div><br clear="none"></div><div>If it is the first problem, then you are actually having 'performance.write-behind on' in both options, and it is the only perf xlator which comes into action during the test you ran.</div><div><br clear="none"></div><div>If it is the second, then please be informed that gluster does client side replication, which means, n/w would be split in half for write operations (like write(), creat() etc), so the number you are getting is almost the maximum with 1GbE.</div><div><br clear="none"></div><div>Regards,</div><div>Amar</div></div><br clear="none"><div class="yiv6089672943elided-text"><div dir="ltr">On Wed, Jan 23, 2019 at 8:38 PM Strahil Nikolov <<a rel="nofollow" shape="rect" ymailto="mailto:hunter86_bg@yahoo.com" target="_blank" href="mailto:hunter86_bg@yahoo.com">hunter86_bg@yahoo.com</a>> wrote:<br clear="none"></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb( 204 , 204 , 204 );padding-left:1ex;">Hello Community,<br clear="none">
<br clear="none">
recently I have built a new lab based on oVirt and CentOS 7.<br clear="none">
During deployment I had some hicups, but now the engine is up and running - but gluster is causing me trouble.<br clear="none">
<br clear="none">
Symptoms: Slow VM install from DVD, poor write performance. The latter has been tested via:<br clear="none">
dd if=/dev/zero of=/rhev/data-center/mnt/glusterSD/ovirt1.localdomain:_data bs=1M count=1000 status=progress<br clear="none">
<br clear="none">
The reported speed is 60MB/s which is way too low for my setup.<br clear="none">
<br clear="none">
My lab design:<br clear="none">
<a rel="nofollow" shape="rect" target="_blank" href="https://drive.google.com/file/d/1SiW21ASPXHRAEuE_jZ50R3FoO-NcnFqT/view?usp=sharing">https://drive.google.com/file/d/1SiW21ASPXHRAEuE_jZ50R3FoO-NcnFqT/view?usp=sharing</a><br clear="none">
Gluster version is 3.12.15<br clear="none">
<br clear="none">
So far I have done:<br clear="none">
<br clear="none">
1. Added 'server.allow-insecure on' (with 'option rpc-auth-allow-insecure on' in glusterd.vol)<br clear="none">
Volume info after that change:<br clear="none">
<br clear="none">
Volume Name: data<br clear="none">
Type: Replicate<br clear="none">
Volume ID: 9b06a1e9-8102-4cd7-bc56-84960a1efaa2<br clear="none">
Status: Started<br clear="none">
Snapshot Count: 0<br clear="none">
Number of Bricks: 1 x (2 + 1) = 3<br clear="none">
Transport-type: tcp<br clear="none">
Bricks:<br clear="none">
Brick1: ovirt1.localdomain:/gluster_bricks/data/data<br clear="none">
Brick2: ovirt2.localdomain:/gluster_bricks/data/data<br clear="none">
Brick3: ovirt3.localdomain:/gluster_bricks/data/data (arbiter)<br clear="none">
Options Reconfigured:<br clear="none">
performance.client-io-threads: off<br clear="none">
nfs.disable: on<br clear="none">
transport.address-family: inet<br clear="none">
performance.quick-read: off<br clear="none">
performance.read-ahead: off<br clear="none">
performance.io-cache: off<br clear="none">
performance.low-prio-threads: 32<br clear="none">
network.remote-dio: off<br clear="none">
cluster.eager-lock: enable<br clear="none">
cluster.quorum-type: auto<br clear="none">
cluster.server-quorum-type: server<br clear="none">
cluster.data-self-heal-algorithm: full<br clear="none">
cluster.locking-scheme: granular<br clear="none">
cluster.shd-max-threads: 8<br clear="none">
cluster.shd-wait-qlength: 10000<br clear="none">
features.shard: on<br clear="none">
user.cifs: off<br clear="none">
storage.owner-uid: 36<br clear="none">
storage.owner-gid: 36<br clear="none">
network.ping-timeout: 30<br clear="none">
performance.strict-o-direct: on<br clear="none">
cluster.granular-entry-heal: enable<br clear="none">
server.allow-insecure: on<br clear="none">
<br clear="none">
Seems no positive or negative effect so far.<br clear="none">
<br clear="none">
2. Tested with tmpfs on all bricks -> ovirt1 mounted gluster volume -> max 60MB/s (bs=1M without 'oflag=direct')<br clear="none">
<br clear="none">
<br clear="none">
[root@ovirt1 data]# dd if=/dev/zero of=large_io bs=1M count=4000 status=progress<br clear="none">
4177526784 bytes (4.2 GB) copied, 70.843409 s, 59.0 MB/s<br clear="none">
4000+0 records in<br clear="none">
4000+0 records out<br clear="none">
4194304000 bytes (4.2 GB) copied, 71.1407 s, 59.0 MB/s<br clear="none">
[root@ovirt1 data]# rm -f large_io<br clear="none">
[root@ovirt1 data]# gluster volume profile data info<br clear="none">
Brick: ovirt1.localdomain:/gluster_bricks/data/data<br clear="none">
---------------------------------------------------<br clear="none">
Cumulative Stats:<br clear="none">
Block Size: 131072b+ <br clear="none">
No. of Reads: 8 <br clear="none">
No. of Writes: 44968 <br clear="none">
%-latency Avg-latency Min-Latency Max-Latency No. of calls Fop<br clear="none">
--------- ----------- ----------- ----------- ------------ ----<br clear="none">
0.00 0.00 us 0.00 us 0.00 us 3 FORGET<br clear="none">
0.00 0.00 us 0.00 us 0.00 us 35 RELEASE<br clear="none">
0.00 0.00 us 0.00 us 0.00 us 28 RELEASEDIR<br clear="none">
0.00 78.00 us 78.00 us 78.00 us 1 FSTAT<br clear="none">
0.00 35.67 us 26.00 us 73.00 us 6 FLUSH<br clear="none">
0.00 324.00 us 324.00 us 324.00 us 1 XATTROP<br clear="none">
0.00 45.80 us 38.00 us 54.00 us 10 STAT<br clear="none">
0.00 227.67 us 216.00 us 242.00 us 3 CREATE<br clear="none">
0.00 113.38 us 68.00 us 381.00 us 8 READ<br clear="none">
0.00 39.82 us 1.00 us 148.00 us 28 OPENDIR<br clear="none">
0.00 67.54 us 10.00 us 283.00 us 24 GETXATTR<br clear="none">
0.00 59.97 us 45.00 us 113.00 us 32 OPEN<br clear="none">
0.00 24.41 us 13.00 us 89.00 us 161 INODELK<br clear="none">
0.00 43.43 us 28.00 us 214.00 us 93 STATFS<br clear="none">
0.00 246.35 us 11.00 us 1155.00 us 20 READDIR<br clear="none">
0.00 283.00 us 233.00 us 353.00 us 18 READDIRP<br clear="none">
0.00 153.23 us 122.00 us 259.00 us 87 MKNOD<br clear="none">
0.01 99.77 us 10.00 us 258.00 us 442 LOOKUP<br clear="none">
0.31 49.22 us 27.00 us 540.00 us 45620 FXATTROP<br clear="none">
0.77 124.24 us 87.00 us 604.00 us 44968 WRITE<br clear="none">
0.93 15767.71 us 15.00 us 305833.00 us 431 ENTRYLK<br clear="none">
1.99 160711.39 us 3332.00 us 406037.00 us 90 UNLINK<br clear="none">
96.00 5167.82 us 18.00 us 55972.00 us 135349 FINODELK<br clear="none">
<br clear="none">
Duration: 380 seconds<br clear="none">
Data Read: 1048576 bytes<br clear="none">
Data Written: 5894045696 bytes<br clear="none">
<br clear="none">
Interval 0 Stats:<br clear="none">
Block Size: 131072b+ <br clear="none">
No. of Reads: 8 <br clear="none">
No. of Writes: 44968 <br clear="none">
%-latency Avg-latency Min-Latency Max-Latency No. of calls Fop<br clear="none">
--------- ----------- ----------- ----------- ------------ ----<br clear="none">
0.00 0.00 us 0.00 us 0.00 us 3 FORGET<br clear="none">
0.00 0.00 us 0.00 us 0.00 us 35 RELEASE<br clear="none">
0.00 0.00 us 0.00 us 0.00 us 28 RELEASEDIR<br clear="none">
0.00 78.00 us 78.00 us 78.00 us 1 FSTAT<br clear="none">
0.00 35.67 us 26.00 us 73.00 us 6 FLUSH<br clear="none">
0.00 324.00 us 324.00 us 324.00 us 1 XATTROP<br clear="none">
0.00 45.80 us 38.00 us 54.00 us 10 STAT<br clear="none">
0.00 227.67 us 216.00 us 242.00 us 3 CREATE<br clear="none">
0.00 113.38 us 68.00 us 381.00 us 8 READ<br clear="none">
0.00 39.82 us 1.00 us 148.00 us 28 OPENDIR<br clear="none">
0.00 67.54 us 10.00 us 283.00 us 24 GETXATTR<br clear="none">
0.00 59.97 us 45.00 us 113.00 us 32 OPEN<br clear="none">
0.00 24.41 us 13.00 us 89.00 us 161 INODELK<br clear="none">
0.00 43.43 us 28.00 us 214.00 us 93 STATFS<br clear="none">
0.00 246.35 us 11.00 us 1155.00 us 20 READDIR<br clear="none">
0.00 283.00 us 233.00 us 353.00 us 18 READDIRP<br clear="none">
0.00 153.23 us 122.00 us 259.00 us 87 MKNOD<br clear="none">
0.01 99.77 us 10.00 us 258.00 us 442 LOOKUP<br clear="none">
0.31 49.22 us 27.00 us 540.00 us 45620 FXATTROP<br clear="none">
0.77 124.24 us 87.00 us 604.00 us 44968 WRITE<br clear="none">
0.93 15767.71 us 15.00 us 305833.00 us 431 ENTRYLK<br clear="none">
1.99 160711.39 us 3332.00 us 406037.00 us 90 UNLINK<br clear="none">
96.00 5167.82 us 18.00 us 55972.00 us 135349 FINODELK<br clear="none">
<br clear="none">
Duration: 380 seconds<br clear="none">
Data Read: 1048576 bytes<br clear="none">
Data Written: 5894045696 bytes<br clear="none">
<br clear="none">
Brick: ovirt3.localdomain:/gluster_bricks/data/data<br clear="none">
---------------------------------------------------<br clear="none">
Cumulative Stats:<br clear="none">
Block Size: 1b+ <br clear="none">
No. of Reads: 0 <br clear="none">
No. of Writes: 39328 <br clear="none">
%-latency Avg-latency Min-Latency Max-Latency No. of calls Fop<br clear="none">
--------- ----------- ----------- ----------- ------------ ----<br clear="none">
0.00 0.00 us 0.00 us 0.00 us 2 FORGET<br clear="none">
0.00 0.00 us 0.00 us 0.00 us 12 RELEASE<br clear="none">
0.00 0.00 us 0.00 us 0.00 us 17 RELEASEDIR<br clear="none">
0.00 101.00 us 101.00 us 101.00 us 1 FSTAT<br clear="none">
0.00 51.50 us 20.00 us 81.00 us 4 FLUSH<br clear="none">
0.01 219.50 us 188.00 us 251.00 us 2 CREATE<br clear="none">
0.01 43.45 us 11.00 us 90.00 us 11 GETXATTR<br clear="none">
0.01 62.30 us 38.00 us 119.00 us 10 OPEN<br clear="none">
0.01 50.59 us 1.00 us 102.00 us 17 OPENDIR<br clear="none">
0.01 24.60 us 12.00 us 64.00 us 40 INODELK<br clear="none">
0.02 176.30 us 10.00 us 765.00 us 10 READDIR<br clear="none">
0.07 63.08 us 39.00 us 133.00 us 78 UNLINK<br clear="none">
0.13 27.35 us 10.00 us 91.00 us 333 ENTRYLK<br clear="none">
0.13 126.89 us 99.00 us 179.00 us 76 MKNOD<br clear="none">
0.42 116.70 us 8.00 us 8661.00 us 261 LOOKUP<br clear="none">
28.73 51.79 us 22.00 us 2574.00 us 39822 FXATTROP<br clear="none">
29.52 53.87 us 16.00 us 3290.00 us 39328 WRITE<br clear="none">
40.92 24.71 us 10.00 us 3224.00 us 118864 FINODELK<br clear="none">
<br clear="none">
Duration: 189 seconds<br clear="none">
Data Read: 0 bytes<br clear="none">
Data Written: 39328 bytes<br clear="none">
<br clear="none">
Interval 0 Stats:<br clear="none">
Block Size: 1b+ <br clear="none">
No. of Reads: 0 <br clear="none">
No. of Writes: 39328 <br clear="none">
%-latency Avg-latency Min-Latency Max-Latency No. of calls Fop<br clear="none">
--------- ----------- ----------- ----------- ------------ ----<br clear="none">
0.00 0.00 us 0.00 us 0.00 us 2 FORGET<br clear="none">
0.00 0.00 us 0.00 us 0.00 us 12 RELEASE<br clear="none">
0.00 0.00 us 0.00 us 0.00 us 17 RELEASEDIR<br clear="none">
0.00 101.00 us 101.00 us 101.00 us 1 FSTAT<br clear="none">
0.00 51.50 us 20.00 us 81.00 us 4 FLUSH<br clear="none">
0.01 219.50 us 188.00 us 251.00 us 2 CREATE<br clear="none">
0.01 43.45 us 11.00 us 90.00 us 11 GETXATTR<br clear="none">
0.01 62.30 us 38.00 us 119.00 us 10 OPEN<br clear="none">
0.01 50.59 us 1.00 us 102.00 us 17 OPENDIR<br clear="none">
0.01 24.60 us 12.00 us 64.00 us 40 INODELK<br clear="none">
0.02 176.30 us 10.00 us 765.00 us 10 READDIR<br clear="none">
0.07 63.08 us 39.00 us 133.00 us 78 UNLINK<br clear="none">
0.13 27.35 us 10.00 us 91.00 us 333 ENTRYLK<br clear="none">
0.13 126.89 us 99.00 us 179.00 us 76 MKNOD<br clear="none">
0.42 116.70 us 8.00 us 8661.00 us 261 LOOKUP<br clear="none">
28.73 51.79 us 22.00 us 2574.00 us 39822 FXATTROP<br clear="none">
29.52 53.87 us 16.00 us 3290.00 us 39328 WRITE<br clear="none">
40.92 24.71 us 10.00 us 3224.00 us 118864 FINODELK<br clear="none">
<br clear="none">
Duration: 189 seconds<br clear="none">
Data Read: 0 bytes<br clear="none">
Data Written: 39328 bytes<br clear="none">
<br clear="none">
Brick: ovirt2.localdomain:/gluster_bricks/data/data<br clear="none">
---------------------------------------------------<br clear="none">
Cumulative Stats:<br clear="none">
Block Size: 512b+ 131072b+ <br clear="none">
No. of Reads: 0 0 <br clear="none">
No. of Writes: 36 76758 <br clear="none">
%-latency Avg-latency Min-Latency Max-Latency No. of calls Fop<br clear="none">
--------- ----------- ----------- ----------- ------------ ----<br clear="none">
0.00 0.00 us 0.00 us 0.00 us 6 FORGET<br clear="none">
0.00 0.00 us 0.00 us 0.00 us 87 RELEASE<br clear="none">
0.00 0.00 us 0.00 us 0.00 us 96 RELEASEDIR<br clear="none">
0.00 100.50 us 80.00 us 121.00 us 2 REMOVEXATTR<br clear="none">
0.00 101.00 us 101.00 us 101.00 us 2 SETXATTR<br clear="none">
0.00 36.18 us 22.00 us 62.00 us 11 FLUSH<br clear="none">
0.00 57.44 us 42.00 us 77.00 us 9 FTRUNCATE<br clear="none">
0.00 82.56 us 59.00 us 138.00 us 9 FSTAT<br clear="none">
0.00 89.42 us 67.00 us 161.00 us 12 SETATTR<br clear="none">
0.00 272.40 us 235.00 us 296.00 us 5 CREATE<br clear="none">
0.01 154.28 us 88.00 us 320.00 us 18 XATTROP<br clear="none">
0.01 45.29 us 1.00 us 319.00 us 96 OPENDIR<br clear="none">
0.01 86.69 us 30.00 us 379.00 us 62 STAT<br clear="none">
0.01 64.30 us 47.00 us 169.00 us 84 OPEN<br clear="none">
0.02 107.34 us 23.00 us 273.00 us 73 READDIRP<br clear="none">
0.02 4688.00 us 86.00 us 9290.00 us 2 TRUNCATE<br clear="none">
0.02 59.29 us 13.00 us 394.00 us 165 GETXATTR<br clear="none">
0.03 128.51 us 27.00 us 338.00 us 96 FSYNC<br clear="none">
0.03 240.75 us 14.00 us 1943.00 us 52 READDIR<br clear="none">
0.04 65.59 us 26.00 us 293.00 us 279 STATFS<br clear="none">
0.06 180.77 us 118.00 us 306.00 us 148 MKNOD<br clear="none">
0.14 37.98 us 17.00 us 192.00 us 1598 INODELK<br clear="none">
0.67 91.68 us 12.00 us 1141.00 us 3186 LOOKUP<br clear="none">
10.10 55.92 us 28.00 us 1658.00 us 78608 FXATTROP<br clear="none">
11.89 6814.76 us 18.00 us 301246.00 us 760 ENTRYLK<br clear="none">
19.44 36.55 us 14.00 us 2353.00 us 231535 FINODELK<br clear="none">
25.21 142.92 us 62.00 us 593.00 us 76794 WRITE<br clear="none">
32.28 91283.68 us 28.00 us 316658.00 us 154 UNLINK<br clear="none">
<br clear="none">
Duration: 1206 seconds<br clear="none">
Data Read: 0 bytes<br clear="none">
Data Written: 10060843008 bytes<br clear="none">
<br clear="none">
Interval 0 Stats:<br clear="none">
Block Size: 512b+ 131072b+ <br clear="none">
No. of Reads: 0 0 <br clear="none">
No. of Writes: 36 76758 <br clear="none">
%-latency Avg-latency Min-Latency Max-Latency No. of calls Fop<br clear="none">
--------- ----------- ----------- ----------- ------------ ----<br clear="none">
0.00 0.00 us 0.00 us 0.00 us 6 FORGET<br clear="none">
0.00 0.00 us 0.00 us 0.00 us 87 RELEASE<br clear="none">
0.00 0.00 us 0.00 us 0.00 us 96 RELEASEDIR<br clear="none">
0.00 100.50 us 80.00 us 121.00 us 2 REMOVEXATTR<br clear="none">
0.00 101.00 us 101.00 us 101.00 us 2 SETXATTR<br clear="none">
0.00 36.18 us 22.00 us 62.00 us 11 FLUSH<br clear="none">
0.00 57.44 us 42.00 us 77.00 us 9 FTRUNCATE<br clear="none">
0.00 82.56 us 59.00 us 138.00 us 9 FSTAT<br clear="none">
0.00 89.42 us 67.00 us 161.00 us 12 SETATTR<br clear="none">
0.00 272.40 us 235.00 us 296.00 us 5 CREATE<br clear="none">
0.01 154.28 us 88.00 us 320.00 us 18 XATTROP<br clear="none">
0.01 45.29 us 1.00 us 319.00 us 96 OPENDIR<br clear="none">
0.01 86.69 us 30.00 us 379.00 us 62 STAT<br clear="none">
0.01 64.30 us 47.00 us 169.00 us 84 OPEN<br clear="none">
0.02 107.34 us 23.00 us 273.00 us 73 READDIRP<br clear="none">
0.02 4688.00 us 86.00 us 9290.00 us 2 TRUNCATE<br clear="none">
0.02 59.29 us 13.00 us 394.00 us 165 GETXATTR<br clear="none">
0.03 128.51 us 27.00 us 338.00 us 96 FSYNC<br clear="none">
0.03 240.75 us 14.00 us 1943.00 us 52 READDIR<br clear="none">
0.04 65.59 us 26.00 us 293.00 us 279 STATFS<br clear="none">
0.06 180.77 us 118.00 us 306.00 us 148 MKNOD<br clear="none">
0.14 37.98 us 17.00 us 192.00 us 1598 INODELK<br clear="none">
0.67 91.66 us 12.00 us 1141.00 us 3186 LOOKUP<br clear="none">
10.10 55.92 us 28.00 us 1658.00 us 78608 FXATTROP<br clear="none">
11.89 6814.76 us 18.00 us 301246.00 us 760 ENTRYLK<br clear="none">
19.44 36.55 us 14.00 us 2353.00 us 231535 FINODELK<br clear="none">
25.21 142.92 us 62.00 us 593.00 us 76794 WRITE<br clear="none">
32.28 91283.68 us 28.00 us 316658.00 us 154 UNLINK<br clear="none">
<br clear="none">
Duration: 1206 seconds<br clear="none">
Data Read: 0 bytes<br clear="none">
Data Written: 10060843008 bytes<br clear="none">
<br clear="none">
<br clear="none">
<br clear="none">
This indicates to me that it's not a problem in Disk/LVM/FileSystem layout.<br clear="none">
<br clear="none">
Most probably I haven't created the volume properly or some option/feature is disabled ?!?<br clear="none">
Network shows OK for a gigabit:<br clear="none">
[root@ovirt1 data]# dd if=/dev/zero status=progress | nc ovirt2 9999<br clear="none">
3569227264 bytes (3.6 GB) copied, 29.001052 s, 123 MB/s^C<br clear="none">
7180980+0 records in<br clear="none">
7180979+0 records out<br clear="none">
3676661248 bytes (3.7 GB) copied, 29.8739 s, 123 MB/s<br clear="none">
<br clear="none">
<br clear="none">
I'm looking for any help... you can share your volume info also.<br clear="none">
<br clear="none">
Thanks in advance.<br clear="none">
<br clear="none">
Best Regards,<br clear="none">
Strahil Nikolov<br clear="none">
_______________________________________________<br clear="none">
Gluster-users mailing list<br clear="none">
<a rel="nofollow" shape="rect" ymailto="mailto:Gluster-users@gluster.org" target="_blank" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br clear="none">
<a rel="nofollow" shape="rect" target="_blank" href="https://lists.gluster.org/mailman/listinfo/gluster-users">https://lists.gluster.org/mailman/listinfo/gluster-users</a><br clear="none">
</blockquote></div><br clear="all"><div><br clear="none"></div>-- <br clear="none"><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div>Amar Tumballi (amarts)<br clear="none"></div></div></div></div></div>
</blockquote></div><br clear="none"></div></div></div></div></div><div class="yqt8450526150" id="yqt99954">Dear Amar,<br clear="none"><br clear="none">Thanks for your email.<br clear="none"><br clear="none">Actually my concerns were on both topics.<br clear="none">Would you recommend any perf options that will be suitable ?<br clear="none"><br clear="none">After mentioning the network usage, I just checked it and it seems duringthe test session, ovirt1 (both client and host) is using no more than 455Mbit/s which is half the network bandwidth.<br clear="none"><br clear="none">I'm still in the middle of nowhere, so any ideas are welcome.<br clear="none"><br clear="none">Best Regards,<br clear="none">Strahil Nikolov<br clear="none"><br clear="none">On Jan 23, 2019 17:49, Amar Tumballi Suryanarayan <<a shape="rect" ymailto="mailto:atumball@redhat.com" href="mailto:atumball@redhat.com">atumball@redhat.com</a>> wrote:<br clear="none">><br clear="none">> I didn't understand the issue properly. Mostly I missed something.<br clear="none">><br clear="none">> Are you concerned the performance is 49MB/s with and without perf options? or are you expecting it to be 123MB/s as over the n/w you get that speed?<br clear="none">><br clear="none">> If it is the first problem, then you are actually having 'performance.write-behind on' in both options, and it is the only perf xlator which comes into action during the test you ran.<br clear="none">><br clear="none">> If it is the second, then please be informed that gluster does client side replication, which means, n/w would be split in half for write operations (like write(), creat() etc), so the number you are getting is almost the maximum with 1GbE.<br clear="none">><br clear="none">> Regards,<br clear="none">> Amar<br clear="none">><br clear="none">> On Wed, Jan 23, 2019 at 8:38 PM Strahil Nikolov <<a shape="rect" ymailto="mailto:hunter86_bg@yahoo.com" href="mailto:hunter86_bg@yahoo.com">hunter86_bg@yahoo.com</a>> wrote:<br clear="none">>><br clear="none">>> Hello Community,<br clear="none">>><br clear="none">>> recently I have built a new lab based on oVirt and CentOS 7.<br clear="none">>> During deployment I had some hicups, but now the engine is up and running - but gluster is causing me trouble.<br clear="none">>><br clear="none">>> Symptoms: Slow VM install from DVD, poor write performance. The latter has been tested via:<br clear="none">>> dd if=/dev/zero of=/rhev/data-center/mnt/glusterSD/ovirt1.localdomain:_data bs=1M count=1000 status=progress<br clear="none">>><br clear="none">>> The reported speed is 60MB/s which is way too low for my setup.<br clear="none">>><br clear="none">>> My lab design:<br clear="none">>> <a shape="rect" href="https://drive.google.com/file/d/1SiW21ASPXHRAEuE_jZ50R3FoO-NcnFqT/view?usp=sharing" target="_blank">https://drive.google.com/file/d/1SiW21ASPXHRAEuE_jZ50R3FoO-NcnFqT/view?usp=sharing</a><br clear="none">>> Gluster version is 3.12.15<br clear="none">>><br clear="none">>> So far I have done:<br clear="none">>><br clear="none">>> 1. Added 'server.allow-insecure on' (with 'option rpc-auth-allow-insecure on' in glusterd.vol)<br clear="none">>> Volume info after that change:<br clear="none">>><br clear="none">>> Volume Name: data<br clear="none">>> Type: Replicate<br clear="none">>> Volume ID: 9b06a1e9-8102-4cd7-bc56-84960a1efaa2<br clear="none">>> Status: Started<br clear="none">>> Snapshot Count: 0<br clear="none">>> Number of Bricks: 1 x (2 + 1) = 3<br clear="none">>> Transport-type: tcp<br clear="none">>> Bricks:<br clear="none">>> Brick1: ovirt1.localdomain:/gluster_bricks/data/data<br clear="none">>> Brick2: ovirt2.localdomain:/gluster_bricks/data/data<br clear="none">>> Brick3: ovirt3.localdomain:/gluster_bricks/data/data (arbiter)<br clear="none">>> Options Reconfigured:<br clear="none">>> performance.client-io-threads: off<br clear="none">>> nfs.disable: on<br clear="none">>> transport.address-family: inet<br clear="none">>> performance.quick-read: off<br clear="none">>> performance.read-ahead: off<br clear="none">>> performance.io-cache: off<br clear="none">>> performance.low-prio-threads: 32<br clear="none">>> network.remote-dio: off<br clear="none">>> cluster.eager-lock: enable<br clear="none">>> cluster.quorum-type: auto<br clear="none">>> cluster.server-quorum-type: server<br clear="none">>> cluster.data-self-heal-algorithm: full<br clear="none">>> cluster.locking-scheme: granular<br clear="none">>> cluster.shd-max-threads: 8<br clear="none">>> cluster.shd-wait-qlength: 10000<br clear="none">>> features.shard: on<br clear="none">>> user.cifs: off<br clear="none">>> storage.owner-uid: 36<br clear="none">>> storage.owner-gid: 36<br clear="none">>> network.ping-timeout: 30<br clear="none">>> performance.strict-o-direct: on<br clear="none">>> cluster.granular-entry-heal: enable<br clear="none">>> server.allow-insecure: on<br clear="none">>><br clear="none">>> Seems no positive or negative effect so far.<br clear="none">>><br clear="none">>> 2. Tested with tmpfs on all bricks -> ovirt1 mounted gluster volume -> max 60MB/s (bs=1M without 'oflag=direct')<br clear="none">>><br clear="none">>><br clear="none">>> [<a shape="rect" ymailto="mailto:root@ovirt1" href="mailto:root@ovirt1">root@ovirt1</a> data]# dd if=/dev/zero of=large_io bs=1M count=4000 status=progress<br clear="none">>> 4177526784 bytes (4.2 GB) copied, 70.843409 s, 59.0 MB/s<br clear="none">>> 4000+0 records in<br clear="none">>> 4000+0 records out<br clear="none">>> 4194304000 bytes (4.2 GB) copied, 71.1407 s, 59.0 MB/s<br clear="none">>> [<a shape="rect" ymailto="mailto:root@ovirt1" href="mailto:root@ovirt1">root@ovirt1</a> data]# rm -f large_io<br clear="none">>> [<a shape="rect" ymailto="mailto:root@ovirt1" href="mailto:root@ovirt1">root@ovirt1</a> data]# gluster volume profile data info<br clear="none">>> Brick: ovirt1.localdomain:/gluster_bricks/data/data<br clear="none">>> ---------------------------------------------------<br clear="none">>> Cumulative Stats:<br clear="none">>> Block Size: 131072b+ <br clear="none">>> No. of Reads: 8 <br clear="none">>> No. of Writes: 44968 <br clear="none">>> %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop<br clear="none">>> --------- ----------- ----------- ----------- ------------ ----<br clear="none">>> 0.00 0.00 us 0.00 us 0.00 us 3 FORGET<br clear="none">>> 0.00 0.00 us 0.00 us 0.00 us 35 RELEASE<br clear="none">>> 0.00 0.00 us 0.00 us 0.00 us 28 RELEASEDIR<br clear="none">>> 0.00 78.00 us 78.00 us 78.00 us 1 FSTAT<br clear="none">>> 0.00 35.67 us 26.00 us 73.00 us 6 FLUSH<br clear="none">>> 0.00 324.00 us 324.00 us 324.00 us 1 XATTROP<br clear="none">>> 0.00 45.80 us 38.00 us 54.00 us 10 STAT<br clear="none">>> 0.00 227.67 us 216.00 us 242.00 us 3 CREATE<br clear="none">>> 0.00 113.38 us 68.00 us 381.00 us 8 READ<br clear="none">>> 0.00 39.82 us 1.00 us 148.00 us 28 OPENDIR<br clear="none">>> 0.00 67.54 us 10.00 us 283.00 us 24 GETXATTR<br clear="none">>> 0.00 59.97 us 45.00 us 113.00 us 32 OPEN<br clear="none">>> 0.00 24.41 us 13.00 us 89.00 us 161 INODELK<br clear="none">>> 0.00 43.43 us 28.00 us 214.00 us 93 STATFS<br clear="none">>> 0.00 246.35 us 11.00 us 1155.00 us 20 READDIR<br clear="none">>> 0.00 283.00 us 233.00 us 353.00 us 18 READDIRP<br clear="none">>> 0.00 153.23 us 122.00 us 259.00 us 87 MKNOD<br clear="none">>> 0.01 99.77 us 10.00 us 258.00 us 442 LOOKUP<br clear="none">>> 0.31 49.22 us 27.00 us 540.00 us 45620 FXATTROP<br clear="none">>> 0.77 124.24 us 87.00 us 604.00 us 44968 WRITE<br clear="none">>> 0.93 15767.71 us 15.00 us 305833.00 us 431 ENTRYLK<br clear="none">>> 1.99 160711.39 us 3332.00 us 406037.00 us 90 UNLINK<br clear="none">>> 96.00 5167.82 us 18.00 us 55972.00 us 135349 FINODELK<br clear="none">>><br clear="none">>> Duration: 380 seconds<br clear="none">>> Data Read: 1048576 bytes<br clear="none">>> Data Written: 5894045696 bytes<br clear="none">>><br clear="none">>> Interval 0 Stats:<br clear="none">>> Block Size: 131072b+ <br clear="none">>> No. of Reads: 8 <br clear="none">>> No. of Writes: 44968 <br clear="none">>> %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop<br clear="none">>> --------- ----------- ----------- ----------- ------------ ----<br clear="none">>> 0.00 0.00 us 0.00 us 0.00 us 3 FORGET<br clear="none">>> 0.00 0.00 us 0.00 us 0.00 us 35 RELEASE<br clear="none">>> 0.00 0.00 us 0.00 us 0.00 us 28 RELEASEDIR<br clear="none">>> 0.00 78.00 us 78.00 us 78.00 us 1 FSTAT<br clear="none">>> 0.00 35.67 us 26.00 us 73.00 us 6 FLUSH<br clear="none">>> 0.00 324.00 us 324.00 us 324.00 us 1 XATTROP<br clear="none">>> 0.00 45.80 us 38.00 us 54.00 us 10 STAT<br clear="none">>> 0.00 227.67 us 216.00 us 242.00 us 3 CREATE<br clear="none">>> 0.00 113.38 us 68.00 us 381.00 us 8 READ<br clear="none">>> 0.00 39.82 us 1.00 us 148.00 us 28 OPENDIR<br clear="none">>> 0.00 67.54 us 10.00 us 283.00 us 24 GETXATTR<br clear="none">>> 0.00 59.97 us 45.00 us 113.00 us 32 OPEN<br clear="none">>> 0.00 24.41 us 13.00 us 89.00 us 161 INODELK<br clear="none">>> 0.00 43.43 us 28.00 us 214.00 us 93 STATFS<br clear="none">>> 0.00 246.35 us 11.00 us 1155.00 us 20 READDIR<br clear="none">>> 0.00 283.00 us 233.00 us 353.00 us 18 READDIRP<br clear="none">>> 0.00 153.23 us 122.00 us 259.00 us 87 MKNOD<br clear="none">>> 0.01 99.77 us 10.00 us 258.00 us 442 LOOKUP<br clear="none">>> 0.31 49.22 us 27.00 us 540.00 us 45620 FXATTROP<br clear="none">>> 0.77 124.24 us 87.00 us 604.00 us 44968 WRITE<br clear="none">>> 0.93 15767.71 us 15.00 us 305833.00 us 431 ENTRYLK<br clear="none">>> 1.99 160711.39 us 3332.00 us 406037.00 us 90 UNLINK<br clear="none">>> 96.00 5167.82 us 18.00 us 55972.00 us 135349 FINODELK<br clear="none">>><br clear="none">>> Duration: 380 seconds<br clear="none">>> Data Read: 1048576 bytes<br clear="none">>> Data Written: 5894045696 bytes<br clear="none">>><br clear="none">>> Brick: ovirt3.localdomain:/gluster_bricks/data/data<br clear="none">>> ---------------------------------------------------<br clear="none">>> Cumulative Stats:<br clear="none">>> Block Size: 1b+ <br clear="none">>> No. of Reads: 0 <br clear="none">>> No. of Writes: 39328 <br clear="none">>> %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop<br clear="none">>> --------- ----------- ----------- ----------- ------------ ----<br clear="none">>> 0.00 0.00 us 0.00 us 0.00 us 2 FORGET<br clear="none">>> 0.00 0.00 us 0.00 us 0.00 us 12 RELEASE<br clear="none">>> 0.00 0.00 us 0.00 us 0.00 us 17 RELEASEDIR<br clear="none">>> 0.00 101.00 us 101.00 us 101.00 us 1 FSTAT<br clear="none">>> 0.00 51.50 us 20.00 us 81.00 us 4 FLUSH<br clear="none">>> 0.01 219.50 us 188.00 us 251.00 us 2 CREATE<br clear="none">>> 0.01 43.45 us 11.00 us 90.00 us 11 GETXATTR<br clear="none">>> 0.01 62.30 us 38.00 us 119.00 us 10 OPEN<br clear="none">>> 0.01 50.59 us 1.00 us 102.00 us 17 OPENDIR<br clear="none">>> 0.01 24.60 us 12.00 us 64.00 us 40 INODELK<br clear="none">>> 0.02 176.30 us 10.00 us 765.00 us 10 READDIR<br clear="none">>> 0.07 63.08 us 39.00 us 133.00 us 78 UNLINK<br clear="none">>> 0.13 27.35 us 10.00 us 91.00 us 333 ENTRYLK<br clear="none">>> 0.13 126.89 us 99.00 us 179.00 us 76 MKNOD<br clear="none">>> 0.42 116.70 us 8.00 us 8661.00 us 261 LOOKUP<br clear="none">>> 28.73 51.79 us 22.00 us 2574.00 us 39822 FXATTROP<br clear="none">>> 29.52 53.87 us 16.00 us 3290.00 us 39328 WRITE<br clear="none">>> 40.92 24.71 us 10.00 us 3224.00 us 118864 FINODELK<br clear="none">>><br clear="none">>> Duration: 189 seconds<br clear="none">>> Data Read: 0 bytes<br clear="none">>> Data Written: 39328 bytes<br clear="none">>><br clear="none">>> Interval 0 Stats:<br clear="none">>> Block Size: 1b+ <br clear="none">>> No. of Reads: 0 <br clear="none">>> No. of Writes: 39328 <br clear="none">>> %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop<br clear="none">>> --------- ----------- ----------- ----------- ------------ ----<br clear="none">>> 0.00 0.00 us 0.00 us 0.00 us 2 FORGET<br clear="none">>> 0.00 0.00 us 0.00 us 0.00 us 12 RELEASE<br clear="none">>> 0.00 0.00 us 0.00 us 0.00 us 17 RELEASEDIR<br clear="none">>> 0.00 101.00 us 101.00 us 101.00 us 1 FSTAT<br clear="none">>> 0.00 51.50 us 20.00 us 81.00 us 4 FLUSH<br clear="none">>> 0.01 219.50 us 188.00 us 251.00 us 2 CREATE<br clear="none">>> 0.01 43.45 us 11.00 us 90.00 us 11 GETXATTR<br clear="none">>> 0.01 62.30 us 38.00 us 119.00 us 10 OPEN<br clear="none">>> 0.01 50.59 us 1.00 us 102.00 us 17 OPENDIR<br clear="none">>> 0.01 24.60 us 12.00 us 64.00 us 40 INODELK<br clear="none">>> 0.02 176.30 us 10.00 us 765.00 us 10 READDIR<br clear="none">>> 0.07 63.08 us 39.00 us 133.00 us 78 UNLINK<br clear="none">>> 0.13 27.35 us 10.00 us 91.00 us 333 ENTRYLK<br clear="none">>> 0.13 126.89 us 99.00 us 179.00 us 76 MKNOD<br clear="none">>> 0.42 116.70 us 8.00 us 8661.00 us 261 LOOKUP<br clear="none">>> 28.73 51.79 us 22.00 us 2574.00 us 39822 FXATTROP<br clear="none">>> 29.52 53.87 us 16.00 us 3290.00 us 39328 WRITE<br clear="none">>> 40.92 24.71 us 10.00 us 3224.00 us 118864 FINODELK<br clear="none">>><br clear="none">>> Duration: 189 seconds<br clear="none">>> Data Read: 0 bytes<br clear="none">>> Data Written: 39328 bytes<br clear="none">>><br clear="none">>> Brick: ovirt2.localdomain:/gluster_bricks/data/data<br clear="none">>> ---------------------------------------------------<br clear="none">>> Cumulative Stats:<br clear="none">>> Block Size: 512b+ 131072b+ <br clear="none">>> No. of Reads: 0 0 <br clear="none">>> No. of Writes: 36 76758 <br clear="none">>> %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop<br clear="none">>> --------- ----------- ----------- ----------- ------------ ----<br clear="none">>> 0.00 0.00 us 0.00 us 0.00 us 6 FORGET<br clear="none">>> 0.00 0.00 us 0.00 us 0.00 us 87 RELEASE<br clear="none">>> 0.00 0.00 us 0.00 us 0.00 us 96 RELEASEDIR<br clear="none">>> 0.00 100.50 us 80.00 us 121.00 us 2 REMOVEXATTR<br clear="none">>> 0.00 101.00 us 101.00 us 101.00 us 2 SETXATTR<br clear="none">>> 0.00 36.18 us 22.00 us 62.00 us 11 FLUSH<br clear="none">>> 0.00 57.44 us 42.00 us 77.00 us 9 FTRUNCATE<br clear="none">>> 0.00 82.56 us 59.00 us 138.00 us 9 FSTAT<br clear="none">>> 0.00 89.42 us 67.00 us 161.00 us 12 SETATTR<br clear="none">>> 0.00 272.40 us 235.00 us 296.00 us 5 CREATE<br clear="none">>> 0.01 154.28 us 88.00 us 320.00 us 18 XATTROP<br clear="none">>> 0.01 45.29 us 1.00 us 319.00 us 96 OPENDIR<br clear="none">>> 0.01 86.69 us 30.00 us 379.00 us 62 STAT<br clear="none">>> 0.01 64.30 us 47.00 us 169.00 us 84 OPEN<br clear="none">>> 0.02 107.34 us 23.00 us 273.00 us 73 READDIRP<br clear="none">>> 0.02 4688.00 us 86.00 us 9290.00 us 2 TRUNCATE<br clear="none">>> 0.02 59.29 us 13.00 us 394.00 us 165 GETXATTR<br clear="none">>> 0.03 128.51 us 27.00 us 338.00 us 96 FSYNC<br clear="none">>> 0.03 240.75 us 14.00 us 1943.00 us 52 READDIR<br clear="none">>> 0.04 65.59 us 26.00 us 293.00 us 279 STATFS<br clear="none">>> 0.06 180.77 us 118.00 us 306.00 us 148 MKNOD<br clear="none">>> 0.14 37.98 us 17.00 us 192.00 us 1598 INODELK<br clear="none">>> 0.67 91.68 us 12.00 us 1141.00 us 3186 LOOKUP<br clear="none">>> 10.10 55.92 us 28.00 us 1658.00 us 78608 FXATTROP<br clear="none">>> 11.89 6814.76 us 18.00 us 301246.00 us 760 ENTRYLK<br clear="none">>> 19.44 36.55 us 14.00 us 2353.00 us 231535 FINODELK<br clear="none">>> 25.21 142.92 us 62.00 us 593.00 us 76794 WRITE<br clear="none">>> 32.28 91283.68 us 28.00 us 316658.00 us 154 UNLINK<br clear="none">>><br clear="none">>> Duration: 1206 seconds<br clear="none">>> Data Read: 0 bytes<br clear="none">>> Data Written: 10060843008 bytes<br clear="none">>><br clear="none">>> Interval 0 Stats:<br clear="none">>> Block Size: 512b+ 131072b+ <br clear="none">>> No. of Reads: 0 0 <br clear="none">>> No. of Writes: 36 76758 <br clear="none">>> %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop<br clear="none">>> --------- ----------- ----------- ----------- ------------ ----<br clear="none">>> 0.00 0.00 us 0.00 us 0.00 us 6 FORGET<br clear="none">>> 0.00 0.00 us 0.00 us 0.00 us 87 RELEASE<br clear="none">>> 0.00 0.00 us 0.00 us 0.00 us 96 RELEASEDIR<br clear="none">>> 0.00 100.50 us 80.00 us 121.00 us 2 REMOVEXATTR<br clear="none">>> 0.00 101.00 us 101.00 us 101.00 us 2 SETXATTR<br clear="none">>> 0.00 36.18 us 22.00 us 62.00 us 11 FLUSH<br clear="none">>> 0.00 57.44 us 42.00 us 77.00 us 9 FTRUNCATE<br clear="none">>> 0.00 82.56 us 59.00 us 138.00 us 9 FSTAT<br clear="none">>> 0.00 89.42 us 67.00 us 161.00 us 12 SETATTR<br clear="none">>> 0.00 272.40 us 235.00 us 296.00 us 5 CREATE<br clear="none">>> 0.01 154.28 us 88.00 us 320.00 us 18 XATTROP<br clear="none">>> 0.01 45.29 us 1.00 us 319.00 us 96 OPENDIR<br clear="none">>> 0.01 86.69 us 30.00 us 379.00 us 62 STAT<br clear="none">>> 0.01 64.30 us 47.00 us 169.00 us 84 OPEN<br clear="none">>> 0.02 107.34 us 23.00 us 273.00 us 73 READDIRP<br clear="none">>> 0.02 4688.00 us 86.00 us 9290.00 us 2 TRUNCATE<br clear="none">>> 0.02 59.29 us 13.00 us 394.00 us 165 GETXATTR<br clear="none">>> 0.03 128.51 us 27.00 us 338.00 us 96 FSYNC<br clear="none">>> 0.03 240.75 us 14.00 us 1943.00 us 52 READDIR<br clear="none">>> 0.04 65.59 us 26.00 us 293.00 us 279 STATFS<br clear="none">>> 0.06 180.77 us 118.00 us 306.00 us 148 MKNOD<br clear="none">>> 0.14 37.98 us 17.00 us 192.00 us 1598 INODELK<br clear="none">>> 0.67 91.66 us 12.00 us 1141.00 us 3186 LOOKUP<br clear="none">>> 10.10 55.92 us 28.00 us 1658.00 us 78608 FXATTROP<br clear="none">>> 11.89 6814.76 us 18.00 us 301246.00 us 760 ENTRYLK<br clear="none">>> 19.44 36.55 us 14.00 us 2353.00 us 231535 FINODELK<br clear="none">>> 25.21 142.92 us 62.00 us 593.00 us 76794 WRITE<br clear="none">>> 32.28 91283.68 us 28.00 us 316658.00 us 154 UNLINK<br clear="none">>><br clear="none">>> Duration: 1206 seconds<br clear="none">>> Data Read: 0 bytes<br clear="none">>> Data Written: 10060843008 bytes<br clear="none">>><br clear="none">>><br clear="none">>><br clear="none">>> This indicates to me that it's not a problem in Disk/LVM/FileSystem layout.<br clear="none">>><br clear="none">>> Most probably I haven't created the volume properly or some option/feature is disabled ?!?<br clear="none">>> Network shows OK for a gigabit:<br clear="none">>> [<a shape="rect" ymailto="mailto:root@ovirt1" href="mailto:root@ovirt1">root@ovirt1</a> data]# dd if=/dev/zero status=progress | nc ovirt2 9999<br clear="none">>> 3569227264 bytes (3.6 GB) copied, 29.001052 s, 123 MB/s^C<br clear="none">>> 7180980+0 records in<br clear="none">>> 7180979+0 records out<br clear="none">>> 3676661248 bytes (3.7 GB) copied, 29.8739 s, 123 MB/s<br clear="none">>><br clear="none">>><br clear="none">>> I'm looking for any help... you can share your volume info also.<br clear="none">>><br clear="none">>> Thanks in advance.<br clear="none">>><br clear="none">>> Best Regards,<br clear="none">>> Strahil Nikolov<br clear="none">>> _______________________________________________<br clear="none">>> Gluster-users mailing list<br clear="none">>> <a shape="rect" ymailto="mailto:Gluster-users@gluster.org" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br clear="none">>> <a shape="rect" href="https://lists.gluster.org/mailman/listinfo/gluster-users" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-users</a><br clear="none">><br clear="none">><br clear="none">><br clear="none">> -- <br clear="none">> Amar Tumballi (amarts)<br clear="none"><br clear="none"><br clear="none"></div><br><br></div> </div> </div> </div></div></body></html>