<html><body><div style="font-family: trebuchet ms,sans-serif; font-size: 10pt; color: #000000"><div>OK, I see.</div><div><br data-mce-bogus="1"></div><div>I'm using the recommended ovirt configuration:</div><div># cat /var/lib/glusterd/groups/virt<br>performance.quick-read=off<br>performance.read-ahead=off<br>performance.io-cache=off<br>performance.low-prio-threads=32<br>network.remote-dio=enable<br>cluster.eager-lock=enable<br>cluster.quorum-type=auto<br>cluster.server-quorum-type=server<br>cluster.data-self-heal-algorithm=full<br>cluster.locking-scheme=granular<br>cluster.shd-max-threads=8<br>cluster.shd-wait-qlength=10000<br>features.shard=on<br>user.cifs=off<br>cluster.choose-local=off<br>client.event-threads=4<br>server.event-threads=4<br>performance.client-io-threads=on<br></div><div><br data-mce-bogus="1"></div><div>Changing the parameters the performance doesn't improve much.</div><div>Maybe for smaller installations is better to use NFS? When you don't need to scale out or get high availability.</div><div>I don't know how to make a real world workload test.</div><div><br data-mce-bogus="1"></div><div>Thanks</div><div><br data-mce-bogus="1"></div><div>José</div><div><br data-mce-bogus="1"></div><div><br></div><hr id="zwchr" data-marker="__DIVIDER__"><div data-marker="__HEADERS__"><b>De: </b>"Strahil Nikolov" <hunter86_bg@yahoo.com><br><b>Para: </b>gluster-users@gluster.org, "jf" <jf@logicworks.pt><br><b>Enviadas: </b>Sábado, 10 De Abril de 2021 14:04:01<br><b>Assunto: </b>Re: [Gluster-users] Gluster performance<br></div><br><div data-marker="__QUOTED_TEXT__">Yes, FUSE is slower as it's in user space - which leads to more system calls than when using NFS.Amar has explained it in https://lists.gluster.org/pipermail/gluster-users/2020-November/038996.html . Yet, gluster can tolerate a single node failure , while NFS requires cluster software like corosync/pacemaker + a shared storage device.<br><br>First of all you can start testing with real world workload.<br>Next, take profile data from server and client.<br>Verify that client is connected to all bricks.<br><br>Next ensure that if you use hardware raid ,LVM and XFS are aligned properly.<br><br>Another idea that comes to my mind is to test with different settings for client and server threads (more is not always better).<br><br><br>Best Regards,<br>Strahil Nikolov<br><br><br><br><br><br><br><br><br><br>В четвъртък, 8 април 2021 г., 22:28:10 ч. Гринуич+3, José Ferradeira <jf@logicworks.pt> написа: <br><br><br><br><br><br>Hello,<br><br>I'm running oVirt with Glusterfs as storage.<br>On a 10GB network.<br><br>Gluster version: glusterfs 6.10<br><br>Configuration:<br># gluster volume info data2<br><br>Volume Name: data2<br>Type: Distribute<br>Volume ID: 3fc4d067-f845-47bc-beae-2be0106116b9<br>Status: Started<br>Snapshot Count: 0<br>Number of Bricks: 1<br>Transport-type: tcp<br>Bricks:<br>Brick1: engine.pt.ags.corp:/storage/brick1<br>Options Reconfigured:<br>performance.client-io-threads: off<br>server.event-threads: 2<br>client.event-threads: 4<br>cluster.choose-local: on<br>user.cifs: off<br>features.shard: on<br>cluster.shd-wait-qlength: 10000<br>cluster.locking-scheme: granular<br>cluster.data-self-heal-algorithm: full<br>cluster.server-quorum-type: server<br>cluster.quorum-type: auto<br>cluster.eager-lock: enable<br>network.remote-dio: enable<br>performance.low-prio-threads: 32<br>performance.io-cache: off<br>performance.read-ahead: off<br>performance.quick-read: off<br>transport.address-family: inet<br>nfs.disable: on<br><br><br>On the same disks I did these tests:<br><br>On a NFS storage:<br><br># dd if=/dev/zero of=test5.img bs=1G count=1 oflag=dsync <br>1+0 registos dentro <br>1+0 registos fora <br>1073741824 bytes (1,1 GB) copiados, 1,70022 s, 632 MB/s<br><br># dd if=/dev/zero of=test5.img bs=5G count=1 oflag=dsync <br>0+1 registos dentro <br>0+1 registos fora <br>2147479552 bytes (2,1 GB) copiados, 2,94035 s, 730 MB/s<br><br><br>On the Gluster Storage<br><br>dd if=/dev/zero of=test5.img bs=1G count=1 oflag=dsync <br>1+0 registos dentro <br>1+0 registos fora <br>1073741824 bytes (1,1 GB) copiados, 2,22618 s, 482 MB/s<br><br># dd if=/dev/zero of=test5.img bs=5G count=1 oflag=dsync <br>0+1 registos dentro <br>0+1 registos fora <br>2147479552 bytes (2,1 GB) copiados, 4,10337 s, 523 MB/s<br><br><br>This means that NFS is faster than Gluster?<br>How can I improve Gluster performance?<br><br>Thanks<br><br>-- <br>Com os melhores cumprimentos | Kind Regards | Meilleures salutations | Met vriendelijke groeten,<br> <br>Descubra AQUI os novos superpoderes do Zimbra 9<br><br>José Ferradeira<br>T.: +351 214 261 698 <br>---------------------------------------------<br>Logicworks Tecnologias de Informática<br>http://www.logicworks.pt<br>www.acloud.pt<br>serviços de alojamento e virtualização<br><br><br><br><br>________<br><br><br><br>Community Meeting Calendar:<br><br>Schedule -<br>Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC<br>Bridge: https://meet.google.com/cpu-eiue-hvk<br>Gluster-users mailing list<br>Gluster-users@gluster.org<br>https://lists.gluster.org/mailman/listinfo/gluster-users<br></div><br><div data-marker="__SIG_POST__">-- <br></div><div><div style="color: #000000; font-family: 'trebuchet ms', sans-serif; font-size: 13.3333px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;" data-mce-style="color: #000000; font-family: 'trebuchet ms', sans-serif; font-size: 13.3333px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;"><span style="color: #000000;" data-mce-style="color: #000000;"><span style="color: #000000;" data-mce-style="color: #000000;"><a href="http://webmail.acloud.pt/home/info@logicworks.pt/Assinatura/index.html" data-mce-href="http://webmail.acloud.pt/home/info@logicworks.pt/Assinatura/index.html"></a>Com os melhores cumprimentos | Kind Regards |<span> </span></span></span><span face="arial, sans-serif" style="font-family: arial,sans-serif;" data-mce-style="font-family: arial,sans-serif;">Meilleures salutations | Met vriendelijke groeten,</span><span style="color: #000000;" data-mce-style="color: #000000;"><span style="color: #000000;" data-mce-style="color: #000000;"></span></span><div><span face="arial, sans-serif" style="font-family: arial,sans-serif;" data-mce-style="font-family: arial,sans-serif;"> </span></div><div><strong><span face="arial, sans-serif" style="font-family: arial, sans-serif; color: #ff6600;" data-mce-style="font-family: arial, sans-serif; color: #ff6600;">Descubra <a href="https://youtu.be/q4Y1mWbqezo" target="_blank" data-mce-href="https://youtu.be/q4Y1mWbqezo">AQUI</a> os novos superpoderes do Zimbra 9</span></strong></div></div><div style="color: #000000; font-family: 'trebuchet ms', sans-serif; font-size: 13.3333px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;" data-mce-style="color: #000000; font-family: 'trebuchet ms', sans-serif; font-size: 13.3333px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;"><span style="color: #000000;" data-mce-style="color: #000000;"></span><br></div><div style="color: #000000; font-family: 'trebuchet ms', sans-serif; font-size: 13.3333px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;" data-mce-style="color: #000000; font-family: 'trebuchet ms', sans-serif; font-size: 13.3333px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;"><span style="color: #000000;" data-mce-style="color: #000000;">José Ferradeira</span></div><div style="color: #000000; font-family: 'trebuchet ms', sans-serif; font-size: 13.3333px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;" data-mce-style="color: #000000; font-family: 'trebuchet ms', sans-serif; font-size: 13.3333px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;"><span style="color: #000000;" data-mce-style="color: #000000;">T.: +351 214 261 698<span> </span></span></div><div style="color: #000000; font-family: 'trebuchet ms', sans-serif; font-size: 13.3333px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;" data-mce-style="color: #000000; font-family: 'trebuchet ms', sans-serif; font-size: 13.3333px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;"><span style="color: #fdbc55;" data-mce-style="color: #fdbc55;">---------------------------------------------</span><br style="color: #fdbc55;" data-mce-style="color: #fdbc55;"><span style="color: #fdbc55;" data-mce-style="color: #fdbc55;">Logicworks Tecnologias de Informática</span><br style="color: #fdbc55;" data-mce-style="color: #fdbc55;"><a href="http://www.logicworks.pt/" data-mce-href="http://www.logicworks.pt/"><span style="color: #fdbc55;" data-mce-style="color: #fdbc55;"></span></a><a href="http://www.logicworks.pt" data-mce-href="http://www.logicworks.pt">http://www.logicworks.pt</a></div><div style="color: #000000; font-family: 'trebuchet ms', sans-serif; font-size: 13.3333px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;" data-mce-style="color: #000000; font-family: 'trebuchet ms', sans-serif; font-size: 13.3333px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;"><div><span style="color: #003366;" data-mce-style="color: #003366;"><span class="Object" id="OBJ_PREFIX_DWT1480_com_zimbra_url"><span class="Object" id="OBJ_PREFIX_DWT1483_com_zimbra_url"><a href="http://www.acloud.pt" target="_blank" data-mce-href="http://www.acloud.pt"><span style="color: #003366;" data-mce-style="color: #003366;">www.acloud.pt</span></a></span></span></span></div><span style="color: #fdbc55;" data-mce-style="color: #fdbc55;"><span style="color: #003366;" data-mce-style="color: #003366;">serviços de alojamento e virtualização</span></span></div><div style="color: #000000; font-family: 'trebuchet ms', sans-serif; font-size: 13.3333px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;" data-mce-style="color: #000000; font-family: 'trebuchet ms', sans-serif; font-size: 13.3333px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;"><img src="cid:7694dcfbf74309564ef2d29d251cfff70f696706@zimbra" data-mce-src="/home/jf@logicworks.pt/Briefcase/Zimbra_Individual_SalesCertified@60.png" doc="Briefcase/Zimbra_Individual_SalesCertified@60.png"><br><br></div><div><img src="https://webmail.acloud.pt/home/info@logicworks.pt/Assinatura/ciberseg.png" data-mce-src="https://webmail.acloud.pt/home/info@logicworks.pt/Assinatura/ciberseg.png"></div><div style="color: #000000; font-family: 'trebuchet ms', sans-serif; font-size: 13.3333px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;" data-mce-style="color: #000000; font-family: 'trebuchet ms', sans-serif; font-size: 13.3333px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;"><br></div><div style="color: #000000; font-family: 'trebuchet ms', sans-serif; font-size: 13.3333px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;" data-mce-style="color: #000000; font-family: 'trebuchet ms', sans-serif; font-size: 13.3333px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;"><br></div><div><span></span><span></span><br></div></div></div></body></html>