[Gluster-users] Gluster performance

Tue Apr 13 10:24:05 UTC 2021

OK, I see. 

I'm using the recommended ovirt configuration: 
# cat /var/lib/glusterd/groups/virt 
performance.quick-read=off 
performance.read-ahead=off 
performance.io-cache=off 
performance.low-prio-threads=32 
network.remote-dio=enable 
cluster.eager-lock=enable 
cluster.quorum-type=auto 
cluster.server-quorum-type=server 
cluster.data-self-heal-algorithm=full 
cluster.locking-scheme=granular 
cluster.shd-max-threads=8 
cluster.shd-wait-qlength=10000 
features.shard=on 
user.cifs=off 
cluster.choose-local=off 
client.event-threads=4 
server.event-threads=4 
performance.client-io-threads=on 

Changing the parameters the performance doesn't improve much. 
Maybe for smaller installations is better to use NFS? When you don't need to scale out or get high availability. 
I don't know how to make a real world workload test. 

Thanks 

José 

De: "Strahil Nikolov" <hunter86_bg at yahoo.com> 
Para: gluster-users at gluster.org, "jf" <jf at logicworks.pt> 
Enviadas: Sábado, 10 De Abril de 2021 14:04:01 
Assunto: Re: [Gluster-users] Gluster performance 

Yes, FUSE is slower as it's in user space - which leads to more system calls than when using NFS.Amar has explained it in https://lists.gluster.org/pipermail/gluster-users/2020-November/038996.html . Yet, gluster can tolerate a single node failure , while NFS requires cluster software like corosync/pacemaker + a shared storage device. 

First of all you can start testing with real world workload. 
Next, take profile data from server and client. 
Verify that client is connected to all bricks. 

Next ensure that if you use hardware raid ,LVM and XFS are aligned properly. 

Another idea that comes to my mind is to test with different settings for client and server threads (more is not always better). 

Best Regards, 
Strahil Nikolov 

В четвъртък, 8 април 2021 г., 22:28:10 ч. Гринуич+3, José Ferradeira <jf at logicworks.pt> написа: 

Hello, 

I'm running oVirt with Glusterfs as storage. 
On a 10GB network. 

Gluster version: glusterfs 6.10 

Configuration: 
# gluster volume info data2 

Volume Name: data2 
Type: Distribute 
Volume ID: 3fc4d067-f845-47bc-beae-2be0106116b9 
Status: Started 
Snapshot Count: 0 
Number of Bricks: 1 
Transport-type: tcp 
Bricks: 
Brick1: engine.pt.ags.corp:/storage/brick1 
Options Reconfigured: 
performance.client-io-threads: off 
server.event-threads: 2 
client.event-threads: 4 
cluster.choose-local: on 
user.cifs: off 
features.shard: on 
cluster.shd-wait-qlength: 10000 
cluster.locking-scheme: granular 
cluster.data-self-heal-algorithm: full 
cluster.server-quorum-type: server 
cluster.quorum-type: auto 
cluster.eager-lock: enable 
network.remote-dio: enable 
performance.low-prio-threads: 32 
performance.io-cache: off 
performance.read-ahead: off 
performance.quick-read: off 
transport.address-family: inet 
nfs.disable: on 

On the same disks I did these tests: 

On a NFS storage: 

# dd if=/dev/zero of=test5.img bs=1G count=1 oflag=dsync 
1+0 registos dentro 
1+0 registos fora 
1073741824 bytes (1,1 GB) copiados, 1,70022 s, 632 MB/s 

# dd if=/dev/zero of=test5.img bs=5G count=1 oflag=dsync 
0+1 registos dentro 
0+1 registos fora 
2147479552 bytes (2,1 GB) copiados, 2,94035 s, 730 MB/s 

On the Gluster Storage 

dd if=/dev/zero of=test5.img bs=1G count=1 oflag=dsync 
1+0 registos dentro 
1+0 registos fora 
1073741824 bytes (1,1 GB) copiados, 2,22618 s, 482 MB/s 

# dd if=/dev/zero of=test5.img bs=5G count=1 oflag=dsync 
0+1 registos dentro 
0+1 registos fora 
2147479552 bytes (2,1 GB) copiados, 4,10337 s, 523 MB/s 

This means that NFS is faster than Gluster? 
How can I improve Gluster performance? 

Thanks 

-- 
Com os melhores cumprimentos | Kind Regards | Meilleures salutations | Met vriendelijke groeten, 

Descubra AQUI os novos superpoderes do Zimbra 9 

José Ferradeira 
T.: +351 214 261 698 
--------------------------------------------- 
Logicworks Tecnologias de Informática 
http://www.logicworks.pt 
www.acloud.pt 
serviços de alojamento e virtualização 

________ 

Community Meeting Calendar: 

Schedule - 
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC 
Bridge: https://meet.google.com/cpu-eiue-hvk 
Gluster-users mailing list 
Gluster-users at gluster.org 
https://lists.gluster.org/mailman/listinfo/gluster-users 

-- 
Com os melhores cumprimentos | Kind Regards | Meilleures salutations | Met vriendelijke groeten, 
Descubra AQUI os novos superpoderes do Zimbra 9 

José Ferradeira 
T.: +351 214 261 698 
--------------------------------------------- 
Logicworks Tecnologias de Informática 
http://www.logicworks.pt 
www.acloud.pt 
serviços de alojamento e virtualização 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20210413/91af731b/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Zimbra_Individual_SalesCertified at 60.png
Type: image/png
Size: 2795 bytes
Desc: not available
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20210413/91af731b/attachment.png>