<html><head></head><body><div class="ydpf9fb2747yahoo-style-wrap" style="font-family:courier new, courier, monaco, monospace, sans-serif;font-size:16px;" dir="ltr" data-setdir="false"><div></div>
<div dir="ltr" data-setdir="false">These are just a small sample of the virt group settings that you are not using:<br><br><div>performance.quick-read=off<br>performance.read-ahead=off<br>performance.io-cache=off</div><span>network.remote-dio=disable<br><span>performance.strict-o-direct=on<br><span>cluster.lookup-optimize=off</span></span></span></div><div dir="ltr" data-setdir="false"><span><span><span><br></span></span></span></div><div dir="ltr" data-setdir="false">Here is an example file <a href="https://github.com/gluster/glusterfs/blob/devel/extras/group-virt.example" class="" target="_blank" rel="noreferrer noopener">https://github.com/gluster/glusterfs/blob/devel/extras/group-virt.example</a> <br></div><div dir="ltr" data-setdir="false">Also glusterfs-server provides a lot more groups of settings in <span>/var/lib/glusterd/groups directory.</span></div><span><br>Best Regards,<br>Strahil Nikolov</span><div><br></div><div><br></div><div><br></div>
</div><div id="yahoo_quoted_1390514700" class="yahoo_quoted">
<div style="font-family:'Helvetica Neue', Helvetica, Arial, sans-serif;font-size:13px;color:#26282a;">
<div>
В събота, 22 февруари 2025 г. в 14:56:14 ч. Гринуич+2, Gilberto Ferreira <gilberto.nunes32@gmail.com> написа:
</div>
<div><br></div>
<div><br></div>
<div><div id="yiv5546900620"><div dir="ltr"><div>Hi there.<br clear="all"></div><div><br></div><div>I'd like to know if there are any issues with GlusterFS and NVME.</div><div>This week I got two customer where I build 2 Proxmox VE with GlusterFS 11.<br>I had have created with:</div><div>On both nodes I do:</div><div><br></div><div>mkdir /data1</div><div>mkdir /data2</div><div>mkfs.xfs /dev/nvme1</div><div>mkfs.xfs /dev/nvme2</div><div>mount /dev/nvme1 /data1</div><div>mount /dev/nvme2 /data2<br>After install glusterfs and do the peer probe, I do</div><div>gluster vol create VMS replica 2 gluster1:/data1/vms gluster2:/data1/vms gluster1:/data2/vms gluster2:/data/vms</div><div><br></div><div>To solve the split-brain issue, I applied this configurations:</div><div><font face="arial, sans-serif"><span style="color:rgb(0,0,0);">gluster vol set VMS cluster.heal-timeout 5</span><span style="color:rgb(0,0,0);">
</span><br><span style="color:rgb(0,0,0);">gluster vol heal VMS enable</span><span style="color:rgb(0,0,0);">
</span><br><span style="color:rgb(0,0,0);">gluster vol set VMS cluster.quorum-reads false</span><span style="color:rgb(0,0,0);">
</span><br><span style="color:rgb(0,0,0);">gluster vol set VMS cluster.quorum-count 1</span><span style="color:rgb(0,0,0);">
</span><br><span style="color:rgb(0,0,0);">gluster vol set VMS network.ping-timeout 2</span><span style="color:rgb(0,0,0);">
</span><br><span style="color:rgb(0,0,0);">gluster vol set VMS cluster.favorite-child-policy mtime</span><span style="color:rgb(0,0,0);">
</span><br><span style="color:rgb(0,0,0);">gluster vol heal VMS granular-entry-heal enable</span><span style="color:rgb(0,0,0);">
</span><br><span style="color:rgb(0,0,0);">gluster vol set VMS cluster.data-self-heal-algorithm full</span><span style="color:rgb(0,0,0);">
</span><br><span style="color:rgb(0,0,0);">gluster vol set VMS features.shard on</span><span style="color:rgb(0,0,0);">
</span><br><span style="color:rgb(0,0,0);">gluster vol set VMS performance.write-behind off</span><span style="color:rgb(0,0,0);">
</span><br><span style="color:rgb(0,0,0);">gluster vol set VMS performance.flush-behind off</span></font></div><div><font color="#000000" face="arial, sans-serif"><br></font></div><div><font color="#000000" face="arial, sans-serif">So this configuration allows me to power down the first server and the VMs restart on the secondary server, with no issues at all.<br>I have the very same scenario in another customer, but there we are working wih SSD DC600M Kingston.<br><br>Turns out that in the servers with NVME I got a lot of disk corruption inside the VM.</font></div><div><font color="#000000" face="arial, sans-serif">If I reboot, things go worse.<br><br>Does anybody know any cases about gluster and nvme issues like that?</font></div><div><font color="#000000" face="arial, sans-serif">Is there any fix for that?</font></div><div><font color="#000000" face="arial, sans-serif"><br></font></div><div><font color="#000000" face="arial, sans-serif">Thanks</font></div><div><br></div><div><span style="font-family:monospace;color:rgb(0,0,0);">
</span><br></div><div><div dir="ltr" class="yiv5546900620gmail_signature"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div>---</div><div><br></div><div><img src="https://ci3.googleusercontent.com/mail-sig/AIorK4yHG-wUIelCTtozPBQoS83ZcRg8ukTeTVlsRkm1MmU-3Xy2S-2myWu_idYMDhBGeBoDo33pV0UOwMIl"><br></div><div><div><div>Gilberto Nunes Ferreira</div></div><div><span style="font-size:12.8px;">(47) 99676-7530 - Whatsapp / Telegram</span><br></div><div><p style="font-size:12.8px;margin:0px;"></p><p style="font-size:12.8px;margin:0px;"><br></p><p style="font-size:12.8px;margin:0px;"><br></p></div></div><div><br></div></div></div></div></div></div></div></div></div>
</div>________<br><br><br><br>Community Meeting Calendar:<br><br>Schedule -<br>Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC<br>Bridge: <a href="https://meet.google.com/cpu-eiue-hvk" target="_blank" rel="noreferrer noopener">https://meet.google.com/cpu-eiue-hvk</a><br>Gluster-users mailing list<br><a ymailto="mailto:Gluster-users@gluster.org" href="mailto:Gluster-users@gluster.org" target="_blank" rel="noreferrer noopener">Gluster-users@gluster.org</a><br><a href="https://lists.gluster.org/mailman/listinfo/gluster-users" target="_blank" rel="noreferrer noopener">https://lists.gluster.org/mailman/listinfo/gluster-users</a><br></div>
</div>
</div></body></html>