[Gluster-users] glustefs as vmware datastore in production

[Jonathan Archer]

 Hi,I've gone through a bit of testing around using Gluster as a VMWare datastore, here are my findings:
running VMWare vSphere 6.5 with ESXi nodesGluster running on Supermicro kit, 6 SAS disks with 2 SSD's for caching all carved up using LVM on to of CentOS 7.
I set up a 4 node cluster, ultimately to scale to 12 should this become usable, initially just using 3 nodes to satisfy quorum requirements without playing too much with default values.The idea was to preset via NFS, have a cluster VIP and heartbeat. Obviously NFS is no longer a viable option direct from Gluster so the route I took was down the NFS-Ganesha route. Had no end of troubles using the Ganesha tools with gluster 3.10 so ended up doing the whole thing manually which seems to stand up ok.For this I used Gluster, NFS-Ganesha with VFS connections, Pacemaker and Corrosync (still not 100% happy with the config and it needs tweaking but it does what it needs so far).
Next job was connecting from VMWare, I had to ensure that relevant read/write permissions were set both on the filesystem and in the Ganesha config but connection succeeded. From here was where I'm suffering a few issues.
If you migrate a VM into the NFS volume and then away again it doesn't clear the parent directory, it removes everything else but that, SSHing onto the ESXi server and manually trying to remove it results in 2 errors:
rm directory - cannot remove as it is a directoryrm -rf directory - cannot remove as it isn't a directory
second issue is to do with failover, The VIP fails over extremely fast, but VMWare always loses connection to the volume and a weird oddity of VMware seems to be that it will not remount an NFS volume so stays unavailable.
I have not got over these hurdles as yet, but will persevere as performance wise and cost wise its brilliant.
Hope this helps, or if anyone has any clues to the above issues I'd be most grateful.
Regards
Jon
    On Friday, 25 May 2018, 18:33:09 BST, <joao at 7lan.net> wrote:  
 
  
Hi,

Does anyone have glusterfs as vmware datastore working in production in a real world case? How to serve the glusterfs cluster? As iscsi, NFS?

Thanks in advance _______________________________________________
Gluster-users mailing list
Gluster-users at gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180529/b2b49aa2/attachment.html>