[Gluster-users] GlusterFS with iSCSI and PaceMaker

[Jon Heese]

Justin,


I do something similar, but not quite as complex.


I have a replicated (x2) Gluster volume where I drop thin-provisioned iSCSI volume files to be served up via tgtd on CentOS6.  I mount the Gluster volume locally on both Gluster servers (FUSE driver), then point the tgtd daemons to the image files.  I use this as a back-end for a VMware ESXi datastore, so I'm using the multipath iSCSI functionality of ESXi to handle the failover between the nodes.


I was previously doing this with DRBD/Pacemaker/Corosync, but ESXi freaks out when all paths to a datastore go down, and it takes ~2-5 seconds for the entire cluster stack to go down and come back up during an orderly failover (to say nothing of a catastrophic failover), so that model just didn't work for me.


I've since done some testing with just a simple VIP in Keepalived on top of Gluster using LIO and the libgfapi stuff on CentOS7, and that seemed to work great -- but I have some other incompatibilities with CentOS7, so I decided not to pursue that for this project -- maybe another one on the horizon.


I briefly tried testing the libgfapi driver on CentOS6 with my current production setup, but when I started the rebuild tgtd instance it gave my iSCSI LUNs different LUN numbers so my ESXi cluster didn't recognize them as different paths to the same LUN.  I couldn't be bothered to work out the reason for this change, so I just switched back in the meantime.  I will probably play with it a bit in a test environment when I have time -- the libgfapi stuff should be faster and more efficient than going through the FUSE stuff.


What would the Pacemaker CRM handle for you, besides a shared VIP?  Would you want it to start/stop the iSCSI target daemon as well?  (If so, why?)  Is there any reason to use a full CRM for this versus a simple VIP in something like keepalived?


Good luck, and let us know how you get on!


Regards,

Jon Heese



________________________________
From: gluster-users-bounces at gluster.org <gluster-users-bounces at gluster.org> on behalf of Justin Chin-You <justin.chinyou at gmail.com>
Sent: Monday, April 20, 2015 10:14 PM
To: gluster-users at gluster.org
Subject: [Gluster-users] GlusterFS with iSCSI and PaceMaker

Hi All,

I'm new to Gluster and I'm trying to work through some test configurations.

What I am trying to do is use Gluster to create a mirrored and scalable storage array. My plan is once I setup Gluster is to then export the GlusterFS via an iSCSI interface which will have an Active/Standby failover managed via Pacemaker.

I am curious if anyone has tried anything similar and if you have what iSCSI driver are you using.

Thanks,

Justin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150421/4967f560/attachment.html>