[Gluster-users] Infiniband

[Joe Landman]

On 01/09/2011 10:06 PM, Bryan McGuire wrote:
> Hello,
>
> I am looking into GlusterFS as a high availability solution for our
> email servers. I am new to Infiniband but find it could possibly
> provide us with the necessary speed.

Hi Bryan

  We've done this for various ISP/email hosting customers.

>
> Could someone describe what I would need in the way of Infiniband
> hardware / software to complete the following.
>
> Two to 4 front end email servers with each being a client and server
> for the GlusterFS file system performing replication of the data.

Depends on what sort of machine you use for your front end, and what the 
software is.  Are these servers for IMAP/POP or are these postfix/exim etc?

> I think I would need the necessary Infiniband cards in each server
> along with an Infiniband switch. But do not have any background to
> determine which or even if this is correct.

Simplest architecture is a small IB switch, IB HCA's in each node, an IB 
stack (OFED) in each node, a subnet manager (OpenSM) daemon, IB cables, 
and then Gluster built against your stack.  Make sure your time daemon 
is up, running, and correct between the nodes.  Give serious 
consideration to really fast disk in each node (fast IOP, so SSD, in 
RAID10).

Once you have that, you are ready to build your volumes (probably 
replicated distributed, so 4 way at least).

Do beware that there are some gotchas in configurating MTA/MUA software 
for Gluster.  Its doable, but such software often abhors shared storage 
of any sort.  You need to make sure that you turn down some of the 
caching.

>
> Thanks in advance.
>
> Bryan McGuire Senior Network Engineer NewNet 66
>
> 918.231.8063 bmcguire at newnet66.org

Regards,

Joe

-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics, Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/sicluster
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615