[Gluster-users] Shared VM disk/image on gluster for redundancy?

[Jeff Darcy]

On 06/30/2010 07:53 AM, Emmanuel Noobadmin wrote:
> On Wed, Jun 30, 2010 at 7:25 PM, Jeff Darcy <jdarcy at redhat.com> wrote:
> 
>> Another option, since you do have a fast interconnect, would be
>> to place all of the permanent storage on the data nodes and use storage
>> on the app nodes only for caching (as we had discussed).  Replicate
>> pair-wise or diagonally between data nodes, distribute across the
>> replica sets, and you'd have a pretty good solution to handle future
>> expansion.
> 
> I think I'll probably go with this since you mention the replicate
> over distribute doesn't work that well and I like to keep the app and
> storage separate. But might change my mind if testing indicates the
> performance level is not acceptable.
> 
> As for fast interconnect, does that imply 10GbE/FC kind of speeds or
> would normal GbE work?

Hm, it appears I was confusing this thread with another one where the
person had mentioned using DDR IB.  By "fast interconnect" (having
worked with interconnects up to 48Gb/s/node) I usually mean at least
10GbE and preferably some form of IB.  Accessing all storage over a GbE
network can work, but often requires more careful tuning and selection
of equipment to get adequate performance.  A lot depends on how much you
can benefit from things like read-ahead and io-cache, or how much data
you're willing to leave in write-behind buffers.  It might well be the
case that replicate over nufa/distribute will work better for your
environment after all despite the issues with app-node "crosstalk" or
the "inversion" of replicate vs. distribute.  I think it's time to
experiment with some of the options and see how they do.