[Gluster-users] how well will this work

[Jeff Darcy]

On 12/27/12 6:47 PM, Miles Fidelman wrote:
> John Mark Walker wrote:
>> In general, I don't recommend any distributed filesystems for VM
>> images, but I can also see that this is the wave of the future.
> Ok.  I can see that.
> 
> Let's say that I take a slightly looser approach to high-availability:
> - keep the static parts of my installs on local disk
> - share and replicate dynamic data using gluster

That, in a nutshell, is the approach that I (and others) often advocate.  Block
storage should be used sparingly, e.g. for booting and for data served to
others at a higher level.  I'd say that's true in general, but it's especially
true for any kind of network block storage.  When network latencies are
involved, going "up the stack" where operations are expressed at a high
semantic level will almost always work out better than blocks and locks.

> In this scenario, how well does gluster work when:
> - storage and processing are inter-mixed on the same nodes

That works fine and is a common deployment model for the community code, though
RHS demands a "separate server and client" model.  The main thing to watch out
for is CPU/memory contention between application and Gluster processes.  Those
can be addressed in all the standard ways, from cgroups to containers to
virtualization.

> - data is triply replicated (allow for 2-node failures)

Unfortunately, three-way replication is still a bit of a work in progress.
Some (such as Joe Julian) use it successfully, but they also use it very
carefully.  I've had to make a few fixes in this area myself recently, and I
expect to make a few more before I'd say that it's really up to snuff for
general use.