[Gluster-users] gluster with xfs

[Craig Carl]

On 11/15/2010 06:46 AM, David Lloyd wrote:
> Thanks for that. With the hardware we've got we're hoping to get an
> order of magnitude more throughput than that though.
>
> We have:
>
> 4 nodes
> 12 sata disks hardware raid in each node
> 10Gbit ethernet
> I'm getting over 1GByte/s reads and writes on the xfs filesystems locally.
>
> typical filesize will be 10MByte, but lots of bigger and smaller files too.
>
> there will be about 50 clients, all mounting glusterfs.
>
> 'Introduction to Gluster' also ominously states:  'There are known
> challenges with other filesystems' (which would include xfs). Can
> anyone expand on that?
>
> Cheers
> David
>
> On Mon, Nov 15, 2010 at 11:09 AM, Liam Slusser<lslusser at gmail.com>  wrote:
>> We run two somewhat large gluster clusters in production on xfs with great
>> success.  I had to go with xfs as ext4 doesn't support large enough file
>> systems.  Make sure you mount your xfs partitions with 64bit inode support
>> and use only 64bit OS's.
>>
>> I'm still running 2.0.9 however the performance is pretty good.  We use ours
>> to store media for our website and with our smaller two server four brick
>> 60tb cluster I can easily push 800mbit of http traffic with an average
>> object size of 2-3megs.  Not bad for a bunch of slow sata disks!
>>
>> Liam
>>
David -
    From a previous XFS & Gluster discussion -

"The reason we usually recommend Ext3 instead of XFS is because the 
implementation of extended attributes in Ext3 is significantly faster 
than in XFS. GlusterFS makes use of extended attributes quite a bit, 
especially the replicate translator. In an environment with lots of 
small files and many creation/deletion operations, using XFS with 
replication will be slower than Ext3.

However, if your workload consists of mostly large files and relatively 
fewer create/delete operations, you might find that the performance XFS 
delivers is acceptable. We have many successful deployments that use XFS 
in just this way."


Thanks,

Craig

-->
Craig Carl
Senior Systems Engineer
Gluster