[Gluster-users] Performance with small files (again)

[Andrew Séguin]

Hi,

I made the transition in a hurry a week ago from an NFS server to
Gluster for home folder exports (story should be in the mail archives
of the list). We have about 40 PCs with Linux, which now mount the
home folders directly from Gluster (via GlusterFS and no longer via
NFS). Gluster also exports the storage for our virtual machines.

The performance on the first-day for the desktops was a flop, not so
much because of Gluster, but because of the disks. After changing from
low-rpm "green" hard disks to better disks set in raid, the desktop
experience came back to normal -- no really noticeable drop in
performance. One week later, and nobody is complaining...

Looking at iostat on the underlying Gluster "bricks", the disk is not
getting loaded very much if you look at how much data is being written
-- maybe a few hundred KB per second  -- but the transactions per
second iostat shows, never drops below 30-40, and I've seen it as high
as 300-400.

Since the performance at the desktop for now is acceptable, I haven't
put more time into this for now. I plan on making two tests when time
permits: mounting NFS from Gluster (Gluster's NFS export); trying to
set up Gluster with client-side caching.

I think a longer term solution for me will come down to creating a
virtual machine: user files will be stored inside, and for Gluster, it
will become just one big file on it's storage. This will also probably
become necessary as security comes back to mind.


Regards,
Andrew



On Thu, Jan 27, 2011 at 12:15 AM, David Lloyd
<david.lloyd at v-consultants.co.uk> wrote:
> So, we're trying to serve home directories, and some directories with
> applications from our gluster cluster, mounting as glusterfs. This is stuff
> we've done quite happily from nfs previously.
>
> The main use of the gluster is to serve large image files and it's great for
> that but the piddling little files that in the homedirs and applications are
> much slower (~10x) than the previous nfs. This is measured as the time it
> takes for the application to start, rather than anything low-level.
>
> It feels to me that it's the same kind of thing that's been mentioned here
> before, but it doesn't seem to have been answered.
>
> Should we expect this behaviour?
> Is there anything we can do to improve things?
>
> We're on 3.1.1, haven't done much in the way of tuning from defaults.
>
> Cheers
> David
>
> --
> David Lloyd
> V Consultants
> www.v-consultants.co.uk
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>
>