[Gluster-devel] accessing glusterfs mounted share - really slow

[Anand Avati]

Matthias,
 which is your fuse version? You could try a patched fuse from the Gluster
project available at -

http://ftp.zresearch.com/pub/gluster/glusterfs/fuse/

If you happen to use this fuse, use a page-size of 1MB in client read-ahead
(with a lower page-count, of 2)

avati

2007/8/31, Anand Avati <avati at zresearch.com>:
>
>
>
>
> So I guess, I should add a readahead and writebehind translator on
> > server side?
>
>
> Generally just io-threads => storage/posix, or write-behind (aggregate
> upto 16MB) => io-threads => stroage/posix on the server-side will do good.
> io-threads is the important one on server.
>
> io-cache can also help you as well if your I/O pattern has a lot of
> re-reads.
>
> Please use 1.3.1 tarball or the latest TLA checkout as they have quite a
> few important fixes.
>
>  avati
>
> Regards,
> >
> >   Matthias
> >
> > > avati
> > >
> > > 2007/8/31, Matthias Albert <gluster at linux4experts.de
> > > <mailto: gluster at linux4experts.de>>:
> > >
> > >     Hi Krishna,
> > >
> > >
> > >     Krishna Srinivas schrieb:
> > >     > Hi Matthias,
> > >     >
> > >     > If I understand correctly, for you all the operations are fine,
> > but
> > >     > when a "cp" is being done and simultaneously you do "ls" from
> > >     > another client, the "ls" is slow?
> > >     >
> > >     yepp, absolutly correct. Only If I do a "cp or dd for example" the
> >
> > >     ls or
> > >     tab completion is really slow and only in the glusterfs mounted
> > share.
> > >
> > >     Matthias
> > >
> > >     > Krishna
> > >     >
> > >     > On 8/31/07, Matthias Albert < gluster at linux4experts.de
> > >     <mailto:gluster at linux4experts.de>> wrote:
> > >     >
> > >     >> Hi all,
> > >     >>
> > >     >> first of all, I've to say that gluterfs is really cool and
> > >     absolutly
> > >     >> great. I'm not a cluster filesystem specialist but I
> > >     tested/configured
> > >     >> openafs and lustre and both of them are so huge and
> > complicated.
> > >     >> As I saw glusterfs and played a little bit with it, I was
> > really
> > >     >> surprised how easy it is to setup a cluster filesystem without
> > >     extra
> > >     >> acl's, without formatting the new filesystem without a
> > >     >> metadata/objectserver :-). Thanks a lot for this.
> > >     >>
> > >     >> Of course I've some questions :-).
> > >     >>
> > >     >> I've setup 4 glusterfsd server, each of them with a storage of
> > >     about
> > >     >> 400-500 Gig pre-tax.
> > >     >> On client side  I made different afr's over my remote volumes
> > and
> > >     >> finally a unify over the afr's. Readahead and writebehind is
> > >     also enabled.
> > >     >>
> > >     >> Everything is working fine. I can copy "tons" of Gigabytes in
> > my
> > >     >> glusterfs without any problms and also my performance is
> > >     absolutly great.
> > >     >>
> > >     >> But every time I start a "cp" or do a "dd test (to write some
> > >     testfiles
> > >     >> in the gluster storage) on some of my clients (I've 3 glusterfs
> > >     clients
> > >     >> one of them is a bacula server which uses the glusterfs as
> > >     storage)
> > >     >> all access from my glusterfs clients to the mounted share is
> > really
> > >     >> slow. It takes sometimes about 3-4 seconds till my ls is
> > >     printing the
> > >     >> output of the directory.
> > >     >>
> > >     >> e.g.
> > >     >> ---snip---
> > >     >> bash# df -h
> > >     >> glusterfs             892G   84G  809G  10% /backup
> > >     >>
> > >     >> gsx:/backup/vmware-images # time ll
> > >     >> ...
> > >     >> ...
> > >     >> real     0m2.863s
> > >     >> user    0m0.004s
> > >     >> sys     0m0.005s
> > >     >> gsx:/backup/vmware-images #
> > >     >> ---snap---
> > >     >>
> > >     >> Also the "tab completion" in the mounted glusterfs share is
> > >     really slow.
> > >     >> Access of not mounted glusterfs share is just normal (accessing
> >
> > >     /etc
> > >     >> /usr/ /root etc. )
> > >     >>
> > >     >> Does anyone know these "phenomenon"?
> > >     >>
> > >     >> I'm using Debian as distro for all of my servers and Debian and
> >
> > >     SuSE on
> > >     >> Client side.
> > >     >>
> > >     >> glusterfs version: glusterfs--mainline--2.5 patch-459
> > >     >> fuse: fuse-2.7.0-glfs3
> > >     >>
> > >     >> If needed I can post my configs, strace outputs of ls -la and
> > >     so on.
> > >     >>
> > >     >> Regards,
> > >     >>
> > >     >>   Matthias
> > >     >>
> > >     >>
> > >     >>
> > >     >>
> > >     >>
> > >     >>
> > >     >>
> > >     >>
> > >     >>
> > >     >>
> > >     >>
> > >     >>
> > >     >> _______________________________________________
> > >     >> Gluster-devel mailing list
> > >     >> Gluster-devel at nongnu.org <mailto:Gluster-devel at nongnu.org>
> > >     >> http://lists.nongnu.org/mailman/listinfo/gluster-devel
> > >     <http://lists.nongnu.org/mailman/listinfo/gluster-devel>
> > >     >>
> > >     >>
> > >
> > >
> > >
> > >     _______________________________________________
> > >     Gluster-devel mailing list
> > >     Gluster-devel at nongnu.org <mailto:Gluster-devel at nongnu.org>
> > >     http://lists.nongnu.org/mailman/listinfo/gluster-devel
> > >
> > >
> > >
> > >
> > > --
> > > It always takes longer than you expect, even when you take into
> > > account Hofstadter's Law.
> > >
> > > -- Hofstadter's Law
> >
> >
> >
> > _______________________________________________
> > Gluster-devel mailing list
> > Gluster-devel at nongnu.org
> > http://lists.nongnu.org/mailman/listinfo/gluster-devel
> >
>
>
>
> --
> It always takes longer than you expect, even when you take into account
> Hofstadter's Law.
>
> -- Hofstadter's Law
>



-- 
It always takes longer than you expect, even when you take into account
Hofstadter's Law.

-- Hofstadter's Law