[Gluster-devel] accessing glusterfs mounted share - really slow
Matthias Albert
gluster at linux4experts.de
Fri Aug 31 10:03:53 UTC 2007
Hi,
thx a lot for all your replys. I will read the "Optimizing Glusterfs
Guide" and do these changes and also update to the latest tla version.
After finishing this, I will post my results here.
Regards,
Matthias
>
>
>
> So I guess, I should add a readahead and writebehind translator on
> server side?
>
>
> Generally just io-threads => storage/posix, or write-behind (aggregate
> upto 16MB) => io-threads => stroage/posix on the server-side will do
> good. io-threads is the important one on server.
>
> io-cache can also help you as well if your I/O pattern has a lot of
> re-reads.
>
> Please use 1.3.1 tarball or the latest TLA checkout as they have quite
> a few important fixes.
>
> avati
>
> Regards,
>
> Matthias
>
> > avati
> >
> > 2007/8/31, Matthias Albert <gluster at linux4experts.de
> <mailto:gluster at linux4experts.de>
> > <mailto: gluster at linux4experts.de
> <mailto:gluster at linux4experts.de>>>:
> >
> > Hi Krishna,
> >
> >
> > Krishna Srinivas schrieb:
> > > Hi Matthias,
> > >
> > > If I understand correctly, for you all the operations are
> fine, but
> > > when a "cp" is being done and simultaneously you do "ls" from
> > > another client, the "ls" is slow?
> > >
> > yepp, absolutly correct. Only If I do a "cp or dd for
> example" the
> > ls or
> > tab completion is really slow and only in the glusterfs
> mounted share.
> >
> > Matthias
> >
> > > Krishna
> > >
> > > On 8/31/07, Matthias Albert < gluster at linux4experts.de
> <mailto:gluster at linux4experts.de>
> > <mailto:gluster at linux4experts.de
> <mailto:gluster at linux4experts.de>>> wrote:
> > >
> > >> Hi all,
> > >>
> > >> first of all, I've to say that gluterfs is really cool and
> > absolutly
> > >> great. I'm not a cluster filesystem specialist but I
> > tested/configured
> > >> openafs and lustre and both of them are so huge and
> complicated.
> > >> As I saw glusterfs and played a little bit with it, I was
> really
> > >> surprised how easy it is to setup a cluster filesystem
> without
> > extra
> > >> acl's, without formatting the new filesystem without a
> > >> metadata/objectserver :-). Thanks a lot for this.
> > >>
> > >> Of course I've some questions :-).
> > >>
> > >> I've setup 4 glusterfsd server, each of them with a
> storage of
> > about
> > >> 400-500 Gig pre-tax.
> > >> On client side I made different afr's over my remote
> volumes and
> > >> finally a unify over the afr's. Readahead and writebehind is
> > also enabled.
> > >>
> > >> Everything is working fine. I can copy "tons" of
> Gigabytes in my
> > >> glusterfs without any problms and also my performance is
> > absolutly great.
> > >>
> > >> But every time I start a "cp" or do a "dd test (to write
> some
> > testfiles
> > >> in the gluster storage) on some of my clients (I've 3
> glusterfs
> > clients
> > >> one of them is a bacula server which uses the glusterfs as
> > storage)
> > >> all access from my glusterfs clients to the mounted share
> is really
> > >> slow. It takes sometimes about 3-4 seconds till my ls is
> > printing the
> > >> output of the directory.
> > >>
> > >> e.g.
> > >> ---snip---
> > >> bash# df -h
> > >> glusterfs 892G 84G 809G 10% /backup
> > >>
> > >> gsx:/backup/vmware-images # time ll
> > >> ...
> > >> ...
> > >> real 0m2.863s
> > >> user 0m0.004s
> > >> sys 0m0.005s
> > >> gsx:/backup/vmware-images #
> > >> ---snap---
> > >>
> > >> Also the "tab completion" in the mounted glusterfs share is
> > really slow.
> > >> Access of not mounted glusterfs share is just normal
> (accessing
> > /etc
> > >> /usr/ /root etc. )
> > >>
> > >> Does anyone know these "phenomenon"?
> > >>
> > >> I'm using Debian as distro for all of my servers and
> Debian and
> > SuSE on
> > >> Client side.
> > >>
> > >> glusterfs version: glusterfs--mainline--2.5 patch-459
> > >> fuse: fuse-2.7.0-glfs3
> > >>
> > >> If needed I can post my configs, strace outputs of ls -la
> and
> > so on.
> > >>
> > >> Regards,
> > >>
> > >> Matthias
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> _______________________________________________
> > >> Gluster-devel mailing list
> > >> Gluster-devel at nongnu.org
> <mailto:Gluster-devel at nongnu.org> <mailto:Gluster-devel at nongnu.org
> <mailto:Gluster-devel at nongnu.org>>
> > >> http://lists.nongnu.org/mailman/listinfo/gluster-devel
> > <http://lists.nongnu.org/mailman/listinfo/gluster-devel>
> > >>
> > >>
> >
> >
> >
> > _______________________________________________
> > Gluster-devel mailing list
> > Gluster-devel at nongnu.org <mailto:Gluster-devel at nongnu.org>
> <mailto:Gluster-devel at nongnu.org <mailto:Gluster-devel at nongnu.org>>
> > http://lists.nongnu.org/mailman/listinfo/gluster-devel
> <http://lists.nongnu.org/mailman/listinfo/gluster-devel>
> >
> >
> >
> >
> > --
> > It always takes longer than you expect, even when you take into
> > account Hofstadter's Law.
> >
> > -- Hofstadter's Law
>
>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org <mailto:Gluster-devel at nongnu.org>
> http://lists.nongnu.org/mailman/listinfo/gluster-devel
> <http://lists.nongnu.org/mailman/listinfo/gluster-devel>
>
>
>
>
> --
> It always takes longer than you expect, even when you take into
> account Hofstadter's Law.
>
> -- Hofstadter's Law
More information about the Gluster-devel
mailing list