[Gluster-users] Problem with performance

[Roland Rabben]

Thanks for replying. Comments below.

2011/2/16 Luis Cerezo <lec at luiscerezo.org>:
> is there a difference in speed between "ls" and /bin/ls ?

ls on my root folder containing 999 sub folders takes about 5 minutes
the first time I run ls. Then about a minute 15 seconds the following
times. I guess caching comes into play.

/bin/ls on the same root folder takes less than 2 seconds. Pretty
significant change!

> most linux systems have the pretty console colors, and there is an alias set
> for ls to ls --color=tty. this make ls read extra attributes. this takes
> forever on dirs with tons of files. if you do a long listing, it has to do
> lookups against NIS/LDAP for uid info if you use directory services too.
> how large are these files?

> do you have atime on? do you need atime on?

Bricks are mounted with noatime parameter, so atime should be disabled.

> what are your bricks formatted as? ext3/4? Have you tuned them?

My bricks are formatted as ext4. I have not really tuned them. I need
some intelligence on how to tune them.


> what is the network performance like outside of gluster? have you tested the
> network with iperf etc?

iperf on the production environment shows on average about 350-400
Mbps throughput.

Regards

Roland

> hope this helps..
> -luis
> On Feb 16, 2011, at 7:56 AM, Roland Rabben wrote:
>
> No answer to this at all?
>
> Regards
>
> Roland Rabben
>
> 2011/2/13 Roland Rabben <roland at jotta.no>:
>
> I have a few Glusterfs clusters. They are all running version 3.0.5.
>
> Both servers and clients are on Ubuntu 64 bit systems.
>
> My clusters are configured as Distributed and Replicated over Gigabit
>
> network. About 5 clients are reading and writing millions of files in
>
> all sizes to my filesystem.
>
> The volumes are exported using the fuse-based Glusterfs client. Volume
>
> size is usually 115 TB.
>
> I am experiencing performance degradation as the number of folders and
>
> files on my gluster filesystem grows.  Running a simple command like
>
> ls can take minutes to complete.
>
> Cocurrent write performance is pretty poor.
>
> Is this performance degradation expected behavior?
>
> Are there any performance improvements related to concurrency and
>
> small files in Glusterfs 3.1?
>
> Any operating system or RAID configurations the community can
>
> recommend to help with performance? (We are mostly using Adaptec 5805
>
> RAID controllers on or storage nodes.)
>
>
> Best regards,
>
> Roland Rabben
>
> Founder & CEO Jotta AS
>
> Cell: +47 90 85 85 39
>
> Phone: +47 21 04 29 00
>
> Email: roland at jotta.no
>
>
>
>
> --
> Roland Rabben
> Founder & CEO Jotta AS
> Cell: +47 90 85 85 39
> Phone: +47 21 04 29 00
> Email: roland at jotta.no
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>
>
> Luis E. Cerezo
>
> http://www.luiscerezo.org
> http://twitter.com/luiscerezo
> http://flickr.com/photos/luiscerezo
> Voice: 412 223 7396
>



-- 
Roland Rabben
Founder & CEO Jotta AS
Cell: +47 90 85 85 39
Phone: +47 21 04 29 00
Email: roland at jotta.no