[Gluster-users] heavy metadata issues?

[Moore, Michael]

Hi Joe,

  We've been seeing the same thing here with several GlusterFS setups.  It is really bad with GigE under load.  From looking at our setups, it appears to be a network latency/contention issue.  It is definitely not a disk contention issue as there is not a lot of load on the disks at the time of the metadata performance issues.

  Like you, we've tried a number of different combinations to help improve the performance.  statprefetch will help with subsequent accesses to the same metadata, but it doesn't help with the initial access.

  We have not tried Infiniband yet to see if it improves the performance, but I am concerned that in a high IOps environment (like we run here), that even IB will not be able to sustain performance.

- Mike


Michael Moore
Staff Engineer - Network
SOLiD Advanced Research and Collaborations
500 Cummings Center
Suite 2400
Beverly, MA 01915
T: 978-232-7886

Michael.Moore at lifetech.com
Mike.Moore at appliedbiosystems.com




-----Original Message-----
From: gluster-users-bounces at gluster.org [mailto:gluster-users-bounces at gluster.org] On Behalf Of Joe Landman
Sent: Tuesday, April 27, 2010 7:19 PM
To: gluster-users at gluster.org
Subject: [Gluster-users] heavy metadata issues?

Hi folks

  Seeing some metadata slowdowns in some cases on GlusterFS 3.0.3. 
These metadata slowdowns manifest in terms of long ls, long stat, long 
rm times for directories with a few thousand files.

  We tried statprefetch at the end of the stack, as well as right after 
the protocol/client, tried increasing read-ahead, etc.

  Is there anything in particular we should be looking to do, in order 
to get reasonable metadata performance under load?

  Thanks!

Joe

-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/jackrabbit
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615
_______________________________________________
Gluster-users mailing list
Gluster-users at gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users