[Gluster-users] very bad performance on small files

[Burnash, James]

Hello.

Gluster - and in fact most (all?) parallel filesystems are optimized for very large files. That being the case, small files are not retrieved as efficiently, and result in a larger number of file operations in total because there are a fixed number for each file accessed.

While the Gluster Development team has put forth a major effort to remedy this as much as possible (using translators), small files simply are not what a filesystem like this was designed for.

I have seen people reporting reasonable success in dealing with this issue on this mailing list - I encourage you to carefully search through it and see what they have done.

"When all you have is a hammer, everything looks like a nail" :-)

James Burnash, Unix Engineering


-----Original Message-----
From: gluster-users-bounces at gluster.org [mailto:gluster-users-bounces at gluster.org] On Behalf Of ???(yongjoon kong)/Cloud Computing ????
Sent: Thursday, January 13, 2011 11:54 PM
To: gluster-users at gluster.org
Subject: Re: [Gluster-users] very bad performance on small files

Maybe it's related with make the Metadata. And I have the same issue here. 

When I untar the kernel source( which has 34000 files) it took over 15 minute on replicated gluster but not more than
1 minute on local filesystem.

Maybe this is because when gluster create metadata using DHT,it should contact  brick to find the leftover space using 'du'

Maybe it is to be a problem. 

But I don't know how to improve the performance  

Best Regards, 
Andrew 

Cloud Computing Business Team
Andrew Kong  Manager | andrew.kong at sk.com | T : +82-2-6400-4328  | M : +82-010-8776-5025
SK u-Tower, 25-1, Jeongja-dong, Bundang-gu, Seongnam-si, Gyeonggi-do, 463-844, Korea




-----Original Message-----
From: gluster-users-bounces at gluster.org [mailto:gluster-users-bounces at gluster.org] On Behalf Of Max Ivanov
Sent: Friday, January 14, 2011 7:32 AM
To: gluster-users
Subject: [Gluster-users] very bad performance on small files

Hi! I've deployed glusterfs on 2 nodes using replication mode and done
some tests.

Peroformance drop was significant and I've no idea if its possible to
improve it or not. Volume was mounted on one of nodes using both FUSE
and NFS glusterfs clients.

 1.1G of small files are stored on volume. To make command shorter M
symbol is used as mountpoint label. Native performance is performance
of command issued over native FS on one of the bricks. No other
activities on both node happened.

Here is the results per command:
dd if=/dev/zero of=M/tmp bs=1M count=16384 69.2 MB/se (Native) 69.2
MB/sec(FUSE) 52 MB/sec (NFS)
dd if=/dev/zero of=M/tmp bs=1K count=163840000  88.1 MB/sec  (Native)
1.1MB/sec (FUSE) 52.4 MB/sec (NFS)
time tar cf - M | pv > /dev/null 15.8 MB/sec (native) 3.48MB/sec
(FUSE) 254 Kb/sec (NFS)

I use default configuration, no adjustments to /etc/gluster* where
made. I use ext4 on bricks.
_______________________________________________
Gluster-users mailing list
Gluster-users at gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
Gluster-users at gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users


DISCLAIMER: 
This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this in error, please immediately notify me and permanently delete the original and any copy of any e-mail and any printout thereof. E-mail transmission cannot be guaranteed to be secure or error-free. The sender therefore does not accept liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission. 
NOTICE REGARDING PRIVACY AND CONFIDENTIALITY Knight Capital Group may, at its discretion, monitor and review the content of all e-mail communications. http://www.knight.com