[Gluster-users] Very slow directory listing and high CPU usage on replicated volume
jonathan.lefman at essess.com
Mon Nov 5 12:58:22 UTC 2012
I take it back. Things deteriorated pretty quickly after I began dumping
data onto my volume from multiple clients. Initially my transfer rates were
okay, not fast, but livable. However after about an hour of copying several
terabytes from 3-4 client machines, the rates of transfer often dropped to
lb/s. Sometimes I would see a couple second burst of good transfer rates.
Anyone have ideas on how to address this effectively? I'm at a loss.
On Nov 2, 2012 1:21 PM, "Jonathan Lefman" <jonathan.lefman at essess.com>
> I should have also said that my volume is working well now and all is well.
> On Fri, Nov 2, 2012 at 1:21 PM, Jonathan Lefman <
> jonathan.lefman at essess.com> wrote:
>> Thank you Brian. I'm happy to hear that this behavior is not typical. I
>> am now using xfs on all of my drives. I also wiped out the entire
>> /etc/glusterd directory for good measure. I bet that there was residual
>> information from a previous attempt at a gluster volume that must have
>> caused problems. Or moving to xfs from ext4 is an amazing fix, but I think
>> this is less likely.
>> I appreciate your time responding to me.
>> On Nov 2, 2012 4:44 AM, "Brian Candler" <B.Candler at pobox.com> wrote:
>>> On Thu, Nov 01, 2012 at 08:03:21PM -0400, Jonathan Lefman wrote:
>>> > Soon after loading up about 100 MB of small files (about 300kb
>>> > the drive usage is at 1.1T.
>>> That is very odd. What do you get if you run du and df on the individual
>>> bricks themselves? 100MB is only ~330 files of 300KB each.
>>> Did you specify any special options to mkfs.ext4? Maybe -l 512 would
>>> as the xattrs are more likely to sit within the indoes themselves.
>>> If you start everything from scratch, it would be interesting to see df
>>> stats when the filesystem is empty. It may be that a huge amount of
>>> has been allocated to inodes. If you expect most of your files >16KB
>>> you could add -i 16384 to mkfs.ext4 to reduce the space reserved for
>>> But using xfs would be better, as it doesn't reserve any space for
>>> it allocates it dynamically.
>>> Ignore the comment that glusterfs is "not designed for handling large
>>> small files" - 300KB is not small.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Gluster-users