[Gluster-users] How to partition directory structur for 300K files?

[Mathieu Chateau]

Re,

putting back mailing list so they keep up.

With newer version, fuse perform much better, and provide transparent
failover. As both brick are 2 VM in same host, latency will not be an issue.
Most important is to ensure you have drivers /tools inside VM to get best
perf.

on client and servers, I use this in /etc/sysctl.conf

vm.swappiness=0
net.core.rmem_max=67108864
net.core.wmem_max=67108864
# increase Linux autotuning TCP buffer limit to 32MB
net.ipv4.tcp_rmem="4096 87380 33554432"
net.ipv4.tcp_wmem="4096 65536 33554432"
# increase the length of the processor input queue
net.core.netdev_max_backlog=30000
# recommended default congestion control is htcp
net.ipv4.tcp_congestion_control=htcp


options I set on gluster volumes:

server.allow-insecure: on
performance.client-io-threads: on
performance.read-ahead: on
performance.readdir-ahead: enable
performance.cache-size: 1GB
performance.io-thread-count: 16

Options I set on brick in fstab for XFS mounted volumes used by gluster:

defaults,noatime,nodiratime,logbufs=8,logbsize=256k,largeio,inode64,swalloc,allocsize=131072k,nobarrier

Cordialement,
Mathieu CHATEAU
http://www.lotp.fr

2015-08-24 16:11 GMT+02:00 Merlin Morgenstern <merlin.morgenstern at gmail.com>
:

> re your questions:
>
> > did you do some basic tuning to help anyway ?
>
> no, this is a basica setup. Can you please direct me to the most important
> tuning parameters to look at?
>
> > using latest version ?
> glusterfs 3.7.3 built on Jul 28 2015 15:14:43
>
> > in replication or only distributed ?
> re 2
>
> > Why using NFS and not native fuse client to mount volume?
> I was reading that the NFS-Client is better with small files. (typical
> 2-20KB in my case)
>
>
> > did you install VM tools (if using VMware fusion) ?
> I am using virtualbox 5.0.3 on Mac OS X 10.10
>
>
>
> 2015-08-24 16:01 GMT+02:00 Mathieu Chateau <mathieu.chateau at lotp.fr>:
>
>> Hello,
>>
>> did you do some basic tuning to help anyway ?
>> using latest version ?
>> in replication or only distributed ?
>> Why using NFS and not native fuse client to mount volume?
>> did you install VM tools (if using VMware fusion) ?
>>
>> Cordialement,
>> Mathieu CHATEAU
>> http://www.lotp.fr
>>
>> 2015-08-24 15:20 GMT+02:00 Merlin Morgenstern <
>> merlin.morgenstern at gmail.com>:
>>
>>> I am running into trouble while syncing (rsync, cp ... ) my files to
>>> glusterfs. After about 50K files, one machine dies and has to be rebooted.
>>>
>>> As there are about 300K files in one directory, I am thinking about to
>>> cluster that in a directory structure in order to overcome that problem.
>>>
>>> e.g. /0001/filename /0002/filename
>>>
>>> That would cut down the amount of files in one directory. However this
>>> is something I would like to avoid if possible due to SEO - changing the
>>> url of the file brings a lot of trouble.
>>>
>>> The system underneith are 2 seperate VM instances, each running ubuntu
>>> 14.04. Cluster NFS client on same machine as Gluster server. Macbook Pro 13
>>> retina with capable SSD and 1G internal network between the VMs.
>>>
>>> Thank you for any help on this.
>>>
>>> _______________________________________________
>>> Gluster-users mailing list
>>> Gluster-users at gluster.org
>>> http://www.gluster.org/mailman/listinfo/gluster-users
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150824/8fb666ae/attachment.html>