[Gluster-users] drop in write performance

[Ason Hu]

Hi Pranith,


Gluster Version :
glusterfs 3.6.2 built on Jan 22 2015 12:58:10

   - 100,000 file test in XFS directly

[root at glusterfs-node1 gfs_data]# java FileCreationAndSearchtest 10w_test
100000
Creation finished!! Time: *8961(ms)*
File Search Finished!! 33223 searches, time: 124ms, misses:0
NIO Search Finished!! 33223 searches, time: 367ms, misses:0



   - 100,000 file test from client (Distributed Replicated)

[root at glusterfs-client01 drvol]# java FileCreationAndSearchtest 10w_test01
100000
Start!!
Creation finished!! Time: *265713(ms)*
File Search Finished!! 33453 searches, time: 18403(ms), misses:0
NIO Search Finished!! 33453 searches, time: 17827(ms), misses:0



after many small file created (≒200 million), it cost more than 1000000(ms)
to finish 100,000 file test

I've flushed all volumes and create new one to reproduce the condition
again.



2015-02-05 19:19 GMT+08:00 Pranith Kumar Karampuri <pkarampu at redhat.com>:

>
> On 02/02/2015 02:42 PM, Ason Hu wrote:
>
> Hi,
>
>  In my small file test, 3 types of volume are created in same GlusterFS
> Cluster (4 nodes)
>
>    - Distritubed
>     - Distributed Replicated
>     - Distributed Striped
>
>
> 10 million small files (8 byte) create and search each test cycle life
>
>
>  Nodes info :
>
>    - OS : CentOS 6.6 64bit
>    - RAM : 64GB
>     - HDD : 4TB, 7200RPM, SAS *6 RAID 0
>     - FileSystem : XFS, BS=4K
>
>
>
>  Problem :
>
>  It occur when file reach ≒200 million (sum of 3 volumes)
>  files created speed drop from  ≒1500 to  ≒100
>
> If you do this test directly on the XFS partition, what is the kind of
> performance difference you see?
>
> Pranith
>
>
>  Though the speed of creation become slow but search speed and throughput
> still good
> Seems each types of volume have same problem, I don't know what cause this
> problem..
>
>
>  Workaround :
>
>  I try to create new volume ( Distributed | Distributed Replicated |
> Distributed Striped ), mount it,
> problem solved!
> But the old volume write speed still sad..
>
>
>
>  I'd  try to increase below volume setting, but don't work at all
>
>    - performance.cache-size
>     - performance.io-thread-count
>
>
>  I didn't enable the volume qouta option.
> Does glusterfs volume have some kind of limitation of file creation?
>
>
>
>  Regards
> Nosa Hu,
>
>
>
>
> _______________________________________________
> Gluster-users mailing listGluster-users at gluster.orghttp://www.gluster.org/mailman/listinfo/gluster-users
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150206/9b71728a/attachment.html>