[Gluster-devel] Problem during reproducing smallfile experiment on Gluster 10

박현승 hspark0582 at gluesys.com
Thu Jan 20 06:53:54 UTC 2022


Dear Gluster developers,

 

This is Hyunseung Park at Gluesys, South Korea.

  

We are trying to replicate the test in https://github.com/gluster/glusterfs/issues/2771 but to no avail.

In our experiments, Gluster version 10 unfortunately did not perform noticably better than version 9.

  

 


 
  
  
  
  
  
 
 
  
   v9 2x2
     
     
     
   average
  
  
   create
   3399.99
   3484.79
   2702.57
   3195.783333
  
  
   ls -l
   65605.2
   64930.6
   72018.7
   67518.16667
  
  
   chmod
   4858.95
   4965.29
   5597.73
   5140.656667
  
  
   stat
   7334.88
   7755.89
   8335.11
   7808.626667
  
  
   read
   7015.64
   8255.48
   7007.01
   7426.043333
  
  
   append
   2554.93
   2777.65
   2572.57
   2635.05
  
  
   mkdir
   1800.29
   1865.07
   1805.48
   1823.613333
  
  
   rmdir
   1854.09
   1722.89
   1876.81
   1817.93
  
  
   cleanup
   2402.02
   2447.36
   2438.71
   2429.363333
  
  
     
     
     
     
     
  
  
   v10 2x2
     
     
     
   average
  
  
   create
   3741.39
   3174.82
   3234.42
   3383.543333
  
  
   ls -l
   71543.7
   67275.9
   72975.1
   70598.23333
  
  
   chmod
   5441.11
   5109.22
   5004.08
   5184.803333
  
  
   stat
   7746.37
   7677.99
   7885.72
   7770.026667
  
  
   read
   7061.12
   7165.21
   7121.07
   7115.8
  
  
   append
   3458.93
   2641.84
   2887.46
   2996.076667
  
  
   mkdir
   2685.22
   1879.35
   1970.91
   2178.493333
  
  
   rmdir
   2240.11
   1648.37
   1602.16
   1830.213333
  
  
   cleanup
   3739.68
   2407.57
   2403.48
   2850.243333
  
 


 

The result above is from the test that deployed 32 threads on each of the 4 clients.

Some results were better than others, but it is not good enough when compared to the result in the aforementioned link.

We are wondering what we can do to get the full potential of the new version.

 

We have been running tests with varying file sizes, number of threads, different volume topology, etc. but we were not able to see data conclusive enough.

We were also not able to find meaningful output from running tests using other benchmark tools such as bonnie++ and FIO.

 

To find the potential cause we tried to look into the program by calling malloc_stats() and using perf.

However, we also could not find something noteworthy from the result.

Here is the data recorded during one set of smallfile test (from create to cleanup): https://drive.google.com/drive/folders/1NMXNjgOZ7svDd4-YvKCU4UAp43tm15dC?usp=sharing

  

Below is our test environment:

  

Basic HW info: VM (vSphere), 2 core CPU, 4G RAM. 4 servers and 4 clients.

OS: Centos 7

kernel version: 3.10.0-1160

Gluster version: built from git, checked out from branch 'release-9' and 'release-10' respectively (commits 7094da and a804f8)

build option: default setting except "./configure --disable-linux-io_uring" (kernel does not support io_uring)


Configuration result (case of version 10):

GlusterFS configure summary
===========================
FUSE client          : yes
epoll IO multiplex   : yes
fusermount           : yes
readline             : no
georeplication       : yes
Linux-AIO            : yes
Linux io_uring       : no
Use liburing         : no
Enable Debug         : no
Run with Valgrind    : no
Sanitizer enabled    : none
XML output           : yes
Unit Tests           : no
Track priv ports     : yes
POSIX ACLs           : yes
SELinux features     : yes
firewalld-config     : no
Events               : yes
EC dynamic support   : x64 sse avx
Use memory pools     : no
Nanosecond m/atimes  : yes
Server components    : yes
Legacy gNFS server   : no
IPV6 default         : no
Use TIRPC            : yes
With Python          : 3.6
Cloudsync            : yes
Metadata dispersal   : no
Link with TCMALLOC   : yes
Enable Brick Mux     : no
Building with LTO    : no

  

  


 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-devel/attachments/20220120/18d35c36/attachment-0001.html>


More information about the Gluster-devel mailing list