<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Hi All,</p>
<p>So we've got a cluster currently running on glusterfs 7.3</p>
<p>Heal performance is absolutely terrible (it starts of great, runs
for a while then completely trips up).</p>
<p>Once the SHD trips up, access speeds to the cluster also becomes
poor. And with poor I mean directory listings that normally takes
well under 500ms takes upwards of 10s, but I've seen upwards of
40s (although a follow up ls immediately after is nice and fast
again, temporarily). With all SHDs killed this particular ls is
as per normal (around 30-50ms).<br>
<br>
The SHD logs doesn't provide any usable clues, the only variance
in logs is this, just before all daemons across the cluster crawls
to a near halt.</p>
<div class="de1">[2020-05-02 19:55:58.627036] I [MSGID: 108026]
[afr-self-heal-common.c:1747:afr_log_selfheal]
0-vpbx_shared-replicate-0: Completed entry selfheal on
5b9aa3d8-5085-46e2-9500-145b8cbf9a61. sources= sinks=0 1 </div>
<div class="de1">[2020-05-02 19:55:58.627058] I [MSGID: 108026]
[afr-self-heal-common.c:1747:afr_log_selfheal]
0-vpbx_shared-replicate-0: Completed entry selfheal on
c736c9b6-f116-468b-adbc-f102f5b57d01. sources= sinks=0 1 </div>
<div class="de1"> </div>
<div class="de1">[2020-05-02 19:55:59.125001] W [MSGID: 114031]
[client-rpc-fops_v2.c:1983:client4_0_setattr_cbk]
0-vpbx_shared-client-1: remote operation failed [Invalid argument]</div>
<div class="de2"> </div>
<div class="de1">[2020-05-02 19:55:59.339803] I [MSGID: 108026]
[afr-self-heal-common.c:1747:afr_log_selfheal]
0-vpbx_shared-replicate-0: Completed entry selfheal on
2e128eeb-e991-4544-8316-4ddfcb4942da. sources= sinks=0 1 <br>
[2020-05-02 19:55:59.490387] I [MSGID: 108026]
[afr-self-heal-metadata.c:51:__afr_selfheal_metadata_do]
0-vpbx_shared-replicate-0: performing metadata selfheal on
527a63a3-9acd-4c3f-b95c-5c719325c973</div>
<p>A few more entries will complete overall and then it'll be a file
every other second. Once I saw this I ran a few more tests, and
on every test just prior to slowing to a crawl, that entry is
there.</p>
<p>There is a perceived improvement when lowering the thread count,
but not significant enough to confirm a correlation. Even with<b>
</b>cluster.shd-max-threads=1 this still happens (gets a few tens
thousand files completed instead of few hundred, at 32 it gets to
maybe 400 to 500 before it stalls, 16 does maybe a thousand).<b><br>
</b></p>
<p>There are probably performant bursts hereafter, because from
experience we know this process takes about two months. But this
time around it seems to be much, much worse. Might be because
that was healing a single brick to another brick, so less locking
going? At the moment the replicas are basically trying to
populate the newly added arbiters. Also, we haven't messed as
greatly with the health check settings (queues and threads and
locks nearly as much as since the arbiter addition because
performance has simply dropped to non-acceptable levels.<br>
</p>
<p>Unfortunately granular entry heal isn't on at the moment (nor can
I switch it on due to a few million outstanding heals). This used
to be a 2x2 cluster. We then moved one of the bricks to a newly
added server using replace-brick. What a frightening process this
is. We then proceeded to add the arbiters.</p>
<p>Some of the settings below was absolute shots in the dark, based
on the little information we've been able to track down so far.<br>
</p>
<p>Kind Regards,<br>
Jaco<br>
</p>
<p>The current gluster volume info (this used to be 2x2
distributed-replicate, aiming to get it to 3x (2 + 1) ... at
current rates we might get there by 2021):<br>
<br>
Volume Name: glvol<br>
Type: Distributed-Replicate<br>
Volume ID: 87cfd38f-d72e-4cc2-b644-9e772d2059bb<br>
Status: Started<br>
Snapshot Count: 0<br>
Number of Bricks: 2 x (2 + 1) = 6<br>
Transport-type: tcp<br>
Bricks:<br>
Brick1: zoidberg:/mnt/gluster/a<br>
Brick2: flexo:/mnt/gluster/b<br>
Brick3: bender:/mnt/gluster/c (arbiter)<br>
Brick4: bender:/mnt/gluster/a<br>
Brick5: zoidberg:/mnt/gluster/b<br>
Brick6: flexo:/mnt/gluster/c (arbiter)<br>
Options Reconfigured:<br>
<b>cluster.heal-timeout: 5 <-- hoping this would at least
shorten the blockages.</b><br>
cluster.locking-scheme: granular<br>
cluster.heal-wait-queue-length: 10240<br>
<b>diagnostics.count-fop-hits: on</b><b><br>
</b><b>diagnostics.latency-measurement: on <-- was hoping
profiling might provide some insight. Going to switch this off
now.</b><br>
cluster.background-self-heal-count: 128<br>
<b>cluster.shd-wait-qlength: 131072</b><b><br>
</b><b>cluster.shd-max-threads: 32 <-- even with this disk IO
never goes over 10% utilization. CPU goes to about 2 cores. So
it's lock contention that's slowing it down we reckon.</b><br>
<b>cluster.metadata-self-heal: off</b><b><br>
</b><b>cluster.entry-self-heal: off</b><b><br>
</b><b>cluster.data-self-heal: off</b><b> <-- these kill
performance.<br>
</b>transport.address-family: inet<br>
nfs.disable: on<br>
performance.client-io-threads: on<br>
features.cache-invalidation: on<br>
features.cache-invalidation-timeout: 600<br>
performance.stat-prefetch: on<br>
performance.cache-invalidation: on<br>
performance.md-cache-timeout: 600<br>
<b>cluster.self-heal-daemon: yes <-- disabling/enabling this
doesn't help, have to kill the shd and start ... force to get
things moving along again.</b><br>
performance.io-thread-count: 16<br>
performance.high-prio-threads: 16<br>
performance.normal-prio-threads: 16<br>
performance.low-prio-threads: 16<br>
performance.least-prio-threads: 4<br>
cluster.self-heal-window-size: 512<br>
cluster.self-heal-readdir-size: 131072<br>
client.event-threads: 8<br>
server.event-threads: 16<br>
network.ping-timeout: 2<br>
<b>cluster.data-self-heal-algorithm: full <-- most files are
small (<1MB), on dual 1gbit links (so up to 2gbit, but b/w
never goes over 5mbit anyway).</b><br>
</p>
<p>performance info:</p>
<p><tt>Brick: zoidberg:/mnt/gluster/a</tt><tt><br>
</tt><tt>-----------------------------------------</tt><tt><br>
</tt><tt>Cumulative Stats:</tt><tt><br>
</tt><tt> Block Size: 1b+
8b+ 16b+ </tt><tt><br>
</tt><tt> No. of Reads: 0
0 0 </tt><tt><br>
</tt><tt>No. of Writes: 117
195 367 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 32b+
64b+ 128b+ </tt><tt><br>
</tt><tt> No. of Reads: 306
0 6 </tt><tt><br>
</tt><tt>No. of Writes: 1016
9 134 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 256b+
512b+ 1024b+ </tt><tt><br>
</tt><tt> No. of Reads: 107
58 485 </tt><tt><br>
</tt><tt>No. of Writes: 731
73 168 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 2048b+
4096b+ 8192b+ </tt><tt><br>
</tt><tt> No. of Reads: 43
59 383 </tt><tt><br>
</tt><tt>No. of Writes: 295
475 10562 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 16384b+
32768b+ 65536b+ </tt><tt><br>
</tt><tt> No. of Reads: 347
380 634 </tt><tt><br>
</tt><tt>No. of Writes: 1487
6369 3970 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 131072b+ </tt><tt><br>
</tt><tt> No. of Reads: 35704 </tt><tt><br>
</tt><tt>No. of Writes: 12398 </tt><tt><br>
</tt><tt> %-latency Avg-latency Min-Latency Max-Latency
No. of calls Fop</tt><tt><br>
</tt><tt> --------- ----------- ----------- -----------
------------ ----</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 957293 FORGET</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 1455091 RELEASE</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 69951 RELEASEDIR</tt><tt><br>
</tt><tt> 0.00 16.20 us 5.43 us 100.95
us 113 IPC</tt><tt><br>
</tt><tt> 0.00 687.33 us 80.53 us 15414.03
us 30 REMOVEXATTR</tt><tt><br>
</tt><tt> 0.00 840.76 us 72.71 us 37648.39
us 52 FTRUNCATE</tt><tt><br>
</tt><tt> 0.00 138.49 us 36.66 us 25280.24
us 1385 STAT</tt><tt><br>
</tt><tt> 0.00 364.93 us 63.45 us 15957.66
us 810 FGETXATTR</tt><tt><br>
</tt><tt> 0.00 2208.82 us 127.23 us 267092.17
us 137 LINK</tt><tt><br>
</tt><tt> 0.00 37595.33 us 1677.46 us 218313.32
us 12 FSYNC</tt><tt><br>
</tt><tt> 0.00 675.61 us 106.82 us 114351.01
us 698 TRUNCATE</tt><tt><br>
</tt><tt> 0.00 3995.66 us 147.00 us 109544.46
us 188 RENAME</tt><tt><br>
</tt><tt> 0.00 453.11 us 53.37 us 104848.94
us 1680 SETATTR</tt><tt><br>
</tt><tt> 0.00 6004.32 us 229.33 us 151322.58
us 127 MKNOD</tt><tt><br>
</tt><tt> 0.00 68.74 us 16.11 us 61050.93
us 11412 FINODELK</tt><tt><br>
</tt><tt> 0.00 3793.44 us 68.61 us 411523.70
us 221 SETXATTR</tt><tt><br>
</tt><tt> 0.00 28669.99 us 227.28 us 138988.41
us 30 SYMLINK</tt><tt><br>
</tt><tt> 0.00 56.18 us 7.31 us 41484.37
us 22640 FLUSH</tt><tt><br>
</tt><tt> 0.00 105.64 us 19.34 us 351055.72
us 21693 STATFS</tt><tt><br>
</tt><tt> 0.00 23642.90 us 244.19 us 234912.25
us 178 MKDIR</tt><tt><br>
</tt><tt> 0.00 537.38 us 31.73 us 296911.67
us 9653 FSTAT</tt><tt><br>
</tt><tt> 0.00 214.87 us 25.71 us 1712020.99
us 39054 READ</tt><tt><br>
</tt><tt> 0.01 147.08 us 1.47 us 241611.80
us 69951 OPENDIR</tt><tt><br>
</tt><tt> 0.01 3899.84 us 117.22 us 503422.21
us 3180 RMDIR</tt><tt><br>
</tt><tt> 0.01 327.08 us 55.69 us 420376.80
us 38366 WRITE</tt><tt><br>
</tt><tt> 0.02 5580.36 us 210.11 us 1029702.30
us 6169 CREATE</tt><tt><br>
</tt><tt> 0.05 92.69 us 29.94 us 687955.86
us 1159936 READLINK</tt><tt><br>
</tt><tt> 0.40 543.50 us 36.45 us 2430653.54
us 1449307 OPEN</tt><tt><br>
</tt><tt> 0.48 87643.67 us 70.20 us 661999.51
us 10787 FXATTROP</tt><tt><br>
</tt><tt> 0.53 2342.72 us 20.91 us 4157542.43
us 447821 UNLINK</tt><tt><br>
</tt><tt> 0.53 558734.79 us 24.40 us 1399487.63
us 1889 LK</tt><tt><br>
</tt><tt> 0.78 145786.31 us 19.06 us 37490825.73
us 10687 READDIR</tt><tt><br>
</tt><tt> 0.99 2426.23 us 12.47 us 24176123.13
us 809430 GETXATTR</tt><tt><br>
</tt><tt> 1.45 676.83 us 9.30 us 193177067.25
us 4260021 ENTRYLK</tt><tt><br>
</tt><tt> 1.76 38325.38 us 21.44 us 46693575.93
us 91439 READDIRP</tt><tt><br>
</tt><tt> 4.44 1200.31 us 9.10 us 2559377.90
us 7352811 INODELK</tt><tt><br>
</tt><tt> 9.30 1849.24 us 9.28 us 35299050.00
us 9989221 LOOKUP</tt><tt><br>
</tt><tt> 79.24 47791.46 us 76.77 us 10488824.26
us 3294642 XATTROP</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 3194806 UPCALL</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 5311 CI_IATT</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 54 CI_XATTR</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 1907 CI_UNLINK</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 3189457 CI_FORGET</tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Duration: 209524 seconds</tt><tt><br>
</tt><tt> Data Read: 4775365531 bytes</tt><tt><br>
</tt><tt>Data Written: 2354108664 bytes</tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt>Interval 0 Stats:</tt><tt><br>
</tt><tt> Block Size: 1b+
8b+ 16b+ </tt><tt><br>
</tt><tt> No. of Reads: 0
0 0 </tt><tt><br>
</tt><tt>No. of Writes: 117
195 367 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 32b+
64b+ 128b+ </tt><tt><br>
</tt><tt> No. of Reads: 306
0 6 </tt><tt><br>
</tt><tt>No. of Writes: 1016
9 134 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 256b+
512b+ 1024b+ </tt><tt><br>
</tt><tt> No. of Reads: 107
58 485 </tt><tt><br>
</tt><tt>No. of Writes: 731
73 168 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 2048b+
4096b+ 8192b+ </tt><tt><br>
</tt><tt> No. of Reads: 43
59 383 </tt><tt><br>
</tt><tt>No. of Writes: 295
475 10562 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 16384b+
32768b+ 65536b+ </tt><tt><br>
</tt><tt> No. of Reads: 347
380 634 </tt><tt><br>
</tt><tt>No. of Writes: 1487
6369 3970 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 131072b+ </tt><tt><br>
</tt><tt> No. of Reads: 35704 </tt><tt><br>
</tt><tt>No. of Writes: 12398 </tt><tt><br>
</tt><tt> %-latency Avg-latency Min-Latency Max-Latency
No. of calls Fop</tt><tt><br>
</tt><tt> --------- ----------- ----------- -----------
------------ ----</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 957293 FORGET</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 1455091 RELEASE</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 69951 RELEASEDIR</tt><tt><br>
</tt><tt> 0.00 16.20 us 5.43 us 100.95
us 113 IPC</tt><tt><br>
</tt><tt> 0.00 687.33 us 80.53 us 15414.03
us 30 REMOVEXATTR</tt><tt><br>
</tt><tt> 0.00 840.76 us 72.71 us 37648.39
us 52 FTRUNCATE</tt><tt><br>
</tt><tt> 0.00 138.49 us 36.66 us 25280.24
us 1385 STAT</tt><tt><br>
</tt><tt> 0.00 363.20 us 63.45 us 15957.66
us 810 FGETXATTR</tt><tt><br>
</tt><tt> 0.00 2208.82 us 127.23 us 267092.17
us 137 LINK</tt><tt><br>
</tt><tt> 0.00 37595.33 us 1677.46 us 218313.32
us 12 FSYNC</tt><tt><br>
</tt><tt> 0.00 675.61 us 106.82 us 114351.01
us 698 TRUNCATE</tt><tt><br>
</tt><tt> 0.00 3995.66 us 147.00 us 109544.46
us 188 RENAME</tt><tt><br>
</tt><tt> 0.00 453.11 us 53.37 us 104848.94
us 1680 SETATTR</tt><tt><br>
</tt><tt> 0.00 6004.32 us 229.33 us 151322.58
us 127 MKNOD</tt><tt><br>
</tt><tt> 0.00 68.74 us 16.11 us 61050.93
us 11412 FINODELK</tt><tt><br>
</tt><tt> 0.00 3793.44 us 68.61 us 411523.70
us 221 SETXATTR</tt><tt><br>
</tt><tt> 0.00 28669.99 us 227.28 us 138988.41
us 30 SYMLINK</tt><tt><br>
</tt><tt> 0.00 56.19 us 7.31 us 41484.37
us 22640 FLUSH</tt><tt><br>
</tt><tt> 0.00 105.61 us 19.34 us 351055.72
us 21693 STATFS</tt><tt><br>
</tt><tt> 0.00 23642.90 us 244.19 us 234912.25
us 178 MKDIR</tt><tt><br>
</tt><tt> 0.00 537.38 us 31.73 us 296911.67
us 9653 FSTAT</tt><tt><br>
</tt><tt> 0.00 214.91 us 25.71 us 1712020.99
us 39054 READ</tt><tt><br>
</tt><tt> 0.01 147.08 us 1.47 us 241611.80
us 69951 OPENDIR</tt><tt><br>
</tt><tt> 0.01 3899.84 us 117.22 us 503422.21
us 3180 RMDIR</tt><tt><br>
</tt><tt> 0.01 327.13 us 55.69 us 420376.80
us 38366 WRITE</tt><tt><br>
</tt><tt> 0.02 5580.36 us 210.11 us 1029702.30
us 6169 CREATE</tt><tt><br>
</tt><tt> 0.05 92.69 us 29.94 us 687955.86
us 1159936 READLINK</tt><tt><br>
</tt><tt> 0.40 543.49 us 36.45 us 2430653.54
us 1449307 OPEN</tt><tt><br>
</tt><tt> 0.48 87643.67 us 70.20 us 661999.51
us 10787 FXATTROP</tt><tt><br>
</tt><tt> 0.53 2342.57 us 20.91 us 4157542.43
us 447821 UNLINK</tt><tt><br>
</tt><tt> 0.53 558416.10 us 24.40 us 1399487.63
us 1889 LK</tt><tt><br>
</tt><tt> 0.78 145786.31 us 19.06 us 37490825.73
us 10687 READDIR</tt><tt><br>
</tt><tt> 0.99 2426.24 us 12.47 us 24176123.13
us 809430 GETXATTR</tt><tt><br>
</tt><tt> 1.45 676.75 us 9.30 us 193177067.25
us 4260021 ENTRYLK</tt><tt><br>
</tt><tt> 1.76 38324.59 us 21.44 us 46693575.93
us 91439 READDIRP</tt><tt><br>
</tt><tt> 4.44 1200.37 us 9.10 us 2559377.90
us 7352811 INODELK</tt><tt><br>
</tt><tt> 9.29 1848.94 us 9.28 us 35299050.00
us 9989221 LOOKUP</tt><tt><br>
</tt><tt> 79.24 47792.42 us 76.77 us 10488824.26
us 3294642 XATTROP</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 3194806 UPCALL</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 5311 CI_IATT</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 54 CI_XATTR</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 1907 CI_UNLINK</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 3189457 CI_FORGET</tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Duration: 209524 seconds</tt><tt><br>
</tt><tt> Data Read: 4775365531 bytes</tt><tt><br>
</tt><tt>Data Written: 2354108664 bytes</tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt>Brick: zoidberg:/mnt/gluster/b</tt><tt><br>
</tt><tt>-----------------------------------------</tt><tt><br>
</tt><tt>Cumulative Stats:</tt><tt><br>
</tt><tt> Block Size: 1b+
2b+ 8b+ </tt><tt><br>
</tt><tt> No. of Reads: 0
0 0 </tt><tt><br>
</tt><tt>No. of Writes: 53
1 60 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 16b+
32b+ 64b+ </tt><tt><br>
</tt><tt> No. of Reads: 0
213 2 </tt><tt><br>
</tt><tt>No. of Writes: 14
1057 15 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 128b+
256b+ 512b+ </tt><tt><br>
</tt><tt> No. of Reads: 0
41 146 </tt><tt><br>
</tt><tt>No. of Writes: 59
715 80 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 1024b+
2048b+ 4096b+ </tt><tt><br>
</tt><tt> No. of Reads: 224
9 15 </tt><tt><br>
</tt><tt>No. of Writes: 163
282 404 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 8192b+
16384b+ 32768b+ </tt><tt><br>
</tt><tt> No. of Reads: 52
74 210 </tt><tt><br>
</tt><tt>No. of Writes: 7415
1430 6860 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 65536b+
131072b+ 262144b+ </tt><tt><br>
</tt><tt> No. of Reads: 299
15313 1 </tt><tt><br>
</tt><tt>No. of Writes: 2458
12378 1 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> %-latency Avg-latency Min-Latency Max-Latency
No. of calls Fop</tt><tt><br>
</tt><tt> --------- ----------- ----------- -----------
------------ ----</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 686733 FORGET</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 837434 RELEASE</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 87338 RELEASEDIR</tt><tt><br>
</tt><tt> 0.00 22.69 us 7.16 us 332.61
us 116 IPC</tt><tt><br>
</tt><tt> 0.00 119.93 us 70.56 us 272.28
us 57 FTRUNCATE</tt><tt><br>
</tt><tt> 0.00 152.25 us 86.82 us 340.63
us 65 FGETXATTR</tt><tt><br>
</tt><tt> 0.00 2336.06 us 90.05 us 20509.81
us 23 REMOVEXATTR</tt><tt><br>
</tt><tt> 0.00 631.26 us 123.17 us 51803.77
us 121 LINK</tt><tt><br>
</tt><tt> 0.00 369.27 us 22.64 us 145090.48
us 490 LK</tt><tt><br>
</tt><tt> 0.00 360.85 us 45.14 us 46846.98
us 540 STAT</tt><tt><br>
</tt><tt> 0.00 1190.96 us 117.40 us 69781.85
us 200 TRUNCATE</tt><tt><br>
</tt><tt> 0.00 245.72 us 54.91 us 123985.76
us 1482 SETATTR</tt><tt><br>
</tt><tt> 0.00 42139.10 us 3313.56 us 312273.21
us 12 FSYNC</tt><tt><br>
</tt><tt> 0.00 55.83 us 8.26 us 49649.35
us 10047 FLUSH</tt><tt><br>
</tt><tt> 0.00 57.52 us 14.62 us 820.59
us 9977 FINODELK</tt><tt><br>
</tt><tt> 0.00 3191.60 us 143.06 us 96214.75
us 188 RENAME</tt><tt><br>
</tt><tt> 0.00 32862.86 us 215.38 us 83851.13
us 25 SYMLINK</tt><tt><br>
</tt><tt> 0.00 11392.03 us 230.70 us 105471.90
us 118 MKNOD</tt><tt><br>
</tt><tt> 0.00 134.63 us 28.08 us 36326.43
us 12587 FSTAT</tt><tt><br>
</tt><tt> 0.00 112.32 us 19.91 us 381241.34
us 21695 STATFS</tt><tt><br>
</tt><tt> 0.00 307.86 us 28.00 us 292666.83
us 16841 READ</tt><tt><br>
</tt><tt> 0.00 29625.60 us 239.57 us 239532.26
us 178 MKDIR</tt><tt><br>
</tt><tt> 0.00 319.24 us 51.19 us 136353.39
us 33445 WRITE</tt><tt><br>
</tt><tt> 0.01 175.83 us 1.57 us 513635.22
us 87338 OPENDIR</tt><tt><br>
</tt><tt> 0.01 249.96 us 58.42 us 594774.86
us 85089 SETXATTR</tt><tt><br>
</tt><tt> 0.01 11001.25 us 119.87 us 612765.03
us 3180 RMDIR</tt><tt><br>
</tt><tt> 0.04 14240.66 us 203.89 us 1376758.99
us 6067 CREATE</tt><tt><br>
</tt><tt> 0.05 89.45 us 30.14 us 687961.60
us 1219542 READLINK</tt><tt><br>
</tt><tt> 0.39 1114.30 us 44.29 us 2412169.89
us 831371 OPEN</tt><tt><br>
</tt><tt> 0.51 127078.45 us 78.55 us 694364.09
us 9493 FXATTROP</tt><tt><br>
</tt><tt> 0.63 399.16 us 8.43 us 3815689.36
us 3764714 ENTRYLK</tt><tt><br>
</tt><tt> 0.86 91973.00 us 19.89 us 24258016.33
us 22253 READDIR</tt><tt><br>
</tt><tt> 1.06 6913.71 us 11.58 us 36968532.72
us 364963 GETXATTR</tt><tt><br>
</tt><tt> 1.29 62130.12 us 36.89 us 8649871.11
us 49472 READDIRP</tt><tt><br>
</tt><tt> 2.89 1477.53 us 11.10 us 2807637.64
us 4650500 INODELK</tt><tt><br>
</tt><tt> 4.12 23681.31 us 20.98 us 23291414.50
us 414352 UNLINK</tt><tt><br>
</tt><tt> 16.09 5000.31 us 9.04 us 23866016.88
us 7655621 LOOKUP</tt><tt><br>
</tt><tt> 72.04 68953.31 us 68.40 us 27542499.65
us 2486126 XATTROP</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 2433405 UPCALL</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 2453 CI_IATT</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 69 CI_XATTR</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 1544 CI_UNLINK</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 2430885 CI_FORGET</tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Duration: 209524 seconds</tt><tt><br>
</tt><tt> Data Read: 2051388524 bytes</tt><tt><br>
</tt><tt>Data Written: 2208261499 bytes</tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt>Interval 0 Stats:</tt><tt><br>
</tt><tt> Block Size: 1b+
2b+ 8b+ </tt><tt><br>
</tt><tt> No. of Reads: 0
0 0 </tt><tt><br>
</tt><tt>No. of Writes: 53
1 60 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 16b+
32b+ 64b+ </tt><tt><br>
</tt><tt> No. of Reads: 0
213 2 </tt><tt><br>
</tt><tt>No. of Writes: 14
1057 15 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 128b+
256b+ 512b+ </tt><tt><br>
</tt><tt> No. of Reads: 0
41 146 </tt><tt><br>
</tt><tt>No. of Writes: 59
715 80 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 1024b+
2048b+ 4096b+ </tt><tt><br>
</tt><tt> No. of Reads: 224
9 15 </tt><tt><br>
</tt><tt>No. of Writes: 163
282 404 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 8192b+
16384b+ 32768b+ </tt><tt><br>
</tt><tt> No. of Reads: 52
74 210 </tt><tt><br>
</tt><tt>No. of Writes: 7415
1430 6860 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 65536b+
131072b+ 262144b+ </tt><tt><br>
</tt><tt> No. of Reads: 299
15313 1 </tt><tt><br>
</tt><tt>No. of Writes: 2458
12378 1 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> %-latency Avg-latency Min-Latency Max-Latency
No. of calls Fop</tt><tt><br>
</tt><tt> --------- ----------- ----------- -----------
------------ ----</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 686733 FORGET</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 837434 RELEASE</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 87338 RELEASEDIR</tt><tt><br>
</tt><tt> 0.00 22.69 us 7.16 us 332.61
us 116 IPC</tt><tt><br>
</tt><tt> 0.00 119.93 us 70.56 us 272.28
us 57 FTRUNCATE</tt><tt><br>
</tt><tt> 0.00 152.25 us 86.82 us 340.63
us 65 FGETXATTR</tt><tt><br>
</tt><tt> 0.00 2336.06 us 90.05 us 20509.81
us 23 REMOVEXATTR</tt><tt><br>
</tt><tt> 0.00 631.26 us 123.17 us 51803.77
us 121 LINK</tt><tt><br>
</tt><tt> 0.00 369.27 us 22.64 us 145090.48
us 490 LK</tt><tt><br>
</tt><tt> 0.00 360.85 us 45.14 us 46846.98
us 540 STAT</tt><tt><br>
</tt><tt> 0.00 1190.96 us 117.40 us 69781.85
us 200 TRUNCATE</tt><tt><br>
</tt><tt> 0.00 245.72 us 54.91 us 123985.76
us 1482 SETATTR</tt><tt><br>
</tt><tt> 0.00 42139.10 us 3313.56 us 312273.21
us 12 FSYNC</tt><tt><br>
</tt><tt> 0.00 55.83 us 8.26 us 49649.35
us 10047 FLUSH</tt><tt><br>
</tt><tt> 0.00 57.52 us 14.62 us 820.59
us 9977 FINODELK</tt><tt><br>
</tt><tt> 0.00 3191.60 us 143.06 us 96214.75
us 188 RENAME</tt><tt><br>
</tt><tt> 0.00 32862.86 us 215.38 us 83851.13
us 25 SYMLINK</tt><tt><br>
</tt><tt> 0.00 11392.03 us 230.70 us 105471.90
us 118 MKNOD</tt><tt><br>
</tt><tt> 0.00 134.63 us 28.08 us 36326.43
us 12587 FSTAT</tt><tt><br>
</tt><tt> 0.00 112.32 us 19.91 us 381241.34
us 21695 STATFS</tt><tt><br>
</tt><tt> 0.00 307.91 us 28.00 us 292666.83
us 16841 READ</tt><tt><br>
</tt><tt> 0.00 29625.60 us 239.57 us 239532.26
us 178 MKDIR</tt><tt><br>
</tt><tt> 0.00 319.22 us 51.19 us 136353.39
us 33445 WRITE</tt><tt><br>
</tt><tt> 0.01 175.83 us 1.57 us 513635.22
us 87338 OPENDIR</tt><tt><br>
</tt><tt> 0.01 249.95 us 58.42 us 594774.86
us 85089 SETXATTR</tt><tt><br>
</tt><tt> 0.01 11001.25 us 119.87 us 612765.03
us 3180 RMDIR</tt><tt><br>
</tt><tt> 0.04 14240.66 us 203.89 us 1376758.99
us 6067 CREATE</tt><tt><br>
</tt><tt> 0.05 89.46 us 30.14 us 687961.60
us 1219542 READLINK</tt><tt><br>
</tt><tt> 0.39 1114.30 us 44.29 us 2412169.89
us 831371 OPEN</tt><tt><br>
</tt><tt> 0.51 127078.45 us 78.55 us 694364.09
us 9493 FXATTROP</tt><tt><br>
</tt><tt> 0.63 399.16 us 8.43 us 3815689.36
us 3764714 ENTRYLK</tt><tt><br>
</tt><tt> 0.86 91973.00 us 19.89 us 24258016.33
us 22253 READDIR</tt><tt><br>
</tt><tt> 1.06 6913.26 us 11.58 us 36968532.72
us 364963 GETXATTR</tt><tt><br>
</tt><tt> 1.29 62121.95 us 36.89 us 8649871.11
us 49472 READDIRP</tt><tt><br>
</tt><tt> 2.89 1477.58 us 11.10 us 2807637.64
us 4650500 INODELK</tt><tt><br>
</tt><tt> 4.12 23681.29 us 20.98 us 23291414.50
us 414352 UNLINK</tt><tt><br>
</tt><tt> 16.09 5000.02 us 9.04 us 23866016.88
us 7655621 LOOKUP</tt><tt><br>
</tt><tt> 72.04 68956.14 us 68.40 us 27542499.65
us 2486126 XATTROP</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 2433405 UPCALL</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 2453 CI_IATT</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 69 CI_XATTR</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 1544 CI_UNLINK</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 2430885 CI_FORGET</tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Duration: 209524 seconds</tt><tt><br>
</tt><tt> Data Read: 2051388524 bytes</tt><tt><br>
</tt><tt>Data Written: 2208261499 bytes</tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt>Brick: flexo:/mnt/gluster/b</tt><tt><br>
</tt><tt>--------------------------------------</tt><tt><br>
</tt><tt>Cumulative Stats:</tt><tt><br>
</tt><tt> Block Size: 1b+
8b+ 16b+ </tt><tt><br>
</tt><tt> No. of Reads: 0
0 0 </tt><tt><br>
</tt><tt>No. of Writes: 117
195 367 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 32b+
64b+ 128b+ </tt><tt><br>
</tt><tt> No. of Reads: 130
0 3 </tt><tt><br>
</tt><tt>No. of Writes: 1012
9 134 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 256b+
512b+ 1024b+ </tt><tt><br>
</tt><tt> No. of Reads: 47
8 257 </tt><tt><br>
</tt><tt>No. of Writes: 729
73 168 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 2048b+
4096b+ 8192b+ </tt><tt><br>
</tt><tt> No. of Reads: 10
27 240 </tt><tt><br>
</tt><tt>No. of Writes: 294
466 10472 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 16384b+
32768b+ 65536b+ </tt><tt><br>
</tt><tt> No. of Reads: 139
119 135 </tt><tt><br>
</tt><tt>No. of Writes: 1480
6273 3935 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 131072b+ </tt><tt><br>
</tt><tt> No. of Reads: 7892 </tt><tt><br>
</tt><tt>No. of Writes: 12327 </tt><tt><br>
</tt><tt> %-latency Avg-latency Min-Latency Max-Latency
No. of calls Fop</tt><tt><br>
</tt><tt> --------- ----------- ----------- -----------
------------ ----</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 955058 FORGET</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 1450749 RELEASE</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 81377 RELEASEDIR</tt><tt><br>
</tt><tt> 0.00 15.55 us 5.35 us 31.88
us 108 IPC</tt><tt><br>
</tt><tt> 0.00 125.85 us 59.77 us 224.15
us 44 FTRUNCATE</tt><tt><br>
</tt><tt> 0.00 198.38 us 81.68 us 1332.45
us 30 REMOVEXATTR</tt><tt><br>
</tt><tt> 0.00 4254.16 us 621.52 us 10602.99
us 4 FSYNC</tt><tt><br>
</tt><tt> 0.00 795.08 us 190.30 us 16518.64
us 30 SYMLINK</tt><tt><br>
</tt><tt> 0.00 195.90 us 111.02 us 953.39
us 137 LINK</tt><tt><br>
</tt><tt> 0.00 286.07 us 179.64 us 437.82
us 119 MKNOD</tt><tt><br>
</tt><tt> 0.00 162.61 us 65.62 us 3864.77
us 221 SETXATTR</tt><tt><br>
</tt><tt> 0.00 117.29 us 48.50 us 317.17
us 354 STAT</tt><tt><br>
</tt><tt> 0.00 70.35 us 23.10 us 317.32
us 777 LK</tt><tt><br>
</tt><tt> 0.00 461.72 us 114.54 us 16013.81
us 188 RENAME</tt><tt><br>
</tt><tt> 0.00 150.40 us 75.59 us 749.54
us 810 FGETXATTR</tt><tt><br>
</tt><tt> 0.00 267.80 us 99.67 us 994.58
us 698 TRUNCATE</tt><tt><br>
</tt><tt> 0.00 1176.00 us 194.89 us 78261.12
us 178 MKDIR</tt><tt><br>
</tt><tt> 0.00 439.83 us 54.38 us 75270.51
us 1665 SETATTR</tt><tt><br>
</tt><tt> 0.00 65.89 us 17.39 us 354.77
us 11372 FINODELK</tt><tt><br>
</tt><tt> 0.00 131.57 us 32.68 us 2107.52
us 7398 FSTAT</tt><tt><br>
</tt><tt> 0.00 120.71 us 31.43 us 847.97
us 9094 READ</tt><tt><br>
</tt><tt> 0.00 60.75 us 7.25 us 22520.98
us 22614 FLUSH</tt><tt><br>
</tt><tt> 0.00 73.03 us 22.41 us 750.17
us 21656 STATFS</tt><tt><br>
</tt><tt> 0.00 323.63 us 166.56 us 23034.44
us 6145 CREATE</tt><tt><br>
</tt><tt> 0.00 217.55 us 50.60 us 30734.99
us 38051 WRITE</tt><tt><br>
</tt><tt> 0.01 146.39 us 1.80 us 318612.45
us 81377 OPENDIR</tt><tt><br>
</tt><tt> 0.01 5641.37 us 108.84 us 468960.33
us 3177 RMDIR</tt><tt><br>
</tt><tt> 0.03 100.05 us 30.93 us 49756.93
us 432439 READLINK</tt><tt><br>
</tt><tt> 0.13 483.47 us 16.81 us 2608013.14
us 448381 UNLINK</tt><tt><br>
</tt><tt> 0.21 7101.56 us 22.26 us 2554390.49
us 52185 READDIRP</tt><tt><br>
</tt><tt> 0.35 414.05 us 36.58 us 1038229.93
us 1444606 OPEN</tt><tt><br>
</tt><tt> 0.49 78518.02 us 73.39 us 373999.31
us 10745 FXATTROP</tt><tt><br>
</tt><tt> 0.72 776.21 us 11.24 us 4271792.57
us 1605822 GETXATTR</tt><tt><br>
</tt><tt> 0.77 97829.38 us 18.84 us 9477446.11
us 13532 READDIR</tt><tt><br>
</tt><tt> 0.79 320.44 us 7.71 us 2369511.06
us 4257297 ENTRYLK</tt><tt><br>
</tt><tt> 1.97 464.02 us 9.38 us 1038460.80
us 7333722 INODELK</tt><tt><br>
</tt><tt> 13.73 2372.50 us 8.33 us 6258159.87
us 9979799 LOOKUP</tt><tt><br>
</tt><tt> 80.79 42372.04 us 61.49 us 2238308.13
us 3289103 XATTROP</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 2250268 UPCALL</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 3756 CI_IATT</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 167 CI_XATTR</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 1908 CI_UNLINK</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 2246352 CI_FORGET</tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Duration: 209334 seconds</tt><tt><br>
</tt><tt> Data Read: 1059777395 bytes</tt><tt><br>
</tt><tt>Data Written: 2336921014 bytes</tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt>Interval 0 Stats:</tt><tt><br>
</tt><tt> Block Size: 1b+
8b+ 16b+ </tt><tt><br>
</tt><tt> No. of Reads: 0
0 0 </tt><tt><br>
</tt><tt>No. of Writes: 117
195 367 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 32b+
64b+ 128b+ </tt><tt><br>
</tt><tt> No. of Reads: 130
0 3 </tt><tt><br>
</tt><tt>No. of Writes: 1012
9 134 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 256b+
512b+ 1024b+ </tt><tt><br>
</tt><tt> No. of Reads: 47
8 257 </tt><tt><br>
</tt><tt>No. of Writes: 729
73 168 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 2048b+
4096b+ 8192b+ </tt><tt><br>
</tt><tt> No. of Reads: 10
27 240 </tt><tt><br>
</tt><tt>No. of Writes: 294
466 10472 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 16384b+
32768b+ 65536b+ </tt><tt><br>
</tt><tt> No. of Reads: 139
119 135 </tt><tt><br>
</tt><tt>No. of Writes: 1480
6273 3935 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 131072b+ </tt><tt><br>
</tt><tt> No. of Reads: 7892 </tt><tt><br>
</tt><tt>No. of Writes: 12327 </tt><tt><br>
</tt><tt> %-latency Avg-latency Min-Latency Max-Latency
No. of calls Fop</tt><tt><br>
</tt><tt> --------- ----------- ----------- -----------
------------ ----</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 955058 FORGET</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 1450749 RELEASE</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 81377 RELEASEDIR</tt><tt><br>
</tt><tt> 0.00 15.55 us 5.35 us 31.88
us 108 IPC</tt><tt><br>
</tt><tt> 0.00 125.85 us 59.77 us 224.15
us 44 FTRUNCATE</tt><tt><br>
</tt><tt> 0.00 198.38 us 81.68 us 1332.45
us 30 REMOVEXATTR</tt><tt><br>
</tt><tt> 0.00 4254.16 us 621.52 us 10602.99
us 4 FSYNC</tt><tt><br>
</tt><tt> 0.00 795.08 us 190.30 us 16518.64
us 30 SYMLINK</tt><tt><br>
</tt><tt> 0.00 195.90 us 111.02 us 953.39
us 137 LINK</tt><tt><br>
</tt><tt> 0.00 286.07 us 179.64 us 437.82
us 119 MKNOD</tt><tt><br>
</tt><tt> 0.00 162.61 us 65.62 us 3864.77
us 221 SETXATTR</tt><tt><br>
</tt><tt> 0.00 117.29 us 48.50 us 317.17
us 354 STAT</tt><tt><br>
</tt><tt> 0.00 70.35 us 23.10 us 317.32
us 777 LK</tt><tt><br>
</tt><tt> 0.00 461.72 us 114.54 us 16013.81
us 188 RENAME</tt><tt><br>
</tt><tt> 0.00 150.43 us 75.59 us 749.54
us 810 FGETXATTR</tt><tt><br>
</tt><tt> 0.00 267.80 us 99.67 us 994.58
us 698 TRUNCATE</tt><tt><br>
</tt><tt> 0.00 1176.00 us 194.89 us 78261.12
us 178 MKDIR</tt><tt><br>
</tt><tt> 0.00 439.83 us 54.38 us 75270.51
us 1665 SETATTR</tt><tt><br>
</tt><tt> 0.00 65.89 us 17.39 us 354.77
us 11372 FINODELK</tt><tt><br>
</tt><tt> 0.00 131.57 us 32.68 us 2107.52
us 7398 FSTAT</tt><tt><br>
</tt><tt> 0.00 120.74 us 31.43 us 847.97
us 9094 READ</tt><tt><br>
</tt><tt> 0.00 60.75 us 7.25 us 22520.98
us 22614 FLUSH</tt><tt><br>
</tt><tt> 0.00 73.03 us 22.41 us 750.17
us 21656 STATFS</tt><tt><br>
</tt><tt> 0.00 323.63 us 166.56 us 23034.44
us 6145 CREATE</tt><tt><br>
</tt><tt> 0.00 217.50 us 50.60 us 30734.99
us 38051 WRITE</tt><tt><br>
</tt><tt> 0.01 146.39 us 1.80 us 318612.45
us 81377 OPENDIR</tt><tt><br>
</tt><tt> 0.01 5641.37 us 108.84 us 468960.33
us 3177 RMDIR</tt><tt><br>
</tt><tt> 0.03 100.06 us 30.93 us 49756.93
us 432439 READLINK</tt><tt><br>
</tt><tt> 0.13 483.47 us 16.81 us 2608013.14
us 448381 UNLINK</tt><tt><br>
</tt><tt> 0.21 7101.56 us 22.26 us 2554390.49
us 52185 READDIRP</tt><tt><br>
</tt><tt> 0.35 414.01 us 36.58 us 1038229.93
us 1444606 OPEN</tt><tt><br>
</tt><tt> 0.49 78518.02 us 73.39 us 373999.31
us 10745 FXATTROP</tt><tt><br>
</tt><tt> 0.72 776.24 us 11.24 us 4271792.57
us 1605822 GETXATTR</tt><tt><br>
</tt><tt> 0.77 97829.38 us 18.84 us 9477446.11
us 13532 READDIR</tt><tt><br>
</tt><tt> 0.79 320.45 us 7.71 us 2369511.06
us 4257297 ENTRYLK</tt><tt><br>
</tt><tt> 1.97 464.07 us 9.38 us 1038460.80
us 7333722 INODELK</tt><tt><br>
</tt><tt> 13.72 2372.42 us 8.33 us 6258159.87
us 9979799 LOOKUP</tt><tt><br>
</tt><tt> 80.79 42373.50 us 61.49 us 2238308.13
us 3289103 XATTROP</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 2250268 UPCALL</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 3756 CI_IATT</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 167 CI_XATTR</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 1908 CI_UNLINK</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 2246352 CI_FORGET</tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Duration: 209334 seconds</tt><tt><br>
</tt><tt> Data Read: 1059777395 bytes</tt><tt><br>
</tt><tt>Data Written: 2336921014 bytes</tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt>Brick: flexo:/mnt/gluster/c</tt><tt><br>
</tt><tt>--------------------------------------</tt><tt><br>
</tt><tt>Cumulative Stats:</tt><tt><br>
</tt><tt> Block Size: 1b+ </tt><tt><br>
</tt><tt> No. of Reads: 0 </tt><tt><br>
</tt><tt>No. of Writes: 32843 </tt><tt><br>
</tt><tt> %-latency Avg-latency Min-Latency Max-Latency
No. of calls Fop</tt><tt><br>
</tt><tt> --------- ----------- ----------- -----------
------------ ----</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 686562 FORGET</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 834596 RELEASE</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 93757 RELEASEDIR</tt><tt><br>
</tt><tt> 0.00 18.50 us 8.21 us 50.20
us 107 IPC</tt><tt><br>
</tt><tt> 0.00 205.83 us 126.30 us 412.06
us 36 LINK</tt><tt><br>
</tt><tt> 0.00 167.33 us 75.01 us 272.47
us 65 FGETXATTR</tt><tt><br>
</tt><tt> 0.00 203.58 us 117.35 us 648.28
us 88 RENAME</tt><tt><br>
</tt><tt> 0.00 203.20 us 66.60 us 809.96
us 153 TRUNCATE</tt><tt><br>
</tt><tt> 0.00 67.06 us 23.64 us 555.59
us 491 LK</tt><tt><br>
</tt><tt> 0.01 51.73 us 7.70 us 1174.64
us 9339 FLUSH</tt><tt><br>
</tt><tt> 0.01 62.41 us 15.13 us 636.35
us 9011 FINODELK</tt><tt><br>
</tt><tt> 0.03 126.23 us 43.15 us 4760.14
us 10824 FSTAT</tt><tt><br>
</tt><tt> 0.03 571.94 us 94.85 us 51697.03
us 2594 MKDIR</tt><tt><br>
</tt><tt> 0.05 264.87 us 81.58 us 53152.06
us 9009 FXATTROP</tt><tt><br>
</tt><tt> 0.05 423.05 us 38.26 us 548686.55
us 5754 CREATE</tt><tt><br>
</tt><tt> 0.05 78.14 us 19.17 us 678.32
us 32843 WRITE</tt><tt><br>
</tt><tt> 0.06 526.39 us 130.84 us 41405.46
us 5798 SYMLINK</tt><tt><br>
</tt><tt> 0.10 133.73 us 16.25 us 676494.72
us 34904 READDIR</tt><tt><br>
</tt><tt> 0.12 2054.48 us 99.28 us 88061.83
us 2732 RMDIR</tt><tt><br>
</tt><tt> 0.21 110.60 us 1.19 us 16833.78
us 93757 OPENDIR</tt><tt><br>
</tt><tt> 0.72 205.89 us 33.28 us 261023.99
us 170292 UNLINK</tt><tt><br>
</tt><tt> 0.85 2971.45 us 21.24 us 257839.19
us 13840 READDIRP</tt><tt><br>
</tt><tt> 1.94 200.88 us 25.49 us 461654.32
us 468842 FTRUNCATE</tt><tt><br>
</tt><tt> 2.13 212.44 us 54.02 us 461878.34
us 486882 REMOVEXATTR</tt><tt><br>
</tt><tt> 2.27 223.01 us 52.03 us 178897.40
us 493240 SETXATTR</tt><tt><br>
</tt><tt> 3.53 155.31 us 7.96 us 67873.00
us 1101338 GETXATTR</tt><tt><br>
</tt><tt> 3.63 212.32 us 44.42 us 551264.02
us 829002 OPEN</tt><tt><br>
</tt><tt> 5.77 76.36 us 7.76 us 962634.73
us 3660425 ENTRYLK</tt><tt><br>
</tt><tt> 6.23 304.93 us 42.51 us 661394.43
us 990644 SETATTR</tt><tt><br>
</tt><tt> 7.81 382.20 us 107.42 us 846502.78
us 990766 XATTROP</tt><tt><br>
</tt><tt> 12.39 549.11 us 135.57 us 1659697.83
us 1093387 MKNOD</tt><tt><br>
</tt><tt> 12.47 139.78 us 9.76 us 552067.56
us 4325281 INODELK</tt><tt><br>
</tt><tt> 39.53 206.94 us 8.56 us 964823.33
us 9259429 LOOKUP</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 1856377 UPCALL</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 933300 CI_IATT</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 251 CI_XATTR</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 293 CI_UNLINK</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 923077 CI_FORGET</tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Duration: 209334 seconds</tt><tt><br>
</tt><tt> Data Read: 0 bytes</tt><tt><br>
</tt><tt>Data Written: 32843 bytes</tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt>Interval 0 Stats:</tt><tt><br>
</tt><tt> Block Size: 1b+ </tt><tt><br>
</tt><tt> No. of Reads: 0 </tt><tt><br>
</tt><tt>No. of Writes: 32843 </tt><tt><br>
</tt><tt> %-latency Avg-latency Min-Latency Max-Latency
No. of calls Fop</tt><tt><br>
</tt><tt> --------- ----------- ----------- -----------
------------ ----</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 686562 FORGET</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 834596 RELEASE</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 93757 RELEASEDIR</tt><tt><br>
</tt><tt> 0.00 18.50 us 8.21 us 50.20
us 107 IPC</tt><tt><br>
</tt><tt> 0.00 205.83 us 126.30 us 412.06
us 36 LINK</tt><tt><br>
</tt><tt> 0.00 167.33 us 75.01 us 272.47
us 65 FGETXATTR</tt><tt><br>
</tt><tt> 0.00 203.58 us 117.35 us 648.28
us 88 RENAME</tt><tt><br>
</tt><tt> 0.00 203.20 us 66.60 us 809.96
us 153 TRUNCATE</tt><tt><br>
</tt><tt> 0.00 67.06 us 23.64 us 555.59
us 491 LK</tt><tt><br>
</tt><tt> 0.01 51.71 us 7.70 us 1174.64
us 9339 FLUSH</tt><tt><br>
</tt><tt> 0.01 62.41 us 15.13 us 636.35
us 9011 FINODELK</tt><tt><br>
</tt><tt> 0.03 126.23 us 43.15 us 4760.14
us 10824 FSTAT</tt><tt><br>
</tt><tt> 0.03 571.94 us 94.85 us 51697.03
us 2594 MKDIR</tt><tt><br>
</tt><tt> 0.05 264.87 us 81.58 us 53152.06
us 9009 FXATTROP</tt><tt><br>
</tt><tt> 0.05 423.05 us 38.26 us 548686.55
us 5754 CREATE</tt><tt><br>
</tt><tt> 0.05 78.14 us 19.17 us 678.32
us 32843 WRITE</tt><tt><br>
</tt><tt> 0.06 526.39 us 130.84 us 41405.46
us 5798 SYMLINK</tt><tt><br>
</tt><tt> 0.10 133.73 us 16.25 us 676494.72
us 34904 READDIR</tt><tt><br>
</tt><tt> 0.12 2054.48 us 99.28 us 88061.83
us 2732 RMDIR</tt><tt><br>
</tt><tt> 0.21 110.60 us 1.19 us 16833.78
us 93757 OPENDIR</tt><tt><br>
</tt><tt> 0.72 205.89 us 33.28 us 261023.99
us 170292 UNLINK</tt><tt><br>
</tt><tt> 0.85 2971.45 us 21.24 us 257839.19
us 13840 READDIRP</tt><tt><br>
</tt><tt> 1.94 200.87 us 25.49 us 461654.32
us 468842 FTRUNCATE</tt><tt><br>
</tt><tt> 2.13 212.44 us 54.02 us 461878.34
us 486882 REMOVEXATTR</tt><tt><br>
</tt><tt> 2.27 223.00 us 52.03 us 178897.40
us 493240 SETXATTR</tt><tt><br>
</tt><tt> 3.53 155.32 us 7.96 us 67873.00
us 1101338 GETXATTR</tt><tt><br>
</tt><tt> 3.63 212.32 us 44.42 us 551264.02
us 829002 OPEN</tt><tt><br>
</tt><tt> 5.77 76.36 us 7.76 us 962634.73
us 3660425 ENTRYLK</tt><tt><br>
</tt><tt> 6.23 304.88 us 42.51 us 661394.43
us 990644 SETATTR</tt><tt><br>
</tt><tt> 7.81 382.20 us 107.42 us 846502.78
us 990766 XATTROP</tt><tt><br>
</tt><tt> 12.39 549.25 us 135.57 us 1659697.83
us 1093387 MKNOD</tt><tt><br>
</tt><tt> 12.47 139.78 us 9.76 us 552067.56
us 4325281 INODELK</tt><tt><br>
</tt><tt> 39.53 206.95 us 8.56 us 964823.33
us 9259429 LOOKUP</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 1856377 UPCALL</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 933300 CI_IATT</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 251 CI_XATTR</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 293 CI_UNLINK</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 923077 CI_FORGET</tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Duration: 209334 seconds</tt><tt><br>
</tt><tt> Data Read: 0 bytes</tt><tt><br>
</tt><tt>Data Written: 32843 bytes</tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt>Brick: bender:/mnt/gluster/c</tt><tt><br>
</tt><tt>---------------------------------------</tt><tt><br>
</tt><tt>Cumulative Stats:</tt><tt><br>
</tt><tt> Block Size: 1b+ </tt><tt><br>
</tt><tt> No. of Reads: 0 </tt><tt><br>
</tt><tt>No. of Writes: 37050 </tt><tt><br>
</tt><tt> %-latency Avg-latency Min-Latency Max-Latency
No. of calls Fop</tt><tt><br>
</tt><tt> --------- ----------- ----------- -----------
------------ ----</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 928119 FORGET</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 1401524 RELEASE</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 70092 RELEASEDIR</tt><tt><br>
</tt><tt> 0.00 19.49 us 8.28 us 180.61
us 100 IPC</tt><tt><br>
</tt><tt> 0.00 238.88 us 163.97 us 486.93
us 38 LINK</tt><tt><br>
</tt><tt> 0.00 306.43 us 196.67 us 788.36
us 30 SYMLINK</tt><tt><br>
</tt><tt> 0.00 2670.18 us 765.93 us 4151.64
us 4 FSYNC</tt><tt><br>
</tt><tt> 0.00 350.65 us 141.92 us 10573.64
us 122 RENAME</tt><tt><br>
</tt><tt> 0.00 67.02 us 20.66 us 453.90
us 777 LK</tt><tt><br>
</tt><tt> 0.00 158.32 us 64.22 us 532.21
us 810 FGETXATTR</tt><tt><br>
</tt><tt> 0.00 229.87 us 66.47 us 826.31
us 603 TRUNCATE</tt><tt><br>
</tt><tt> 0.01 1742.85 us 202.81 us 47471.71
us 302 MKDIR</tt><tt><br>
</tt><tt> 0.01 59.05 us 15.40 us 1813.56
us 10060 FINODELK</tt><tt><br>
</tt><tt> 0.01 118.54 us 36.65 us 4002.58
us 6409 FSTAT</tt><tt><br>
</tt><tt> 0.01 49.38 us 8.08 us 1264.41
us 21719 FLUSH</tt><tt><br>
</tt><tt> 0.03 74.76 us 20.86 us 2008.41
us 37050 WRITE</tt><tt><br>
</tt><tt> 0.04 553.12 us 40.81 us 386485.29
us 5901 CREATE</tt><tt><br>
</tt><tt> 0.06 534.01 us 100.86 us 527770.44
us 10057 FXATTROP</tt><tt><br>
</tt><tt> 0.12 138.96 us 1.00 us 601208.70
us 70092 OPENDIR</tt><tt><br>
</tt><tt> 0.12 947.76 us 18.54 us 712568.33
us 10779 READDIR</tt><tt><br>
</tt><tt> 0.15 4099.07 us 124.69 us 421939.47
us 3008 RMDIR</tt><tt><br>
</tt><tt> 0.61 465.30 us 20.67 us 936959.86
us 109478 UNLINK</tt><tt><br>
</tt><tt> 1.57 182.41 us 49.54 us 73938.09
us 722669 REMOVEXATTR</tt><tt><br>
</tt><tt> 1.61 171.41 us 6.98 us 83809.88
us 789294 GETXATTR</tt><tt><br>
</tt><tt> 1.86 217.88 us 26.40 us 556446.19
us 718835 FTRUNCATE</tt><tt><br>
</tt><tt> 1.89 219.47 us 49.92 us 506767.78
us 722856 SETXATTR</tt><tt><br>
</tt><tt> 3.02 181.66 us 44.40 us 344656.69
us 1395957 OPEN</tt><tt><br>
</tt><tt> 5.56 322.08 us 43.37 us 634069.61
us 1450160 SETATTR</tt><tt><br>
</tt><tt> 5.84 115.88 us 7.67 us 747387.51
us 4233520 ENTRYLK</tt><tt><br>
</tt><tt> 6.53 378.33 us 122.84 us 766939.14
us 1449601 XATTROP</tt><tt><br>
</tt><tt> 9.75 115.48 us 10.70 us 556675.13
us 7097226 INODELK</tt><tt><br>
</tt><tt> 19.37 1104.98 us 101.28 us 1612346.75
us 1472813 MKNOD</tt><tt><br>
</tt><tt> 41.85 295.61 us 8.98 us 996805.54
us 11896091 LOOKUP</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 3531139 UPCALL</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 1862470 CI_IATT</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 244 CI_XATTR</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 632 CI_UNLINK</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 1668669 CI_FORGET</tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Duration: 208643 seconds</tt><tt><br>
</tt><tt> Data Read: 0 bytes</tt><tt><br>
</tt><tt>Data Written: 37050 bytes</tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt>Interval 0 Stats:</tt><tt><br>
</tt><tt> Block Size: 1b+ </tt><tt><br>
</tt><tt> No. of Reads: 0 </tt><tt><br>
</tt><tt>No. of Writes: 37050 </tt><tt><br>
</tt><tt> %-latency Avg-latency Min-Latency Max-Latency
No. of calls Fop</tt><tt><br>
</tt><tt> --------- ----------- ----------- -----------
------------ ----</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 928119 FORGET</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 1401524 RELEASE</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 70092 RELEASEDIR</tt><tt><br>
</tt><tt> 0.00 19.49 us 8.28 us 180.61
us 100 IPC</tt><tt><br>
</tt><tt> 0.00 238.88 us 163.97 us 486.93
us 38 LINK</tt><tt><br>
</tt><tt> 0.00 306.43 us 196.67 us 788.36
us 30 SYMLINK</tt><tt><br>
</tt><tt> 0.00 2670.18 us 765.93 us 4151.64
us 4 FSYNC</tt><tt><br>
</tt><tt> 0.00 350.65 us 141.92 us 10573.64
us 122 RENAME</tt><tt><br>
</tt><tt> 0.00 67.02 us 20.66 us 453.90
us 777 LK</tt><tt><br>
</tt><tt> 0.00 158.32 us 64.22 us 532.21
us 810 FGETXATTR</tt><tt><br>
</tt><tt> 0.00 229.87 us 66.47 us 826.31
us 603 TRUNCATE</tt><tt><br>
</tt><tt> 0.01 1742.85 us 202.81 us 47471.71
us 302 MKDIR</tt><tt><br>
</tt><tt> 0.01 59.05 us 15.40 us 1813.56
us 10060 FINODELK</tt><tt><br>
</tt><tt> 0.01 118.54 us 36.65 us 4002.58
us 6409 FSTAT</tt><tt><br>
</tt><tt> 0.01 49.39 us 8.08 us 1264.41
us 21719 FLUSH</tt><tt><br>
</tt><tt> 0.03 74.77 us 20.86 us 2008.41
us 37050 WRITE</tt><tt><br>
</tt><tt> 0.04 553.12 us 40.81 us 386485.29
us 5901 CREATE</tt><tt><br>
</tt><tt> 0.06 534.01 us 100.86 us 527770.44
us 10057 FXATTROP</tt><tt><br>
</tt><tt> 0.12 138.96 us 1.00 us 601208.70
us 70092 OPENDIR</tt><tt><br>
</tt><tt> 0.12 947.76 us 18.54 us 712568.33
us 10779 READDIR</tt><tt><br>
</tt><tt> 0.15 4099.07 us 124.69 us 421939.47
us 3008 RMDIR</tt><tt><br>
</tt><tt> 0.61 465.30 us 20.67 us 936959.86
us 109478 UNLINK</tt><tt><br>
</tt><tt> 1.57 182.41 us 49.54 us 73938.09
us 722669 REMOVEXATTR</tt><tt><br>
</tt><tt> 1.61 171.41 us 6.98 us 83809.88
us 789294 GETXATTR</tt><tt><br>
</tt><tt> 1.86 217.88 us 26.40 us 556446.19
us 718835 FTRUNCATE</tt><tt><br>
</tt><tt> 1.89 219.48 us 49.92 us 506767.78
us 722856 SETXATTR</tt><tt><br>
</tt><tt> 3.02 181.66 us 44.40 us 344656.69
us 1395957 OPEN</tt><tt><br>
</tt><tt> 5.56 322.09 us 43.37 us 634069.61
us 1450160 SETATTR</tt><tt><br>
</tt><tt> 5.84 115.88 us 7.67 us 747387.51
us 4233520 ENTRYLK</tt><tt><br>
</tt><tt> 6.53 378.33 us 122.84 us 766939.14
us 1449601 XATTROP</tt><tt><br>
</tt><tt> 9.75 115.48 us 10.70 us 556675.13
us 7097226 INODELK</tt><tt><br>
</tt><tt> 19.37 1105.06 us 101.28 us 1612346.75
us 1472813 MKNOD</tt><tt><br>
</tt><tt> 41.84 295.54 us 8.98 us 996805.54
us 11896091 LOOKUP</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 3531139 UPCALL</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 1862470 CI_IATT</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 244 CI_XATTR</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 632 CI_UNLINK</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 1668669 CI_FORGET</tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Duration: 208643 seconds</tt><tt><br>
</tt><tt> Data Read: 0 bytes</tt><tt><br>
</tt><tt>Data Written: 37050 bytes</tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt>Brick: bender:/mnt/gluster/a</tt><tt><br>
</tt><tt>---------------------------------------</tt><tt><br>
</tt><tt>Cumulative Stats:</tt><tt><br>
</tt><tt> Block Size: 1b+
2b+ 8b+ </tt><tt><br>
</tt><tt> No. of Reads: 0
0 0 </tt><tt><br>
</tt><tt>No. of Writes: 53
1 60 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 16b+
32b+ 64b+ </tt><tt><br>
</tt><tt> No. of Reads: 0
235 4 </tt><tt><br>
</tt><tt>No. of Writes: 14
1033 15 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 128b+
256b+ 512b+ </tt><tt><br>
</tt><tt> No. of Reads: 4
115 81 </tt><tt><br>
</tt><tt>No. of Writes: 58
703 80 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 1024b+
2048b+ 4096b+ </tt><tt><br>
</tt><tt> No. of Reads: 181
26 74 </tt><tt><br>
</tt><tt>No. of Writes: 163
278 397 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 8192b+
16384b+ 32768b+ </tt><tt><br>
</tt><tt> No. of Reads: 131
161 267 </tt><tt><br>
</tt><tt>No. of Writes: 7264
1398 6742 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 65536b+
131072b+ 262144b+ </tt><tt><br>
</tt><tt> No. of Reads: 443
25528 0 </tt><tt><br>
</tt><tt>No. of Writes: 2426
12240 1 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> %-latency Avg-latency Min-Latency Max-Latency
No. of calls Fop</tt><tt><br>
</tt><tt> --------- ----------- ----------- -----------
------------ ----</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 684942 FORGET</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 837092 RELEASE</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 87297 RELEASEDIR</tt><tt><br>
</tt><tt> 0.00 219.22 us 121.08 us 306.12
us 8 REMOVEXATTR</tt><tt><br>
</tt><tt> 0.00 21.70 us 7.24 us 196.07
us 97 IPC</tt><tt><br>
</tt><tt> 0.00 152.77 us 78.90 us 1059.82
us 46 FTRUNCATE</tt><tt><br>
</tt><tt> 0.00 295.73 us 167.10 us 606.96
us 65 FGETXATTR</tt><tt><br>
</tt><tt> 0.00 243.66 us 140.50 us 3392.16
us 121 LINK</tt><tt><br>
</tt><tt> 0.00 1216.47 us 194.37 us 23411.42
us 25 SYMLINK</tt><tt><br>
</tt><tt> 0.00 327.87 us 234.27 us 551.71
us 107 MKNOD</tt><tt><br>
</tt><tt> 0.00 36094.61 us 36094.61 us 36094.61
us 1 FSYNC</tt><tt><br>
</tt><tt> 0.00 219.63 us 147.34 us 783.10
us 188 RENAME</tt><tt><br>
</tt><tt> 0.00 104.27 us 40.34 us 595.29
us 745 STAT</tt><tt><br>
</tt><tt> 0.00 531.71 us 113.57 us 53973.08
us 200 TRUNCATE</tt><tt><br>
</tt><tt> 0.00 289.05 us 63.10 us 70107.41
us 1444 SETATTR</tt><tt><br>
</tt><tt> 0.00 2430.15 us 232.78 us 68120.54
us 177 MKDIR</tt><tt><br>
</tt><tt> 0.00 52.68 us 7.28 us 826.01
us 9890 FLUSH</tt><tt><br>
</tt><tt> 0.00 60.96 us 17.54 us 1393.95
us 9792 FINODELK</tt><tt><br>
</tt><tt> 0.00 74.26 us 20.00 us 1522.39
us 21521 STATFS</tt><tt><br>
</tt><tt> 0.00 129.21 us 34.15 us 9068.67
us 13094 FSTAT</tt><tt><br>
</tt><tt> 0.00 456.99 us 195.50 us 130182.02
us 5967 CREATE</tt><tt><br>
</tt><tt> 0.00 112.06 us 28.14 us 9545.53
us 27474 READ</tt><tt><br>
</tt><tt> 0.01 313.76 us 52.98 us 71635.74
us 32926 WRITE</tt><tt><br>
</tt><tt> 0.01 135.84 us 1.52 us 671373.08
us 87297 OPENDIR</tt><tt><br>
</tt><tt> 0.01 5225.28 us 131.28 us 951323.69
us 3177 RMDIR</tt><tt><br>
</tt><tt> 0.01 203.34 us 56.13 us 486756.90
us 85073 SETXATTR</tt><tt><br>
</tt><tt> 0.03 3521.00 us 145.28 us 3396306.00
us 15090 READDIRP</tt><tt><br>
</tt><tt> 0.04 113758.50 us 26.72 us 1147074.44
us 564 LK</tt><tt><br>
</tt><tt> 0.05 92.18 us 29.71 us 39069.99
us 850938 READLINK</tt><tt><br>
</tt><tt> 0.23 453.79 us 41.84 us 655999.23
us 831212 OPEN</tt><tt><br>
</tt><tt> 0.41 29425.34 us 19.20 us 1526187.08
us 22482 READDIR</tt><tt><br>
</tt><tt> 0.50 1954.70 us 21.51 us 1526772.28
us 415226 UNLINK</tt><tt><br>
</tt><tt> 0.54 1272.83 us 9.66 us 1604368.93
us 686945 GETXATTR</tt><tt><br>
</tt><tt> 0.55 237.57 us 7.91 us 1277047.91
us 3739718 ENTRYLK</tt><tt><br>
</tt><tt> 0.60 103888.49 us 89.32 us 761509.35
us 9311 FXATTROP</tt><tt><br>
</tt><tt> 2.60 909.04 us 9.68 us 2272778.49
us 4649291 INODELK</tt><tt><br>
</tt><tt> 5.82 1240.20 us 8.78 us 1526868.50
us 7625383 LOOKUP</tt><tt><br>
</tt><tt> 88.60 58005.98 us 67.31 us 1448810.06
us 2481914 XATTROP</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 1396875 UPCALL</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 1854 CI_IATT</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 28 CI_XATTR</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 1051 CI_UNLINK</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 1394996 CI_FORGET</tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Duration: 208643 seconds</tt><tt><br>
</tt><tt> Data Read: 3409342848 bytes</tt><tt><br>
</tt><tt>Data Written: 2180686507 bytes</tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt>Interval 0 Stats:</tt><tt><br>
</tt><tt> Block Size: 1b+
2b+ 8b+ </tt><tt><br>
</tt><tt> No. of Reads: 0
0 0 </tt><tt><br>
</tt><tt>No. of Writes: 53
1 60 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 16b+
32b+ 64b+ </tt><tt><br>
</tt><tt> No. of Reads: 0
235 4 </tt><tt><br>
</tt><tt>No. of Writes: 14
1033 15 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 128b+
256b+ 512b+ </tt><tt><br>
</tt><tt> No. of Reads: 4
115 81 </tt><tt><br>
</tt><tt>No. of Writes: 58
703 80 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 1024b+
2048b+ 4096b+ </tt><tt><br>
</tt><tt> No. of Reads: 181
26 74 </tt><tt><br>
</tt><tt>No. of Writes: 163
278 397 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 8192b+
16384b+ 32768b+ </tt><tt><br>
</tt><tt> No. of Reads: 131
161 267 </tt><tt><br>
</tt><tt>No. of Writes: 7264
1398 6742 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 65536b+
131072b+ 262144b+ </tt><tt><br>
</tt><tt> No. of Reads: 443
25528 0 </tt><tt><br>
</tt><tt>No. of Writes: 2426
12240 1 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> %-latency Avg-latency Min-Latency Max-Latency
No. of calls Fop</tt><tt><br>
</tt><tt> --------- ----------- ----------- -----------
------------ ----</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 684942 FORGET</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 837092 RELEASE</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 87297 RELEASEDIR</tt><tt><br>
</tt><tt> 0.00 219.22 us 121.08 us 306.12
us 8 REMOVEXATTR</tt><tt><br>
</tt><tt> 0.00 21.70 us 7.24 us 196.07
us 97 IPC</tt><tt><br>
</tt><tt> 0.00 152.77 us 78.90 us 1059.82
us 46 FTRUNCATE</tt><tt><br>
</tt><tt> 0.00 295.73 us 167.10 us 606.96
us 65 FGETXATTR</tt><tt><br>
</tt><tt> 0.00 243.66 us 140.50 us 3392.16
us 121 LINK</tt><tt><br>
</tt><tt> 0.00 1216.47 us 194.37 us 23411.42
us 25 SYMLINK</tt><tt><br>
</tt><tt> 0.00 327.87 us 234.27 us 551.71
us 107 MKNOD</tt><tt><br>
</tt><tt> 0.00 36094.61 us 36094.61 us 36094.61
us 1 FSYNC</tt><tt><br>
</tt><tt> 0.00 219.63 us 147.34 us 783.10
us 188 RENAME</tt><tt><br>
</tt><tt> 0.00 104.27 us 40.34 us 595.29
us 745 STAT</tt><tt><br>
</tt><tt> 0.00 531.71 us 113.57 us 53973.08
us 200 TRUNCATE</tt><tt><br>
</tt><tt> 0.00 289.05 us 63.10 us 70107.41
us 1444 SETATTR</tt><tt><br>
</tt><tt> 0.00 2430.15 us 232.78 us 68120.54
us 177 MKDIR</tt><tt><br>
</tt><tt> 0.00 52.69 us 7.28 us 826.01
us 9890 FLUSH</tt><tt><br>
</tt><tt> 0.00 60.96 us 17.54 us 1393.95
us 9792 FINODELK</tt><tt><br>
</tt><tt> 0.00 74.25 us 20.00 us 1522.39
us 21521 STATFS</tt><tt><br>
</tt><tt> 0.00 129.23 us 34.15 us 9068.67
us 13094 FSTAT</tt><tt><br>
</tt><tt> 0.00 456.99 us 195.50 us 130182.02
us 5967 CREATE</tt><tt><br>
</tt><tt> 0.00 112.45 us 28.14 us 9545.53
us 27474 READ</tt><tt><br>
</tt><tt> 0.01 313.77 us 52.98 us 71635.74
us 32926 WRITE</tt><tt><br>
</tt><tt> 0.01 135.84 us 1.52 us 671373.08
us 87297 OPENDIR</tt><tt><br>
</tt><tt> 0.01 5225.28 us 131.28 us 951323.69
us 3177 RMDIR</tt><tt><br>
</tt><tt> 0.01 203.34 us 56.13 us 486756.90
us 85073 SETXATTR</tt><tt><br>
</tt><tt> 0.03 3521.00 us 145.28 us 3396306.00
us 15090 READDIRP</tt><tt><br>
</tt><tt> 0.04 113758.50 us 26.72 us 1147074.44
us 564 LK</tt><tt><br>
</tt><tt> 0.05 92.18 us 29.71 us 39069.99
us 850938 READLINK</tt><tt><br>
</tt><tt> 0.23 453.79 us 41.84 us 655999.23
us 831212 OPEN</tt><tt><br>
</tt><tt> 0.41 29425.34 us 19.20 us 1526187.08
us 22482 READDIR</tt><tt><br>
</tt><tt> 0.50 1954.70 us 21.51 us 1526772.28
us 415226 UNLINK</tt><tt><br>
</tt><tt> 0.54 1272.79 us 9.66 us 1604368.93
us 686945 GETXATTR</tt><tt><br>
</tt><tt> 0.55 237.57 us 7.91 us 1277047.91
us 3739718 ENTRYLK</tt><tt><br>
</tt><tt> 0.60 103888.49 us 89.32 us 761509.35
us 9311 FXATTROP</tt><tt><br>
</tt><tt> 2.60 909.04 us 9.68 us 2272778.49
us 4649291 INODELK</tt><tt><br>
</tt><tt> 5.82 1240.10 us 8.78 us 1526868.50
us 7625383 LOOKUP</tt><tt><br>
</tt><tt> 88.60 58006.02 us 67.31 us 1448810.06
us 2481914 XATTROP</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 1396875 UPCALL</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 1854 CI_IATT</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 28 CI_XATTR</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 1051 CI_UNLINK</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 1394996 CI_FORGET</tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Duration: 208643 seconds</tt><tt><br>
</tt><tt> Data Read: 3409342848 bytes</tt><tt><br>
</tt><tt>Data Written: 2180686507 bytes</tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><br>
</p>
</body>
</html>