[Gluster-users] Gluster-users Digest, Vol 62, Issue 17
Charles Cooke
charles at coupa.com
Fri Jun 7 13:53:20 UTC 2013
On Fri, Jun 7, 2013 at 5:00 AM, <gluster-users-request at gluster.org> wrote:
>
> Message: 2
> Date: Thu, 6 Jun 2013 16:07:08 +0200
> From: Stephan von Krawczynski <skraw at ithnet.com>
> To: Pablo <paa.listas at gmail.com>
> Cc: gluster-users at gluster.org
> Subject: Re: [Gluster-users] GlusterFS (3.3.1) - performance issues -
> large number of LOOKUP calls & high CPU usage
> Message-ID: <20130606160708.898f5049.skraw at ithnet.com>
> Content-Type: text/plain; charset=US-ASCII
>
> On Thu, 06 Jun 2013 10:39:21 -0300
> Pablo <paa.listas at gmail.com> wrote:
>
> > I have never try this (In fact I'm just learning a bit more how to
> > administer a Gluster server.), buy you may find it useful.
> >
> >
> http://download.gluster.org/pub/gluster/glusterfs/doc/HA%20and%20Load%20Balancing%20for%20NFS%20and%20SMB.html
> >
> > Pablo.
>
> The thing with this way of failover is though, that you will likely
> corrupt a
> currently written file. If your NFS-server (gluster) node dies while you
> write
> your file will be corrupt. If you use native glusterfs mounts it will not
> (should not). This is why I consider the NFS server feature nothing more
> than
> a bad hack. It does not deliver the safety that glusterfs promises, even if
> you solve the failover problem somehow.
>
> --
> Regards,
> Stephan
>
>
> ------------------------------
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-users
>
> End of Gluster-users Digest, Vol 62, Issue 17
> *********************************************
>
Yes - and in fact, we'd rather NOT use NFS if it can be avoided.
As another point of interest, the GlusterFS FUSE native appears to be due
to the large number of LOOKUP calls as initially indicated (69000s for FUSE
vs. 592 for NFS). We also see a tremendous increase in network overhead
using the FUSE client which I suppose is related to the LOOKUP calls (about
300-400k sustained over 60s for FUSE, 30-50k over about 10s for NFS).
Any idea as to what would be causing this overhead?
===FUSE===
Interval 10 Stats:
Block Size: 1b+ 2b+
4b+
No. of Reads: 0 0
0
No. of Writes: 15 12
20
Block Size: 8b+ 16b+
32b+
No. of Reads: 0 0
0
No. of Writes: 27 17
59
Block Size: 64b+ 128b+
256b+
No. of Reads: 1 0
0
No. of Writes: 68 27
24
Block Size: 512b+ 1024b+
2048b+
No. of Reads: 5 20
5
No. of Writes: 60 0
10
Block Size: 4096b+
No. of Reads: 0
No. of Writes: 30
%-latency Avg-latency Min-Latency Max-Latency No. of calls
Fop
--------- ----------- ----------- ----------- ------------
----
0.00 0.00 us 0.00 us 0.00 us 58
FORGET
0.00 0.00 us 0.00 us 0.00 us 671
RELEASE
0.00 0.00 us 0.00 us 0.00 us 114
RELEASEDIR
0.01 31.20 us 21.00 us 62.00 us 10
LK
0.02 38.45 us 32.00 us 56.00 us 31
READ
0.03 210.17 us 187.00 us 271.00 us 6
STATFS
0.03 287.20 us 264.00 us 313.00 us 5
MKDIR
0.11 62.73 us 48.00 us 168.00 us 82
TRUNCATE
0.11 38.25 us 23.00 us 620.00 us 135
STAT
0.13 52.50 us 37.00 us 268.00 us 114
OPENDIR
0.15 156.36 us 129.00 us 247.00 us 45
UNLINK
0.21 123.66 us 51.00 us 2014.00 us 82
SETATTR
0.23 132.70 us 90.00 us 254.00 us 82
RENAME
0.44 56.75 us 33.00 us 1096.00 us 369
WRITE
0.44 45.49 us 24.00 us 3014.00 us 461
FSTAT
0.69 292.22 us 216.00 us 1777.00 us 112
CREATE
0.75 64.18 us 36.00 us 3763.00 us 559
OPEN
0.94 46.40 us 10.00 us 2253.00 us 963
FLUSH
3.55 669.45 us 83.00 us 3746.00 us 253
READDIRP
4.10 2385.21 us 762.00 us 5424.00 us 82
FSYNC
88.09 60.94 us 40.00 us 21474.00 us 69004
LOOKUP
Duration: 113 seconds
Data Read: 47519 bytes
Data Written: 224013 bytes
===NFS===
Interval 13 Stats:
Block Size: 8b+ 16b+
64b+
No. of Reads: 0 0
0
No. of Writes: 5 10
11
Block Size: 128b+ 256b+
512b+
No. of Reads: 0 0
0
No. of Writes: 6 10
23
Block Size: 1024b+ 2048b+
16384b+
No. of Reads: 0 0
0
No. of Writes: 18 12
5
%-latency Avg-latency Min-Latency Max-Latency No. of calls
Fop
--------- ----------- ----------- ----------- ------------
----
0.00 0.00 us 0.00 us 0.00 us 57
FORGET
0.00 0.00 us 0.00 us 0.00 us 113
RELEASE
0.07 225.00 us 199.00 us 251.00 us 2
STATFS
0.09 58.22 us 43.00 us 137.00 us 9
OPEN
0.12 36.65 us 20.00 us 139.00 us 20
LK
0.24 292.20 us 277.00 us 297.00 us 5
MKDIR
1.01 150.39 us 102.00 us 278.00 us 41
FSTAT
1.09 66.31 us 18.00 us 194.00 us 100
FLUSH
1.36 83.16 us 46.00 us 424.00 us 100
WRITE
1.94 289.24 us 189.00 us 599.00 us 41
READDIRP
2.28 29.13 us 14.00 us 181.00 us 478
ACCESS
2.61 39.76 us 18.00 us 376.00 us 401
FINODELK
2.98 81.78 us 52.00 us 383.00 us 222
SETATTR
2.98 40.78 us 18.00 us 531.00 us 446
INODELK
3.92 322.81 us 95.00 us 3197.00 us 74
RENAME
4.71 143.64 us 83.00 us 303.00 us 200
FXATTROP
4.91 287.68 us 218.00 us 679.00 us 104
CREATE
5.48 54.41 us 16.00 us 10294.00 us 614
ENTRYLK
6.10 35.55 us 23.00 us 299.00 us 1046
STAT
10.15 104.55 us 42.00 us 2412.00 us 592
LOOKUP
13.87 1692.02 us 782.00 us 4956.00 us 50
UNLINK
34.09 196.52 us 91.00 us 14576.00 us 1058
XATTROP
Duration: 39 seconds
Data Read: 0 bytes
Data Written: 222075 bytes
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20130607/37a7b4bc/attachment.html>
More information about the Gluster-users
mailing list