[Gluster-users] not sure how to troubleshoot SMB / CIFS overload when using GlusterFS

[Ken Randall]

Joe,

Thank you for your response.  After seeing what you wrote, I bumped up the
performance.cache-size up to 4096MB, the max allowed, and ran into the same
wall.

I wouldn't think that any SMB caching would help in this case, since the
same Samba server on top of the raw Gluster data wasn't exhibiting any
trouble, or am I deceived?

I haven't used strace before, but I ran it on the glusterfs process, and saw
a lot of:
epoll_wait(3, {{EPOLLIN, {u32=9, u64=9}}}, 257, 4294967295) = 1
readv(9, [{"\200\0\16,", 4}], 1)        = 4
readv(9, [{"\0\n;\227\0\0\0\1", 8}], 1) = 8
readv(9,
[{"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\31\0\0\0\0\0\0\0\1\0\0\0\0"...,
3620}], 1) = 1436
readv(9, 0xa90b1b8, 1)                  = -1 EAGAIN (Resource temporarily
unavailable)

And when I ran it on smbd, I saw a constant stream of this kind of activity:
getdents(29, /* 25 entries */, 32768)   = 840
getdents(29, /* 25 entries */, 32768)   = 856
getdents(29, /* 25 entries */, 32768)   = 848
getdents(29, /* 24 entries */, 32768)   = 856
getdents(29, /* 25 entries */, 32768)   = 864
getdents(29, /* 24 entries */, 32768)   = 832
getdents(29, /* 25 entries */, 32768)   = 832
getdents(29, /* 24 entries */, 32768)   = 856
getdents(29, /* 25 entries */, 32768)   = 840
getdents(29, /* 24 entries */, 32768)   = 832
getdents(29, /* 25 entries */, 32768)   = 784
getdents(29, /* 25 entries */, 32768)   = 824
getdents(29, /* 25 entries */, 32768)   = 808
getdents(29, /* 25 entries */, 32768)   = 840
getdents(29, /* 25 entries */, 32768)   = 864
getdents(29, /* 25 entries */, 32768)   = 872
getdents(29, /* 25 entries */, 32768)   = 832
getdents(29, /* 24 entries */, 32768)   = 832
getdents(29, /* 25 entries */, 32768)   = 840
getdents(29, /* 25 entries */, 32768)   = 824
getdents(29, /* 25 entries */, 32768)   = 824
getdents(29, /* 24 entries */, 32768)   = 864
getdents(29, /* 25 entries */, 32768)   = 848
getdents(29, /* 24 entries */, 32768)   = 840

That chunk would get repeated over and over and over again as fast as the
screen could go, with the occasional (every 5-10 seconds or so), would you
see anything that you'd normally expect to see, such as:
close(29)                               = 0
stat("Storage/01", 0x7fff07dae870) = -1 ENOENT (No such file or directory)
write(23,
"\0\0\0#\377SMB24\0\0\300\210A\310\0\0\0\0\0\0\0\0\0\0\0\0\1\0d\233"..., 39)
= 39
select(38, [5 20 23 27 30 31 35 36 37], [], NULL, {60, 0}) = 1 (in [23],
left {60, 0})
read(23, "\0\0\0x", 4)                  = 4
read(23,
"\377SMB2\0\0\0\0\30\7\310\0\0\0\0\0\0\0\0\0\0\0\0\1\0\250P\273\0[8"...,
120) = 120
stat("Storage", {st_mode=S_IFDIR|0755, st_size=1581056, ...}) = 0
stat("Storage/011235", 0x7fff07dad470) = -1 ENOENT (No such file or
directory)
stat("Storage/011235", 0x7fff07dad470) = -1 ENOENT (No such file or
directory)
open("Storage", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = 29
fcntl(29, F_SETFD, FD_CLOEXEC)          = 0

(The no such file or directory part is expected since some of the image
references don't exist.)

Ken
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20110717/aee41a56/attachment.html>