[Gluster-users] Samba > NFS > Gluster leads to runaway lock problem after many stable months

[Whit Blauvelt]

Hi,

I have a couple of Gluster 3.1.4 shares on the LAN NFS mounted at
192.168.1.242 to a couple of other systems. One of those systems in turn has
access via Samba that includes those shares. This has been a stable system
for a year. Today it went crazy, where the most immediate bad effect was an
immense swelling of the logs of the system with both the Gluster shares
mounted by NFS, and Samba access from other systems.

That swelling was in the neighborhood of 5000 lines per second repeating
approximately this:

Aug 29 18:47:01 system2 smbd[7118]: [2012/08/29 18:47:01, 0] locking/posix.c:posix_fcntl_getlock(244) 
Aug 29 18:47:01 system2 smbd[7118]:   on 32 bit NFS mounted file systems. 
Aug 29 18:47:01 system2 smbd[7118]: [2012/08/29 18:47:01, 0] locking/posix.c:posix_fcntl_getlock(252) 
Aug 29 18:47:01 system2 smbd[7118]:   Offset greater than 31 bits. Returning success. 
Aug 29 18:47:01 system2 smbd[7118]: [2012/08/29 18:47:01, 0] locking/posix.c:posix_fcntl_getlock(242) 
Aug 29 18:47:01 system2 smbd[7118]:   posix_fcntl_getlock: WARNING: lock request at offset 2886282524, length 1 returned 
Aug 29 18:47:01 system2 smbd[7118]: [2012/08/29 18:47:01, 0] locking/posix.c:posix_fcntl_getlock(243) 
Aug 29 18:47:01 system2 smbd[7118]:   an No locks available error. This can happen when using 64 bit lock offsets 
Aug 29 18:47:01 system2 smbd[7118]: [2012/08/29 18:47:01, 0] locking/posix.c:posix_fcntl_getlock(244) 
Aug 29 18:47:01 system2 kernel: [14164811.340891] lockd: couldn't create RPC handle for 192.168.1.242

Not good. The /var partition filled totally, fast. 

Turned Samba off. Cleared logs out of the way. Restarted Samba, and
everything's happy and not complaining. But obviously I need to avoid
whatever set that off like the devil. It's an older Samba on that system,
Version 3.0.28a, if that makes a difference. 

What are my options for avoiding this, both the core problem, and a repeat
of anything throwing 5000 lines into the logs per second? The log level's
set low in Samba. I suppose I could set up something to shut down syslog if
/var gets to 99% full again, as a stop gap. There's 3X the space there that
the logs normally fill.

Thanks,
Whit