[Gluster-users] NFS file locks

[Shehjar Tikoo]

There are a couple of things we can do:

-> Mail us the Glusterfs log files from the NFS server and the glusterfs 
servers when the lock script fails. Do file a bug if you can.

-> On the NFS client machine, before you run the mount command, make 
sure you run the following command.

	$ rpc.statd

-> Run the same perl script but this time at the nfs server over the 
glusterfs mount point, not at the NFS client. If it runs fine, it is 
probably related to locking over NFS and we'll look at other places to 
figure it out.

-Shehjar



Steinmetz, Ian wrote:
> 	I'm seeing an issue where I can't lock files on a NFS exported
> GlusterFS mount.  I have two servers connected to each other doing AFR
> to provide a high available NFS server (mirror the content, one VIP for
> NFS mounts to clients).  Both of the servers have mounted
> "/mnt/glusterfs" using GlusterFS with the client pointing to both
> servers.  I then export the filesyste with NFS.  I grabbed a quick perl
> program that tries to lock a file for testing, which fails only on the
> glusterfs.  When I export a normal directory "/mnt/test" the locking
> works.
> 	Any ideas appreciated.  I have a feeling I've implemented the
> posix/locks option incorrectly.
> 
> Both servers are running Ubuntu with identical setups, below are
> relevant configs.
> root at gluster01:/mnt/glusterfs# uname -a
> Linux gluster01 2.6.31-20-generic-pae #58-Ubuntu SMP Fri Mar 12 06:25:51
> UTC 2010 i686 GNU/Linux
> 
> root at gluster01:/mnt/glusterfs# cat /etc/exports
> /mnt/glusterfs  <ip removed for
> security>/25(rw,no_root_squash,no_all_squash,no_subtree_check,sync,insec
> ure,fsid=10)
> /mnt/test       <ip removed for
> security>/25(rw,no_root_squash,no_all_squash,no_subtree_check,sync,insec
> ure,fsid=11)
> 
> * I've tried async, rsync, removing all options except FSID.
> 
> root at gluster02:/etc/glusterfs# cat glusterfs.vol 
> volume brick1
>  type protocol/client
>  option transport-type tcp/client
>  option remote-host <ip removed for security> # IP address of the remote
> brick
>  option remote-subvolume brick        # name of the remote volume
> end-volume
>  
> volume brick2
>  type protocol/client
>  option transport-type tcp/client
>  option remote-host <ip removed for security>      # IP address of the
> remote brick
>  option remote-subvolume brick        # name of the remote volume
> end-volume
>  
> volume afr1
>  type cluster/afr
>  subvolumes brick1 brick2
> end-volume
>  
> volume writebehind
>   type performance/write-behind
>   option window-size 4MB
>   subvolumes afr1
> end-volume
> 
> volume cache
>   type performance/io-cache
>   option cache-size 512MB
>   subvolumes writebehind
> end-volume
> 
> volume readahead
>    type performance/read-ahead
>    option page-size 128KB     # unit in bytes
>    subvolumes cache
> end-volume
> 
> volume iothreads
>    type performance/io-threads
>    option thread-count 4
>    option cache-size 64MB
>    subvolumes readahead
> end-volume
> 
> root at gluster02:/etc/glusterfs# cat glusterfsd.vol 
> volume posix
>   type storage/posix
>   option directory /data/export
> end-volume
>  
> volume locks
>   type features/posix-locks
>   option manditory on  # tried with and without this, found in a search
> of earlier post
>   subvolumes posix
> end-volume
>  
> volume brick
>   type performance/io-threads
>   option thread-count 8
>   subvolumes locks
> end-volume
>  
> volume server
>   type protocol/server
>   option transport-type tcp
>   option auth.addr.brick.allow *
>   option auth.addr.brick-ns.allow *
>   option transport.socket.nodelay on
>   option auth.ip.locks.allow *
>   subvolumes brick
> end-volume
> 
> 
> * file to test locking...
> root at gluster02:/mnt/glusterfs# cat locktest.pl 
> #!/usr/bin/perl
> 
> use Fcntl qw(:flock);
> 
> my $lock_file = 'lockfile';
> 
> open(LOCKFILE,">>$lock_file") or die "Cannot open $lock_file: $!\n";
> print "Opened file $lock_file\n";
> flock(LOCKFILE, LOCK_SH) or die "Can't get shared lock on $lock_file:
> $!\n";
> print "Got shared lock on file $lock_file\n";
> sleep 2;
> close LOCKFILE;
> print "Closed file $lock_file\n";
> 
> exit;
> 
> *Test run from gluster02 using normal NFS mount:
> root at gluster02:/# mount <ip removed for security>:/mnt/test /mnt/test
> root at gluster02:/# cd /mnt/test
> root at gluster02:/mnt/test# ./locktest.pl 
> Opened file lockfile
> Got shared lock on file lockfile
> Closed file lockfile
> 
> *Test run from gluster02 using gluster exported NFS mount:
> root at gluster02:/# mount 74.81.128.17:/mnt/glusterfs /mnt/test
> root at gluster02:/# cd /mnt/test
> root at gluster02:/mnt/test# ./locktest.pl 
> Opened file lockfile
> Can't get shared lock on lockfile:
> No locks available
> 
> --
> Ian Steinmetz
> Comcast Engineering - Houston
> 713-375-7866
> 
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users