[Gluster-devel] Feedback - Problem with the locks feature

[Anand Avati]

----- "Samuel Hassine" <samuel.hassine at gmail.com> wrote:

> Hi all,
> 
> For the PHP script with little write/read accesses I will try to find
> it (I dont remember exactly the syntax), but for PHP Sessions, the bug
> could be easily reproduced.
> 
> I just test it on a new very simple GlusterFS partition with no trafic
> (juste me), and I reproduced it immediatly.
> 
> Explainations:
> - 2 servers Debian Lenny stable
> - GlusterFS 3.0.0 in distributed mode (one server and multiple
> clients)
> - Lighttpd / PHP5 Fast-CGI
> 
> I juste mount the GlusterFS partition on the /var/www directory.
> 
> First of all, the PHP script you can execute:
> 
> <?php
> session_save_path('.');
> //if you want to verify if it worked
> //echo session_save_path();
> session_start();
> ?>
> 
> Secondly, there are 2 configurations if GlusterFS and, of course, one
> works and one does not.
> The client configuration is the same in the both cases:
> 
> glusterfs.vol
> volume test-1
> type protocol/client
> option transport-type tcp
> option remote-host test
> option transport.socket.nodelay on
> option transport.remote-port 6996
> option remote-subvolume brick1
> end-volume
> 
> volume writebehind
> type performance/write-behind
> option cache-size 4MB
> subvolumes test-1
> end-volume
> 
> volume readahead
> type performance/read-ahead
> option page-count 4
> subvolumes writebehind
> end-volume
> 
> volume iocache
> type performance/io-cache
> option cache-size 1GB
> option cache-timeout 1
> subvolumes readahead
> end-volume
> 
> volume quickread
> type performance/quick-read
> option cache-timeout 1
> option max-file-size 64kB
> subvolumes iocache
> end-volume
> 
> volume statprefetch
> type performance/stat-prefetch
> subvolumes quickread
> end-volume
> 
> Now the server configuration:
> 
> glusterfsd.vol (this doesnt work)
> volume posix1
> type storage/posix
> option directory /data
> end-volume
> 
> volume locks1
> type features/locks
> subvolumes posix1
> end-volume
> 
> volume brick1
> type performance/io-threads
> option thread-count 8
> subvolumes locks1
> end-volume
> 
> volume server-tcp
> type protocol/server
> option transport-type tcp
> option auth.addr.brick1.allow *
> option transport.socket.listen-port 6996
> option transport.socket.nodelay on
> subvolumes brick1
> end-volume
> 
> glusterfsd.vol (this works)
> volume posix1
> type storage/posix
> option directory /data
> end-volume
> 
> #volume locks1
> # type features/locks
> # subvolumes posix1
> #end-volume
> 
> volume brick1
> type performance/io-threads
> option thread-count 8
> subvolumes posix1
> end-volume
> 
> volume server-tcp
> type protocol/server
> option transport-type tcp
> option auth.addr.brick1.allow *
> option transport.socket.listen-port 6996
> option transport.socket.nodelay on
> subvolumes brick1
> end-volume
> 
> So, with the locks translator, you can execute the script one time (it
> will be ok) but the second time the session file is on the file system
> but locked and nobody can access to it. PHP freezes and processes
> coult not be killed.
> 
> When it's happened, I have nothing in client-side logs but I have 2
> kinds of message in the server-side logs:
> When I execute the script:
> [2010-02-04 21:11:22] W [posix.c:246:posix_lstat_with_gen] posix1:
> Access to /data//.. (on dev 2049) is crossing device (64768)
> [2010-02-04 21:11:24] W [posix.c:246:posix_lstat_with_gen] posix1:
> Access to /data//.. (on dev 2049) is crossing device (64768)
> 
> When I try to umount -f (disconnect the gluster):
> [2010-02-04 21:13:45] E [server-protocol.c:339:protocol_server_reply]
> protocol/server: frame 20: failed to submit. op= 26, type= 4
> 
> As I said I will try to find the other PHP script.
> 
> I hope this will help you.

I tried to reproduce the problem with your exact configuration (only changing 'option remote-host') from 1 server and 2 clients. I was not able to hit the problem with the configuration which is breaking for you. I used v3.0.0 as well.

Can you please turn 'option trace on' in the locks translator and give us the server log when the php session hangs?

Thanks,
Avati