[Gluster-users] can't use sqlite3 on gluster mounted as NFS

[Brandon Simmons]

On Fri, Sep 23, 2011 at 4:11 PM, Anand Babu Periasamy <ab at gluster.com> wrote:
> This is a known issue. Gluster NFS doesn't support NLM (locking) yet. 3.4
> may implement this.  Did you try on GlusterFS native mount?

Thanks for that information.

I did test with the native fuse mount, but the results were difficult
to interpret. We have a rails application that writes to multiple
sqlite databases, and a test script that simulates a bunch of random
writes to a specified DB, retrying if it fails.

On NFS this test runs reasonably well: both clients take turns, a
couple retries, all writes complete without failures.

But mounted over gluster (same machines, underlying disk as above) one
client always runs while the other gets locked out (different client
machines depending on which was started first). At some point during
this test the client that was locked out from writing to the DB
actually gets disconnected from gluster and I have to remount:

    $ ls /mnt/gluster
    ls: cannot access /websites/: Transport endpoint is not connected

One client is consistently locked out even if they are writing to
DIFFERENT DBs altogether.

The breakage of the mountpoint happened every time the test was run
concurrently against the SAME DB, but did not seem to occur when
clients were running against different DBs.

But like I said, this was a very high level test with many moving
parts so I'm not sure how useful the above details are for you to
know.

Happy to hear any ideas for testing,
Brandon

/var/log/glusterfs/etc-glusterfs-glusterd.vol.log:
[2011-09-16 19:32:38.122196] W
[socket.c:1494:__socket_proto_state_machine] 0-socket.management:
reading from socket failed. Error (Transport endpoint is not
connected), peer (127.0.0.1:1017)

>
> --AB
>
> On Sep 23, 2011 10:00 AM, "Brandon Simmons" <bsimmons at labarchives.com>
> wrote:
>> I am able to successfully mount a gluster volume using the NFS client
>> on my test servers. Simple reading and writing seems to work, but
>> trying to work with sqlite databases seems to cause the sqlite client
>> and libraries to freeze. I have to send KILL to stop the process.
>>
>> Here is an example, server 1 and 2 are clients mounting gluster volume
>> over NFS:
>>
>> server1# echo "working" > /mnt/gluster/test_simple
>> server2# echo "working" >> /mnt/gluster/test_simple
>> server1# cat /mnt/gluster/test_simple
>> working
>> working
>> server1# sqlite3 /websites/new.sqlite3
>> SQLite version 3.6.10
>> Enter ".help" for instructions
>> Enter SQL statements terminated with a ";"
>> sqlite> create table memos(text, priority INTEGER);
>> (...hangs forever, have to detach screen and do kill -9)
>>
>> the gluster volume was created and NFS-mounted as per the instructions
>> here:
>>
>>
>> http://www.gluster.com/community/documentation/index.php/Gluster_3.2_Filesystem_Administration_Guide
>>
>> If I mount the volume using the nolock option, then things work:
>>
>> mount -t nfs -o nolock server:/test-vol /mnt/gluster
>>
>> So I assume this has something to do with the locking RPC service
>> stufff, which I don't know much about. Here's output from rpc info:
>>
>> server# rpcinfo -p
>> program vers proto port
>> 100000 2 tcp 111 portmapper
>> 100000 2 udp 111 portmapper
>> 100024 1 udp 56286 status
>> 100024 1 tcp 40356 status
>> 100005 3 tcp 38465 mountd
>> 100005 1 tcp 38466 mountd
>> 100003 3 tcp 38467 nfs
>>
>>
>> client1# rpcinfo -p server
>> program vers proto port
>> 100000 2 tcp 111 portmapper
>> 100000 2 udp 111 portmapper
>> 100024 1 udp 56286 status
>> 100024 1 tcp 40356 status
>> 100005 3 tcp 38465 mountd
>> 100005 1 tcp 38466 mountd
>> 100003 3 tcp 38467 nfs
>>
>> client1# # rpcinfo -p
>> program vers proto port
>> 100000 2 tcp 111 portmapper
>> 100000 2 udp 111 portmapper
>> 100024 1 udp 32768 status
>> 100024 1 tcp 58368 status
>>
>> Thanks for any help,
>> Brandon
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>