[Gluster-devel] ls -l in directory doesn't show file under samba with a brick down

m.c.wilkins at massey.ac.nz m.c.wilkins at massey.ac.nz
Tue May 5 23:23:17 UTC 2009


Hi,

Using samba on top of gluster, and when one brick goes down I can't
see files that are actively being made on functional bricks.

Running 2.0.0, I have 2 machines in a NUFA arrangement (log below),
gluster mounted on /mnt, sharing /mnt out from one node via samba.  On
the machine that is doing the samba I do a mkdir /mnt/me, then start
copying a big file into /mnt/me.  I can do a ls -l /mnt/me and see the
file growing in there.  I take down my second node (umount /mnt), back
on the first node everything is fine, the copy continues since the
file is going to a local brick anyway.  I can do a ls -l /mnt/me and
see the file growing there fine.

I repeat the above but I'm copying a file into /mnt/me via samba.
When the second node goes down, the copy continues, but I can't see
the file in /mnt/me (a ls -l /mnt/me/nameoffile shows it, but a ls -l
/mnt/me doesn't).  If I start a new copy while the second node is
down, then a ls -l /mnt/me shows that file fine.  It is only files
that are in the process of being copied while the second node goes
down, and only under samba, that I see the problem.

I thought it might be something to do with samba preallocating the
entire file, then filling it up during the copy.  I set 'strict
allocate = yes' in my smb.conf, which doesn't stop the preallocating
but it does make the preallocation real slow, slow enough that i can
watch the file growing and then take the second node down, but then
wham it is then not visible via a ls -l on its directory.

It might not sound like a huge issue, since the copy works.  But I am
currently testing our gluster and demonstrating to a user that it will
work OK in the situation when a brick other than actively used one on
the samba server goes down.

Please, any help much appreciated.

Matt

---
Version      : glusterfs 2.0.0 built on May  6 2009 09:02:18
TLA Revision : 7b2e459db65edd302aa12476bc73b3b7a17b1410
Starting Time: 2009-05-06 09:05:19
Command line : /usr/local/sbin/glusterfs --log-level=NORMAL --volfile=/usr/local/etc/glusterfs/glusterfsd3.vol /mnt 
PID          : 20866
System name  : Linux
Nodename     : mu-rhdev1
Kernel Release : 2.6.18-128.1.6.el5
Hardware Identifier: x86_64

Given volfile:
+------------------------------------------------------------------------------+
  1: volume posix0
  2:   type storage/posix
  3:   option directory /export/brick0
  4: end-volume
  5: 
  6: volume locks0
  7:   type features/locks
  8:   subvolumes posix0
  9: end-volume
 10: 
 11: volume brick0
 12:   type performance/io-threads
 13:   subvolumes locks0
 14: end-volume
 15: 
 16: volume posix1
 17:   type storage/posix
 18:   option directory /export/brick1
 19: end-volume
 20: 
 21: volume locks1
 22:   type features/locks
 23:   subvolumes posix1
 24: end-volume
 25: 
 26: volume brick1
 27:   type performance/io-threads
 28:   subvolumes locks1
 29: end-volume
 30: 
 31: volume server
 32:   type protocol/server
 33:   option transport-type tcp
 34:   option auth.addr.brick0.allow *
 35:   option auth.addr.brick1.allow *
 36:   subvolumes brick0 brick1
 37: end-volume
 38: 
 39: volume mu-rhdev1-0
 40:   type protocol/client
 41:   option transport-type tcp
 42:   option remote-host mu-rhdev1
 43:   option remote-subvolume brick0
 44: end-volume
 45: 
 46: volume mu-rhdev1-1
 47:   type protocol/client
 48:   option transport-type tcp
 49:   option remote-host mu-rhdev1
 50:   option remote-subvolume brick1
 51: end-volume
 52: 
 53: volume mu-rhdev2-0
 54:   type protocol/client
 55:   option transport-type tcp
 56:   option remote-host mu-rhdev2
 57:   option remote-subvolume brick0
 58: end-volume
 59: 
 60: volume nufa
 61:    type cluster/nufa
 62:    option local-volume-name mu-rhdev1-0
 63:    subvolumes mu-rhdev1-0 mu-rhdev1-1 mu-rhdev2-0
 64: end-volume
 65: 
 66: volume writebehind
 67:    type performance/write-behind
 68:    option cache-size 1MB
 69:    subvolumes nufa
 70: end-volume
 71: 
 72: # before or after writebehind?
 73: volume ra
 74:    type performance/read-ahead
 75:    subvolumes writebehind
 76: end-volume
 77: 
 78: volume cache
 79:    type performance/io-cache
 80:    option cache-size 512MB
 81:    subvolumes ra
 82: end-volume
 83: 

+------------------------------------------------------------------------------+
2009-05-06 10:33:15 N [glusterfsd.c:1152:main] glusterfs: Successfully started
2009-05-06 10:33:15 E [socket.c:743:socket_connect_finish] mu-rhdev2-0: connection failed (Connection refused)
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected
2009-05-06 10:33:15 E [socket.c:743:socket_connect_finish] mu-rhdev2-0: connection failed (Connection refused)
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected
2009-05-06 10:33:15 E [socket.c:743:socket_connect_finish] mu-rhdev1-1: connection failed (Connection refused)
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-1: disconnected
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-1: disconnected
2009-05-06 10:33:15 E [socket.c:743:socket_connect_finish] mu-rhdev1-1: connection failed (Connection refused)
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-1: disconnected
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-1: disconnected
2009-05-06 10:33:15 E [socket.c:743:socket_connect_finish] mu-rhdev1-0: connection failed (Connection refused)
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-0: disconnected
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-0: disconnected
2009-05-06 10:33:15 E [socket.c:743:socket_connect_finish] mu-rhdev1-0: connection failed (Connection refused)
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-0: disconnected
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-0: disconnected
2009-05-06 10:33:16 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.13:1014
2009-05-06 10:33:16 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.13:1017
2009-05-06 10:33:16 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.13:1016
2009-05-06 10:33:16 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.13:1015
2009-05-06 10:33:25 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.2:1019
2009-05-06 10:33:25 W [client-protocol.c:6320:client_setvolume_cbk] mu-rhdev1-0: attaching to the local volume 'brick0'
2009-05-06 10:33:25 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev1-0: connection and handshake succeeded
2009-05-06 10:33:25 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.2:1018
2009-05-06 10:33:25 W [client-protocol.c:6320:client_setvolume_cbk] mu-rhdev1-0: attaching to the local volume 'brick0'
2009-05-06 10:33:25 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev1-0: connection and handshake succeeded
2009-05-06 10:33:25 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.2:1017
2009-05-06 10:33:25 W [client-protocol.c:6320:client_setvolume_cbk] mu-rhdev1-1: attaching to the local volume 'brick1'
2009-05-06 10:33:25 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev1-1: connection and handshake succeeded
2009-05-06 10:33:25 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.2:1016
2009-05-06 10:33:25 W [client-protocol.c:6320:client_setvolume_cbk] mu-rhdev1-1: attaching to the local volume 'brick1'
2009-05-06 10:33:25 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev1-1: connection and handshake succeeded
2009-05-06 10:33:25 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev2-0: connection and handshake succeeded
2009-05-06 10:33:25 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev2-0: connection and handshake succeeded
2009-05-06 10:33:29 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected
2009-05-06 10:33:29 N [server-protocol.c:8272:notify] server: 130.123.131.13:1014 disconnected
2009-05-06 10:33:29 N [server-protocol.c:8272:notify] server: 130.123.131.13:1015 disconnected
2009-05-06 10:33:29 N [server-helpers.c:723:server_connection_destroy] server: destroyed connection of mu-rhdev2-7959-2009/05/06-10:33:16:708168-mu-rhdev1-1
2009-05-06 10:33:29 E [socket.c:102:__socket_rwv] server: readv failed (Connection reset by peer)
2009-05-06 10:33:29 E [socket.c:562:__socket_proto_state_machine] server: read (Connection reset by peer) in state 1 (130.123.131.13:1016)
2009-05-06 10:33:29 N [server-protocol.c:8272:notify] server: 130.123.131.13:1016 disconnected
2009-05-06 10:33:29 N [server-protocol.c:8272:notify] server: 130.123.131.13:1017 disconnected
2009-05-06 10:33:29 N [server-helpers.c:723:server_connection_destroy] server: destroyed connection of mu-rhdev2-7959-2009/05/06-10:33:16:708168-mu-rhdev1-0
2009-05-06 10:33:38 E [socket.c:743:socket_connect_finish] mu-rhdev2-0: connection failed (Connection refused)
2009-05-06 10:33:38 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected
2009-05-06 10:33:38 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected
2009-05-06 10:33:38 E [socket.c:743:socket_connect_finish] mu-rhdev2-0: connection failed (Connection refused)
2009-05-06 10:33:38 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected
2009-05-06 10:33:38 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected
2009-05-06 10:33:43 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.13:1009
2009-05-06 10:33:43 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.13:1016
2009-05-06 10:33:43 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.13:1011
2009-05-06 10:33:43 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.13:1010
2009-05-06 10:33:45 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev2-0: connection and handshake succeeded
2009-05-06 10:33:45 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev2-0: connection and handshake succeeded





More information about the Gluster-devel mailing list