[Gluster-devel] How namespace works in mainline-2.5?
Sebastien LELIEVRE
slelievre at tbs-internet.com
Thu Jun 21 13:51:27 UTC 2007
Hi
I'm getting the exact same problems here. Since I don't need unify, I've
removed it (I'm using it only to specify the afr brick to check if there
is still space left on the volume)
Apparently, self-heal sync the accesed files, am I correct ?
I tried a 'ls -Rl' and the client crashed.
I'm on 2.5 patch 177
here is gdb and 'bt' command :
# gdb glusterfs -c /core.25302
GNU gdb 6.4.90-debian
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i486-linux-gnu"...Using host libthread_db
library "/lib/tls/i686/cmov/libthread_db.so.1".
warning: Can't read pathname for load map: Input/output error.
Reading symbols from /usr/local/lib/libglusterfs.so.0...done.
Loaded symbols for /usr/local/lib/libglusterfs.so.0
Reading symbols from /usr/local/lib/libfuse.so.2...done.
Loaded symbols for /usr/local/lib/libfuse.so.2
Reading symbols from /lib/tls/i686/cmov/libdl.so.2...done.
Loaded symbols for /lib/tls/i686/cmov/libdl.so.2
Reading symbols from /lib/tls/i686/cmov/libpthread.so.0...done.
Loaded symbols for /lib/tls/i686/cmov/libpthread.so.0
Reading symbols from /lib/tls/i686/cmov/libc.so.6...done.
Loaded symbols for /lib/tls/i686/cmov/libc.so.6
Reading symbols from /lib/tls/i686/cmov/librt.so.1...done.
Loaded symbols for /lib/tls/i686/cmov/librt.so.1
Reading symbols from /lib/ld-linux.so.2...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from
/usr/local/lib/glusterfs/1.3.0-pre5/xlator/protocol/client.so...done.
Loaded symbols for
/usr/local/lib/glusterfs/1.3.0-pre5/xlator/protocol/client.so
Reading symbols from
/usr/local/lib/glusterfs/1.3.0-pre5/xlator/cluster/afr.so...done.
Loaded symbols for /usr/local/lib/glusterfs/1.3.0-pre5/xlator/cluster/afr.so
Reading symbols from
/usr/local/lib/glusterfs/1.3.0-pre5/xlator/performance/write-behind.so...done.
Loaded symbols for
/usr/local/lib/glusterfs/1.3.0-pre5/xlator/performance/write-behind.so
Reading symbols from
/usr/local/lib/glusterfs/1.3.0-pre5/xlator/features/fixed-id.so...done.
Loaded symbols for
/usr/local/lib/glusterfs/1.3.0-pre5/xlator/features/fixed-id.so
Reading symbols from
/usr/local/lib/glusterfs/1.3.0-pre5/transport/tcp/client.so...done.
Loaded symbols for
/usr/local/lib/glusterfs/1.3.0-pre5/transport/tcp/client.so
Reading symbols from /lib/libgcc_s.so.1...done.
Loaded symbols for /lib/libgcc_s.so.1
Core was generated by `[glusterfs]
'.
Program terminated with signal 11, Segmentation fault.
#0 0x0804da85 in fuse_xattr_cbk ()
(gdb) bt
#0 0x0804da85 in fuse_xattr_cbk ()
#1 0xb7f32965 in default_getxattr_cbk (frame=0x8094748,
cookie=0x8094790, this=0x8057c38, op_ret=0, op_errno=2, dict=0x80955a0)
at defaults.c:1071
#2 0xb7f32965 in default_getxattr_cbk (frame=0x8094790,
cookie=0x80947d8, this=0x8057610, op_ret=0, op_errno=2, dict=0x80955a0)
at defaults.c:1071
#3 0xb75a13d7 in afr_getxattr_cbk () from
/usr/local/lib/glusterfs/1.3.0-pre5/xlator/cluster/afr.so
#4 0xb75b2e0d in client_getxattr_cbk () from
/usr/local/lib/glusterfs/1.3.0-pre5/xlator/protocol/client.so
#5 0xb75b5102 in notify () from
/usr/local/lib/glusterfs/1.3.0-pre5/xlator/protocol/client.so
#6 0xb7f35ea7 in transport_notify (this=0x80945cc, event=10) at
transport.c:152
#7 0xb7f369c9 in sys_epoll_iteration (ctx=0xbffbf084) at epoll.c:54
#8 0xb7f35f7d in poll_iteration (ctx=0xbffbf084) at transport.c:260
#9 0x0804a110 in main ()
I hope it will be helpful
Here are the actual spec files :
#server1 :
volume brick1
type storage/posix
option directory /home/www
end-volume
volume fixed1
type features/fixed-id
option fixed-uid 33
option fixed-gid 33
subvolumes brick1
end-volume
volume locks1
type features/posix-locks
subvolumes fixed1
end-volume
volume server
type protocol/server
option transport-type tcp/server
subvolumes locks1
option ath.ip.locks1.allow 192.168.0.*
end-volume
volume trace
type debug/trace
subvolumes locks1
option debug on
end-volume
#server2:
volume brick2
type storage/posix
option directory /home/www
end-volume
volume fixed2
type features/fixed-id
option fixed-uid 33
option fixed-gid 33
subvolumes brick2
end-volume
volume locks2
type features/posix-locks
subvolumes fixed2
end-volume
volume server
type protocol/server
option transport-type tcp/server
subvolumes locks2
option auth.ip.locks2.allow 192.168.0.*
end-volume
volume trace
type debug/trace
subvolumes locks2
option debug on
end-volume
client:
volume brick1c
type protocol/client
option transport-type tcp/client
option remote-host 192.168.0.6
option remote-subvolume locks1
end-volume
volume brick2c
type protocol/client
option transport-type tcp/client
option remote-host 192.168.0.7
option remote-subvolume locks2
end-volume
volume afr
type cluster/afr
subvolumes brick1c brick2c
option replicate *:2
end-volume
volume writebehind
type performance/write-behind
option aggregate-size 131072
subvolumes afr
end-volume
volume client-fixed
type features/fixed-id
option fixed-uid 33
option fixed-gid 33
subvolumes writebehind
end-volume
Cheers,
Sebastien.
Amar S. Tumballi a écrit :
> Hi,
> Sorry for less info regarding the changes made to unify in 2.5 branch. We
> haven't yet updated wiki, as release is not yet made. but wrong example in
> doc/* is my mistake :(
>
> Yes, you have figured it right, namespace is NOT a subvolume of unify. Also
> about the schedulers complaining about the not enough space, all the
> schedulers now uses % based min-disk-size, (option
> alu.limits.min-disk-size10 #don't consider the node for scheduling if
> the free disk space is less
> than 10%).
>
> About the 'ls' failing, i may need little more info. First, which patch are
> you using. And, an output of 'bt' after doing gdb glusterfs -c /core.<pid>.
>
> Regards,
> Amar
>
> On 6/21/07, Dale Dude <dale at oc3networks.com> wrote:
>>
>> How do I use the namespace option correctly? In the docs/examples I
>> think your using it wrong.
>>
>> In cluster-client.vol you have volumes client1,client2,client3
>> configured and this line in cluster/unify:
>> option namespace client1 # this will not be storage child of unify.
>> subvolumes client1 client2 client3
>>
>> I get the impression that namespace shouldnt be configured as a
>> subvolume of unify. If I use a namespace of a volume that is a subvolume
>> of unify it complains as such. If I create a volume on the server
>> specifically for the namespace (i.e. 1 storage/posix called
>> volumenamespace) and in the client config use that then it doesnt
>> complain anymore. But I cant even ls on the mounted volume. I get this
>> debug found below. A 'df -h' looks correct (glusterfs 5.9T 400M 5.9T
>> 1% /volumes).
>>
>> Btw, I have to use the ALU scheduler because any other scheduler keeps
>> saying that there isnt enough space on any of the "bricks".
>>
>>
>> DEBUG WHEN DOING AN 'ls -l /volumes':
>> ==> glusterfsd.log <==
>> 2007-06-20 15:19:15 D [server-protocol.c:5536:open_file_cleanup_fn]
>> protocol/server: force releaseing file 0x8052580
>> 2007-06-20 15:19:15 D [server-protocol.c:5583:server_protocol_cleanup]
>> protocol/server: cleaned up xl_private of 0x8051cb8
>> 2007-06-20 15:19:15 C [tcp.c:82:tcp_disconnect] transport/tcp:
>> clusterfs: connection disconnected
>> 2007-06-20 15:19:15 D [tcp-server.c:230:gf_transport_fini] tcp/server:
>> destroying transport object for 127.0.0.1:1023 (fd=7)
>> 2007-06-20 15:19:15 D [server-protocol.c:5536:open_file_cleanup_fn]
>> protocol/server: force releaseing file 0x8052c70
>> 2007-06-20 15:19:15 D [server-protocol.c:5583:server_protocol_cleanup]
>> protocol/server: cleaned up xl_private of 0x8052248
>> 2007-06-20 15:19:15 C [tcp.c:82:tcp_disconnect] transport/tcp:
>> clusterfs: connection disconnected
>> 2007-06-20 15:19:15 D [tcp-server.c:230:gf_transport_fini] tcp/server:
>> destroying transport object for 127.0.0.1:1021 (fd=8)
>> 2007-06-20 15:19:15 D [server-protocol.c:5536:open_file_cleanup_fn]
>> protocol/server: force releaseing file 0x8052b38
>> 2007-06-20 15:19:15 D [server-protocol.c:5583:server_protocol_cleanup]
>> protocol/server: cleaned up xl_private of 0x8051fc8
>> 2007-06-20 15:19:15 C [tcp.c:82:tcp_disconnect] transport/tcp:
>> clusterfs: connection disconnected
>> 2007-06-20 15:19:15 D [tcp-server.c:230:gf_transport_fini] tcp/server:
>> destroying transport object for 127.0.0.1:1022 (fd=5)
>>
>> ==> glusterfs.log <==
>> 2007-06-20 15:19:15 D [fuse-bridge.c:403:fuse_getattr] glusterfs-fuse:
>> GETATTR 1 ()
>> 2007-06-20 15:19:15 C [common-utils.c:205:gf_print_trace]
>> debug-backtrace: Got signal (11), printing backtrace
>> 2007-06-20 15:19:15 C [common-utils.c:207:gf_print_trace]
>> debug-backtrace: /lib/libglusterfs.so.0(gf_print_trace+0x2d) [0xb7fcc1bd]
>> 2007-06-20 15:19:15 C [common-utils.c:207:gf_print_trace]
>> debug-backtrace: [0xffffe420]
>> 2007-06-20 15:19:15 C [common-utils.c:207:gf_print_trace]
>> debug-backtrace: /lib/glusterfs/1.3.0-pre5/xlator/cluster/unify.so
>> [0xb762e325]
>> 2007-06-20 15:19:15 C [common-utils.c:207:gf_print_trace]
>> debug-backtrace: /lib/glusterfs/1.3.0-pre5/xlator/protocol/client.so
>> [0xb763de0d]
>> 2007-06-20 15:19:15 C [common-utils.c:207:gf_print_trace]
>> debug-backtrace:
>> /lib/glusterfs/1.3.0-pre5/xlator/protocol/client.so(notify+0x892)
>> [0xb7640102]
>> 2007-06-20 15:19:15 C [common-utils.c:207:gf_print_trace]
>> debug-backtrace: /lib/libglusterfs.so.0(transport_notify+0x37)
>> [0xb7fcd707]
>> 2007-06-20 15:19:15 C [common-utils.c:207:gf_print_trace]
>> debug-backtrace: /lib/libglusterfs.so.0(sys_epoll_iteration+0xd9)
>> [0xb7fce229]
>> 2007-06-20 15:19:15 C [common-utils.c:207:gf_print_trace]
>> debug-backtrace: /lib/libglusterfs.so.0(poll_iteration+0x1d) [0xb7fcd7dd]
>> 2007-06-20 15:19:15 C [common-utils.c:207:gf_print_trace]
>> debug-backtrace: [glusterfs] [0x804a160]
>> 2007-06-20 15:19:15 C [common-utils.c:207:gf_print_trace]
>> debug-backtrace: /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xdc)
>> [0xb7e65ebc]
>> 2007-06-20 15:19:15 C [common-utils.c:207:gf_print_trace]
>> debug-backtrace: [glusterfs] [0x8049d61]
>>
>>
>>
>>
>>
>> ================================
>> glusterfs-client.vol:
>> volume clusterfs1
>> type storage/posix
>> option directory /volume1
>> end-volume
>>
>> #######
>>
>> volume clusterfs2
>> type storage/posix
>> option directory /volume2
>> end-volume
>>
>> #######
>>
>> volume volumenamespace
>> type storage/posix
>> option directory /volume.namespace
>> end-volume
>>
>> ###
>>
>> volume clusterfs
>> type protocol/server
>> option transport-type tcp/server
>> subvolumes clusterfs1 clusterfs2 volumenamespace
>> option auth.ip.clusterfs1.allow *
>> option auth.ip.clusterfs2.allow *
>> option auth.ip.volumenamespace.allow *
>> end-volume
>> root at mayorsmoney:/etc/glusterfs# cat glusterfs-client.vol
>> volume server1
>> type protocol/client
>> option transport-type tcp/client # for TCP/IP transport
>> option remote-host 127.0.0.1 # IP address of the remote
>> brick
>> option remote-subvolume volumenamespace
>> end-volume
>>
>> volume server1vol1
>> type protocol/client
>> option transport-type tcp/client # for TCP/IP transport
>> option remote-host 127.0.0.1 # IP address of the remote
>> brick
>> option remote-subvolume clusterfs1
>> end-volume
>>
>> volume server1vol2
>> type protocol/client
>> option transport-type tcp/client # for TCP/IP transport
>> option remote-host 127.0.0.1 # IP address of the remote
>> brick
>> option remote-subvolume clusterfs2
>> end-volume
>>
>> ###################
>>
>> volume bricks
>> type cluster/unify
>> option namespace server1
>> option readdir-force-success on # ignore failed mounts
>> subvolumes server1vol1 server1vol2
>>
>> ### ** ALU Scheduler Option **
>> option scheduler alu
>> #option lock-node clusterfs # first child will be
>> lock-node by default
>> option alu.limits.min-free-disk 128GB
>> option alu.limits.max-open-files 10000
>> option alu.order
>> disk-usage:read-usage:write-usage:open-files-usage:disk-speed-usage
>> option alu.disk-usage.entry-threshold 2GB
>> option alu.disk-usage.exit-threshold 128MB
>> option alu.open-files-usage.entry-threshold 1024
>> option alu.open-files-usage.exit-threshold 32
>> option alu.read-usage.entry-threshold 20 #%
>> option alu.read-usage.exit-threshold 4 #%
>> option alu.write-usage.entry-threshold 20 #%
>> option alu.write-usage.exit-threshold 4 #%
>> option alu.disk-speed-usage.entry-threshold 0 # DO NOT SET IT. SPEED
>> IS CONSTANT!!!.
>> option alu.disk-speed-usage.exit-threshold 0 # DO NOT SET IT. SPEED IS
>> CONSTANT!!!.
>> option alu.stat-refresh.interval 10sec
>> option alu.stat-refresh.num-file-create 10
>> end-volume
>>
>>
>>
>> =====================================
>> glusterfs-server.vol:
>> volume clusterfs1
>> type storage/posix
>> option directory /volume1
>> end-volume
>>
>> #######
>>
>> volume clusterfs2
>> type storage/posix
>> option directory /volume2
>> end-volume
>>
>> #######
>>
>> volume volumenamespace
>> type storage/posix
>> option directory /volume.namespace
>> end-volume
>>
>> ###
>>
>> volume clusterfs
>> type protocol/server
>> option transport-type tcp/server
>> subvolumes clusterfs1 clusterfs2 volumenamespace
>> option auth.ip.clusterfs1.allow *
>> option auth.ip.clusterfs2.allow *
>> option auth.ip.volumenamespace.allow *
>> end-volume
>>
>>
>>
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at nongnu.org
>> http://lists.nongnu.org/mailman/listinfo/gluster-devel
>>
>
>
>
More information about the Gluster-devel
mailing list