[Gluster-devel] AFR over NUFA with 1.3.5

Mon Oct 15 07:08:42 UTC 2007

Thierry,
AFR over unify doesn't work yet. However it is in our TODO list.
Thanks
Krishna

On 10/15/07, Thierry EXCOFFIER <exco at bat710.univ-lyon1.fr> wrote:
> Hello.
>
> Does AFR can work over UNIFY ?
>
> I tried with NUFA and RANDOM scheduler but there is
> a core dump when starting the 'glusterfs' mount.
>
> I attached the log with backtrace and the configuration file.
>
> A simple AFR on two bricks works perfectly for me.
>
>
> Thanks to all the glusterfs developpers for this great work.
> Thierry Excoffier.
> --
> Thierry.EXCOFFIER@@liris.univ-lyon1.fr//   GO: Oriental Strategy Game
> LIRIS/UFR Informatique, bat710       //DYLAN: Programming Language
> Universite Claude Bernard LYON 1  __// UNIX: Standard Operating System
> 43 Bd du 11 novembre 1918       //
> 69622 VILLEURBANNE, FRANCE     //WWW: http://www710.univ-lyon1.fr/~exco
>
> 2007-10-14 22:35:55 D [glusterfs.c:131:get_spec_fp] glusterfs: loading spec from /etc/glusterfs/glusterfs-client.vol
> 2007-10-14 22:35:55 W [fuse-bridge.c:2104:fuse_transport_notify] glusterfs-fuse: Ignoring notify event 4
> 2007-10-14 22:35:55 D [spec.y:116:new_section] libglusterfs/parser: New node for 'namespace'
> 2007-10-14 22:35:55 D [spec.y:132:section_type] libglusterfs/parser: Type:namespace:storage/posix
> 2007-10-14 22:35:55 D [xlator.c:102:xlator_set_type] libglusterfs/xlator: attempt to load type storage/posix
> 2007-10-14 22:35:55 D [xlator.c:109:xlator_set_type] libglusterfs/xlator: attempt to load file //lib/glusterfs/1.3.5/xlator/storage/posix.so
> 2007-10-14 22:35:55 D [spec.y:152:section_option] libglusterfs/parser: Option:namespace:directory:/tmp/namespace
> 2007-10-14 22:35:55 D [spec.y:216:section_end] libglusterfs/parser: end:namespace
> 2007-10-14 22:35:55 D [spec.y:116:new_section] libglusterfs/parser: New node for 'big_brick'
> 2007-10-14 22:35:55 D [spec.y:132:section_type] libglusterfs/parser: Type:big_brick:protocol/client
> 2007-10-14 22:35:55 D [xlator.c:102:xlator_set_type] libglusterfs/xlator: attempt to load type protocol/client
> 2007-10-14 22:35:55 D [xlator.c:109:xlator_set_type] libglusterfs/xlator: attempt to load file //lib/glusterfs/1.3.5/xlator/protocol/client.so
> 2007-10-14 22:35:55 D [spec.y:152:section_option] libglusterfs/parser: Option:big_brick:transport-type:tcp/client
> 2007-10-14 22:35:55 D [spec.y:152:section_option] libglusterfs/parser: Option:big_brick:remote-host:192.168.0.8
> 2007-10-14 22:35:55 D [spec.y:152:section_option] libglusterfs/parser: Option:big_brick:remote-subvolume:brick
> 2007-10-14 22:35:55 D [spec.y:152:section_option] libglusterfs/parser: Option:big_brick:transport-timeout:5
> 2007-10-14 22:35:55 D [spec.y:216:section_end] libglusterfs/parser: end:big_brick
> 2007-10-14 22:35:55 D [spec.y:116:new_section] libglusterfs/parser: New node for 'small_brick_1'
> 2007-10-14 22:35:55 D [spec.y:132:section_type] libglusterfs/parser: Type:small_brick_1:protocol/client
> 2007-10-14 22:35:55 D [xlator.c:102:xlator_set_type] libglusterfs/xlator: attempt to load type protocol/client
> 2007-10-14 22:35:55 D [xlator.c:109:xlator_set_type] libglusterfs/xlator: attempt to load file //lib/glusterfs/1.3.5/xlator/protocol/client.so
> 2007-10-14 22:35:55 D [spec.y:152:section_option] libglusterfs/parser: Option:small_brick_1:transport-type:tcp/client
> 2007-10-14 22:35:55 D [spec.y:152:section_option] libglusterfs/parser: Option:small_brick_1:remote-host:192.168.0.7
> 2007-10-14 22:35:55 D [spec.y:152:section_option] libglusterfs/parser: Option:small_brick_1:remote-subvolume:brick
> 2007-10-14 22:35:55 D [spec.y:152:section_option] libglusterfs/parser: Option:small_brick_1:transport-timeout:5
> 2007-10-14 22:35:55 D [spec.y:216:section_end] libglusterfs/parser: end:small_brick_1
> 2007-10-14 22:35:55 D [spec.y:116:new_section] libglusterfs/parser: New node for 'small_brick_2'
> 2007-10-14 22:35:55 D [spec.y:132:section_type] libglusterfs/parser: Type:small_brick_2:protocol/client
> 2007-10-14 22:35:55 D [xlator.c:102:xlator_set_type] libglusterfs/xlator: attempt to load type protocol/client
> 2007-10-14 22:35:55 D [xlator.c:109:xlator_set_type] libglusterfs/xlator: attempt to load file //lib/glusterfs/1.3.5/xlator/protocol/client.so
> 2007-10-14 22:35:55 D [spec.y:152:section_option] libglusterfs/parser: Option:small_brick_2:transport-type:tcp/client
> 2007-10-14 22:35:55 D [spec.y:152:section_option] libglusterfs/parser: Option:small_brick_2:remote-host:192.168.0.1
> 2007-10-14 22:35:55 D [spec.y:152:section_option] libglusterfs/parser: Option:small_brick_2:remote-subvolume:brick
> 2007-10-14 22:35:55 D [spec.y:152:section_option] libglusterfs/parser: Option:small_brick_2:transport-timeout:5
> 2007-10-14 22:35:55 D [spec.y:216:section_end] libglusterfs/parser: end:small_brick_2
> 2007-10-14 22:35:55 D [spec.y:116:new_section] libglusterfs/parser: New node for 'nufa'
> 2007-10-14 22:35:55 D [spec.y:132:section_type] libglusterfs/parser: Type:nufa:cluster/unify
> 2007-10-14 22:35:55 D [xlator.c:102:xlator_set_type] libglusterfs/xlator: attempt to load type cluster/unify
> 2007-10-14 22:35:55 D [xlator.c:109:xlator_set_type] libglusterfs/xlator: attempt to load file //lib/glusterfs/1.3.5/xlator/cluster/unify.so
> 2007-10-14 22:35:55 D [spec.y:201:section_sub] liglusterfs/parser: child:nufa->small_brick_1
> 2007-10-14 22:35:55 D [spec.y:201:section_sub] liglusterfs/parser: child:nufa->small_brick_2
> 2007-10-14 22:35:55 D [spec.y:152:section_option] libglusterfs/parser: Option:nufa:scheduler:nufa
> 2007-10-14 22:35:55 D [spec.y:152:section_option] libglusterfs/parser: Option:nufa:nufa.local-volume-name:small_brick_2
> 2007-10-14 22:35:55 D [spec.y:152:section_option] libglusterfs/parser: Option:nufa:nufa.limits.min-free-disk:5%
> 2007-10-14 22:35:55 D [spec.y:152:section_option] libglusterfs/parser: Option:nufa:namespace:namespace
> 2007-10-14 22:35:55 D [spec.y:216:section_end] libglusterfs/parser: end:nufa
> 2007-10-14 22:35:55 W [inode.c:1099:inode_table_new] fuse: creating new inode table with lru_limit=1024, sizeof(inode_t)=96
> 2007-10-14 22:35:55 D [inode.c:559:__create_inode] fuse/inode: create inode(1)
> 2007-10-14 22:35:55 D [inode.c:351:__active_inode] fuse/inode: activating inode(1), lru=0/1024
> 2007-10-14 22:35:55 D [client-protocol.c:4549:init] small_brick_1: missing 'inode-lru-limit'. defaulting to 1000
> 2007-10-14 22:35:55 D [client-protocol.c:4562:init] small_brick_1: setting transport-timeout to 5
> 2007-10-14 22:35:55 D [transport.c:83:transport_load] libglusterfs/transport: attempt to load type tcp/client
> 2007-10-14 22:35:55 D [transport.c:88:transport_load] libglusterfs/transport: attempt to load file //lib/glusterfs/1.3.5/transport/tcp/client.so
> 2007-10-14 22:35:55 D [client-protocol.c:4549:init] small_brick_2: missing 'inode-lru-limit'. defaulting to 1000
> 2007-10-14 22:35:55 D [client-protocol.c:4562:init] small_brick_2: setting transport-timeout to 5
> 2007-10-14 22:35:55 D [transport.c:83:transport_load] libglusterfs/transport: attempt to load type tcp/client
> 2007-10-14 22:35:55 D [transport.c:88:transport_load] libglusterfs/transport: attempt to load file //lib/glusterfs/1.3.5/transport/tcp/client.so
> 2007-10-14 22:35:55 D [unify.c:3872:init] nufa: namespace node specified as namespace
> 2007-10-14 22:35:55 D [scheduler.c:36:get_scheduler] libglusterfs/scheduler: attempt to load file nufa.so
>
> 2007-10-14 22:35:55 D [unify.c:3890:init] nufa: Child node count is 2
> 2007-10-14 22:35:55 D [client-protocol.c:4847:notify] small_brick_1: got GF_EVENT_PARENT_UP, attempting connect on transport
> 2007-10-14 22:35:55 D [client-protocol.c:4847:notify] small_brick_2: got GF_EVENT_PARENT_UP, attempting connect on transport
> 2007-10-14 22:35:55 D [client-protocol.c:4294:client_protocol_reconnect] small_brick_1: attempting reconnect
> 2007-10-14 22:35:55 D [tcp-client.c:70:tcp_connect] small_brick_1: socket fd = 4
> 2007-10-14 22:35:55 D [tcp-client.c:88:tcp_connect] small_brick_1: finalized on port `1023'
> 2007-10-14 22:35:55 D [tcp-client.c:109:tcp_connect] small_brick_1: defaulting remote-port to 6996
> 2007-10-14 22:35:55 D [common-utils.c:161:gf_resolve_ip] resolver: DNS cache not present, freshly probing hostname: 192.168.0.7
> 2007-10-14 22:35:55 D [common-utils.c:186:gf_resolve_ip] resolver: returning IP:192.168.0.7[0] for hostname: 192.168.0.7
> 2007-10-14 22:35:55 D [common-utils.c:194:gf_resolve_ip] resolver: flushing DNS cache
> 2007-10-14 22:35:55 D [tcp-client.c:142:tcp_connect] small_brick_1: connect on 4 in progress (non-blocking)
> 2007-10-14 22:35:55 D [tcp-client.c:179:tcp_connect] small_brick_1: connection on 4 still in progress - try later
> 2007-10-14 22:35:55 D [client-protocol.c:4294:client_protocol_reconnect] small_brick_2: attempting reconnect
> 2007-10-14 22:35:55 D [tcp-client.c:70:tcp_connect] small_brick_2: socket fd = 7
> 2007-10-14 22:35:55 D [tcp-client.c:88:tcp_connect] small_brick_2: finalized on port `1022'
> 2007-10-14 22:35:55 D [tcp-client.c:109:tcp_connect] small_brick_2: defaulting remote-port to 6996
> 2007-10-14 22:35:55 D [common-utils.c:161:gf_resolve_ip] resolver: DNS cache not present, freshly probing hostname: 192.168.0.1
> 2007-10-14 22:35:55 D [common-utils.c:186:gf_resolve_ip] resolver: returning IP:192.168.0.1[0] for hostname: 192.168.0.1
> 2007-10-14 22:35:55 D [common-utils.c:194:gf_resolve_ip] resolver: flushing DNS cache
> 2007-10-14 22:35:55 D [tcp-client.c:142:tcp_connect] small_brick_2: connect on 7 in progress (non-blocking)
> 2007-10-14 22:35:55 D [tcp-client.c:186:tcp_connect] small_brick_2: connection on 7 success
> 2007-10-14 22:35:55 D [client-protocol.c:4869:notify] small_brick_2: got GF_EVENT_CHILD_UP
> 2007-10-14 22:35:55 D [client-protocol.c:4627:client_protocol_handshake_reply] small_brick_2: reply frame has callid: 424242
> 2007-10-14 22:35:55 D [client-protocol.c:4661:client_protocol_handshake_reply] small_brick_2: SETVOLUME on remote-host succeeded
> 2007-10-14 22:35:56 D [client-protocol.c:4294:client_protocol_reconnect] small_brick_1: attempting reconnect
> 2007-10-14 22:35:56 D [tcp-client.c:179:tcp_connect] small_brick_1: connection on 4 still in progress - try later
> 2007-10-14 22:35:56 D [client-protocol.c:4301:client_protocol_reconnect] small_brick_2: breaking reconnect chain
> 2007-10-14 22:35:58 D [client-protocol.c:4294:client_protocol_reconnect] small_brick_1: attempting reconnect
> 2007-10-14 22:35:58 E [tcp-client.c:171:tcp_connect] small_brick_1: non-blocking connect() returned: 113 (No route to host)
> 2007-10-14 22:36:00 D [tcp-client.c:70:tcp_connect] small_brick_1: socket fd = 4
> 2007-10-14 22:36:00 D [tcp-client.c:88:tcp_connect] small_brick_1: finalized on port `1023'
> 2007-10-14 22:36:00 D [tcp-client.c:109:tcp_connect] small_brick_1: defaulting remote-port to 6996
> 2007-10-14 22:36:00 D [common-utils.c:161:gf_resolve_ip] resolver: DNS cache not present, freshly probing hostname: 192.168.0.7
> 2007-10-14 22:36:00 D [common-utils.c:186:gf_resolve_ip] resolver: returning IP:192.168.0.7[0] for hostname: 192.168.0.7
> 2007-10-14 22:36:00 D [common-utils.c:194:gf_resolve_ip] resolver: flushing DNS cache
> 2007-10-14 22:36:00 D [tcp-client.c:142:tcp_connect] small_brick_1: connect on 4 in progress (non-blocking)
> 2007-10-14 22:36:01 D [tcp-client.c:179:tcp_connect] small_brick_1: connection on 4 still in progress - try later
> 2007-10-14 22:36:01 W [client-protocol.c:344:client_protocol_xfer] small_brick_1: not connected at the moment to submit frame type(0) op(34)
> 2007-10-14 22:36:01 W [unify-self-heal.c:261:unify_sh_readdir_cbk] nufa: found mismatch in mode/uid/gid for .
> 2007-10-14 22:36:01 D [client-protocol.c:4294:client_protocol_reconnect] small_brick_1: attempting reconnect
> 2007-10-14 22:36:01 D [tcp-client.c:179:tcp_connect] small_brick_1: connection on 4 still in progress - try later
> 2007-10-14 22:36:06 D [client-protocol.c:4294:client_protocol_reconnect] small_brick_1: attempting reconnect
> 2007-10-14 22:36:06 E [tcp-client.c:171:tcp_connect] small_brick_1: non-blocking connect() returned: 113 (No route to host)
> 2007-10-14 22:36:14 D [client-protocol.c:4294:client_protocol_reconnect] small_brick_1: attempting reconnect
> 2007-10-14 22:36:14 D [tcp-client.c:70:tcp_connect] small_brick_1: socket fd = 4
> 2007-10-14 22:36:14 D [tcp-client.c:88:tcp_connect] small_brick_1: finalized on port `1023'
> 2007-10-14 22:36:14 D [tcp-client.c:109:tcp_connect] small_brick_1: defaulting remote-port to 6996
> 2007-10-14 22:36:14 D [common-utils.c:161:gf_resolve_ip] resolver: DNS cache not present, freshly probing hostname: 192.168.0.7
> 2007-10-14 22:36:14 D [common-utils.c:186:gf_resolve_ip] resolver: returning IP:192.168.0.7[0] for hostname: 192.168.0.7
> 2007-10-14 22:36:14 D [common-utils.c:194:gf_resolve_ip] resolver: flushing DNS cache
> 2007-10-14 22:36:14 D [tcp-client.c:142:tcp_connect] small_brick_1: connect on 4 in progress (non-blocking)
> 2007-10-14 22:36:14 D [tcp-client.c:179:tcp_connect] small_brick_1: connection on 4 still in progress - try later
> 2007-10-14 22:36:27 D [client-protocol.c:4294:client_protocol_reconnect] small_brick_1: attempting reconnect
> 2007-10-14 22:36:27 E [tcp-client.c:171:tcp_connect] small_brick_1: non-blocking connect() returned: 113 (No route to host)
> 2007-10-14 22:36:48 D [client-protocol.c:4294:client_protocol_reconnect] small_brick_1: attempting reconnect
> 2007-10-14 22:36:48 D [tcp-client.c:70:tcp_connect] small_brick_1: socket fd = 4
> 2007-10-14 22:36:48 D [tcp-client.c:88:tcp_connect] small_brick_1: finalized on port `1023'
> 2007-10-14 22:36:48 D [tcp-client.c:109:tcp_connect] small_brick_1: defaulting remote-port to 6996
> 2007-10-14 22:36:48 D [common-utils.c:161:gf_resolve_ip] resolver: DNS cache not present, freshly probing hostname: 192.168.0.7
> 2007-10-14 22:36:48 D [common-utils.c:186:gf_resolve_ip] resolver: returning IP:192.168.0.7[0] for hostname: 192.168.0.7
> 2007-10-14 22:36:48 D [common-utils.c:194:gf_resolve_ip] resolver: flushing DNS cache
> 2007-10-14 22:36:48 D [tcp-client.c:142:tcp_connect] small_brick_1: connect on 4 in progress (non-blocking)
> 2007-10-14 22:36:49 D [tcp-client.c:179:tcp_connect] small_brick_1: connection on 4 still in progress - try later
> 2007-10-14 22:37:23 D [client-protocol.c:4294:client_protocol_reconnect] small_brick_1: attempting reconnect
> 2007-10-14 22:37:23 E [tcp-client.c:171:tcp_connect] small_brick_1: non-blocking connect() returned: 113 (No route to host)
> 2007-10-14 22:38:18 D [client-protocol.c:4294:client_protocol_reconnect] small_brick_1: attempting reconnect
> 2007-10-14 22:38:18 D [tcp-client.c:70:tcp_connect] small_brick_1: socket fd = 4
> 2007-10-14 22:38:18 D [tcp-client.c:88:tcp_connect] small_brick_1: finalized on port `1023'
> 2007-10-14 22:38:18 D [tcp-client.c:109:tcp_connect] small_brick_1: defaulting remote-port to 6996
> 2007-10-14 22:38:18 D [common-utils.c:161:gf_resolve_ip] resolver: DNS cache not present, freshly probing hostname: 192.168.0.7
> 2007-10-14 22:38:18 D [common-utils.c:186:gf_resolve_ip] resolver: returning IP:192.168.0.7[0] for hostname: 192.168.0.7
> 2007-10-14 22:38:18 D [common-utils.c:194:gf_resolve_ip] resolver: flushing DNS cache
> 2007-10-14 22:38:18 D [tcp-client.c:142:tcp_connect] small_brick_1: connect on 4 in progress (non-blocking)
> 2007-10-14 22:38:18 D [tcp-client.c:179:tcp_connect] small_brick_1: connection on 4 still in progress - try later
> 2007-10-14 22:39:47 D [client-protocol.c:4294:client_protocol_reconnect] small_brick_1: attempting reconnect
> 2007-10-14 22:39:47 E [tcp-client.c:171:tcp_connect] small_brick_1: non-blocking connect() returned: 113 (No route to host)
> 2007-10-14 22:42:11 D [client-protocol.c:4294:client_protocol_reconnect] small_brick_1: attempting reconnect
> 2007-10-14 22:42:11 D [tcp-client.c:70:tcp_connect] small_brick_1: socket fd = 4
> 2007-10-14 22:42:11 D [tcp-client.c:88:tcp_connect] small_brick_1: finalized on port `1023'
> 2007-10-14 22:42:11 D [tcp-client.c:109:tcp_connect] small_brick_1: defaulting remote-port to 6996
> 2007-10-14 22:42:11 D [common-utils.c:161:gf_resolve_ip] resolver: DNS cache not present, freshly probing hostname: 192.168.0.7
> 2007-10-14 22:42:11 D [common-utils.c:186:gf_resolve_ip] resolver: returning IP:192.168.0.7[0] for hostname: 192.168.0.7
> 2007-10-14 22:42:11 D [common-utils.c:194:gf_resolve_ip] resolver: flushing DNS cache
> 2007-10-14 22:42:11 D [tcp-client.c:142:tcp_connect] small_brick_1: connect on 4 in progress (non-blocking)
> 2007-10-14 22:42:11 D [tcp-client.c:179:tcp_connect] small_brick_1: connection on 4 still in progress - try later
>
>
> volume namespace
>   type storage/posix
>   option directory /tmp/namespace
> end-volume
>
> volume big_brick
>   type protocol/client
>   option transport-type tcp/client
>   option remote-host 192.168.0.8
>   option remote-subvolume brick
>   option transport-timeout 5
> end-volume
>
> volume small_brick_1
>   type protocol/client
>   option transport-type tcp/client
>   option remote-host 192.168.0.7
>   option remote-subvolume brick
>   option transport-timeout 5
> end-volume
>
> volume small_brick_2
>   type protocol/client
>   option transport-type tcp/client
>   option remote-host 192.168.0.1
>   option remote-subvolume brick
>   option transport-timeout 5
> end-volume
>
> volume nufa
>   type cluster/unify
>   subvolumes small_brick_1 small_brick_2
>   option scheduler nufa # or random
>   option nufa.local-volume-name small_brick_2
>   option nufa.limits.min-free-disk 5%
>   option namespace namespace
> end-volume
>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> http://lists.nongnu.org/mailman/listinfo/gluster-devel
>
>