[Gluster-devel] glusterfs--mainline--3.0 patch-556
Steve
steeeeeveee at gmx.net
Thu Nov 6 15:20:00 UTC 2008
It's reproducable. As soon as I try to read the GlusterFS mounted share I get the error.
GCC information (that's there where I run the client and the server. On the other server I use gcc 3.4.6):
--------------------
nemesis ~ # gcc -v
Using built-in specs.
Target: i686-pc-linux-gnu
Configured with: /var/tmp/portage/sys-devel/gcc-4.3.2/work/gcc-4.3.2/configure --prefix=/usr --bindir=/usr/i686-pc-linux-gnu/gcc-bin/4.3.2 --includedir=/usr/lib/gcc/i686-pc-linux-gnu/4.3.2/include --datadir=/usr/share/gcc-data/i686-pc-linux-gnu/4.3.2 --mandir=/usr/share/gcc-data/i686-pc-linux-gnu/4.3.2/man --infodir=/usr/share/gcc-data/i686-pc-linux-gnu/4.3.2/info --with-gxx-include-dir=/usr/lib/gcc/i686-pc-linux-gnu/4.3.2/include/g++-v4 --host=i686-pc-linux-gnu --build=i686-pc-linux-gnu --disable-altivec --enable-nls --without-included-gettext --with-system-zlib --disable-checking --disable-werror --enable-secureplt --disable-multilib --enable-libmudflap --disable-libssp --enable-cld --disable-libgcj --with-arch=i686 --enable-languages=c,c++,treelang --enable-shared --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu --with-bugurl=http://bugs.gentoo.org/ --with-pkgversion='Gentoo 4.3.2 p1.0'
Thread model: posix
gcc version 4.3.2 (Gentoo 4.3.2 p1.0)
nemesis ~ #
--------------------
Starting SERVER:
--------------------
2008-11-06 15:38:58 D [glusterfs.c:291:_get_specfp] glusterfs: loading volume specfile /etc/glusterfs/glusterfs-server.vol
Version : glusterfs 1.4.0pre7 built on Nov 6 2008 14:23:21
TLA Revision : glusterfs--mainline--3.0--patch-561
Starting Time: 2008-11-06 15:38:58
Command line : /usr/sbin/glusterfsd -N -l/dev/stdout -L DEBUG -f /etc/glusterfs/glusterfs-server.vol
given volume specfile
+-----
1: ##############################################
2: ### GlusterFS Server Volume Specification ##
3: ### NEMESIS ##
4: ##############################################
5:
6: # dataspace on local
7: volume gfs-ds
8: type storage/posix # POSIX FS translator
9: option directory /local/gfs-brick001 # Export this directoy
10: end-volume
11:
12: # posix locks on local
13: volume gfs-ds-locks
14: type features/posix-locks
15: subvolumes gfs-ds
16: option mandatory on # Enables mandatory locking on all files
17: end-volume
18:
19: # dataspace on remote
20: volume gfs-remote-ds
21: type protocol/client
22: option transport-type tcp/client # For TCP/IP transport
23: option remote-host 192.168.0.115 # IP address of the remote storage
24: option remote-subvolume gfs-ds-locks
25: option transport-timeout 10 # Value in seconds; it should be set relatively low
26: end-volume
27:
28: # automatic file replication translator for dataspace
29: #volume gfs-ds-afr
30: volume gfs
31: type cluster/afr
32: subvolumes gfs-ds-locks gfs-remote-ds # Local and remote dataspaces
33: end-volume
34:
35: # the actual exported volume
36: #volume gfs
37: # type performance/io-threads
38: # option thread-count 8 # Deault is 1
39: # option cache-size 64MB # Default is 64MB
40: # subvolumes gfs-ds-afr
41: #end-volume
42:
43: # server declaration
44: volume server
45: type protocol/server
46: option transport-type tcp/server # For TCP/IP transport
47: subvolumes gfs
48: # storage network access only
49: option auth.addr.gfs-ds-locks.allow 192.168.0.*,127.0.0.1
50: option auth.addr.gfs.allow 192.168.0.*
51: end-volume
+-----
2008-11-06 15:38:58 D [spec.y:178:new_section] parser: New node for 'gfs-ds'
2008-11-06 15:38:58 D [xlator.c:367:xlator_set_type] xlator: attempt to load file /usr/lib/glusterfs/1.4.0pre7/xlator/storage/posix.so
2008-11-06 15:38:58 D [spec.y:202:section_type] parser: Type:gfs-ds:storage/posix
2008-11-06 15:38:58 D [spec.y:268:section_option] parser: Option:gfs-ds:directory:/local/gfs-brick001
2008-11-06 15:38:58 D [spec.y:350:section_end] parser: end:gfs-ds
2008-11-06 15:38:58 D [spec.y:178:new_section] parser: New node for 'gfs-ds-locks'
2008-11-06 15:38:58 D [xlator.c:367:xlator_set_type] xlator: attempt to load file /usr/lib/glusterfs/1.4.0pre7/xlator/features/posix-locks.so
2008-11-06 15:38:58 D [xlator.c:407:xlator_set_type] xlator: dlsym(notify) on /usr/lib/glusterfs/1.4.0pre7/xlator/features/posix-locks.so: undefined symbol: notify -- neglecting
2008-11-06 15:38:58 D [spec.y:202:section_type] parser: Type:gfs-ds-locks:features/posix-locks
2008-11-06 15:38:58 D [spec.y:335:section_sub] parser: child:gfs-ds-locks->gfs-ds
2008-11-06 15:38:58 D [spec.y:268:section_option] parser: Option:gfs-ds-locks:mandatory:on
2008-11-06 15:38:58 D [spec.y:350:section_end] parser: end:gfs-ds-locks
2008-11-06 15:38:58 D [spec.y:178:new_section] parser: New node for 'gfs-remote-ds'
2008-11-06 15:38:58 D [xlator.c:367:xlator_set_type] xlator: attempt to load file /usr/lib/glusterfs/1.4.0pre7/xlator/protocol/client.so
2008-11-06 15:38:58 D [spec.y:202:section_type] parser: Type:gfs-remote-ds:protocol/client
2008-11-06 15:38:58 D [spec.y:268:section_option] parser: Option:gfs-remote-ds:transport-type:tcp/client
2008-11-06 15:38:58 D [spec.y:268:section_option] parser: Option:gfs-remote-ds:remote-host:192.168.0.115
2008-11-06 15:38:58 D [spec.y:268:section_option] parser: Option:gfs-remote-ds:remote-subvolume:gfs-ds-locks
2008-11-06 15:38:58 D [spec.y:268:section_option] parser: Option:gfs-remote-ds:transport-timeout:10
2008-11-06 15:38:58 D [spec.y:350:section_end] parser: end:gfs-remote-ds
2008-11-06 15:38:58 D [spec.y:178:new_section] parser: New node for 'gfs'
2008-11-06 15:38:58 D [xlator.c:367:xlator_set_type] xlator: attempt to load file /usr/lib/glusterfs/1.4.0pre7/xlator/cluster/afr.so
2008-11-06 15:38:58 D [spec.y:202:section_type] parser: Type:gfs:cluster/afr
2008-11-06 15:38:58 D [spec.y:335:section_sub] parser: child:gfs->gfs-ds-locks
2008-11-06 15:38:58 D [spec.y:335:section_sub] parser: child:gfs->gfs-remote-ds
2008-11-06 15:38:58 D [spec.y:350:section_end] parser: end:gfs
2008-11-06 15:38:58 D [spec.y:178:new_section] parser: New node for 'server'
2008-11-06 15:38:58 D [xlator.c:367:xlator_set_type] xlator: attempt to load file /usr/lib/glusterfs/1.4.0pre7/xlator/protocol/server.so
2008-11-06 15:38:58 D [spec.y:202:section_type] parser: Type:server:protocol/server
2008-11-06 15:38:58 D [spec.y:268:section_option] parser: Option:server:transport-type:tcp/server
2008-11-06 15:38:58 D [spec.y:335:section_sub] parser: child:server->gfs
2008-11-06 15:38:58 D [spec.y:268:section_option] parser: Option:server:auth.addr.gfs-ds-locks.allow:192.168.0.*,127.0.0.1
2008-11-06 15:38:58 D [spec.y:268:section_option] parser: Option:server:auth.addr.gfs.allow:192.168.0.*
2008-11-06 15:38:58 D [spec.y:350:section_end] parser: end:server
2008-11-06 15:38:58 D [glusterfs.c:804:main] glusterfs: running in pid 24152
2008-11-06 15:38:58 D [transport.c:104:transport_load] transport: attempt to load file /usr/lib/glusterfs/1.4.0pre7/transport/socket.so
2008-11-06 15:38:58 D [server-protocol.c:7130:init] server: defaulting limits.transaction-size to 4194304
2008-11-06 15:38:58 D [xlator.c:491:xlator_init_rec] gfs-ds: Initialization done
2008-11-06 15:38:58 D [xlator.c:491:xlator_init_rec] gfs-ds-locks: Initialization done
2008-11-06 15:38:58 D [client-protocol.c:4918:init] gfs-remote-ds: setting transport-timeout to 10
2008-11-06 15:38:58 D [transport.c:104:transport_load] transport: attempt to load file /usr/lib/glusterfs/1.4.0pre7/transport/socket.so
2008-11-06 15:38:58 D [client-protocol.c:4962:init] gfs-remote-ds: defaulting limits.transaction-size to 268435456
2008-11-06 15:38:58 D [xlator.c:491:xlator_init_rec] gfs-remote-ds: Initialization done
2008-11-06 15:38:58 D [client-protocol.c:5194:notify] gfs-remote-ds: got GF_EVENT_PARENT_UP, attempting connect on transport
2008-11-06 15:38:58 D [client-protocol.c:5194:notify] gfs-remote-ds: got GF_EVENT_PARENT_UP, attempting connect on transport
2008-11-06 15:38:58 D [client-protocol.c:4610:client_protocol_reconnect] gfs-remote-ds: attempting reconnect
2008-11-06 15:38:58 D [name.c:182:af_inet_client_get_remote_sockaddr] gfs-remote-ds: option remote-port missing in volume gfs-remote-ds. Defaulting to 6996
2008-11-06 15:38:58 D [common-utils.c:213:gf_resolve_ip6] resolver: DNS cache not present, freshly probing hostname: 192.168.0.115
2008-11-06 15:38:58 D [common-utils.c:250:gf_resolve_ip6] resolver: returning ip-192.168.0.115 (port-6996) for hostname: 192.168.0.115 and port: 6996
2008-11-06 15:38:58 D [client-protocol.c:5231:notify] gfs-remote-ds: got GF_EVENT_CHILD_UP
2008-11-06 15:38:58 D [socket.c:924:socket_connect] gfs-remote-ds: connect () called on transport already connected
2008-11-06 15:38:58 D [client-protocol.c:4549:client_setvolume_cbk] gfs-remote-ds: SETVOLUME on remote-host succeeded
2008-11-06 15:38:59 D [client-protocol.c:4616:client_protocol_reconnect] gfs-remote-ds: breaking reconnect chain
2008-11-06 15:39:09 D [addr.c:166:gf_auth] gfs: allowed = "192.168.0.*", received addr = "192.168.0.145"
2008-11-06 15:39:09 D [server-protocol.c:6406:mop_setvolume] server: accepted client from 192.168.0.145:1022
2008-11-06 15:39:09 D [server-protocol.c:6440:mop_setvolume] server: creating inode table with lru_limit=1024, xlator=gfs
2008-11-06 15:39:09 D [inode.c:934:inode_table_new] gfs: creating new inode table with lru_limit=1024
2008-11-06 15:39:09 D [inode.c:443:__inode_create] gfs/inode: create inode(0)
2008-11-06 15:39:19 D [inode.c:268:__inode_activate] gfs/inode: activating inode(1), lru=0/1024 active=1 purge=0
2008-11-06 15:39:19 D [afr.c:335:afr_lookup_cbk] gfs: scaling inode 1 to 3
*** glibc detected *** /usr/sbin/glusterfsd: free(): invalid pointer: 0x08059f4e ***
--------------------
Starting client:
--------------------
2008-11-06 15:39:09 D [glusterfs.c:291:_get_specfp] glusterfs: loading volume specfile /etc/glusterfs/glusterfs-client.vol
Version : glusterfs 1.4.0pre7 built on Nov 6 2008 14:23:21
TLA Revision : glusterfs--mainline--3.0--patch-561
Starting Time: 2008-11-06 15:39:09
Command line : /usr/sbin/glusterfs -N -l/dev/stdout -L DEBUG -f /etc/glusterfs/glusterfs-client.vol /home/vmail/
given volume specfile
+-----
1: #############################################
2: ## GlusterFS Client Volume Specification ##
3: #############################################
4:
5: # the exported volume to mount
6: volume cluster
7: type protocol/client
8: option transport-type tcp/client
9: option remote-host gfs-vmail001.vunet.local # RRDNS
10: option remote-subvolume gfs # Exported volume
11: option transport-timeout 10 # Value in seconds, should be relatively low
12: end-volume
13:
14: # performance block for cluster (Write Behind Translator)
15: #volume writeback
16: # type performance/write-behind
17: # option aggregate-size 1MB # Default is 0bytes
18: # option window-size 3MB # Default is 0bytes
19: # option flush-behind on # Default is 'off'
20: # subvolumes cluster
21: #end-volume
22:
23: # performance block for cluster (Read Ahead Translator)
24: #volume readahead
25: # type performance/read-ahead
26: # option page-size 65KB # 256KB is the default option
27: # option page-count 16 # 2 is default option
28: # option force-atime-update off # Default is off
29: # subvolumes writeback
30: #end-volume
+-----
2008-11-06 15:39:09 D [spec.y:178:new_section] parser: New node for 'cluster'
2008-11-06 15:39:09 D [xlator.c:367:xlator_set_type] xlator: attempt to load file /usr/lib/glusterfs/1.4.0pre7/xlator/protocol/client.so
2008-11-06 15:39:09 D [spec.y:202:section_type] parser: Type:cluster:protocol/client
2008-11-06 15:39:09 D [spec.y:268:section_option] parser: Option:cluster:transport-type:tcp/client
2008-11-06 15:39:09 D [spec.y:268:section_option] parser: Option:cluster:remote-host:gfs-vmail001.vunet.local
2008-11-06 15:39:09 D [spec.y:268:section_option] parser: Option:cluster:remote-subvolume:gfs
2008-11-06 15:39:09 D [spec.y:268:section_option] parser: Option:cluster:transport-timeout:10
2008-11-06 15:39:09 D [spec.y:350:section_end] parser: end:cluster
2008-11-06 15:39:09 D [xlator.c:367:xlator_set_type] xlator: attempt to load file /usr/lib/glusterfs/1.4.0pre7/xlator/mount/fuse.so
2008-11-06 15:39:09 D [glusterfs.c:804:main] glusterfs: running in pid 24155
2008-11-06 15:39:09 D [fuse-options.c:149:fuse_options_validate] fuse-options: using mount-point = /home/vmail/
2008-11-06 15:39:09 D [fuse-options.c:156:fuse_options_validate] fuse-options: using attr-timeout = 1
2008-11-06 15:39:09 D [fuse-options.c:168:fuse_options_validate] fuse-options: using entry-timeout = 1
2008-11-06 15:39:09 D [fuse-options.c:180:fuse_options_validate] fuse-options: using direct-io-mode = 1
2008-11-06 15:39:09 D [client-protocol.c:4918:init] cluster: setting transport-timeout to 10
2008-11-06 15:39:09 D [transport.c:104:transport_load] transport: attempt to load file /usr/lib/glusterfs/1.4.0pre7/transport/socket.so
2008-11-06 15:39:09 D [client-protocol.c:4962:init] cluster: defaulting limits.transaction-size to 268435456
2008-11-06 15:39:09 D [client-protocol.c:5194:notify] cluster: got GF_EVENT_PARENT_UP, attempting connect on transport
2008-11-06 15:39:09 D [inode.c:934:inode_table_new] fuse: creating new inode table with lru_limit=0
2008-11-06 15:39:09 D [inode.c:443:__inode_create] fuse/inode: create inode(0)
2008-11-06 15:39:09 D [client-protocol.c:5194:notify] cluster: got GF_EVENT_PARENT_UP, attempting connect on transport
2008-11-06 15:39:09 D [client-protocol.c:4610:client_protocol_reconnect] cluster: attempting reconnect
2008-11-06 15:39:09 D [name.c:182:af_inet_client_get_remote_sockaddr] cluster: option remote-port missing in volume cluster. Defaulting to 6996
2008-11-06 15:39:09 D [common-utils.c:213:gf_resolve_ip6] resolver: DNS cache not present, freshly probing hostname: gfs-vmail001.vunet.local
2008-11-06 15:39:09 D [common-utils.c:250:gf_resolve_ip6] resolver: returning ip-192.168.0.145 (port-6996) for hostname: gfs-vmail001.vunet.local and port: 6996
2008-11-06 15:39:09 D [common-utils.c:270:gf_resolve_ip6] resolver: next DNS query will return: ip-192.168.0.115 port-6996
2008-11-06 15:39:09 D [client-protocol.c:5231:notify] cluster: got GF_EVENT_CHILD_UP
2008-11-06 15:39:09 D [socket.c:924:socket_connect] cluster: connect () called on transport already connected
2008-11-06 15:39:09 D [client-protocol.c:4549:client_setvolume_cbk] cluster: SETVOLUME on remote-host succeeded
2008-11-06 15:39:10 D [client-protocol.c:4616:client_protocol_reconnect] cluster: breaking reconnect chain
2008-11-06 15:39:19 D [inode.c:268:__inode_activate] fuse/inode: activating inode(1), lru=0/0 active=1 purge=0
2008-11-06 15:39:19 D [fuse-bridge.c:334:fuse_entry_cbk] glusterfs-fuse: 2: LOOKUP() / => 1
2008-11-06 15:39:19 D [fuse-bridge.c:2137:fuse_getxattr] glusterfs-fuse: 3: GETXATTR //1 (system.posix_acl_access)
2008-11-06 15:39:19 D [fuse-bridge.c:1987:fuse_xattr_cbk] glusterfs-fuse: 3: GETXATTR() / => 50
2008-11-06 15:39:19 D [fuse-bridge.c:2137:fuse_getxattr] glusterfs-fuse: 4: GETXATTR //1 (system.posix_acl_default)
2008-11-06 15:39:39 E [client-protocol.c:240:call_bail] cluster: activating bail-out. pending frames = 1. last sent = 2008-11-06 15:39:19. last received = 2008-11-06 15:39:19. transport-timeout = 10
2008-11-06 15:39:39 C [client-protocol.c:247:call_bail] cluster: bailing transport
2008-11-06 15:39:39 D [socket.c:183:__socket_disconnect] cluster: shutdown() returned 0. setting connection state to -1
2008-11-06 15:39:39 D [socket.c:93:__socket_rwv] cluster: EOF from peer 192.168.0.145:6996
2008-11-06 15:39:39 D [socket.c:568:socket_proto_state_machine] cluster: socket read failed (Transport endpoint is not connected) in state 1 (192.168.0.145:6996)
2008-11-06 15:39:39 D [client-protocol.c:4636:protocol_client_cleanup] cluster: cleaning up state in transport object 0x8056218
2008-11-06 15:39:39 E [client-protocol.c:4691:protocol_client_cleanup] cluster: forced unwinding frame type(1) op(GETXATTR) reply=@0x8090800
2008-11-06 15:39:39 E [fuse-bridge.c:2093:fuse_xattr_cbk] glusterfs-fuse: 4: GETXATTR() / => -1 (Transport endpoint is not connected)
2008-11-06 15:39:39 E [socket.c:1187:socket_submit] cluster: transport not connected to submit (priv->connected = 255)
2008-11-06 15:39:39 D [inode.c:443:__inode_create] fuse/inode: create inode(0)
2008-11-06 15:39:39 D [inode.c:268:__inode_activate] fuse/inode: activating inode(0), lru=0/0 active=2 purge=0
2008-11-06 15:39:39 E [fuse-bridge.c:364:fuse_entry_cbk] glusterfs-fuse: 5: LOOKUP() / => -1 (Transport endpoint is not connected)
2008-11-06 15:39:39 D [inode.c:311:__inode_retire] fuse/inode: retiring inode(0) lru=0/0 active=1 purge=1
2008-11-06 15:39:39 D [client-protocol.c:4610:client_protocol_reconnect] cluster: attempting reconnect
2008-11-06 15:39:39 D [name.c:182:af_inet_client_get_remote_sockaddr] cluster: option remote-port missing in volume cluster. Defaulting to 6996
2008-11-06 15:39:39 D [common-utils.c:250:gf_resolve_ip6] resolver: returning ip-192.168.0.115 (port-6996) for hostname: gfs-vmail001.vunet.local and port: 6996
2008-11-06 15:39:39 D [client-protocol.c:5231:notify] cluster: got GF_EVENT_CHILD_UP
2008-11-06 15:39:39 D [socket.c:924:socket_connect] cluster: connect () called on transport already connected
2008-11-06 15:39:39 D [client-protocol.c:4549:client_setvolume_cbk] cluster: SETVOLUME on remote-host succeeded
2008-11-06 15:39:40 D [client-protocol.c:4616:client_protocol_reconnect] cluster: breaking reconnect chain
2008-11-06 15:40:42 W [glusterfs.c:548:cleanup_and_exit] glusterfs: shutting down
2008-11-06 15:40:42 W [fuse-bridge.c:2685:fini] fuse: unmounting /home/vmail/
2008-11-06 15:40:42 W [glusterfs.c:548:cleanup_and_exit] glusterfs: shutting down
2008-11-06 15:40:42 D [glusterfs.c:569:cleanup_and_exit] glusterfs: no graph present
2008-11-06 15:40:42 D [dict.c:353:dict_destroy] dict: @this=(nil)
2008-11-06 15:40:42 D [dict.c:353:dict_destroy] dict: @this=(nil)
--------------------
I used FUSE 2.8.0_pre1 but switched back to FUSE 2.7.4. But this did not help. Same error even after recompiling GlusterFS after the fresh install of FUSE 2.7.4.
I did not have those issues (the crashing stuff) with GlusterFS 1.3.x. I had other issues (with locking files etc) and that is the reason I started to look at the TLA version since I have read in a bug report that a certain patchlevel fixes that particular issue with locking. Well... and now I have other issues and my old issue is still not fixed.
Let me know if you need more info from me.
// Steve
-------- Original-Nachricht --------
> Datum: Thu, 6 Nov 2008 19:22:44 +0530
> Von: "Vikas Gorur" <vikas at zresearch.com>
> An: Steve <steeeeeveee at gmx.net>
> CC: gluster-devel at nongnu.org
> Betreff: Re: [Gluster-devel] glusterfs--mainline--3.0 patch-556
> 2008/11/6 Steve <steeeeeveee at gmx.net>:
> > Hallo Raghavendra
> >
> > Now I get this, wenn I try to read a GlusterFS mounted filesystem:
> > ----
> > 2008-11-06 14:13:31 D [inode.c:268:__inode_activate] gfs/inode:
> activating inode(1), lru=0/1024 active=1 purge=0
> > 2008-11-06 14:13:31 D [afr.c:335:afr_lookup_cbk] gfs: scaling inode 1 to
> 3
> > *** glibc detected *** /usr/sbin/glusterfsd: free(): invalid pointer:
> 0x08091e5e ***
> > ----
> >
> > Do you need more information? What information?
>
> Hi Steve,
>
> Is it easy to reproduce? If so, can you give us your spec files and steps
> to
> reproduce it?
>
> If it is not reproducible, it would help us if you had a core file and
> could get
> a backtrace.
>
> Vikas
> --
> Engineer - Z Research
> http://gluster.com/
--
Psssst! Schon vom neuen GMX MultiMessenger gehört? Der kann`s mit allen: http://www.gmx.net/de/go/multimessenger
More information about the Gluster-devel
mailing list