[Gluster-devel] Error: Transport endpoint is not connected

Sun Apr 26 07:13:53 UTC 2009

Hello.

There wasn't anything interesting:

================================================================================
Version      : glusterfs 2.0.0rc8 built on Apr 20 2009 23:04:47
TLA Revision : 82394d484803e02e28441bc0b988efaaff60dd94
Starting Time: 2009-04-23 17:17:31
Command line : /usr/local/sbin/glusterfs --log-level=NORMAL 
--volfile=/etc/glusterfs/glusterfs-client.vol /
mnt/glusterfs
PID          : 3759
System name  : Linux
Nodename     : f1
Kernel Release : 2.6.29.1
Hardware Identifier: i686

Given volfile:
+------------------------------------------------------------------------------+
   1: volume g1b1
   2:    type protocol/client
   3:    option transport-type tcp/client
   4:    option remote-host 10.10.70.2
   5:    option remote-subvolume brick1
   6: end-volume
   7: volume g1b2
   8:    type protocol/client
   9:    option transport-type tcp
  10:    option remote-host 10.10.70.2
  11:    option remote-subvolume brick2
  12: end-volume
  13: volume g1b3
  14:    type protocol/client
  15:    option transport-type tcp
  16:    option remote-host 10.10.70.2
  17:    option remote-subvolume brick3
  18: end-volume
  19:
  20: volume g2b1
  21:    type protocol/client
  22:    option transport-type tcp/client
  23:    option remote-host 10.10.70.3
  24:    option remote-subvolume brick1
  25: end-volume
  26: volume g2b2
  27:    type protocol/client
  28:    option transport-type tcp
  29:    option remote-host 10.10.70.3
  30:    option remote-subvolume brick2
  31: end-volume
  32: volume g2b3
  33:    type protocol/client
  34:    option transport-type tcp
  35:    option remote-host 10.10.70.3
  36:    option remote-subvolume brick3
  37: end-volume
  38:
  39: volume replicate1
  40:    type cluster/replicate
  41:    subvolumes g1b1 g2b1
  42: end-volume
  43:
  44: volume replicate2
  45:    type cluster/replicate
  46:    subvolumes g1b2 g2b2
  47: end-volume
  48:
  49: volume replicate3
  50:    type cluster/replicate
  51:    subvolumes g1b3 g2b3
  52: end-volume
  53:
  54: volume distribute
  55:   type cluster/distribute
  56:   subvolumes replicate1 replicate2 replicate3
  57: end-volume
  58:

+------------------------------------------------------------------------------+
2009-04-23 17:17:31 N [glusterfsd.c:1152:main] glusterfs: Successfully 
started
2009-04-23 17:17:31 N [client-protocol.c:6327:client_setvolume_cbk] 
g1b1: connection and handshake succeeded
2009-04-23 17:17:31 N [afr.c:2126:notify] replicate1: subvolume g1b1 came up
2009-04-23 17:17:31 N [client-protocol.c:6327:client_setvolume_cbk] 
g1b1: connection and handshake succeeded
2009-04-23 17:17:31 N [afr.c:2126:notify] replicate1: subvolume g1b1 came up
2009-04-23 17:17:31 N [client-protocol.c:6327:client_setvolume_cbk] 
g2b1: connection and handshake succeeded
2009-04-23 17:17:31 N [afr.c:2126:notify] replicate1: subvolume g2b1 came up
2009-04-23 17:17:31 N [client-protocol.c:6327:client_setvolume_cbk] 
g1b2: connection and handshake succeeded
2009-04-23 17:17:31 N [afr.c:2126:notify] replicate2: subvolume g1b2 came up
2009-04-23 17:17:31 N [client-protocol.c:6327:client_setvolume_cbk] 
g2b1: connection and handshake succeeded
2009-04-23 17:17:31 N [afr.c:2126:notify] replicate1: subvolume g2b1 came up
2009-04-23 17:17:31 N [client-protocol.c:6327:client_setvolume_cbk] 
g1b2: connection and handshake succeeded
2009-04-23 17:17:31 N [afr.c:2126:notify] replicate2: subvolume g1b2 came up
2009-04-23 17:17:31 N [client-protocol.c:6327:client_setvolume_cbk] 
g2b2: connection and handshake succeeded
2009-04-23 17:17:31 N [afr.c:2126:notify] replicate2: subvolume g2b2 came up
2009-04-23 17:17:31 N [client-protocol.c:6327:client_setvolume_cbk] 
g2b2: connection and handshake succeeded
2009-04-23 17:17:31 N [afr.c:2126:notify] replicate2: subvolume g2b2 came up
2009-04-23 17:17:31 N [client-protocol.c:6327:client_setvolume_cbk] 
g1b3: connection and handshake succeeded
2009-04-23 17:17:31 N [afr.c:2126:notify] replicate3: subvolume g1b3 came up
2009-04-23 17:17:31 N [client-protocol.c:6327:client_setvolume_cbk] 
g2b3: connection and handshake succeeded
2009-04-23 17:17:31 N [afr.c:2126:notify] replicate3: subvolume g2b3 came up
2009-04-23 17:17:31 N [client-protocol.c:6327:client_setvolume_cbk] 
g1b3: connection and handshake succeeded
2009-04-23 17:17:31 N [afr.c:2126:notify] replicate3: subvolume g1b3 came up
2009-04-23 17:17:31 N [client-protocol.c:6327:client_setvolume_cbk] 
g2b3: connection and handshake succeeded
2009-04-23 17:17:31 N [afr.c:2126:notify] replicate3: subvolume g2b3 came up
pending frames:

patchset: 82394d484803e02e28441bc0b988efaaff60dd94
signal received: 6
configuration details:argp 1
backtrace 1
dlfcn 1
fdatasync 1
libpthread 1
llistxattr 1
setfsid 1
spinlock 1
epoll.h 1
xattr.h 1
st_atim.tv_nsec 1
package-string: glusterfs 2.0.0rc8
[0xb803c400]
/lib/i686/cmov/libc.so.6(abort+0x188)[0xb7eb3018]
/lib/i686/cmov/libc.so.6(__assert_fail+0xee)[0xb7eaa5be]
/lib/i686/cmov/libpthread.so.0(pthread_mutex_lock+0x5d4)[0xb7fe8f54]
/usr/lib/libfuse.so.2[0xb75c0425]
/usr/local/lib/glusterfs/2.0.0rc8/xlator/mount/fuse.so[0xb75d519a]
/usr/lib/libfuse.so.2[0xb75c1cf8]
/usr/lib/libfuse.so.2[0xb75c2211]
/usr/lib/libfuse.so.2(fuse_session_process+0x26)[0xb75c3bf6]
/usr/local/lib/glusterfs/2.0.0rc8/xlator/mount/fuse.so[0xb75d5f31]
/lib/i686/cmov/libpthread.so.0[0xb7fe74c0]
/lib/i686/cmov/libc.so.6(clone+0x5e)[0xb7f666de]
---------

Anyhow - it seems to be working without problems using
git version 689347f278e5acfda95a24f7804a1450043311f3

Also it seems to be working faster without any performance translators, 
however gluster is using only part of dedicated GigE link - no more than 
25% for writing and 30% for reading on client. Half that on each storage 
node. What can I do to improve this?

P.S. sorry for the late response -  your message got into junk folder.

JV

Anand Avati wrote:
> There has to be some more before this. Can you paste the full log?
> 
> Avati
> 
>> in client log is only this:
>>
>> pending frames:
>>
>> patchset: 82394d484803e02e28441bc0b988efaaff60dd94
>> signal received: 6
>> configuration details:argp 1
>> backtrace 1
>> dlfcn 1
>> fdatasync 1
>> libpthread 1
>> llistxattr 1
>> setfsid 1
>> spinlock 1
>> epoll.h 1
>> xattr.h 1
>> st_atim.tv_nsec 1
>> package-string: glusterfs 2.0.0rc8
>> [0xb8067400]
>> /lib/i686/cmov/libc.so.6(abort+0x188)[0xb7ede018]
>> /lib/i686/cmov/libc.so.6(__assert_fail+0xee)[0xb7ed55be]
>> /lib/i686/cmov/libpthread.so.0(pthread_mutex_lock+0x5d4)[0xb8013f54]
>> /usr/lib/libfuse.so.2[0xb75d3147]
>> /usr/lib/libfuse.so.2(fuse_session_process+0x26)[0xb75d4bf6]
>> /usr/local/lib/glusterfs/2.0.0rc8/xlator/mount/fuse.so[0xb75eef31]
>> /lib/i686/cmov/libpthread.so.0[0xb80124c0]
>> /lib/i686/cmov/libc.so.6(clone+0x5e)[0xb7f916de]
>> ---------