[Gluster-users] Glusterfs 2.0 hangs on high load
Maris Ruskulis
maris at chown.lv
Mon Jun 1 07:40:07 UTC 2009
Shehjar Tikoo wrote:
> Ma-ris Ruskulis wrote:
>> Seems now, when autoscaling is off, glusterfs running stable, at
>> least I could not kill it with iozone.
>>
>
> Yes, autoscaling is still under consideration as a feature worth having.
> Please avoid using it for the time being.
>
> Thanks
> Shehjar
>
So if this feature is not stable, than this should be documented, for
example there: http://www.gluster.org/docs/index.php/Translators_v2.0
this will avoid lot of trouble using glusterfs.
Thanks!
>> Maris Ruskulis wrote:
>>> Thank You, for reply! As You can see from config, ping-timeout is
>>> not set - default is asumed. Now started glusterfs with 8 threads on
>>> both server and client (autoscaling switched off).
>>>
>>> Hardware:
>>> *server1:*
>>> lspci
>>> 00:00.0 Host bridge: Intel Corporation E7505 Memory Controller Hub
>>> (rev 03)
>>> 00:00.1 Class ff00: Intel Corporation E7505/E7205 Series RAS
>>> Controller (rev 03)
>>> 00:01.0 PCI bridge: Intel Corporation E7505/E7205 PCI-to-AGP Bridge
>>> (rev 03)
>>> 00:02.0 PCI bridge: Intel Corporation E7505 Hub Interface B
>>> PCI-to-PCI Bridge (rev 03)
>>> 00:02.1 Class ff00: Intel Corporation E7505 Hub Interface B
>>> PCI-to-PCI Bridge RAS Controller (rev 03)
>>> 00:1d.0 USB Controller: Intel Corporation 82801DB/DBL/DBM
>>> (ICH4/ICH4-L/ICH4-M) USB UHCI Controller #1 (rev 02)
>>> 00:1d.1 USB Controller: Intel Corporation 82801DB/DBL/DBM
>>> (ICH4/ICH4-L/ICH4-M) USB UHCI Controller #2 (rev 02)
>>> 00:1d.2 USB Controller: Intel Corporation 82801DB/DBL/DBM
>>> (ICH4/ICH4-L/ICH4-M) USB UHCI Controller #3 (rev 02)
>>> 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 82)
>>> 00:1f.0 ISA bridge: Intel Corporation 82801DB/DBL (ICH4/ICH4-L) LPC
>>> Interface Bridge (rev 02)
>>> 00:1f.1 IDE interface: Intel Corporation 82801DB (ICH4) IDE
>>> Controller (rev 02)
>>> 00:1f.3 SMBus: Intel Corporation 82801DB/DBL/DBM
>>> (ICH4/ICH4-L/ICH4-M) SMBus Controller (rev 02)
>>> 02:1c.0 PIC: Intel Corporation 82870P2 P64H2 I/OxAPIC (rev 04)
>>> 02:1d.0 PCI bridge: Intel Corporation 82870P2 P64H2 Hub PCI Bridge
>>> (rev 04)
>>> 02:1e.0 PIC: Intel Corporation 82870P2 P64H2 I/OxAPIC (rev 04)
>>> 02:1f.0 PCI bridge: Intel Corporation 82870P2 P64H2 Hub PCI Bridge
>>> (rev 04)
>>> 03:01.0 RAID bus controller: Intel Corporation RAID Controller
>>> 04:02.0 Ethernet controller: Intel Corporation 82540EM Gigabit
>>> Ethernet Controller (rev 02)
>>> 05:02.0 VGA compatible controller: ATI Technologies Inc Rage XL (rev
>>> 27)
>>> 05:03.0 Ethernet controller: Intel Corporation 82557/8/9/0/1
>>> Ethernet Pro 100 (rev 0d)
>>>
>>> cat /proc/cpuinfo
>>> processor : 0
>>> vendor_id : GenuineIntel
>>> cpu family : 15
>>> model : 2
>>> model name : Intel(R) Xeon(TM) CPU 2.40GHz
>>> stepping : 5
>>> cpu MHz : 2392.024
>>> cache size : 512 KB
>>> physical id : 0
>>> siblings : 2
>>> core id : 0
>>> cpu cores : 1
>>> apicid : 0
>>> initial apicid : 0
>>> fdiv_bug : no
>>> hlt_bug : no
>>> f00f_bug : no
>>> coma_bug : no
>>> fpu : yes
>>> fpu_exception : yes
>>> cpuid level : 2
>>> wp : yes
>>> flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca
>>> cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe pebs
>>> bts cid xtpr
>>> bogomips : 4784.04
>>> clflush size : 64
>>> power management:
>>>
>>> processor : 1
>>> vendor_id : GenuineIntel
>>> cpu family : 15
>>> model : 2
>>> model name : Intel(R) Xeon(TM) CPU 2.40GHz
>>> stepping : 5
>>> cpu MHz : 2392.024
>>> cache size : 512 KB
>>> physical id : 0
>>> siblings : 2
>>> core id : 0
>>> cpu cores : 1
>>> apicid : 1
>>> initial apicid : 1
>>> fdiv_bug : no
>>> hlt_bug : no
>>> f00f_bug : no
>>> coma_bug : no
>>> fpu : yes
>>> fpu_exception : yes
>>> cpuid level : 2
>>> wp : yes
>>> flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca
>>> cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe pebs
>>> bts cid xtpr
>>> bogomips : 4784.16
>>> clflush size : 64
>>> power management:
>>>
>>> *server2:*
>>> lspci 00:00.0 Host bridge: Intel Corporation E7520 Memory
>>> Controller Hub (rev 0c)
>>> 00:00.1 Class ff00: Intel Corporation E7525/E7520 Error Reporting
>>> Registers (rev 0c)
>>> 00:01.0 System peripheral: Intel Corporation E7520 DMA Controller
>>> (rev 0c)
>>> 00:02.0 PCI bridge: Intel Corporation E7525/E7520/E7320 PCI Express
>>> Port A (rev 0c)
>>> 00:04.0 PCI bridge: Intel Corporation E7525/E7520 PCI Express Port B
>>> (rev 0c)
>>> 00:05.0 PCI bridge: Intel Corporation E7520 PCI Express Port B1 (rev
>>> 0c)
>>> 00:06.0 PCI bridge: Intel Corporation E7520 PCI Express Port C (rev 0c)
>>> 00:1d.0 USB Controller: Intel Corporation 82801EB/ER (ICH5/ICH5R)
>>> USB UHCI Controller #1 (rev 02)
>>> 00:1d.1 USB Controller: Intel Corporation 82801EB/ER (ICH5/ICH5R)
>>> USB UHCI Controller #2 (rev 02)
>>> 00:1d.2 USB Controller: Intel Corporation 82801EB/ER (ICH5/ICH5R)
>>> USB UHCI Controller #3 (rev 02)
>>> 00:1d.7 USB Controller: Intel Corporation 82801EB/ER (ICH5/ICH5R)
>>> USB2 EHCI Controller (rev 02)
>>> 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev c2)
>>> 00:1f.0 ISA bridge: Intel Corporation 82801EB/ER (ICH5/ICH5R) LPC
>>> Interface Bridge (rev 02)
>>> 00:1f.1 IDE interface: Intel Corporation 82801EB/ER (ICH5/ICH5R) IDE
>>> Controller (rev 02)
>>> 00:1f.3 SMBus: Intel Corporation 82801EB/ER (ICH5/ICH5R) SMBus
>>> Controller (rev 02)
>>> 01:00.0 PCI bridge: Intel Corporation 6700PXH PCI Express-to-PCI
>>> Bridge A (rev 09)
>>> 01:00.1 PIC: Intel Corporation 6700/6702PXH I/OxAPIC Interrupt
>>> Controller A (rev 09)
>>> 01:00.2 PCI bridge: Intel Corporation 6700PXH PCI Express-to-PCI
>>> Bridge B (rev 09)
>>> 01:00.3 PIC: Intel Corporation 6700PXH I/OxAPIC Interrupt Controller
>>> B (rev 09)
>>> 02:03.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068
>>> PCI-X Fusion-MPT SAS (rev 01)
>>> 02:05.0 SCSI storage controller: LSI Logic / Symbios Logic 53c1030
>>> PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 08)
>>> 02:05.1 SCSI storage controller: LSI Logic / Symbios Logic 53c1030
>>> PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 08)
>>> 03:01.0 I2O: LSI Logic / Symbios Logic MegaRAID (rev 01)
>>> 05:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8050
>>> PCI-E ASF Gigabit Ethernet Controller (rev 18)
>>> 07:04.0 Ethernet controller: Intel Corporation 82541GI Gigabit
>>> Ethernet Controller (rev 05)
>>> 07:0c.0 VGA compatible controller: ATI Technologies Inc Rage XL (rev
>>> 27)
>>>
>>> cat /proc/cpuinfo
>>> processor : 0
>>> vendor_id : GenuineIntel
>>> cpu family : 15
>>> model : 4
>>> model name : Intel(R) Xeon(TM) CPU 2.80GHz
>>> stepping : 1
>>> cpu MHz : 2792.955
>>> cache size : 1024 KB
>>> physical id : 0
>>> siblings : 2
>>> core id : 0
>>> cpu cores : 1
>>> fpu : yes
>>> fpu_exception : yes
>>> cpuid level : 5
>>> wp : yes
>>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
>>> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe
>>> syscall nx lm constant_tsc pebs bts pni monitor ds_cpl cid cx16 xtpr
>>> bogomips : 5590.46
>>> clflush size : 64
>>> cache_alignment : 128
>>> address sizes : 36 bits physical, 48 bits virtual
>>> power management:
>>>
>>> processor : 1
>>> vendor_id : GenuineIntel
>>> cpu family : 15
>>> model : 4
>>> model name : Intel(R) Xeon(TM) CPU 2.80GHz
>>> stepping : 1
>>> cpu MHz : 2792.955
>>> cache size : 1024 KB
>>> physical id : 3
>>> siblings : 2
>>> core id : 0
>>> cpu cores : 1
>>> fpu : yes
>>> fpu_exception : yes
>>> cpuid level : 5
>>> wp : yes
>>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
>>> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe
>>> syscall nx lm constant_tsc pebs bts pni monitor ds_cpl cid cx16 xtpr
>>> bogomips : 5586.06
>>> clflush size : 64
>>> cache_alignment : 128
>>> address sizes : 36 bits physical, 48 bits virtual
>>> power management:
>>>
>>> processor : 2
>>> vendor_id : GenuineIntel
>>> cpu family : 15
>>> model : 4
>>> model name : Intel(R) Xeon(TM) CPU 2.80GHz
>>> stepping : 1
>>> cpu MHz : 2792.955
>>> cache size : 1024 KB
>>> physical id : 0
>>> siblings : 2
>>> core id : 0
>>> cpu cores : 1
>>> fpu : yes
>>> fpu_exception : yes
>>> cpuid level : 5
>>> wp : yes
>>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
>>> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe
>>> syscall nx lm constant_tsc pebs bts pni monitor ds_cpl cid cx16 xtpr
>>> bogomips : 5586.02
>>> clflush size : 64
>>> cache_alignment : 128
>>> address sizes : 36 bits physical, 48 bits virtual
>>> power management:
>>>
>>> processor : 3
>>> vendor_id : GenuineIntel
>>> cpu family : 15
>>> model : 4
>>> model name : Intel(R) Xeon(TM) CPU 2.80GHz
>>> stepping : 1
>>> cpu MHz : 2792.955
>>> cache size : 1024 KB
>>> physical id : 3
>>> siblings : 2
>>> core id : 0
>>> cpu cores : 1
>>> fpu : yes
>>> fpu_exception : yes
>>> cpuid level : 5
>>> wp : yes
>>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
>>> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe
>>> syscall nx lm constant_tsc pebs bts pni monitor ds_cpl cid cx16 xtpr
>>> bogomips : 5586.05
>>> clflush size : 64
>>> cache_alignment : 128
>>> address sizes : 36 bits physical, 48 bits virtual
>>> power management:
>>>
>>>
>>>
>>> jvanwanrooy at chatventure.nl wrote:
>>>> Hi Maris,
>>>>
>>>> Can you tell me something more about the hardware you use? With our
>>>> tests yesterday we had some troubles with very high load
>>>> inconjunction with autoscaling. You can try a fixed limit of
>>>> threads. What are the ping-timeout settings by the way?
>>>>
>>>> Best Regards Jasper
>>>>
>>>> Jasper van Wanrooy - Chatventure BV
>>>> Technical Manager
>>>> T: +31 (0) 6 47 248 722
>>>> E: jvanwanrooy at chatventure.nl
>>>> W: www.chatventure.nl
>>>>
>>>>
>>>> ----- Original Message -----
>>>> From: "Maris Ruskulis" <maris at chown.lv>
>>>> To: gluster-users at gluster.org
>>>> Sent: Friday, 29 May, 2009 10:11:45 GMT +01:00 Amsterdam / Berlin /
>>>> Bern / Rome / Stockholm / Vienna
>>>> Subject: Re: [Gluster-users] Glusterfs 2.0 hangs on high load
>>>>
>>>> Is there way to solve this issue?
>>>>
>>>> Maris Ruskulis wrote:
>>>>
>>>> I have same issue with same config when both nodes are x64. But
>>>> difference is that, there is no bailout messages in logs.
>>>>
>>>> Jasper van Wanrooy - Chatventure wrote:
>>>>
>>>> Hi Maris,
>>>>
>>>> I regret to hear that. I was also having problems with the
>>>> stability on 32bit platforms. Possibly you should try it on a
>>>> 64bit platform. Is that an option?
>>>>
>>>> Best Regards Jasper
>>>>
>>>>
>>>> On 28 mei 2009, at 09:36, Maris Ruskulis wrote:
>>>>
>>>> Hello!
>>>> After upgrade to version 2.0, now using 2.0.1, I'm
>>>> experiencing problems with glusterfs stability.
>>>> I'm running 2 node setup with cliet side afr, and
>>>> glusterfsd also is running on same servers. Time to time
>>>> glusterfs just hangs, i can reproduce this running iozone
>>>> benchmarking tool. I'm using patched Fuse, but same
>>>> result is with unpatched.
>>>>
>>>>
>>>> ================================================================================
>>>>
>>>> Version : glusterfs 2.0.1 built on May 27 2009
>>>> 16:04:01
>>>> TLA Revision : 5c1d9108c1529a1155963cb1911f8870a674ab5b
>>>> Starting Time: 2009-05-27 16:38:20
>>>> Command line : /usr/sbin/glusterfsd
>>>> --volfile=/etc/glusterfs/glusterfs-server.vol
>>>> --pid-file=/var/run/glusterfsd.pid
>>>> --log-file=/var/log/glusterfsd.log
>>>> PID : 31971
>>>> System name : Linux
>>>> Nodename : weeber.st-inst.lv
>>>> Kernel Release : 2.6.28-hardened-r7
>>>> Hardware Identifier: i686
>>>>
>>>> Given volfile:
>>>>
>>>> +------------------------------------------------------------------------------+
>>>>
>>>> 1: # file: /etc/glusterfs/glusterfs-server.vol
>>>> 2: volume posix
>>>> 3: type storage/posix
>>>> 4: option directory /home/export
>>>> 5: end-volume
>>>> 6:
>>>> 7: volume locks
>>>> 8: type features/locks
>>>> 9: option mandatory-locks on
>>>> 10: subvolumes posix
>>>> 11: end-volume
>>>> 12:
>>>> 13: volume brick
>>>> 14: type performance/io-threads
>>>> 15: option autoscaling on
>>>> 16: subvolumes locks
>>>> 17: end-volume
>>>> 18:
>>>> 19: volume server
>>>> 20: type protocol/server
>>>> 21: option transport-type tcp
>>>> 22: option auth.addr.brick.allow 127.0.0.1,192.168.1.*
>>>> 23: subvolumes brick
>>>> 24: end-volume
>>>>
>>>>
>>>> +------------------------------------------------------------------------------+
>>>>
>>>> [2009-05-27 16:38:20] N [glusterfsd.c:1152:main]
>>>> glusterfs: Successfully started
>>>> [2009-05-27 16:38:33] N
>>>> [server-protocol.c:7035:mop_setvolume] server: accepted
>>>> client from 192.168.1.233:1021
>>>> [2009-05-27 16:38:33] N
>>>> [server-protocol.c:7035:mop_setvolume] server: accepted
>>>> client from 192.168.1.233:1020
>>>> [2009-05-27 16:38:46] N
>>>> [server-protocol.c:7035:mop_setvolume] server: accepted
>>>> client from 192.168.1.252:1021
>>>> [2009-05-27 16:38:46] N
>>>> [server-protocol.c:7035:mop_setvolume] server: accepted
>>>> client from 192.168.1.252:1020
>>>>
>>>>
>>>> ================================================================================
>>>>
>>>> Version : glusterfs 2.0.1 built on May 27 2009
>>>> 16:04:01
>>>> TLA Revision : 5c1d9108c1529a1155963cb1911f8870a674ab5b
>>>> Starting Time: 2009-05-27 16:38:46
>>>> Command line : /usr/sbin/glusterfs -N -f
>>>> /etc/glusterfs/glusterfs-client.vol /mnt/gluster
>>>> PID : 32161
>>>> System name : Linux
>>>> Nodename : weeber.st-inst.lv
>>>> Kernel Release : 2.6.28-hardened-r7
>>>> Hardware Identifier: i686
>>>>
>>>> Given volfile:
>>>>
>>>> +------------------------------------------------------------------------------+
>>>>
>>>> 1: volume xeon
>>>> 2: type protocol/client
>>>> 3: option transport-type tcp
>>>> 4: option remote-host 192.168.1.233
>>>> 5: option remote-subvolume brick
>>>> 6: end-volume
>>>> 7:
>>>> 8: volume weeber
>>>> 9: type protocol/client
>>>> 10: option transport-type tcp
>>>> 11: option remote-host 192.168.1.252
>>>> 12: option remote-subvolume brick
>>>> 13: end-volume
>>>> 14:
>>>> 15: volume replicate
>>>> 16: type cluster/replicate
>>>> 17: subvolumes xeon weeber
>>>> 18: end-volume
>>>> 20: volume readahead
>>>> 21: type performance/read-ahead
>>>> 22: option page-size 128kB
>>>> 23: option page-count 16
>>>> 24: option force-atime-update off
>>>> 25: subvolumes replicate
>>>> 26: end-volume
>>>> 27:
>>>> 28: volume writebehind
>>>> 29: type performance/write-behind
>>>> 30: option aggregate-size 1MB
>>>> 31: option window-size 3MB
>>>> 32: option flush-behind on
>>>> 33: option enable-O_SYNC on
>>>> 34: subvolumes readahead
>>>> 35: end-volume
>>>> 36:
>>>> 37: volume iothreads
>>>> 38: type performance/io-threads
>>>> 39: option autoscaling on
>>>> 40: subvolumes writebehind
>>>> 41: end-volume
>>>> 42:
>>>> 43:
>>>> 44:
>>>> 45: #volume bricks
>>>> 46: #type cluster/distribute
>>>> 47: #option lookup-unhashed yes
>>>> 48: #option min-free-disk 20%
>>>> 49: # subvolumes weeber xeon
>>>> 50: #end-volume
>>>>
>>>>
>>>> +------------------------------------------------------------------------------+
>>>>
>>>> [2009-05-27 16:38:46] W
>>>> [xlator.c:555:validate_xlator_volume_options]
>>>> writebehind: option 'window-size' is deprecated,
>>>> preferred is 'cache-size', continuing with correction
>>>> [2009-05-27 16:38:46] W
>>>> [glusterfsd.c:455:_log_if_option_is_invalid] writebehind:
>>>> option 'aggregate-size' is not recognized
>>>> [2009-05-27 16:38:46] W
>>>> [glusterfsd.c:455:_log_if_option_is_invalid] readahead:
>>>> option 'page-size' is not recognized
>>>> [2009-05-27 16:38:46] N [glusterfsd.c:1152:main]
>>>> glusterfs: Successfully started
>>>> [2009-05-27 16:38:46] N
>>>> [client-protocol.c:5557:client_setvolume_cbk] xeon:
>>>> Connected to 192.168.1.233:6996, attached to remote
>>>> volume 'brick'.
>>>> [2009-05-27 16:38:46] N [afr.c:2190:notify] replicate:
>>>> Subvolume 'xeon' came back up; going online.
>>>> [2009-05-27 16:38:46] N
>>>> [client-protocol.c:5557:client_setvolume_cbk] xeon:
>>>> Connected to 192.168.1.233:6996, attached to remote
>>>> volume 'brick'.
>>>> [2009-05-27 16:38:46] N [afr.c:2190:notify] replicate:
>>>> Subvolume 'xeon' came back up; going online.
>>>> [2009-05-27 16:38:46] N
>>>> [client-protocol.c:5557:client_setvolume_cbk] weeber:
>>>> Connected to 192.168.1.252:6996, attached to remote
>>>> volume 'brick'.
>>>> [2009-05-27 18:46:02] E [client-protocol.c:292:call_bail]
>>>> weeber: bailing out frame LOOKUP(32) frame sent =
>>>> 2009-05-27 18:16:01. frame-timeout = 1800
>>>> [2009-05-27 19:16:09] E [client-protocol.c:292:call_bail]
>>>> weeber: bailing out frame LOOKUP(32) frame sent =
>>>> 2009-05-27 18:46:02. frame-timeout = 1800
>>>> [2009-05-27 19:46:18] E [client-protocol.c:292:call_bail]
>>>> weeber: bailing out frame OPEN(12) frame sent =
>>>> 2009-05-27 19:16:09. frame-timeout = 1800
>>>> [2009-05-27 20:16:25] E [client-protocol.c:292:call_bail]
>>>> weeber: bailing out frame LOOKUP(32) frame sent =
>>>> 2009-05-27 19:46:18. frame-timeout = 1800
>>>> [2009-05-27 20:46:34] E [client-protocol.c:292:call_bail]
>>>> weeber: bailing out frame LOOKUP(32) frame sent =
>>>> 2009-05-27 20:16:25. frame-timeout = 1800
>>>> [2009-05-27 21:16:41] E [client-protocol.c:292:call_bail]
>>>> weeber: bailing out frame OPEN(12) frame sent =
>>>> 2009-05-27 20:46:34. frame-timeout = 1800
>>>> [2009-05-27 21:47:00] E [client-protocol.c:292:call_bail]
>>>> weeber: bailing out frame LOOKUP(32) frame sent =
>>>> 2009-05-27 21:16:53. frame-timeout = 1800
>>>> [2009-05-27 22:17:07] E [client-protocol.c:292:call_bail]
>>>> weeber: bailing out frame LOOKUP(32) frame sent =
>>>> 2009-05-27 21:47:00. frame-timeout = 1800
>>>> [2009-05-27 22:47:15] E [client-protocol.c:292:call_bail]
>>>> weeber: bailing out frame OPENDIR(21) frame sent =
>>>> 2009-05-27 22:17:07. frame-timeout = 1800
>>>> [2009-05-27 23:17:23] E [client-protocol.c:292:call_bail]
>>>> weeber: bailing out frame LOOKUP(32) frame sent =
>>>> 2009-05-27 22:47:15. frame-timeout = 1800
>>>> [2009-05-27 23:47:31] E [client-protocol.c:292:call_bail]
>>>> weeber: bailing out frame OPEN(12) frame sent =
>>>> 2009-05-27 23:17:23. frame-timeout = 1800
>>>> [2009-05-28 00:17:39] E [client-protocol.c:292:call_bail]
>>>> weeber: bailing out frame LOOKUP(32) frame sent =
>>>> 2009-05-27 23:47:32. frame-timeout = 1800
>>>> [2009-05-28 00:47:47] E [client-protocol.c:292:call_bail]
>>>> weeber: bailing out frame LOOKUP(32) frame sent =
>>>> 2009-05-28 00:17:39. frame-timeout = 1800
>>>> [2009-05-28 01:17:55] E [client-protocol.c:292:call_bail]
>>>> weeber: bailing out frame OPENDIR(21) frame sent =
>>>> 2009-05-28 00:47:47. frame-timeout = 1800
>>>> [2009-05-28 01:48:03] E [client-protocol.c:292:call_bail]
>>>> weeber: bailing out frame LOOKUP(32) frame sent =
>>>> 2009-05-28 01:17:55. frame-timeout = 1800
>>>> [2009-05-28 02:18:11] E [client-protocol.c:292:call_bail]
>>>> weeber: bailing out frame OPEN(12) frame sent =
>>>> 2009-05-28 01:48:03. frame-timeout = 1800
>>>> [2009-05-28 02:48:29] E [client-protocol.c:292:call_bail]
>>>> weeber: bailing out frame LOOKUP(32) frame sent =
>>>> 2009-05-28 02:18:24. frame-timeout = 1800
>>>> [2009-05-28 03:18:37] E [client-protocol.c:292:call_bail]
>>>> weeber: bailing out frame LOOKUP(32) frame sent =
>>>> 2009-05-28 02:48:29. frame-timeout = 1800
>>>> [2009-05-28 03:48:45] E [client-protocol.c:292:call_bail]
>>>> weeber: bailing out frame LOOKUP(32) frame sent =
>>>> 2009-05-28 03:18:37. frame-timeout = 1800
>>>> [2009-05-28 04:18:53] E [client-protocol.c:292:call_bail]
>>>> weeber: bailing out frame XATTROP(40) frame sent =
>>>> 2009-05-28 03:48:45. frame-timeout = 1800
>>>> [2009-05-28 04:49:01] E [client-protocol.c:292:call_bail]
>>>> weeber: bailing out frame LOOKUP(32) frame sent =
>>>> 2009-05-28 04:18:53. frame-timeout = 1800
>>>> [2009-05-28 05:19:09] E [client-protocol.c:292:call_bail]
>>>> weeber: bailing out frame OPENDIR(21) frame sent =
>>>> 2009-05-28 04:49:01. frame-timeout = 1800
>>>> [2009-05-28 05:49:17] E [client-protocol.c:292:call_bail]
>>>> weeber: bailing out frame LOOKUP(32) frame sent =
>>>> 2009-05-28 05:19:09. frame-timeout = 1800
>>>> [2009-05-28 06:19:25] E [client-protocol.c:292:call_bail]
>>>> weeber: bailing out frame LOOKUP(32) frame sent =
>>>> 2009-05-28 05:49:17. frame-timeout = 1800
>>>> [2009-05-28 06:49:33] E [client-protocol.c:292:call_bail]
>>>> weeber: bailing out frame XATTROP(40) frame sent =
>>>> 2009-05-28 06:19:25. frame-timeout = 1800
>>>> [2009-05-28 07:19:40] E [client-protocol.c:292:call_bail]
>>>> weeber: bailing out frame LOOKUP(32) frame sent =
>>>> 2009-05-28 06:49:33. frame-timeout = 1800
>>>> [2009-05-28 07:49:48] E [client-protocol.c:292:call_bail]
>>>> weeber: bailing out frame LOOKUP(32) frame sent =
>>>> 2009-05-28 07:19:40. frame-timeout = 1800
>>>> [2009-05-28 08:19:56] E [client-protocol.c:292:call_bail]
>>>> weeber: bailing out frame LOOKUP(32) frame sent =
>>>> 2009-05-28 07:49:48. frame-timeout = 1800
>>>>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: maris.vcf
Type: text/x-vcard
Size: 206 bytes
Desc: not available
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20090601/2d3f8b6d/attachment.vcf>
More information about the Gluster-users
mailing list