Re: [Gluster-devel] Сrash - 2.0.git-2009.06.16

[NovA]

Thanks for the advice. I'll try to switch to more stable GlusterFS
version when there will be critical tasks for our cluster. But for now
the cluster is rather idle, so I can test checked-out versions.
Moreover, from my previous experience the repository code was not less
stable then released one... Anyway, I hope that the backtraces
reported will help to release next GlusterFS version sooner :)

Best wishes,
  Andrey

26 June 2009 22:42 Shehjar Tikoo (shehjart at gluster.com) wrote:
> NovA wrote:
>>
>> Hi!
>>
>> GlusterFS continue crashing in my setup. :(
>
> We highly appreciate your reporting these crashes and I'll be
> looking at them soon, but if you'd really like something to work
> better, perhaps you should consider using one of the more stable
> releases like 2.0.2, or in another few days 2.0.3.
>
> Thanks
> Shehjar
>
>>
>> The client crashed again while Grid Engine attempted to create file
>> for redirected stdout (no writing), but this time backtrace is
>> different:
>> ------
>> Core was generated by `glusterfs -f /etc/glusterfs/client.vol -l
>> /var/log/glusterfs/client.log /home'.
>>
>> Program terminated with signal 11, Segmentation fault.
>>
>> #0  0x00002b978ba6656c in gf_print_trace (signum=11) at common-utils.c:381
>>
>> 381     common-utils.c: No such file or directory.
>>
>>        in common-utils.c
>>
>> (gdb) bt
>>
>> #0  0x00002b978ba6656c in gf_print_trace (signum=11) at common-utils.c:381
>>
>> #1  <signal handler called>
>>
>> #2  0x00002b978ba6656c in gf_print_trace (signum=6) at common-utils.c:381
>>
>> #3  <signal handler called>
>>
>> #4  0x00002b978c0d6b45 in raise () from /lib64/libc.so.6
>>
>> #5  0x00002b978c0d80e0 in abort () from /lib64/libc.so.6
>>
>> #6  0x00002b978c10efbb in ?? () from /lib64/libc.so.6
>>
>> #7  0x00002b978c11421d in ?? () from /lib64/libc.so.6
>>
>> #8  0x00002b978c116506 in ?? () from /lib64/libc.so.6
>>
>> #9  0x00002b978c117c0c in malloc () from /lib64/libc.so.6
>>
>> #10 0x00002b978d2a41e7 in wb_sync (frame=0x657060, file=0x653660,
>> winds=0x7fff1f27af20)
>>
>>    at write-behind.c:381
>>
>> #11 0x00002b978d2a44ff in wb_do_ops (frame=0x657060, file=0x653660,
>> winds=0x7fff1f27af20,
>>
>>    unwinds=<value optimized out>, other_requests=0x7fff1f27af00) at
>> write-behind.c:1579
>>
>> #12 0x00002b978d2a4617 in wb_process_queue (frame=0x657060,
>> file=0x653660, flush_all=0 '\0')
>>
>>    at write-behind.c:1624
>>
>> #13 0x00002b978d2a7d81 in wb_sync_cbk (frame=0x657060, cookie=<value
>> optimized out>,
>>
>>    this=<value optimized out>, op_ret=67, op_errno=0, stbuf=<value
>> optimized out>)
>>
>>    at write-behind.c:338
>>
>> #14 0x00002b978d0941e0 in iot_writev_cbk (frame=0x6570c0,
>> cookie=<value optimized out>,
>>
>>    this=<value optimized out>, op_ret=67, op_errno=0,
>> stbuf=0x7fff1f27b060) at io-threads.c:1186
>>
>> #15 0x00002b978ce7daae in dht_writev_cbk (frame=0x2aaaac01d200,
>> cookie=<value optimized out>,
>>
>>    this=<value optimized out>, op_ret=67, op_errno=0,
>> stbuf=0x7fff1f27b060) at dht-common.c:1797
>>
>> #16 0x00002b978cc6a6e9 in client_write_cbk (frame=0x2aaaac01b120,
>> hdr=<value optimized out>,
>>
>>    hdrlen=<value optimized out>, iobuf=<value optimized out>) at
>> client-protocol.c:4363
>>
>> #17 0x00002b978cc5683a in protocol_client_pollin (this=0x611d30,
>> trans=0x625c00)
>>
>>    at client-protocol.c:6230
>>
>> #18 0x00002b978cc610bc in notify (this=0x3f6f, event=<value optimized
>> out>, data=0x625c00)
>>
>>    at client-protocol.c:6274
>>
>> #19 0x00002b978ba5d183 in xlator_notify (xl=0x611d30, event=2,
>> data=0x625c00) at xlator.c:820
>>
>> #20 0x00002aaaaaaaff0b in socket_event_handler (fd=<value optimized
>> out>, idx=28, data=0x625c00,
>>
>>    poll_in=1, poll_out=0, poll_err=0) at socket.c:813
>>
>> #21 0x00002b978ba752aa in event_dispatch_epoll (event_pool=0x6094f0)
>> at event.c:804
>>
>> #22 0x0000000000403f34 in main (argc=6, argv=0x7fff1f27bfc8) at
>> glusterfsd.c:1223
>> ------
>>
>>
>> The server crashed during writing a large case file (~20Mb) by Fluent
>> CFD solver:
>> ---------
>> Core was generated by `/usr/sbin/glusterfsd -f
>> /etc/glusterfs/server.vol -l /var/log/glusterfs/server.'.
>>
>> Program terminated with signal 6, Aborted.
>>
>> #0  0x00002b22d73dbb45 in raise () from /lib64/libc.so.6
>>
>> (gdb) bt
>>
>> #0  0x00002b22d73dbb45 in raise () from /lib64/libc.so.6
>>
>> #1  0x00002b22d73dd0e0 in abort () from /lib64/libc.so.6
>>
>> #2  0x00002b22d7413fbb in ?? () from /lib64/libc.so.6
>>
>> #3  0x00002b22d741921d in ?? () from /lib64/libc.so.6
>>
>> #4  0x00002b22d741b506 in ?? () from /lib64/libc.so.6
>>
>> #5  0x00002b22d741c92b in calloc () from /lib64/libc.so.6
>>
>> #6  0x00002b22d858c14a in get_frame_for_call (trans=0x681d50,
>> hdr=0x659c30)
>>
>>    at ../../../../libglusterfs/src/stack.h:264
>>
>> #7  0x00002b22d858c496 in protocol_server_interpret (this=0x60fb20,
>> trans=0x681d50,
>>
>>    hdr_p=0x659c30 "", hdrlen=60, iobuf=0x66b320) at server-protocol.c:7481
>>
>> #8  0x00002b22d858c720 in protocol_server_pollin (this=0x60fb20,
>> trans=0x681d50)
>>
>>    at server-protocol.c:7763
>>
>> #9  0x00002b22d858c7b2 in notify (this=0x60fb20, event=<value
>> optimized out>, data=0x6)
>>
>>    at server-protocol.c:7819
>>
>> #10 0x00002b22d6d62183 in xlator_notify (xl=0x60fb20, event=2,
>> data=0x681d50) at xlator.c:820
>>
>> #11 0x00002aaaaaaaff0b in socket_event_handler (fd=<value optimized
>> out>, idx=47, data=0x681d50,
>>
>>    poll_in=1, poll_out=0, poll_err=0) at socket.c:813
>>
>> #12 0x00002b22d6d7a2aa in event_dispatch_epoll (event_pool=0x6094f0)
>> at event.c:804
>>
>> #13 0x0000000000403f34 in main (argc=7, argv=0x7fffd3f77c48) at
>> glusterfsd.c:1223
>>
>> -----------