Re: [Gluster-devel] Сrash - 2.0.git-2009.06.16

[NovA]

Hi!

GlusterFS continue crashing in my setup. :(

The client crashed again while Grid Engine attempted to create file
for redirected stdout (no writing), but this time backtrace is
different:
------
Core was generated by `glusterfs -f /etc/glusterfs/client.vol -l
/var/log/glusterfs/client.log /home'.

Program terminated with signal 11, Segmentation fault.

#0  0x00002b978ba6656c in gf_print_trace (signum=11) at common-utils.c:381

381     common-utils.c: No such file or directory.

        in common-utils.c

(gdb) bt

#0  0x00002b978ba6656c in gf_print_trace (signum=11) at common-utils.c:381

#1  <signal handler called>

#2  0x00002b978ba6656c in gf_print_trace (signum=6) at common-utils.c:381

#3  <signal handler called>

#4  0x00002b978c0d6b45 in raise () from /lib64/libc.so.6

#5  0x00002b978c0d80e0 in abort () from /lib64/libc.so.6

#6  0x00002b978c10efbb in ?? () from /lib64/libc.so.6

#7  0x00002b978c11421d in ?? () from /lib64/libc.so.6

#8  0x00002b978c116506 in ?? () from /lib64/libc.so.6

#9  0x00002b978c117c0c in malloc () from /lib64/libc.so.6

#10 0x00002b978d2a41e7 in wb_sync (frame=0x657060, file=0x653660,
winds=0x7fff1f27af20)

    at write-behind.c:381

#11 0x00002b978d2a44ff in wb_do_ops (frame=0x657060, file=0x653660,
winds=0x7fff1f27af20,

    unwinds=<value optimized out>, other_requests=0x7fff1f27af00) at
write-behind.c:1579

#12 0x00002b978d2a4617 in wb_process_queue (frame=0x657060,
file=0x653660, flush_all=0 '\0')

    at write-behind.c:1624

#13 0x00002b978d2a7d81 in wb_sync_cbk (frame=0x657060, cookie=<value
optimized out>,

    this=<value optimized out>, op_ret=67, op_errno=0, stbuf=<value
optimized out>)

    at write-behind.c:338

#14 0x00002b978d0941e0 in iot_writev_cbk (frame=0x6570c0,
cookie=<value optimized out>,

    this=<value optimized out>, op_ret=67, op_errno=0,
stbuf=0x7fff1f27b060) at io-threads.c:1186

#15 0x00002b978ce7daae in dht_writev_cbk (frame=0x2aaaac01d200,
cookie=<value optimized out>,

    this=<value optimized out>, op_ret=67, op_errno=0,
stbuf=0x7fff1f27b060) at dht-common.c:1797

#16 0x00002b978cc6a6e9 in client_write_cbk (frame=0x2aaaac01b120,
hdr=<value optimized out>,

    hdrlen=<value optimized out>, iobuf=<value optimized out>) at
client-protocol.c:4363

#17 0x00002b978cc5683a in protocol_client_pollin (this=0x611d30, trans=0x625c00)

    at client-protocol.c:6230

#18 0x00002b978cc610bc in notify (this=0x3f6f, event=<value optimized
out>, data=0x625c00)

    at client-protocol.c:6274

#19 0x00002b978ba5d183 in xlator_notify (xl=0x611d30, event=2,
data=0x625c00) at xlator.c:820

#20 0x00002aaaaaaaff0b in socket_event_handler (fd=<value optimized
out>, idx=28, data=0x625c00,

    poll_in=1, poll_out=0, poll_err=0) at socket.c:813

#21 0x00002b978ba752aa in event_dispatch_epoll (event_pool=0x6094f0)
at event.c:804

#22 0x0000000000403f34 in main (argc=6, argv=0x7fff1f27bfc8) at
glusterfsd.c:1223
------


The server crashed during writing a large case file (~20Mb) by Fluent
CFD solver:
---------
Core was generated by `/usr/sbin/glusterfsd -f
/etc/glusterfs/server.vol -l /var/log/glusterfs/server.'.

Program terminated with signal 6, Aborted.

#0  0x00002b22d73dbb45 in raise () from /lib64/libc.so.6

(gdb) bt

#0  0x00002b22d73dbb45 in raise () from /lib64/libc.so.6

#1  0x00002b22d73dd0e0 in abort () from /lib64/libc.so.6

#2  0x00002b22d7413fbb in ?? () from /lib64/libc.so.6

#3  0x00002b22d741921d in ?? () from /lib64/libc.so.6

#4  0x00002b22d741b506 in ?? () from /lib64/libc.so.6

#5  0x00002b22d741c92b in calloc () from /lib64/libc.so.6

#6  0x00002b22d858c14a in get_frame_for_call (trans=0x681d50, hdr=0x659c30)

    at ../../../../libglusterfs/src/stack.h:264

#7  0x00002b22d858c496 in protocol_server_interpret (this=0x60fb20,
trans=0x681d50,

    hdr_p=0x659c30 "", hdrlen=60, iobuf=0x66b320) at server-protocol.c:7481

#8  0x00002b22d858c720 in protocol_server_pollin (this=0x60fb20, trans=0x681d50)

    at server-protocol.c:7763

#9  0x00002b22d858c7b2 in notify (this=0x60fb20, event=<value
optimized out>, data=0x6)

    at server-protocol.c:7819

#10 0x00002b22d6d62183 in xlator_notify (xl=0x60fb20, event=2,
data=0x681d50) at xlator.c:820

#11 0x00002aaaaaaaff0b in socket_event_handler (fd=<value optimized
out>, idx=47, data=0x681d50,

    poll_in=1, poll_out=0, poll_err=0) at socket.c:813

#12 0x00002b22d6d7a2aa in event_dispatch_epoll (event_pool=0x6094f0)
at event.c:804

#13 0x0000000000403f34 in main (argc=7, argv=0x7fffd3f77c48) at
glusterfsd.c:1223

-----------

Regards,
  Andrey