[Gluster-devel] Core from gNFS process
Jiffin Tony Thottan
jthottan at redhat.com
Thu Jan 14 08:58:56 UTC 2016
Hi,
The core generated when encryption xlator is enabled
[2016-01-14 08:13:15.740835] E [crypt.c:4298:master_set_master_vol_key]
0-test1-crypt: FATAL: missing master key
[2016-01-14 08:13:15.740859] E [MSGID: 101019]
[xlator.c:429:xlator_init] 0-test1-crypt: Initialization of volume
'test1-crypt' failed, review your volfile again
[2016-01-14 08:13:15.740890] E [MSGID: 101066]
[graph.c:324:glusterfs_graph_init] 0-test1-crypt: initializing
translator failed
[2016-01-14 08:13:15.740904] E [MSGID: 101176]
[graph.c:670:glusterfs_graph_activate] 0-graph: init failed
[2016-01-14 08:13:15.741676] W [glusterfsd.c:1231:cleanup_and_exit]
(-->/usr/sbin/glusterfs(mgmt_getspec_cbk+0x307) [0x40d287]
-->/usr/sbin/glusterfs(glusterfs_process_volfp+0x117) [0x4086c7]
-->/usr/sbin/glusterfs(cleanup_and_exit+0x4d) [0x407e1d] ) 0-: received
signum (0), shutting down
With regards,
Jiffin
On 14/01/16 12:28, Raghavendra Talur wrote:
> Hi Jiffin and Soumya,
>
> Ravishankar told me about core generated by gNFS process
> during ./tests/bugs/snapshot/bug-1140162-file-snapshot-features-encrypt-opts-validation.t.
>
>
> Here is console output:
> https://build.gluster.org/job/rackspace-regression-2GB-triggered/17525/console
>
> And here is the backtrace for convenience
>
> (gdb) thread apply all bt
>
> Thread 9 (LWP 12499):
> #0 0x00007f622f4fda0e in pthread_cond_timedwait@@GLIBC_2.3.2 () from
> ./lib64/libpthread.so.0
> #1 0x00007f6230258a61 in syncenv_task (proc=0x7f621c0332f0)
> at
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/syncop.c:603
> #2 0x00007f6230258d08 in syncenv_processor (thdata=0x7f621c0332f0)
> at
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/syncop.c:695
> #3 0x00007f622f4f9a51 in start_thread () from ./lib64/libpthread.so.0
> #4 0x00007f622ee6393d in clone () from ./lib64/libc.so.6
>
> Thread 8 (LWP 12497):
> #0 0x00007f622edc2e2c in vfprintf () from ./lib64/libc.so.6
> #1 0x00007f622edea752 in vsnprintf () from ./lib64/libc.so.6
> #2 0x00007f6230243f70 in gf_vasprintf (string_ptr=0x7f6220a66ba8,
> format=0x7f62302aeacd "[%s] %s [%s:%d:%s] %d-%s: ",
> arg=0x7f6220a66a70)
> at
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/mem-pool.c:219
> #3 0x00007f62302440ad in gf_asprintf (string_ptr=0x7f6220a66ba8,
> format=0x7f62302aeacd "[%s] %s [%s:%d:%s] %d-%s: ")
> at
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/mem-pool.c:239
> #4 0x00007f623021d387 in _gf_log (domain=0x7f621c00cde0 "d_exit+0x87)
> [0x407cdf]",
> file=0x7f622272b468 <error: Cannot access memory at address
> 0x7f622272b468>,
> function=0x7f622272d130 <error: Cannot access memory at address
> 0x7f622272d130>, line=2895,
> level=GF_LOG_INFO, fmt=0x7f622272c690 <error: Cannot access memory
> at address 0x7f622272c690>)
> at
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/logging.c:2216
> #5 0x00007f6222725d99 in ?? ()
> #6 0x0000000000000005 in ?? ()
> #7 0x0000000000000000 in ?? ()
>
> Thread 7 (LWP 12460):
> #0 0x00007f622f4fda0e in pthread_cond_timedwait@@GLIBC_2.3.2 () from
> ./lib64/libpthread.so.0
> #1 0x00007f6230258a61 in syncenv_task (proc=0x241b210)
> at
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/syncop.c:603
> #2 0x00007f6230258d08 in syncenv_processor (thdata=0x241b210)
> at
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/syncop.c:695
> #3 0x00007f622f4f9a51 in start_thread () from ./lib64/libpthread.so.0
> #4 0x00007f622ee6393d in clone () from ./lib64/libc.so.6
>
> Thread 6 (LWP 12476):
> #0 0x00007f622f5002e4 in __lll_lock_wait () from ./lib64/libpthread.so.0
> #1 0x00007f622f4fb588 in _L_lock_854 () from ./lib64/libpthread.so.0
> #2 0x00007f622f4fb457 in pthread_mutex_lock () from
> ./lib64/libpthread.so.0
> #3 0x00007f623021ca6c in _gf_msg (domain=0x4117ef <error: Cannot
> access memory at address 0x4117ef>,
> ---Type <return> to continue, or q <return> to quit---
> file=0x411468 <error: Cannot access memory at address 0x411468>,
> function=0x4125d0 <__FUNCTION__.18918> <error: Cannot access
> memory at address 0x4125d0>, line=1231,
> level=GF_LOG_WARNING, errnum=0, trace=1, msgid=100032,
> fmt=0x411bb0 <error: Cannot access memory at address 0x411bb0>)
> at
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/logging.c:2055
> #4 0x0000000000407cdf in cleanup_and_exit (signum=0)
> at
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/glusterfsd/src/glusterfsd.c:1231
> #5 0x0000000000409ee4 in glusterfs_process_volfp (ctx=0x23f6010,
> fp=0x7f621c001400)
> at
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/glusterfsd/src/glusterfsd.c:2202
> #6 0x000000000040e71d in mgmt_getspec_cbk (req=0x7f621c001a4c,
> iov=0x7f621c001a8c, count=1,
> myframe=0x7f621c00135c)
> at
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/glusterfsd/src/glusterfsd-mgmt.c:1640
> #7 0x00007f622ffe242a in rpc_clnt_handle_reply (clnt=0x244e450,
> pollin=0x7f621c0026c0)
> at
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/rpc/rpc-lib/src/rpc-clnt.c:759
> #8 0x00007f622ffe28c8 in rpc_clnt_notify (trans=0x244e8d0,
> mydata=0x244e480,
> event=RPC_TRANSPORT_MSG_RECEIVED, data=0x7f621c0026c0)
> at
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/rpc/rpc-lib/src/rpc-clnt.c:900
> #9 0x00007f622ffdeb5a in rpc_transport_notify (this=0x244e8d0,
> event=RPC_TRANSPORT_MSG_RECEIVED,
> data=0x7f621c0026c0)
> at
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/rpc/rpc-lib/src/rpc-transport.c:541
> #10 0x00007f62257d0dcb in socket_event_poll_in (this=0x244e8d0)
> at
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/rpc/rpc-transport/socket/src/socket.c:2231
> #11 0x00007f62257d1321 in socket_event_handler (fd=9, idx=1,
> data=0x244e8d0, poll_in=1, poll_out=0,
> poll_err=0)
> at
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/rpc/rpc-transport/socket/src/socket.c:2344
> #12 0x00007f623027a1c0 in event_dispatch_epoll_handler
> (event_pool=0x2414ce0, event=0x7f62241a8e70)
> at
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/event-epoll.c:571
> #13 0x00007f623027a5ae in event_dispatch_epoll_worker (data=0x244faa0)
> at
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/event-epoll.c:674
> #14 0x00007f622f4f9a51 in start_thread () from ./lib64/libpthread.so.0
> #15 0x00007f622ee6393d in clone () from ./lib64/libc.so.6
>
> Thread 5 (LWP 12461):
> #0 0x00007f622f4fda0e in pthread_cond_timedwait@@GLIBC_2.3.2 () from
> ./lib64/libpthread.so.0
> #1 0x00007f6230258a61 in syncenv_task (proc=0x241b5d0)
> at
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/syncop.c:603
> #2 0x00007f6230258d08 in syncenv_processor (thdata=0x241b5d0)
> at
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/syncop.c:695
> #3 0x00007f622f4f9a51 in start_thread () from ./lib64/libpthread.so.0
> ---Type <return> to continue, or q <return> to quit---
> #4 0x00007f622ee6393d in clone () from ./lib64/libc.so.6
>
> Thread 4 (LWP 12459):
> #0 0x00007f622f501535 in sigwait () from ./lib64/libpthread.so.0
> #1 0x000000000040983d in glusterfs_sigwaiter (arg=0x7ffd0a272f50) at
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/glusterfsd/src/glusterfsd.c:2001
> #2 0x00007f622f4f9a51 in start_thread () from ./lib64/libpthread.so.0
> #3 0x00007f622ee6393d in clone () from ./lib64/libc.so.6
>
> Thread 3 (LWP 12457):
> #0 0x00007f622f4fa2ad in pthread_join () from ./lib64/libpthread.so.0
> #1 0x00007f623027a7d6 in event_dispatch_epoll (event_pool=0x2414ce0)
> at
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/event-epoll.c:758
> #2 0x00007f6230243082 in event_dispatch (event_pool=0x2414ce0) at
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/event.c:124
> #3 0x000000000040a3aa in main (argc=11, argv=0x7ffd0a2741b8) at
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/glusterfsd/src/glusterfsd.c:2345
>
> Thread 2 (LWP 12458):
> #0 0x00007f622f500fbd in nanosleep () from ./lib64/libpthread.so.0
> #1 0x00007f623022814f in gf_timer_proc (ctx=0x23f6010) at
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/timer.c:188
> #2 0x00007f622f4f9a51 in start_thread () from ./lib64/libpthread.so.0
> #3 0x00007f622ee6393d in clone () from ./lib64/libc.so.6
>
> Thread 1 (LWP 12496):
> #0 0x00007f62235a2867 in ?? () from ./lib64/libgcc_s.so.1
> #1 0x00007f62235a3119 in _Unwind_Backtrace () from ./lib64/libgcc_s.so.1
> #2 0x00007f622ee79936 in backtrace () from ./lib64/libc.so.6
> #3 0x00007f623021a143 in _gf_msg_backtrace_nomem (level=GF_LOG_ALERT,
> stacksize=200)
> at
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/logging.c:1090
> #4 0x00007f623021ff08 in gf_print_trace (signum=11, ctx=0x23f6010) at
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/common-utils.c:740
> #5 0x00000000004098d6 in glusterfsd_print_trace (signum=11) at
> /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/glusterfsd/src/glusterfsd.c:2033
> #6 <signal handler called>
> #7 0x00007f62224ea5ed in ?? ()
> #8 0x0000000000000000 in ?? ()
>
More information about the Gluster-devel
mailing list