[Gluster-devel] Re; System Crash

[Gareth Bult]

Hi, 

Can anyone comment on this ?? 

Jan 18 14:00:31 nodec kernel: [ 362.689029] glusterfs: page allocation failure. order:3, mode:0x4020 
Jan 18 14:00:31 nodec kernel: [ 362.689044] 
Jan 18 14:00:31 nodec kernel: [ 362.689045] Call Trace: 
Jan 18 14:00:31 nodec kernel: [ 362.689048] <IRQ> [__alloc_pages+806/848] __alloc_pages+0x326/0x350 
Jan 18 14:00:31 nodec kernel: [ 362.689092] [__slab_alloc+329/1456] __slab_alloc+0x149/0x5b0 
Jan 18 14:00:31 nodec kernel: [ 362.689108] [pskb_expand_head+66/352] pskb_expand_head+0x42/0x160 
Jan 18 14:00:31 nodec kernel: [ 362.689126] [__kmalloc+207/224] __kmalloc+0xcf/0xe0 
Jan 18 14:00:31 nodec kernel: [ 362.689139] [pskb_expand_head+66/352] pskb_expand_head+0x42/0x160 
Jan 18 14:00:31 nodec kernel: [ 362.689171] [_end+129765542/2130180472] :bridge:nf_bridge_copy_header+0x5e/0x110 
Jan 18 14:00:31 nodec kernel: [ 362.689190] [_end+129743461/2130180472] :bridge:br_dev_queue_push_xmit+0x4d/0xb0 
Jan 18 14:00:31 nodec kernel: [ 362.689211] [_end+129763180/2130180472] :bridge:br_nf_post_routing+0x144/0x240 
Jan 18 14:00:31 nodec kernel: [ 362.689235] [nf_iterate+112/160] nf_iterate+0x70/0xa0 
Jan 18 14:00:31 nodec kernel: [ 362.689259] [_end+129743384/2130180472] :bridge:br_dev_queue_push_xmit+0x0/0xb0 
Jan 18 14:00:31 nodec kernel: [ 362.689271] [nf_hook_slow+113/240] nf_hook_slow+0x71/0xf0 
Jan 18 14:00:31 nodec kernel: [ 362.689285] [_end+129743384/2130180472] :bridge:br_dev_queue_push_xmit+0x0/0xb0 
Jan 18 14:00:31 nodec kernel: [ 362.689300] [nf_hook_slow+113/240] nf_hook_slow+0x71/0xf0 
Jan 18 14:00:31 nodec kernel: [ 362.689327] [_end+129743635/2130180472] :bridge:br_forward_finish+0x4b/0x60 
Jan 18 14:00:31 nodec kernel: [ 362.689352] [_end+129763618/2130180472] :bridge:br_nf_forward_finish+0xba/0x190 
Jan 18 14:00:31 nodec kernel: [ 362.689375] [_end+129766003/2130180472] :bridge:br_nf_forward_ip+0x11b/0x1d0 
Jan 18 14:00:31 nodec kernel: [ 362.689391] [_end+129763618/2130180472] :bridge:br_nf_forward_finish+0xba/0x190 
Jan 18 14:00:31 nodec kernel: [ 362.689405] [nf_iterate+112/160] nf_iterate+0x70/0xa0 
Jan 18 14:00:31 nodec kernel: [ 362.689427] [_end+129743560/2130180472] :bridge:br_forward_finish+0x0/0x60 
Jan 18 14:00:31 nodec kernel: [ 362.689440] [nf_hook_slow+113/240] nf_hook_slow+0x71/0xf0 
Jan 18 14:00:31 nodec kernel: [ 362.689456] [_end+129743560/2130180472] :bridge:br_forward_finish+0x0/0x60 
Jan 18 14:00:31 nodec kernel: [ 362.689488] [_end+129743656/2130180472] :bridge:__br_forward+0x0/0x90 
Jan 18 14:00:31 nodec kernel: [ 362.689504] [_end+129743763/2130180472] :bridge:__br_forward+0x6b/0x90 
Jan 18 14:00:31 nodec kernel: [ 362.689516] [skb_clone+50/784] skb_clone+0x32/0x310 
Jan 18 14:00:31 nodec kernel: [ 362.689534] [_end+129743237/2130180472] :bridge:br_flood+0x8d/0x100 
Jan 18 14:00:31 nodec kernel: [ 362.689562] [_end+129746911/2130180472] :bridge:br_handle_frame_finish+0x127/0x130 
Jan 18 14:00:31 nodec kernel: [ 362.689587] [_end+129764853/2130180472] :bridge:br_nf_pre_routing_finish+0x1bd/0x3d0 
Jan 18 14:00:31 nodec kernel: [ 362.689607] [_end+129764408/2130180472] :bridge:br_nf_pre_routing_finish+0x0/0x3d0 
Jan 18 14:00:31 nodec kernel: [ 362.689622] [nf_hook_slow+113/240] nf_hook_slow+0x71/0xf0 
Jan 18 14:00:31 nodec kernel: [ 362.689638] [_end+129764408/2130180472] :bridge:br_nf_pre_routing_finish+0x0/0x3d0 
Jan 18 14:00:31 nodec kernel: [ 362.689672] [_end+129768044/2130180472] :bridge:br_nf_pre_routing+0x454/0x800 
Jan 18 14:00:31 nodec kernel: [ 362.689688] [_end+129882151/2130180472] :ip_tables:ipt_do_table+0x23f/0x380 
Jan 18 14:00:31 nodec kernel: [ 362.689706] [nf_iterate+112/160] nf_iterate+0x70/0xa0 
Jan 18 14:00:31 nodec kernel: [ 362.689727] [_end+129746616/2130180472] :bridge:br_handle_frame_finish+0x0/0x130 
Jan 18 14:00:31 nodec kernel: [ 362.689741] [nf_hook_slow+113/240] nf_hook_slow+0x71/0xf0 
Jan 18 14:00:31 nodec kernel: [ 362.689755] [_end+129746616/2130180472] :bridge:br_handle_frame_finish+0x0/0x130 
Jan 18 14:00:31 nodec kernel: [ 362.689787] [_end+129747254/2130180472] :bridge:br_handle_frame+0x14e/0x1e0 
Jan 18 14:00:31 nodec kernel: [ 362.689802] [netif_receive_skb+371/1136] netif_receive_skb+0x173/0x470 
Jan 18 14:00:31 nodec kernel: [ 362.689832] [_end+128027301/2130180472] :e1000:e1000_clean_rx_irq+0x19d/0x580 
Jan 18 14:00:31 nodec kernel: [ 362.689864] [_end+128023320/2130180472] :e1000:e1000_clean+0x270/0x320 
Jan 18 14:00:31 nodec kernel: [ 362.689874] [net_rx_action+250/688] net_rx_action+0xfa/0x2b0 
Jan 18 14:00:31 nodec kernel: [ 362.689881] [__wake_up+67/112] __wake_up+0x43/0x70 
Jan 18 14:00:31 nodec kernel: [ 362.689891] [__do_softirq+138/288] __do_softirq+0x8a/0x120 
Jan 18 14:00:31 nodec kernel: [ 362.689900] [call_softirq+28/48] call_softirq+0x1c/0x30 
Jan 18 14:00:31 nodec kernel: [ 362.689906] [do_softirq+165/256] do_softirq+0xa5/0x100 
Jan 18 14:00:31 nodec kernel: [ 362.689913] [do_IRQ+98/192] do_IRQ+0x62/0xc0 
Jan 18 14:00:31 nodec kernel: [ 362.689922] [evtchn_do_upcall+258/336] evtchn_do_upcall+0x102/0x150 
Jan 18 14:00:31 nodec kernel: [ 362.689934] [do_hypervisor_callback+30/48] do_hypervisor_callback+0x1e/0x30 
Jan 18 14:00:31 nodec kernel: [ 362.689938] <EOI> [force_evtchn_callback+10/16] force_evtchn_callback+0xa/0x10 
Jan 18 14:00:31 nodec kernel: [ 362.689958] [do_page_fault+2867/4816] do_page_fault+0xb33/0x12d0 
Jan 18 14:00:31 nodec kernel: [ 362.689971] [copy_user_generic_unrolled+131/320] copy_user_generic_unrolled+0x83/0x140 
Jan 18 14:00:31 nodec kernel: [ 362.689979] [ep_poll_callback+128/224] ep_poll_callback+0x80/0xe0 
Jan 18 14:00:31 nodec kernel: [ 362.689991] [error_exit+0/121] error_exit+0x0/0x79 
Jan 18 14:00:31 nodec kernel: [ 362.690004] [copy_user_generic_unrolled+78/320] copy_user_generic_unrolled+0x4e/0x140 
Jan 18 14:00:31 nodec kernel: [ 362.690012] [memcpy_toiovec+77/112] memcpy_toiovec+0x4d/0x70 
Jan 18 14:00:31 nodec kernel: [ 362.690020] [skb_copy_datagram_iovec+453/640] skb_copy_datagram_iovec+0x1c5/0x280 
Jan 18 14:00:31 nodec kernel: [ 362.690033] [tcp_recvmsg+2755/3232] tcp_recvmsg+0xac3/0xca0 
Jan 18 14:00:31 nodec kernel: [ 362.690073] [sock_common_recvmsg+48/80] sock_common_recvmsg+0x30/0x50 
Jan 18 14:00:31 nodec kernel: [ 362.690089] [sock_aio_read+372/384] sock_aio_read+0x174/0x180 
Jan 18 14:00:31 nodec kernel: [ 362.690125] [do_sync_read+217/288] do_sync_read+0xd9/0x120 
Jan 18 14:00:31 nodec kernel: [ 362.690139] [autoremove_wake_function+0/48] autoremove_wake_function+0x0/0x30 
Jan 18 14:00:31 nodec kernel: [ 362.690150] [do_mmap_pgoff+1560/2128] do_mmap_pgoff+0x618/0x850 
Jan 18 14:00:31 nodec kernel: [ 362.690163] [vfs_read+382/400] vfs_read+0x17e/0x190 
Jan 18 14:00:31 nodec kernel: [ 362.690170] [sys_read+83/144] sys_read+0x53/0x90 
Jan 18 14:00:31 nodec kernel: [ 362.690178] [system_call+134/139] system_call+0x86/0x8b 
Jan 18 14:00:31 nodec kernel: [ 362.690185] [system_call+0/139] system_call+0x0/0x8b 
Jan 18 14:00:31 nodec kernel: [ 362.690192] 
Jan 18 14:00:31 nodec kernel: [ 362.690194] Mem-info: 
Jan 18 14:00:31 nodec kernel: [ 362.690196] DMA per-cpu: 
Jan 18 14:00:31 nodec kernel: [ 362.690199] CPU 0: Hot: hi: 186, btch: 31 usd: 13 Cold: hi: 62, btch: 15 usd: 52 
Jan 18 14:00:31 nodec kernel: [ 362.690203] CPU 1: Hot: hi: 186, btch: 31 usd: 163 Cold: hi: 62, btch: 15 usd: 9 
Jan 18 14:00:31 nodec kernel: [ 362.690208] CPU 2: Hot: hi: 186, btch: 31 usd: 36 Cold: hi: 62, btch: 15 usd: 49 
Jan 18 14:00:31 nodec kernel: [ 362.690212] CPU 3: Hot: hi: 186, btch: 31 usd: 97 Cold: hi: 62, btch: 15 usd: 9 
Jan 18 14:00:31 nodec kernel: [ 362.690218] Active:58006 inactive:40628 dirty:1 writeback:0 unstable:0 
Jan 18 14:00:31 nodec kernel: [ 362.690220] free:272 slab:7704 mapped:2557 pagetables:675 bounce:0 
Jan 18 14:00:31 nodec kernel: [ 362.690230] DMA free:1088kB min:2896kB low:3620kB high:4344kB active:232024kB inactive:162512kB present:525200kB pages_scanned:0 all_unreclaimable? no 
Jan 18 14:00:31 nodec kernel: [ 362.690237] lowmem_reserve[]: 0 0 0 
Jan 18 14:00:31 nodec kernel: [ 362.690242] DMA: 1*4kB 0*8kB 1*16kB 0*32kB 1*64kB 0*128kB 0*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 1108kB 
Jan 18 14:00:31 nodec kernel: [ 362.690265] Swap cache: add 0, delete 0, find 0/0, race 0+0 
Jan 18 14:00:31 nodec kernel: [ 362.690269] Free swap = 2097144kB 
Jan 18 14:00:31 nodec kernel: [ 362.690272] Total swap = 2097144kB 
Jan 18 14:00:31 nodec kernel: [ 362.690277] Free swap: 2097144kB 
Jan 18 14:00:31 nodec kernel: [ 362.693502] 133120 pages of RAM 
Jan 18 14:00:31 nodec kernel: [ 362.693507] 20456 reserved pages 
Jan 18 14:00:31 nodec kernel: [ 362.693509] 31576 pages shared 
Jan 18 14:00:31 nodec kernel: [ 362.693511] 0 pages swap cached