Network problem? Lots of kernel page allocation failures

SUSE Linux Enterprise Server SLES Networking
1

Hi,

These message appear every second day:

[2012-12-20 07:45:51] The following is only an harmless informational message. [2012-12-20 07:45:51] Unless you get a _continuous_flood_ of these messages it means [2012-12-20 07:45:51] everything is working fine. Allocations from irqs cannot be [2012-12-20 07:45:51] perfectly reliable and the kernel is designed to handle that. [2012-12-20 07:45:51] postmaster: page allocation failure: order:1, mode:0x20 [2012-12-20 07:45:51] Pid: 5182, comm: postmaster Tainted: G X 3.0.42-0.7-default #1 [2012-12-20 07:45:51] Call Trace: [2012-12-20 07:45:51] [<ffffffff810048a5>] dump_trace+0x75/0x300 [2012-12-20 07:45:51] [<ffffffff81440073>] dump_stack+0x69/0x6f [2012-12-20 07:45:51] [<ffffffff810f6e52>] warn_alloc_failed+0x102/0x1a0 [2012-12-20 07:45:51] [<ffffffff810f8909>] __alloc_pages_slowpath+0x549/0x7f0 [2012-12-20 07:45:51] [<ffffffff810f8d99>] __alloc_pages_nodemask+0x1e9/0x200 [2012-12-20 07:45:51] [<ffffffff81139a76>] kmem_getpages+0x56/0x170 [2012-12-20 07:45:51] [<ffffffff8113a8bb>] fallback_alloc+0x19b/0x270 [2012-12-20 07:45:51] [<ffffffff8113b4b7>] kmem_cache_alloc+0x207/0x2b0 [2012-12-20 07:45:51] [<ffffffff81377772>] sk_prot_alloc+0x42/0x190 [2012-12-20 07:45:51] [<ffffffff81378d9a>] sk_clone+0x1a/0x320 [2012-12-20 07:45:51] [<ffffffff813c41a0>] inet_csk_clone+0x10/0xb0 [2012-12-20 07:45:51] [<ffffffff813dd551>] tcp_create_openreq_child+0x21/0x4b0 [2012-12-20 07:45:51] [<ffffffff813dbad7>] tcp_v4_syn_recv_sock+0x47/0x240 [2012-12-20 07:45:51] [<ffffffff813dd3bf>] tcp_check_req+0x2df/0x450 [2012-12-20 07:45:51] [<ffffffff813dc457>] tcp_v4_hnd_req+0x67/0x130 [2012-12-20 07:45:51] [<ffffffff813dc695>] tcp_v4_do_rcv+0x175/0x1e0 [2012-12-20 07:45:51] [<ffffffff8137836c>] release_sock+0x5c/0x120 [2012-12-20 07:45:51] [<ffffffff813c4e1e>] inet_csk_accept+0x8e/0x170 [2012-12-20 07:45:51] [<ffffffff813ebd05>] inet_accept+0x25/0x110 [2012-12-20 07:45:51] [<ffffffff81373087>] sys_accept4+0x177/0x260 [2012-12-20 07:45:51] [<ffffffff8144ac92>] system_call_fastpath+0x16/0x1b [2012-12-20 07:45:51] [<00007fbbb765b8a0>] 0x7fbbb765b89f

or

[2012-12-21 11:28:29] The following is only an harmless informational message. [2012-12-21 11:28:29] Unless you get a _continuous_flood_ of these messages it means [2012-12-21 11:28:29] everything is working fine. Allocations from irqs cannot be [2012-12-21 11:28:29] perfectly reliable and the kernel is designed to handle that. [2012-12-21 11:28:29] java: page allocation failure: order:1, mode:0x20 [2012-12-21 11:28:29] Pid: 11018, comm: java Tainted: G X 3.0.42-0.7-default #1 [2012-12-21 11:28:29] Call Trace: [2012-12-21 11:28:29] [<ffffffff810048a5>] dump_trace+0x75/0x300 [2012-12-21 11:28:29] [<ffffffff81440073>] dump_stack+0x69/0x6f [2012-12-21 11:28:29] [<ffffffff810f6e52>] warn_alloc_failed+0x102/0x1a0 [2012-12-21 11:28:29] [<ffffffff810f8909>] __alloc_pages_slowpath+0x549/0x7f0 [2012-12-21 11:28:29] [<ffffffff810f8d99>] __alloc_pages_nodemask+0x1e9/0x200 [2012-12-21 11:28:29] [<ffffffff81139a76>] kmem_getpages+0x56/0x170 [2012-12-21 11:28:29] [<ffffffff8113a8bb>] fallback_alloc+0x19b/0x270 [2012-12-21 11:28:29] [<ffffffff8113b4b7>] kmem_cache_alloc+0x207/0x2b0 [2012-12-21 11:28:29] [<ffffffff81377772>] sk_prot_alloc+0x42/0x190 [2012-12-21 11:28:29] [<ffffffff81378d9a>] sk_clone+0x1a/0x320 [2012-12-21 11:28:29] [<ffffffff813c41a0>] inet_csk_clone+0x10/0xb0 [2012-12-21 11:28:29] [<ffffffff813dd551>] tcp_create_openreq_child+0x21/0x4b0 [2012-12-21 11:28:29] [<ffffffff813dbad7>] tcp_v4_syn_recv_sock+0x47/0x240 [2012-12-21 11:28:29] [<ffffffff813dd3bf>] tcp_check_req+0x2df/0x450 [2012-12-21 11:28:29] [<ffffffff813dc457>] tcp_v4_hnd_req+0x67/0x130 [2012-12-21 11:28:29] [<ffffffff813dc695>] tcp_v4_do_rcv+0x175/0x1e0 [2012-12-21 11:28:29] [<ffffffff813dccbb>] tcp_v4_rcv+0x5bb/0x850 [2012-12-21 11:28:29] [<ffffffff813b99c8>] ip_local_deliver_finish+0x128/0x210 [2012-12-21 11:28:29] [<ffffffff81384db2>] __netif_receive_skb+0x622/0x670 [2012-12-21 11:28:29] [<ffffffff81384f03>] process_backlog+0x103/0x200 [2012-12-21 11:28:29] [<ffffffff8138ac1a>] net_rx_action+0x10a/0x2c0 [2012-12-21 11:28:29] [<ffffffff810604bf>] __do_softirq+0xef/0x220 [2012-12-21 11:28:29] [<ffffffff8144bebc>] call_softirq+0x1c/0x30 [2012-12-21 11:28:29] [<ffffffff810043a5>] do_softirq+0x65/0xa0 [2012-12-21 11:28:29] [<ffffffff81060e49>] local_bh_enable_ip+0x99/0xa0 [2012-12-21 11:28:29] [<ffffffff813ebe92>] inet_wait_for_connect+0xa2/0xf0 [2012-12-21 11:28:29] [<ffffffff813ebfe3>] inet_stream_connect+0x103/0x1e0 [2012-12-21 11:28:29] [<ffffffff81372a67>] sys_connect+0xe7/0xf0 [2012-12-21 11:28:29] [<ffffffff8144ac92>] system_call_fastpath+0x16/0x1b [2012-12-21 11:28:29] [<00007fe7439c7aad>] 0x7fe7439c7aac

I am using kernel 3.0.42-0.7-default #1 SMP Tue Oct 9 11:58:45 UTC 2012 (a8dc443) x86_64 x86_64 x86_64 GNU/Linux

I know it’s only a harmless informational message, but it comes up quite often…

Looks like there is something wrong in the TCP stack?

Any ideas?

Thx!

2

Have you tried getting a LAN trace to see, if TCP/IP-related, what may be
causing this?

sudo /usr/sbin/tcpdump -n -s 0 -i any -w /tmp/tcpip.cap

Good luck.

3

[QUOTE=ab;10852]Have you tried getting a LAN trace to see, if TCP/IP-related, what may be
causing this?

sudo /usr/sbin/tcpdump -n -s 0 -i any -w /tmp/tcpip.cap

Good luck.[/QUOTE]

Thanks for the suggestion. But the trace grows very fast - in a few seconds already 2 GB. So this is not feasible as I don’t know when this issue will happen again :frowning:

4

If troubleshooting isn’t feasible, or at least not worth it, perhaps live
with this “informational message” until something more obvious is wrong.

Good luck.