[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#610118: swapper: page allocation failure. order:0, mode:0x4020



Package: linux-image-2.6.32-5-xen-amd64
Version: 2.6.32-23~bpo50+1

Hi,

Something like this was mentioned misplaced in #592497, and about a
different network driver, so I'm filing a new bug because it should
be unrelated to both issues over there :)

I've just seen something similar with a tg3, randomly during normal
operation (after >100 days of uptime), but also on a Xen dom0.
Nothing very bad seems to have happened, the domUs haven't complained
at all, but it's suspicious.

Jan 15 13:39:10 virgo kernel: [9658533.157616] swapper: page allocation failure. order:0, mode:0x4020
Jan 15 13:39:10 virgo kernel: [9658533.157634] Pid: 0, comm: swapper Not tainted 2.6.32-bpo.5-xen-amd64 #1
Jan 15 13:39:10 virgo kernel: [9658533.157642] Call Trace:
Jan 15 13:39:10 virgo kernel: [9658533.157649]  <IRQ>  [<ffffffff810baed4>] ? __alloc_pages_nodemask+0x55b/0x5cf
Jan 15 13:39:10 virgo kernel: [9658533.157677]  [<ffffffff81295624>] ? tcp_rcv_established+0x688/0x6d9
Jan 15 13:39:10 virgo kernel: [9658533.157690]  [<ffffffff810e70d2>] ? new_slab+0x5b/0x1ca
Jan 15 13:39:10 virgo kernel: [9658533.157700]  [<ffffffff810e7431>] ? __slab_alloc+0x1f0/0x39b
Jan 15 13:39:10 virgo kernel: [9658533.157712]  [<ffffffff81258103>] ? __netdev_alloc_skb+0x29/0x43
Jan 15 13:39:10 virgo kernel: [9658533.157723]  [<ffffffff810e7e63>] ? __kmalloc_node_track_caller+0xbb/0x11b
Jan 15 13:39:10 virgo kernel: [9658533.157734]  [<ffffffff81258103>] ? __netdev_alloc_skb+0x29/0x43
Jan 15 13:39:10 virgo kernel: [9658533.157744]  [<ffffffff81257518>] ? __alloc_skb+0x69/0x15a
Jan 15 13:39:10 virgo kernel: [9658533.157754]  [<ffffffff81258103>] ? __netdev_alloc_skb+0x29/0x43
Jan 15 13:39:10 virgo kernel: [9658533.157785]  [<ffffffffa0019c25>] ? tg3_alloc_rx_skb+0xd2/0x146 [tg3]
Jan 15 13:39:10 virgo kernel: [9658533.157805]  [<ffffffffa0021292>] ? tg3_poll+0x484/0x93d [tg3]
Jan 15 13:39:10 virgo kernel: [9658533.157818]  [<ffffffff8100e5b5>] ? xen_force_evtchn_callback+0x9/0xa
Jan 15 13:39:10 virgo kernel: [9658533.157829]  [<ffffffff8100ec72>] ? check_events+0x12/0x20
Jan 15 13:39:10 virgo kernel: [9658533.157840]  [<ffffffff8125e633>] ? net_rx_action+0xae/0x1c9
Jan 15 13:39:10 virgo kernel: [9658533.157852]  [<ffffffff810548ca>] ? __do_softirq+0xdd/0x19f
Jan 15 13:39:10 virgo kernel: [9658533.157863]  [<ffffffff81012cac>] ? call_softirq+0x1c/0x30
Jan 15 13:39:10 virgo kernel: [9658533.157873]  [<ffffffff8101422b>] ? do_softirq+0x3f/0x7c
Jan 15 13:39:10 virgo kernel: [9658533.157883]  [<ffffffff81054739>] ? irq_exit+0x36/0x76
Jan 15 13:39:10 virgo kernel: [9658533.157894]  [<ffffffff811f14b1>] ? xen_evtchn_do_upcall+0x33/0x42
Jan 15 13:39:10 virgo kernel: [9658533.157905]  [<ffffffff81012cfe>] ? xen_do_hypervisor_callback+0x1e/0x30
Jan 15 13:39:10 virgo kernel: [9658533.157913]  <EOI>  [<ffffffff810093aa>] ? hypercall_page+0x3aa/0x1001
Jan 15 13:39:10 virgo kernel: [9658533.157929]  [<ffffffff810093aa>] ? hypercall_page+0x3aa/0x1001
Jan 15 13:39:10 virgo kernel: [9658533.157940]  [<ffffffff8100e633>] ? xen_safe_halt+0xc/0x15
Jan 15 13:39:10 virgo kernel: [9658533.157950]  [<ffffffff8100bf3f>] ? xen_idle+0x37/0x40
Jan 15 13:39:10 virgo kernel: [9658533.157959]  [<ffffffff81010eb1>] ? cpu_idle+0xa2/0xda
Jan 15 13:39:10 virgo kernel: [9658533.157977]  [<ffffffff81502cd1>] ? start_kernel+0x3dc/0x3e8
Jan 15 13:39:10 virgo kernel: [9658533.157987]  [<ffffffff81504c7d>] ? xen_start_kernel+0x57c/0x581
Jan 15 13:39:10 virgo kernel: [9658533.157995] Mem-Info:
Jan 15 13:39:10 virgo kernel: [9658533.158000] Node 0 DMA per-cpu:
Jan 15 13:39:10 virgo kernel: [9658533.158009] CPU    0: hi:    0, btch:   1 usd:   0
Jan 15 13:39:10 virgo kernel: [9658533.158017] CPU    1: hi:    0, btch:   1 usd:   0
Jan 15 13:39:10 virgo kernel: [9658533.158024] CPU    2: hi:    0, btch:   1 usd:   0
Jan 15 13:39:10 virgo kernel: [9658533.158031] CPU    3: hi:    0, btch:   1 usd:   0
Jan 15 13:39:10 virgo kernel: [9658533.158038] Node 0 DMA32 per-cpu:
Jan 15 13:39:10 virgo kernel: [9658533.158046] CPU    0: hi:  186, btch:  31 usd: 197
Jan 15 13:39:10 virgo kernel: [9658533.158053] CPU    1: hi:  186, btch:  31 usd: 182
Jan 15 13:39:10 virgo kernel: [9658533.158061] CPU    2: hi:  186, btch:  31 usd: 140
Jan 15 13:39:10 virgo kernel: [9658533.158068] CPU    3: hi:  186, btch:  31 usd:  85
Jan 15 13:39:10 virgo kernel: [9658533.158080] active_anon:5831 inactive_anon:8962 isolated_anon:0
Jan 15 13:39:10 virgo kernel: [9658533.158082]  active_file:36954 inactive_file:24010 isolated_file:0
Jan 15 13:39:10 virgo kernel: [9658533.158084]  unevictable:5 dirty:2821 writeback:0 unstable:0
Jan 15 13:39:10 virgo kernel: [9658533.158086]  free:753 slab_reclaimable:25777 slab_unreclaimable:3367
Jan 15 13:39:10 virgo kernel: [9658533.158088]  mapped:4481 shmem:151 pagetables:489 bounce:0
Jan 15 13:39:10 virgo kernel: [9658533.158109] Node 0 DMA free:1976kB min:80kB low:100kB high:120kB active_anon:308kB inactive_anon:512kB active_file:5712kB inactive_file:3932kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:14856kB mlocked:0kB dirty:0kB writeback:0kB mapped:164kB shmem:0kB slab_reclaimable:2432kB slab_unreclaimable:160kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jan 15 13:39:10 virgo kernel: [9658533.158147] lowmem_reserve[]: 0 489 489 489
Jan 15 13:39:10 virgo kernel: [9658533.158163] Node 0 DMA32 free:1036kB min:2788kB low:3484kB high:4180kB active_anon:23016kB inactive_anon:35336kB active_file:142104kB inactive_file:92108kB unevictable:20kB isolated(anon):0kB isolated(file):0kB present:500960kB mlocked:20kB dirty:11284kB writeback:0kB mapped:17760kB shmem:604kB slab_reclaimable:100676kB slab_unreclaimable:13308kB kernel_stack:1072kB pagetables:1956kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:234 all_unreclaimable? no
Jan 15 13:39:10 virgo kernel: [9658533.158218] lowmem_reserve[]: 0 0 0 0
Jan 15 13:39:10 virgo kernel: [9658533.158234] Node 0 DMA: 208*4kB 1*8kB 1*16kB 5*32kB 5*64kB 5*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1976kB
Jan 15 13:39:10 virgo kernel: [9658533.158271] Node 0 DMA32: 147*4kB 0*8kB 1*16kB 2*32kB 0*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1052kB
Jan 15 13:39:10 virgo kernel: [9658533.158309] 61725 total pagecache pages
Jan 15 13:39:10 virgo kernel: [9658533.158328] 559 pages in swap cache
Jan 15 13:39:10 virgo kernel: [9658533.158335] Swap cache stats: add 3779, delete 3220, find 17662478/17662698
Jan 15 13:39:10 virgo kernel: [9658533.158343] Free swap  = 968684kB
Jan 15 13:39:10 virgo kernel: [9658533.158348] Total swap = 979956kB
Jan 15 13:39:10 virgo kernel: [9658533.161600] 131072 pages RAM
Jan 15 13:39:10 virgo kernel: [9658533.161600] 20624 pages reserved
Jan 15 13:39:10 virgo kernel: [9658533.161600] 55796 pages shared
Jan 15 13:39:10 virgo kernel: [9658533.161600] 58358 pages non-shared
Jan 15 13:39:10 virgo kernel: [9658533.161600] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
Jan 15 13:39:10 virgo kernel: [9658533.161600]   cache: kmalloc-4096, object size: 4096, buffer size: 4096, default order: 3, min order: 0
Jan 15 13:39:10 virgo kernel: [9658533.161600]   node 0: slabs: 583, objs: 1213, free: 0
Jan 15 13:39:10 virgo kernel: [9658533.161600] swapper: page allocation failure. order:0, mode:0x4020
Jan 15 13:39:10 virgo kernel: [9658533.161600] Pid: 0, comm: swapper Not tainted 2.6.32-bpo.5-xen-amd64 #1
Jan 15 13:39:10 virgo kernel: [9658533.161600] Call Trace:
Jan 15 13:39:10 virgo kernel: [9658533.161600]  <IRQ>  [<ffffffff810baed4>] ? __alloc_pages_nodemask+0x55b/0x5cf
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff8100ec5f>] ? xen_restore_fl_direct_end+0x0/0x1
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff810e70d2>] ? new_slab+0x5b/0x1ca
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff810e7431>] ? __slab_alloc+0x1f0/0x39b
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81258103>] ? __netdev_alloc_skb+0x29/0x43
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff810e7e63>] ? __kmalloc_node_track_caller+0xbb/0x11b
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81258103>] ? __netdev_alloc_skb+0x29/0x43
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81257518>] ? __alloc_skb+0x69/0x15a
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81258103>] ? __netdev_alloc_skb+0x29/0x43
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffffa0019c25>] ? tg3_alloc_rx_skb+0xd2/0x146 [tg3]
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffffa0021292>] ? tg3_poll+0x484/0x93d [tg3]
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff8100e5b5>] ? xen_force_evtchn_callback+0x9/0xa
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff8100ec72>] ? check_events+0x12/0x20
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff8125e633>] ? net_rx_action+0xae/0x1c9
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff810548ca>] ? __do_softirq+0xdd/0x19f
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81012cac>] ? call_softirq+0x1c/0x30
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff8101422b>] ? do_softirq+0x3f/0x7c
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81054739>] ? irq_exit+0x36/0x76
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff811f14b1>] ? xen_evtchn_do_upcall+0x33/0x42
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81012cfe>] ? xen_do_hypervisor_callback+0x1e/0x30
Jan 15 13:39:10 virgo kernel: [9658533.161600]  <EOI>  [<ffffffff810093aa>] ? hypercall_page+0x3aa/0x1001
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff810093aa>] ? hypercall_page+0x3aa/0x1001
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff8100e633>] ? xen_safe_halt+0xc/0x15
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff8100bf3f>] ? xen_idle+0x37/0x40
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81010eb1>] ? cpu_idle+0xa2/0xda
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81502cd1>] ? start_kernel+0x3dc/0x3e8
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81504c7d>] ? xen_start_kernel+0x57c/0x581
Jan 15 13:39:10 virgo kernel: [9658533.161600] Mem-Info:
Jan 15 13:39:10 virgo kernel: [9658533.161600] Node 0 DMA per-cpu:
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    0: hi:    0, btch:   1 usd:   0
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    1: hi:    0, btch:   1 usd:   0
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    2: hi:    0, btch:   1 usd:   0
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    3: hi:    0, btch:   1 usd:   0
Jan 15 13:39:10 virgo kernel: [9658533.161600] Node 0 DMA32 per-cpu:
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    0: hi:  186, btch:  31 usd: 197
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    1: hi:  186, btch:  31 usd: 182
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    2: hi:  186, btch:  31 usd: 140
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    3: hi:  186, btch:  31 usd:  85
Jan 15 13:39:10 virgo kernel: [9658533.161600] active_anon:5831 inactive_anon:8962 isolated_anon:0
Jan 15 13:39:10 virgo kernel: [9658533.161600]  active_file:36954 inactive_file:24010 isolated_file:0
Jan 15 13:39:10 virgo kernel: [9658533.161600]  unevictable:5 dirty:2821 writeback:0 unstable:0
Jan 15 13:39:10 virgo kernel: [9658533.161600]  free:753 slab_reclaimable:25777 slab_unreclaimable:3367
Jan 15 13:39:10 virgo kernel: [9658533.161600]  mapped:4481 shmem:151 pagetables:489 bounce:0
Jan 15 13:39:10 virgo kernel: [9658533.161600] Node 0 DMA free:1976kB min:80kB low:100kB high:120kB active_anon:308kB inactive_anon:512kB active_file:5712kB inactive_file:3932kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:14856kB mlocked:0kB dirty:0kB writeback:0kB mapped:164kB shmem:0kB slab_reclaimable:2432kB slab_unreclaimable:160kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jan 15 13:39:10 virgo kernel: [9658533.161600] lowmem_reserve[]: 0 489 489 489
Jan 15 13:39:10 virgo kernel: [9658533.161600] Node 0 DMA32 free:1036kB min:2788kB low:3484kB high:4180kB active_anon:23016kB inactive_anon:35336kB active_file:142104kB inactive_file:92108kB unevictable:20kB isolated(anon):0kB isolated(file):0kB present:500960kB mlocked:20kB dirty:11284kB writeback:0kB mapped:17760kB shmem:604kB slab_reclaimable:100676kB slab_unreclaimable:13308kB kernel_stack:1072kB pagetables:1956kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:234 all_unreclaimable? no
Jan 15 13:39:10 virgo kernel: [9658533.161600] lowmem_reserve[]: 0 0 0 0
Jan 15 13:39:10 virgo kernel: [9658533.161600] Node 0 DMA: 208*4kB 1*8kB 1*16kB 5*32kB 5*64kB 5*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1976kB
Jan 15 13:39:10 virgo kernel: [9658533.161600] Node 0 DMA32: 147*4kB 0*8kB 1*16kB 2*32kB 0*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1052kB
Jan 15 13:39:10 virgo kernel: [9658533.161600] 61725 total pagecache pages
Jan 15 13:39:10 virgo kernel: [9658533.161600] 559 pages in swap cache
Jan 15 13:39:10 virgo kernel: [9658533.161600] Swap cache stats: add 3779, delete 3220, find 17662478/17662698
Jan 15 13:39:10 virgo kernel: [9658533.161600] Free swap  = 968684kB
Jan 15 13:39:10 virgo kernel: [9658533.161600] Total swap = 979956kB
Jan 15 13:39:10 virgo kernel: [9658533.161600] 131072 pages RAM
Jan 15 13:39:10 virgo kernel: [9658533.161600] 20624 pages reserved
Jan 15 13:39:10 virgo kernel: [9658533.161600] 55796 pages shared
Jan 15 13:39:10 virgo kernel: [9658533.161600] 58358 pages non-shared
Jan 15 13:39:10 virgo kernel: [9658533.161600] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
Jan 15 13:39:10 virgo kernel: [9658533.161600]   cache: kmalloc-4096, object size: 4096, buffer size: 4096, default order: 3, min order: 0
Jan 15 13:39:10 virgo kernel: [9658533.161600]   node 0: slabs: 583, objs: 1213, free: 0
Jan 15 13:39:10 virgo kernel: [9658533.161600] swapper: page allocation failure. order:0, mode:0x4020
Jan 15 13:39:10 virgo kernel: [9658533.161600] Pid: 0, comm: swapper Not tainted 2.6.32-bpo.5-xen-amd64 #1
Jan 15 13:39:10 virgo kernel: [9658533.161600] Call Trace:
Jan 15 13:39:10 virgo kernel: [9658533.161600]  <IRQ>  [<ffffffff810baed4>] ? __alloc_pages_nodemask+0x55b/0x5cf
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff8100ec5f>] ? xen_restore_fl_direct_end+0x0/0x1
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff810e70d2>] ? new_slab+0x5b/0x1ca
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff810e7431>] ? __slab_alloc+0x1f0/0x39b
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81258103>] ? __netdev_alloc_skb+0x29/0x43
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff810e7e63>] ? __kmalloc_node_track_caller+0xbb/0x11b
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81258103>] ? __netdev_alloc_skb+0x29/0x43
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81257518>] ? __alloc_skb+0x69/0x15a
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81258103>] ? __netdev_alloc_skb+0x29/0x43
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffffa0019c25>] ? tg3_alloc_rx_skb+0xd2/0x146 [tg3]
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffffa0021292>] ? tg3_poll+0x484/0x93d [tg3]
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff8100e5b5>] ? xen_force_evtchn_callback+0x9/0xa
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff8100ec72>] ? check_events+0x12/0x20
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff8125e633>] ? net_rx_action+0xae/0x1c9
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff810548ca>] ? __do_softirq+0xdd/0x19f
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81012cac>] ? call_softirq+0x1c/0x30
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff8101422b>] ? do_softirq+0x3f/0x7c
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81054739>] ? irq_exit+0x36/0x76
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff811f14b1>] ? xen_evtchn_do_upcall+0x33/0x42
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81012cfe>] ? xen_do_hypervisor_callback+0x1e/0x30
Jan 15 13:39:10 virgo kernel: [9658533.161600]  <EOI>  [<ffffffff810093aa>] ? hypercall_page+0x3aa/0x1001
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff810093aa>] ? hypercall_page+0x3aa/0x1001
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff8100e633>] ? xen_safe_halt+0xc/0x15
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff8100bf3f>] ? xen_idle+0x37/0x40
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81010eb1>] ? cpu_idle+0xa2/0xda
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81502cd1>] ? start_kernel+0x3dc/0x3e8
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81504c7d>] ? xen_start_kernel+0x57c/0x581
Jan 15 13:39:10 virgo kernel: [9658533.161600] Mem-Info:
Jan 15 13:39:10 virgo kernel: [9658533.161600] Node 0 DMA per-cpu:
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    0: hi:    0, btch:   1 usd:   0
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    1: hi:    0, btch:   1 usd:   0
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    2: hi:    0, btch:   1 usd:   0
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    3: hi:    0, btch:   1 usd:   0
Jan 15 13:39:10 virgo kernel: [9658533.161600] Node 0 DMA32 per-cpu:
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    0: hi:  186, btch:  31 usd: 197
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    1: hi:  186, btch:  31 usd: 182
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    2: hi:  186, btch:  31 usd: 140
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    3: hi:  186, btch:  31 usd:  85
Jan 15 13:39:10 virgo kernel: [9658533.161600] active_anon:5831 inactive_anon:8962 isolated_anon:0
Jan 15 13:39:10 virgo kernel: [9658533.161600]  active_file:36954 inactive_file:24010 isolated_file:0
Jan 15 13:39:10 virgo kernel: [9658533.161600]  unevictable:5 dirty:2821 writeback:0 unstable:0
Jan 15 13:39:10 virgo kernel: [9658533.161600]  free:753 slab_reclaimable:25777 slab_unreclaimable:3367
Jan 15 13:39:10 virgo kernel: [9658533.161600]  mapped:4481 shmem:151 pagetables:489 bounce:0
Jan 15 13:39:10 virgo kernel: [9658533.161600] Node 0 DMA free:1976kB min:80kB low:100kB high:120kB active_anon:308kB inactive_anon:512kB active_file:5712kB inactive_file:3932kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:14856kB mlocked:0kB dirty:0kB writeback:0kB mapped:164kB shmem:0kB slab_reclaimable:2432kB slab_unreclaimable:160kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jan 15 13:39:10 virgo kernel: [9658533.161600] lowmem_reserve[]: 0 489 489 489
Jan 15 13:39:10 virgo kernel: [9658533.161600] Node 0 DMA32 free:1036kB min:2788kB low:3484kB high:4180kB active_anon:23016kB inactive_anon:35336kB active_file:142104kB inactive_file:92108kB unevictable:20kB isolated(anon):0kB isolated(file):0kB present:500960kB mlocked:20kB dirty:11284kB writeback:0kB mapped:17760kB shmem:604kB slab_reclaimable:100676kB slab_unreclaimable:13308kB kernel_stack:1072kB pagetables:1956kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:234 all_unreclaimable? no
Jan 15 13:39:10 virgo kernel: [9658533.161600] lowmem_reserve[]: 0 0 0 0
Jan 15 13:39:10 virgo kernel: [9658533.161600] Node 0 DMA: 208*4kB 1*8kB 1*16kB 5*32kB 5*64kB 5*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1976kB
Jan 15 13:39:10 virgo kernel: [9658533.161600] Node 0 DMA32: 147*4kB 0*8kB 1*16kB 2*32kB 0*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1052kB
Jan 15 13:39:10 virgo kernel: [9658533.161600] 61725 total pagecache pages
Jan 15 13:39:10 virgo kernel: [9658533.161600] 559 pages in swap cache
Jan 15 13:39:10 virgo kernel: [9658533.161600] Swap cache stats: add 3779, delete 3220, find 17662478/17662698
Jan 15 13:39:10 virgo kernel: [9658533.161600] Free swap  = 968684kB
Jan 15 13:39:10 virgo kernel: [9658533.161600] Total swap = 979956kB
Jan 15 13:39:10 virgo kernel: [9658533.161600] 131072 pages RAM
Jan 15 13:39:10 virgo kernel: [9658533.161600] 20624 pages reserved
Jan 15 13:39:10 virgo kernel: [9658533.161600] 55796 pages shared
Jan 15 13:39:10 virgo kernel: [9658533.161600] 58358 pages non-shared
Jan 15 13:39:10 virgo kernel: [9658533.161600] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
Jan 15 13:39:10 virgo kernel: [9658533.161600]   cache: kmalloc-4096, object size: 4096, buffer size: 4096, default order: 3, min order: 0
Jan 15 13:39:10 virgo kernel: [9658533.161600]   node 0: slabs: 583, objs: 1213, free: 0
Jan 15 13:39:10 virgo kernel: [9658533.161600] swapper: page allocation failure. order:0, mode:0x4020
Jan 15 13:39:10 virgo kernel: [9658533.161600] Pid: 0, comm: swapper Not tainted 2.6.32-bpo.5-xen-amd64 #1
Jan 15 13:39:10 virgo kernel: [9658533.161600] Call Trace:
Jan 15 13:39:10 virgo kernel: [9658533.161600]  <IRQ>  [<ffffffff810baed4>] ? __alloc_pages_nodemask+0x55b/0x5cf
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff8100ec5f>] ? xen_restore_fl_direct_end+0x0/0x1
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff810e70d2>] ? new_slab+0x5b/0x1ca
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff810e7431>] ? __slab_alloc+0x1f0/0x39b
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81258103>] ? __netdev_alloc_skb+0x29/0x43
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff810e7e63>] ? __kmalloc_node_track_caller+0xbb/0x11b
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81258103>] ? __netdev_alloc_skb+0x29/0x43
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81257518>] ? __alloc_skb+0x69/0x15a
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81258103>] ? __netdev_alloc_skb+0x29/0x43
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffffa0019c25>] ? tg3_alloc_rx_skb+0xd2/0x146 [tg3]
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffffa0021292>] ? tg3_poll+0x484/0x93d [tg3]
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff8100e5b5>] ? xen_force_evtchn_callback+0x9/0xa
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff8100ec72>] ? check_events+0x12/0x20
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff8125e633>] ? net_rx_action+0xae/0x1c9
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff810548ca>] ? __do_softirq+0xdd/0x19f
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81012cac>] ? call_softirq+0x1c/0x30
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff8101422b>] ? do_softirq+0x3f/0x7c
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81054739>] ? irq_exit+0x36/0x76
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff811f14b1>] ? xen_evtchn_do_upcall+0x33/0x42
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81012cfe>] ? xen_do_hypervisor_callback+0x1e/0x30
Jan 15 13:39:10 virgo kernel: [9658533.161600]  <EOI>  [<ffffffff810093aa>] ? hypercall_page+0x3aa/0x1001
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff810093aa>] ? hypercall_page+0x3aa/0x1001
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff8100e633>] ? xen_safe_halt+0xc/0x15
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff8100bf3f>] ? xen_idle+0x37/0x40
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81010eb1>] ? cpu_idle+0xa2/0xda
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81502cd1>] ? start_kernel+0x3dc/0x3e8
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81504c7d>] ? xen_start_kernel+0x57c/0x581
Jan 15 13:39:10 virgo kernel: [9658533.161600] Mem-Info:
Jan 15 13:39:10 virgo kernel: [9658533.161600] Node 0 DMA per-cpu:
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    0: hi:    0, btch:   1 usd:   0
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    1: hi:    0, btch:   1 usd:   0
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    2: hi:    0, btch:   1 usd:   0
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    3: hi:    0, btch:   1 usd:   0
Jan 15 13:39:10 virgo kernel: [9658533.161600] Node 0 DMA32 per-cpu:
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    0: hi:  186, btch:  31 usd: 197
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    1: hi:  186, btch:  31 usd: 182
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    2: hi:  186, btch:  31 usd: 140
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    3: hi:  186, btch:  31 usd:  85
Jan 15 13:39:10 virgo kernel: [9658533.161600] active_anon:5831 inactive_anon:8962 isolated_anon:0
Jan 15 13:39:10 virgo kernel: [9658533.161600]  active_file:36954 inactive_file:24010 isolated_file:0
Jan 15 13:39:10 virgo kernel: [9658533.161600]  unevictable:5 dirty:2821 writeback:0 unstable:0
Jan 15 13:39:10 virgo kernel: [9658533.161600]  free:753 slab_reclaimable:25777 slab_unreclaimable:3367
Jan 15 13:39:10 virgo kernel: [9658533.161600]  mapped:4481 shmem:151 pagetables:489 bounce:0
Jan 15 13:39:10 virgo kernel: [9658533.161600] Node 0 DMA free:1976kB min:80kB low:100kB high:120kB active_anon:308kB inactive_anon:512kB active_file:5712kB inactive_file:3932kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:14856kB mlocked:0kB dirty:0kB writeback:0kB mapped:164kB shmem:0kB slab_reclaimable:2432kB slab_unreclaimable:160kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jan 15 13:39:10 virgo kernel: [9658533.161600] lowmem_reserve[]: 0 489 489 489
Jan 15 13:39:10 virgo kernel: [9658533.161600] Node 0 DMA32 free:1036kB min:2788kB low:3484kB high:4180kB active_anon:23016kB inactive_anon:35336kB active_file:142104kB inactive_file:92108kB unevictable:20kB isolated(anon):0kB isolated(file):0kB present:500960kB mlocked:20kB dirty:11284kB writeback:0kB mapped:17760kB shmem:604kB slab_reclaimable:100676kB slab_unreclaimable:13308kB kernel_stack:1072kB pagetables:1956kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:234 all_unreclaimable? no
Jan 15 13:39:10 virgo kernel: [9658533.161600] lowmem_reserve[]: 0 0 0 0
Jan 15 13:39:10 virgo kernel: [9658533.161600] Node 0 DMA: 208*4kB 1*8kB 1*16kB 5*32kB 5*64kB 5*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1976kB
Jan 15 13:39:10 virgo kernel: [9658533.161600] Node 0 DMA32: 147*4kB 0*8kB 1*16kB 2*32kB 0*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1052kB
Jan 15 13:39:10 virgo kernel: [9658533.161600] 61725 total pagecache pages
Jan 15 13:39:10 virgo kernel: [9658533.161600] 559 pages in swap cache
Jan 15 13:39:10 virgo kernel: [9658533.161600] Swap cache stats: add 3779, delete 3220, find 17662478/17662698
Jan 15 13:39:10 virgo kernel: [9658533.161600] Free swap  = 968684kB
Jan 15 13:39:10 virgo kernel: [9658533.161600] Total swap = 979956kB
Jan 15 13:39:10 virgo kernel: [9658533.161600] 131072 pages RAM
Jan 15 13:39:10 virgo kernel: [9658533.161600] 20624 pages reserved
Jan 15 13:39:10 virgo kernel: [9658533.161600] 55796 pages shared
Jan 15 13:39:10 virgo kernel: [9658533.161600] 58358 pages non-shared
Jan 15 13:39:10 virgo kernel: [9658533.161600] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
Jan 15 13:39:10 virgo kernel: [9658533.161600]   cache: kmalloc-4096, object size: 4096, buffer size: 4096, default order: 3, min order: 0
Jan 15 13:39:10 virgo kernel: [9658533.161600]   node 0: slabs: 583, objs: 1213, free: 0
Jan 15 13:39:10 virgo kernel: [9658533.161600] swapper: page allocation failure. order:0, mode:0x4020
Jan 15 13:39:10 virgo kernel: [9658533.161600] Pid: 0, comm: swapper Not tainted 2.6.32-bpo.5-xen-amd64 #1
Jan 15 13:39:10 virgo kernel: [9658533.161600] Call Trace:
Jan 15 13:39:10 virgo kernel: [9658533.161600]  <IRQ>  [<ffffffff810baed4>] ? __alloc_pages_nodemask+0x55b/0x5cf
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff8100ec5f>] ? xen_restore_fl_direct_end+0x0/0x1
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff810e70d2>] ? new_slab+0x5b/0x1ca
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff810e7431>] ? __slab_alloc+0x1f0/0x39b
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81258103>] ? __netdev_alloc_skb+0x29/0x43
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff810e7e63>] ? __kmalloc_node_track_caller+0xbb/0x11b
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81258103>] ? __netdev_alloc_skb+0x29/0x43
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81257518>] ? __alloc_skb+0x69/0x15a
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81258103>] ? __netdev_alloc_skb+0x29/0x43
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffffa0019c25>] ? tg3_alloc_rx_skb+0xd2/0x146 [tg3]
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffffa0021292>] ? tg3_poll+0x484/0x93d [tg3]
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff8100e5b5>] ? xen_force_evtchn_callback+0x9/0xa
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff8100ec72>] ? check_events+0x12/0x20
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff8125e633>] ? net_rx_action+0xae/0x1c9
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff810548ca>] ? __do_softirq+0xdd/0x19f
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81012cac>] ? call_softirq+0x1c/0x30
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff8101422b>] ? do_softirq+0x3f/0x7c
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81054739>] ? irq_exit+0x36/0x76
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff811f14b1>] ? xen_evtchn_do_upcall+0x33/0x42
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81012cfe>] ? xen_do_hypervisor_callback+0x1e/0x30
Jan 15 13:39:10 virgo kernel: [9658533.161600]  <EOI>  [<ffffffff810093aa>] ? hypercall_page+0x3aa/0x1001
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff810093aa>] ? hypercall_page+0x3aa/0x1001
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff8100e633>] ? xen_safe_halt+0xc/0x15
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff8100bf3f>] ? xen_idle+0x37/0x40
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81010eb1>] ? cpu_idle+0xa2/0xda
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81502cd1>] ? start_kernel+0x3dc/0x3e8
Jan 15 13:39:10 virgo kernel: [9658533.161600]  [<ffffffff81504c7d>] ? xen_start_kernel+0x57c/0x581
Jan 15 13:39:10 virgo kernel: [9658533.161600] Mem-Info:
Jan 15 13:39:10 virgo kernel: [9658533.161600] Node 0 DMA per-cpu:
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    0: hi:    0, btch:   1 usd:   0
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    1: hi:    0, btch:   1 usd:   0
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    2: hi:    0, btch:   1 usd:   0
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    3: hi:    0, btch:   1 usd:   0
Jan 15 13:39:10 virgo kernel: [9658533.161600] Node 0 DMA32 per-cpu:
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    0: hi:  186, btch:  31 usd: 197
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    1: hi:  186, btch:  31 usd: 182
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    2: hi:  186, btch:  31 usd: 140
Jan 15 13:39:10 virgo kernel: [9658533.161600] CPU    3: hi:  186, btch:  31 usd:  85
Jan 15 13:39:10 virgo kernel: [9658533.161600] active_anon:5831 inactive_anon:8962 isolated_anon:0
Jan 15 13:39:10 virgo kernel: [9658533.161600]  active_file:36954 inactive_file:24010 isolated_file:0
Jan 15 13:39:10 virgo kernel: [9658533.161600]  unevictable:5 dirty:2821 writeback:0 unstable:0
Jan 15 13:39:10 virgo kernel: [9658533.161600]  free:753 slab_reclaimable:25777 slab_unreclaimable:3367
Jan 15 13:39:10 virgo kernel: [9658533.161600]  mapped:4481 shmem:151 pagetables:489 bounce:0
Jan 15 13:39:10 virgo kernel: [9658533.161600] Node 0 DMA free:1976kB min:80kB low:100kB high:120kB active_anon:308kB inactive_anon:512kB active_file:5712kB inactive_file:3932kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:14856kB mlocked:0kB dirty:0kB writeback:0kB mapped:164kB shmem:0kB slab_reclaimable:2432kB slab_unreclaimable:160kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jan 15 13:39:10 virgo kernel: [9658533.161600] lowmem_reserve[]: 0 489 489 489
Jan 15 13:39:10 virgo kernel: [9658533.161600] Node 0 DMA32 free:1036kB min:2788kB low:3484kB high:4180kB active_anon:23016kB inactive_anon:35336kB active_file:142104kB inactive_file:92108kB unevictable:20kB isolated(anon):0kB isolated(file):0kB present:500960kB mlocked:20kB dirty:11284kB writeback:0kB mapped:17760kB shmem:604kB slab_reclaimable:100676kB slab_unreclaimable:13308kB kernel_stack:1072kB pagetables:1956kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:234 all_unreclaimable? no
Jan 15 13:39:10 virgo kernel: [9658533.161600] lowmem_reserve[]: 0 0 0 0
Jan 15 13:39:10 virgo kernel: [9658533.161600] Node 0 DMA: 208*4kB 1*8kB 1*16kB 5*32kB 5*64kB 5*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1976kB
Jan 15 13:39:10 virgo kernel: [9658533.161600] Node 0 DMA32: 147*4kB 0*8kB 1*16kB 2*32kB 0*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1052kB
Jan 15 13:39:10 virgo kernel: [9658533.161600] 61725 total pagecache pages
Jan 15 13:39:10 virgo kernel: [9658533.161600] 559 pages in swap cache
Jan 15 13:39:10 virgo kernel: [9658533.161600] Swap cache stats: add 3779, delete 3220, find 17662478/17662698
Jan 15 13:39:10 virgo kernel: [9658533.161600] Free swap  = 968684kB
Jan 15 13:39:10 virgo kernel: [9658533.161600] Total swap = 979956kB
Jan 15 13:39:10 virgo kernel: [9658533.161600] 131072 pages RAM
Jan 15 13:39:10 virgo kernel: [9658533.161600] 20624 pages reserved
Jan 15 13:39:10 virgo kernel: [9658533.161600] 55796 pages shared
Jan 15 13:39:10 virgo kernel: [9658533.161600] 58358 pages non-shared
Jan 15 13:39:10 virgo kernel: [9658533.161600] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
Jan 15 13:39:10 virgo kernel: [9658533.161600]   cache: kmalloc-4096, object size: 4096, buffer size: 4096, default order: 3, min order: 0
Jan 15 13:39:10 virgo kernel: [9658533.161600]   node 0: slabs: 583, objs: 1213, free: 0

The dom0 has 512 MB RAM allocated, and the largest domU underneath
has 384 MB RAM. Its firewall rules in the FORWARD chain are:

    pkts      bytes target     prot opt in     out     source               destination
373344874 27936351024 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED PHYSDEV match --physdev-out vif4.0
258240283 103509593798 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           PHYSDEV match --physdev-in vif4.0

-- 
     2. That which causes joy or happiness.



Reply to: