Bug#784688: Thousands of "xen:balloon: Cannot add additional memory (-17) messages" despite dom0 ballooning disabled
On Sunday, 17.05.2015 at 15:14, Ian Campbell wrote:
> On Thu, 2015-05-07 at 21:02 +0200, Martin Lucina wrote:
> > Note that despite the memory setting above, the dom0 has not been allocated
> > exactly the amount asked for:
> >
> > # xl list 0
> > Name ID Mem VCPUs State Time(s)
> > Domain-0 0 4051 8 r----- 5606.1
>
> Could you also post the output of "xenstore-ls -fp | grep target"
> please.
/local/domain/1/memory/target = "2097153" (n0,r1)
/local/domain/76/memory/target = "2097153" (n0,r76)
/local/domain/276/memory/target = "262145" (n0,r276)
/local/domain/314/memory/target = "65537" (n0,r314)
/local/domain/318/memory/target = "65537" (n0,r318)
/local/domain/319/memory/target = "65537" (n0,r319)
/local/domain/323/memory/target = "65537" (n0,r323)
/local/domain/325/memory/target = "65537" (n0,r325)
> You should have a directory /sys/devices/system/xen_memory/xen_memory0.
> Please could you post the contents of each of the files in there. In
> particular target*, *retry_count and info/*.
/sys/devices/system/xen_memory/xen_memory0/info/current_kb:
4163284
/sys/devices/system/xen_memory/xen_memory0/info/low_kb:
0
/sys/devices/system/xen_memory/xen_memory0/info/high_kb:
0
/sys/devices/system/xen_memory/xen_memory0/max_schedule_delay:
20
/sys/devices/system/xen_memory/xen_memory0/retry_count:
1
/sys/devices/system/xen_memory/xen_memory0/power/control:
auto
/sys/devices/system/xen_memory/xen_memory0/power/async:
disabled
/sys/devices/system/xen_memory/xen_memory0/power/runtime_enabled:
disabled
/sys/devices/system/xen_memory/xen_memory0/power/runtime_active_kids:
0
/sys/devices/system/xen_memory/xen_memory0/power/runtime_active_time:
0
/sys/devices/system/xen_memory/xen_memory0/power/autosuspend_delay_ms:
cat: /sys/devices/system/xen_memory/xen_memory0/power/autosuspend_delay_ms: Input/output error
/sys/devices/system/xen_memory/xen_memory0/power/runtime_status:
unsupported
/sys/devices/system/xen_memory/xen_memory0/power/runtime_usage:
0
/sys/devices/system/xen_memory/xen_memory0/power/runtime_suspended_time:
0
/sys/devices/system/xen_memory/xen_memory0/target_kb:
4194304
/sys/devices/system/xen_memory/xen_memory0/target:
4294967296
/sys/devices/system/xen_memory/xen_memory0/uevent:
/sys/devices/system/xen_memory/xen_memory0/schedule_delay:
1
/sys/devices/system/xen_memory/xen_memory0/max_retry_count:
0
> It might also be interesting to see the result of "xl debug-keys q"
> followed by "xl dmesg".
(XEN) 'q' pressed -> dumping domain info (now=0x70765:8E6C9FAB)
(XEN) General information for domain 0:
(XEN) refcnt=3 dying=0 pause_count=0
(XEN) nr_pages=1040821 xenheap_pages=5 shared_pages=0 paged_pages=0 dirty_cpus={3-4} max_pages=1048576
(XEN) handle=00000000-0000-0000-0000-000000000000 vm_assist=0000000d
(XEN) Rangesets belonging to domain 0:
(XEN) I/O Ports { 0-1f, 22-3f, 44-60, 62-9f, a2-cfb, d00-1007, 100c-ffff }
(XEN) Interrupts { 1-51 }
(XEN) I/O Memory { 0-fedff, fef00-ffffffffffffffff }
(XEN) Memory pages belonging to domain 0:
(XEN) DomPage list too long to display
(XEN) XenPage 000000000082e6a4: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 000000000082e6a3: caf=c000000000000001, taf=7400000000000001
(XEN) XenPage 000000000082e6a2: caf=c000000000000001, taf=7400000000000001
(XEN) XenPage 000000000082e6a1: caf=c000000000000001, taf=7400000000000001
(XEN) XenPage 00000000000cfd04: caf=c000000000000002, taf=7400000000000002
(XEN) NODE affinity for domain 0: [0]
(XEN) VCPU information and callbacks for domain 0:
(XEN) VCPU0: CPU2 [has=F] poll=0 upcall_pend = 00, upcall_mask = 00 dirty_cpus={} cpu_affinity={0-7}
(XEN) pause_count=0 pause_flags=1
(XEN) No periodic timer
(XEN) VCPU1: CPU6 [has=F] poll=0 upcall_pend = 00, upcall_mask = 00 dirty_cpus={} cpu_affinity={0-7}
(XEN) pause_count=0 pause_flags=1
(XEN) No periodic timer
(XEN) VCPU2: CPU7 [has=F] poll=0 upcall_pend = 00, upcall_mask = 00 dirty_cpus={} cpu_affinity={0-7}
(XEN) pause_count=0 pause_flags=1
(XEN) No periodic timer
(XEN) VCPU3: CPU5 [has=F] poll=0 upcall_pend = 00, upcall_mask = 00 dirty_cpus={} cpu_affinity={0-7}
(XEN) pause_count=0 pause_flags=1
(XEN) No periodic timer
(XEN) VCPU4: CPU4 [has=F] poll=0 upcall_pend = 00, upcall_mask = 00 dirty_cpus={4} cpu_affinity={0-7}
(XEN) pause_count=0 pause_flags=1
(XEN) No periodic timer
(XEN) VCPU5: CPU3 [has=T] poll=0 upcall_pend = 00, upcall_mask = 00 dirty_cpus={3} cpu_affinity={0-7}
(XEN) pause_count=0 pause_flags=0
(XEN) No periodic timer
(XEN) VCPU6: CPU7 [has=F] poll=0 upcall_pend = 00, upcall_mask = 00 dirty_cpus={} cpu_affinity={0-7}
(XEN) pause_count=0 pause_flags=1
(XEN) No periodic timer
(XEN) VCPU7: CPU4 [has=F] poll=0 upcall_pend = 00, upcall_mask = 00 dirty_cpus={} cpu_affinity={0-7}
(XEN) pause_count=0 pause_flags=1
(XEN) No periodic timer
(XEN) General information for domain 1:
(XEN) refcnt=3 dying=0 pause_count=0
(XEN) nr_pages=524288 xenheap_pages=8 shared_pages=0 paged_pages=0 dirty_cpus={0} max_pages=524544
(XEN) handle=ab336685-f1fb-40c8-ae4d-2fbf122ec3c1 vm_assist=0000000d
(XEN) Rangesets belonging to domain 1:
(XEN) I/O Ports { }
(XEN) Interrupts { }
(XEN) I/O Memory { }
(XEN) Memory pages belonging to domain 1:
(XEN) DomPage list too long to display
(XEN) XenPage 0000000000764c49: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 0000000000764c48: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 00000000006cc6af: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 00000000006cc6ae: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 00000000000cfc7d: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 00000000006cd61b: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 00000000006cd5af: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 000000000080b43d: caf=c000000000000002, taf=7400000000000002
(XEN) NODE affinity for domain 1: [0]
(XEN) VCPU information and callbacks for domain 1:
(XEN) VCPU0: CPU0 [has=F] poll=0 upcall_pend = 00, upcall_mask = 00 dirty_cpus={0} cpu_affinity={0-7}
(XEN) pause_count=0 pause_flags=1
(XEN) No periodic timer
(XEN) General information for domain 76:
(XEN) refcnt=3 dying=0 pause_count=0
(XEN) nr_pages=524288 xenheap_pages=11 shared_pages=0 paged_pages=0 dirty_cpus={} max_pages=524544
(XEN) handle=8602f97a-fae0-46a1-9033-4d901092cc81 vm_assist=0000000d
(XEN) Rangesets belonging to domain 76:
(XEN) I/O Ports { }
(XEN) Interrupts { }
(XEN) I/O Memory { }
(XEN) Memory pages belonging to domain 76:
(XEN) DomPage list too long to display
(XEN) XenPage 000000000082ff6e: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 000000000082fcb6: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 000000000082fd41: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 000000000082fd47: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 00000000000cfd16: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 0000000000804277: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 000000000080445b: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 0000000000806a78: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 0000000000806a16: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 000000000080b780: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 00000000008042a8: caf=c000000000000002, taf=7400000000000002
(XEN) NODE affinity for domain 76: [0]
(XEN) VCPU information and callbacks for domain 76:
(XEN) VCPU0: CPU5 [has=F] poll=0 upcall_pend = 00, upcall_mask = 00 dirty_cpus={} cpu_affinity={0-7}
(XEN) pause_count=0 pause_flags=1
(XEN) No periodic timer
(XEN) VCPU1: CPU3 [has=F] poll=0 upcall_pend = 00, upcall_mask = 00 dirty_cpus={} cpu_affinity={0-7}
(XEN) pause_count=0 pause_flags=1
(XEN) No periodic timer
(XEN) General information for domain 276:
(XEN) refcnt=3 dying=0 pause_count=0
(XEN) nr_pages=65536 xenheap_pages=5 shared_pages=0 paged_pages=0 dirty_cpus={} max_pages=65792
(XEN) handle=3ca350e3-fc2f-4ed7-a0ba-ba675a0ab501 vm_assist=00000000
(XEN) Rangesets belonging to domain 276:
(XEN) I/O Ports { }
(XEN) Interrupts { }
(XEN) I/O Memory { }
(XEN) Memory pages belonging to domain 276:
(XEN) DomPage list too long to display
(XEN) XenPage 0000000000815f7d: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 000000000081608f: caf=c000000000000001, taf=7400000000000001
(XEN) XenPage 0000000000815fb1: caf=c000000000000001, taf=7400000000000001
(XEN) XenPage 0000000000816082: caf=c000000000000001, taf=7400000000000001
(XEN) XenPage 00000000000cfd14: caf=c000000000000002, taf=7400000000000002
(XEN) NODE affinity for domain 276: [0]
(XEN) VCPU information and callbacks for domain 276:
(XEN) VCPU0: CPU4 [has=F] poll=0 upcall_pend = 00, upcall_mask = 00 dirty_cpus={} cpu_affinity={0-7}
(XEN) pause_count=0 pause_flags=1
(XEN) No periodic timer
(XEN) General information for domain 314:
(XEN) refcnt=3 dying=0 pause_count=0
(XEN) nr_pages=16384 xenheap_pages=5 shared_pages=0 paged_pages=0 dirty_cpus={} max_pages=16640
(XEN) handle=86ab55d5-3089-406c-8877-64385a7348ed vm_assist=00000000
(XEN) Rangesets belonging to domain 314:
(XEN) I/O Ports { }
(XEN) Interrupts { }
(XEN) I/O Memory { }
(XEN) Memory pages belonging to domain 314:
(XEN) DomPage list too long to display
(XEN) XenPage 00000000006d332c: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 00000000006d3577: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 00000000006676bf: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 00000000007c26b3: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 00000000000cfc7a: caf=c000000000000002, taf=7400000000000002
(XEN) NODE affinity for domain 314: [0]
(XEN) VCPU information and callbacks for domain 314:
(XEN) VCPU0: CPU7 [has=F] poll=0 upcall_pend = 01, upcall_mask = 01 dirty_cpus={} cpu_affinity={0-7}
(XEN) pause_count=1 pause_flags=0
(XEN) 100 Hz periodic timer (period 10 ms)
(XEN) General information for domain 318:
(XEN) refcnt=3 dying=0 pause_count=0
(XEN) nr_pages=16384 xenheap_pages=5 shared_pages=0 paged_pages=0 dirty_cpus={} max_pages=16640
(XEN) handle=adb17a19-35b1-4b60-9acb-145c13d8cbbb vm_assist=00000000
(XEN) Rangesets belonging to domain 318:
(XEN) I/O Ports { }
(XEN) Interrupts { }
(XEN) I/O Memory { }
(XEN) Memory pages belonging to domain 318:
(XEN) DomPage list too long to display
(XEN) XenPage 00000000007c2e8e: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 00000000007c2e8d: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 00000000007c2e8c: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 00000000007c2e8b: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 00000000000cfd10: caf=c000000000000002, taf=7400000000000002
(XEN) NODE affinity for domain 318: [0]
(XEN) VCPU information and callbacks for domain 318:
(XEN) VCPU0: CPU7 [has=F] poll=0 upcall_pend = 01, upcall_mask = 00 dirty_cpus={} cpu_affinity={0-7}
(XEN) pause_count=1 pause_flags=0
(XEN) 100 Hz periodic timer (period 10 ms)
(XEN) General information for domain 319:
(XEN) refcnt=3 dying=0 pause_count=0
(XEN) nr_pages=16384 xenheap_pages=5 shared_pages=0 paged_pages=0 dirty_cpus={1} max_pages=16640
(XEN) handle=76dabaa0-0b14-4e6b-83c8-f30195d7be5b vm_assist=00000000
(XEN) Rangesets belonging to domain 319:
(XEN) I/O Ports { }
(XEN) Interrupts { }
(XEN) I/O Memory { }
(XEN) Memory pages belonging to domain 319:
(XEN) DomPage list too long to display
(XEN) XenPage 00000000005332b8: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 00000000005332cb: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 00000000005332ca: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 00000000005332c9: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 00000000000cfcae: caf=c000000000000002, taf=7400000000000002
(XEN) NODE affinity for domain 319: [0]
(XEN) VCPU information and callbacks for domain 319:
(XEN) VCPU0: CPU1 [has=F] poll=0 upcall_pend = 00, upcall_mask = 00 dirty_cpus={1} cpu_affinity={0-7}
(XEN) pause_count=0 pause_flags=1
(XEN) 100 Hz periodic timer (period 10 ms)
(XEN) General information for domain 323:
(XEN) refcnt=3 dying=0 pause_count=0
(XEN) nr_pages=16384 xenheap_pages=5 shared_pages=0 paged_pages=0 dirty_cpus={} max_pages=16640
(XEN) handle=9a05b42f-2d7e-477b-93f3-05a4dbe4a728 vm_assist=00000000
(XEN) Rangesets belonging to domain 323:
(XEN) I/O Ports { }
(XEN) Interrupts { }
(XEN) I/O Memory { }
(XEN) Memory pages belonging to domain 323:
(XEN) DomPage list too long to display
(XEN) XenPage 000000000053164f: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 000000000053164e: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 000000000053164d: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 000000000053164c: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 00000000000cfd11: caf=c000000000000002, taf=7400000000000002
(XEN) NODE affinity for domain 323: [0]
(XEN) VCPU information and callbacks for domain 323:
(XEN) VCPU0: CPU6 [has=F] poll=0 upcall_pend = 01, upcall_mask = 01 dirty_cpus={} cpu_affinity={0-7}
(XEN) pause_count=1 pause_flags=0
(XEN) 100 Hz periodic timer (period 10 ms)
(XEN) General information for domain 325:
(XEN) refcnt=3 dying=0 pause_count=0
(XEN) nr_pages=16384 xenheap_pages=5 shared_pages=0 paged_pages=0 dirty_cpus={} max_pages=16640
(XEN) handle=b74db9b5-41d8-4f27-919d-250735c9dad2 vm_assist=00000000
(XEN) Rangesets belonging to domain 325:
(XEN) I/O Ports { }
(XEN) Interrupts { }
(XEN) I/O Memory { }
(XEN) Memory pages belonging to domain 325:
(XEN) DomPage list too long to display
(XEN) XenPage 0000000000530636: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 0000000000530635: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 0000000000530634: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 000000000053d627: caf=c000000000000002, taf=7400000000000002
(XEN) XenPage 00000000000cfcac: caf=c000000000000002, taf=7400000000000002
(XEN) NODE affinity for domain 325: [0]
(XEN) VCPU information and callbacks for domain 325:
(XEN) VCPU0: CPU7 [has=F] poll=0 upcall_pend = 00, upcall_mask = 00 dirty_cpus={} cpu_affinity={0-7}
(XEN) pause_count=0 pause_flags=1
(XEN) 100 Hz periodic timer (period 10 ms)
(XEN) Notifying guest 0:0 (virq 1, port 4)
(XEN) Notifying guest 0:1 (virq 1, port 10)
(XEN) Notifying guest 0:2 (virq 1, port 16)
(XEN) Notifying guest 0:3 (virq 1, port 22)
(XEN) Notifying guest 0:4 (virq 1, port 28)
(XEN) Notifying guest 0:5 (virq 1, port 34)
(XEN) Notifying guest 0:6 (virq 1, port 40)
(XEN) Notifying guest 0:7 (virq 1, port 46)
(XEN) Notifying guest 1:0 (virq 1, port 6)
(XEN) Notifying guest 76:0 (virq 1, port 6)
(XEN) Notifying guest 76:1 (virq 1, port 12)
(XEN) Notifying guest 276:0 (virq 1, port 3)
(XEN) Notifying guest 314:0 (virq 1, port 0)
(XEN) Notifying guest 318:0 (virq 1, port 0)
(XEN) Notifying guest 319:0 (virq 1, port 0)
(XEN) Notifying guest 323:0 (virq 1, port 0)
(XEN) Notifying guest 325:0 (virq 1, port 0)
(XEN) Shared frames 0 -- Saved frames 0
> Does "xl mem-set 0 4051" make the messages stop? If not what about using
> 4050?
I'm not sure if this is due to my rebooting the dom0 since I reported this
issue, it now appears to be using a slightly different amount of memory
just under the target:
Domain-0 0 4065 8 r----- 36437.3
xl mem-set 0 4065 seems to have made the messages stop. Will run with it
for a while and see what happens.
> If you try and balloon up to 4096 (with xl mem-set) does the amount of
> memory in xl list change?
Tried this after xl mem-set 0 4065 and no, the amount of memory in xl list
does not change. It also makes the messages start again, one is triggered
by the mem-set itself and more start appearing after a unikernel domU shuts
down.
Martin
Reply to: