Bug#1120602: hyper-v: BUG: kernel NULL pointer dereference, address: 00000000000000a0
Hi Salvatore,
On Thu, 13 Nov 2025 at 13:57, Salvatore Bonaccorso <carnil@debian.org> wrote:
>
> Control: tags -1 + upstream
> Control: tags -1 - moreinfo
>
> Hi Peter,
>
> On Thu, Nov 13, 2025 at 01:14:37PM +0000, Peter Morrow wrote:
> > Hi Salvatore,
> >
> > Thank you for your reply and maintainership!
>
> Welcome :)
>
> > On Thu, 13 Nov 2025 at 06:20, Salvatore Bonaccorso <carnil@debian.org> wrote:
> > >
> > > Control: tags -1 + moreinfo
> > >
> > > Hi Peter,
> > >
> > > Thanks a lot for the report.
> > >
> > > On Wed, Nov 12, 2025 at 10:56:38PM +0000, Peter Morrow wrote:
> > > > Package: src:linux
> > > > Version: 6.12.57-1
> > > > Severity: important
> > > > X-Debbugs-Cc: pdmorrow@gmail.com
> > > >
> > > > Dear Maintainer,
> > > >
> > > > I'm seeing a kernel crash quite soon after boot on a debian trixie based
> > > > system running 6.12.57+deb13-amd64, unfortunately the kernel panics before
> > > > I can access the system to gather more information. Thus I'll provide details
> > > > of the system using a previously known good version. The panic is happening
> > > > 100% of the time unfortunately. I have access to the serial console however
> > > > so can enable any required verbose logging during boot if necessary.
> > > >
> > > > Crucially the crash is not seen with kernel version 6.12.41+deb13-amd64 with the
> > > > same userspace. We had pinned to that version until very recently to in order
> > > > to work around https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1109676
> > > >
> > > > I'm running a dpdk application here (VPP) on Azure, VM form factor is a
> > > > "Standard DS3 v2 (4 vcpus, 14 GiB memory)".
> > > >
> > > > The only relevant upstream commit in this area (as far as I can see) is:
> > > >
> > > > https://lore.kernel.org/linux-hyperv/1bb599ee-fe28-409d-b430-2fc086268936@linux.microsoft.com/
> > > >
> > > > The comment regarding avoiding races at start adds a bit more weight behind this
> > > > hunch, though it's only a hunch as I am most definitely nowhere near an expert
> > > > in this area.
> > >
> > > I have a couple of questions. As you pinned 6.12.41+deb13-amd64, but
> > > this was not the most recent kernel for trixie, can you confirm you
> > > are seeing or not seeing the issue as well with 6.12.48-1 (this was
> > > released via security update).
> >
> > I installed linux-image-6.12.48+deb13-amd64 and can confirm it's working fine,
> > no crashes on boot.
>
> Thanks for doing so.
>
> > >
> > > This would help narrowing the range further.
> > >
> > > The commit b15b7d2a1b09 ("uio_hv_generic: Let userspace take care of
> > > interrupt mask") was backported to 6.12.53, so if you do not see the
> > > problem with 6.12.48-1 then this further weight for the offending
> > > commit.
> > >
> > > Secondly: Would you have either the possibility to bisect the changes
> > > between the given range of good/bad kernel to isolate the offending
> > > commit with a proof? Alternatively could you build 6.12.57-1 with the
> > > commit reverted only? (you can use the debian/bin/test-patches script
> > > for that, instructions in
> > > https://kernel-team.pages.debian.net/kernel-handbook/ch-common-tasks.html#id-1.6.6.4,
> > > let me know though if you need help to make the patch to be used).
> >
> > I was able to build 6.12.57-1 with b15b7d2a1b09 reverted using test-patches and
> > confirm that the kernel does not panic on boot. So it looks fairly
> > conclusive to me that
> > b15b7d2a1b09 is the culprit for me.
>
> Ok that is already a good result thank you. The next step is to
> forward it upstream, will do shortly, still the following point:
>
> > >
> > > Could you additionally (just to test, then revert back to the regular
> > > trixie kernel series) test the 6.17.7-2 kernel from unstable? This one
> > > would include as well a backport of b15b7d2a1b09.
> >
> > Would you still like this test done? I suspect it's not required now with the
> > data we have on b15b7d2a1b09.
>
> Not necessarily if it is a problem for deploying it for your
> enviornment. If you can, then we have the benefit to confirm to
> upstream that indeed it is still affecting more recent versions, and
> unlikely that some other changes inbetween make the issue disapear
> (even if there is no commit with fixes tag to the orginal commit).
>
> So not necessary, but still helpful to confirm that even newer
> branches are affected.
It was straight forward to try out, Interestingly I am not seeing the
panic on 6.17.7-2:
gnos@vEdgeOlder:~$ uname -a
Linux vEdgeOlder 6.17.7+deb14+1-amd64 #1 SMP PREEMPT_DYNAMIC Debian
6.17.7-2 (2025-11-06) x86_64 GNU/Linux
gnos@vEdgeOlder:~$
I also then retried 6.12.57-1 to make sure I am not hallucinating
today, and it indeed still panic'd on boot:
vEdgeOlder login: [ 30.339054] BUG: kernel NULL pointer dereference,
address: 00000000000000a0
[ 30.340151] #PF: supervisor read access in kernel mode
[ 30.341032] #PF: error_code(0x0000) - not-present page
[ 30.341739] PGD 80000003075ef067 P4D 80000003075ef067 PUD 0
[ 30.342525] Oops: Oops: 0000 [#1] PREEMPT SMP PTI
[ 30.343223] CPU: 1 UID: 0 PID: 1468 Comm: vpp_wk_0 Not tainted
6.12.57+deb13-amd64 #1 Debian 6.12.57-1
[ 30.344492] Hardware name: Microsoft Corporation Virtual
Machine/Virtual Machine, BIOS Hyper-V UEFI Release v4.1 05/13/2024
[ 30.346032] RIP: 0010:hv_uio_channel_cb+0xd/0x20 [uio_hv_generic]
[ 30.346898] Code: 02 00 00 5b 5d e9 53 d8 ef d9 0f 1f 00 90 90 90
90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 0f 1f 44 00 00 48
8b 47 10 <48> 8b b8 a0 00 00 00 f0 83 44 24 fc 00 e9 51 3f ff ff 90 90
90 90
[ 30.349433] RSP: 0000:ffffa92213647ef0 EFLAGS: 00010046
[ 30.350469] RAX: 0000000000000000 RBX: 0000000000000018 RCX: 0000000000000018
[ 30.351752] RDX: 0000000000000001 RSI: ffffffffffffffff RDI: ffff998fdbdfb000
[ 30.352880] RBP: ffff998fc1b2f200 R08: ffff998fc1b2f200 R09: 0000000000000000
[ 30.354030] R10: 0000000000000000 R11: 0000000000000000 R12: ffff9992f1cc1460
[ 30.355004] R13: ffff998fdbdfb000 R14: ffff998fdbdfb2a0 R15: ffffffffc100a160
[ 30.356005] FS: 00007fcea0f2a6c0(0000) GS:ffff9992f1c80000(0000)
knlGS:0000000000000000
[ 30.357069] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 30.357907] CR2: 00000000000000a0 CR3: 0000000150a4e001 CR4: 00000000003706f0
[ 30.358848] Call Trace:
[ 30.359209] <TASK>
[ 30.359568] vmbus_isr+0x1a5/0x210 [hv_vmbus]
[ 30.360162] __sysvec_hyperv_callback+0x32/0x60
[ 30.360911] sysvec_hyperv_callback+0x38/0x90
[ 30.363023] asm_sysvec_hyperv_callback+0x1a/0x20
[ 30.363819] RIP: 0033:0x7fcfedf3cbf6
[ 30.364362] Code: 54 24 18 90 49 8b 04 24 4a 8b 44 e8 40 49 8b 0e
48 85 c9 74 0f 8b 49 f8 eb 0c 66 2e 0f 1f 84 00 00 00 00 00 31 c9 31
d2 f7 f1 <39> c5 75 86 f3 90 eb d2 0f 31 41 8b 76 70 48 8b 4c 24 10 41
3b 76
[ 30.366863] RSP: 002b:00007fcea0f29980 EFLAGS: 00000246
[ 30.367542] RAX: 0000000000000000 RBX: 00007fcf1e01c9c0 RCX: 0000000000000004
[ 30.368579] RDX: 0000000000000003 RSI: 00007fcfee0d1ea0 RDI: 00007fcf1e01c9c0
[ 30.369522] RBP: 0000000000000000 R08: 0000000000000002 R09: 00007fcf1dfdc700
[ 30.370457] R10: 0000000000000000 R11: 00007fcf1da0db90 R12: 00007fcfee0d3f28
[ 30.371407] R13: 0000000000000003 R14: 00007fcfee0d1000 R15: 00007fcfee0ce240
[ 30.372438] </TASK>
[ 30.372768] Modules linked in: uio_hv_generic uio binfmt_misc
dm_crypt intel_rapl_msr intel_rapl_common rpcrdma
intel_uncore_frequency_common isst_if_mbox_msr sunrpc isst_if_common
rdma_ucm ib_iser rdma_cm skx_edac_common nfit ib_umad iw_cm libnvdimm
ib_ipoib libiscsi crct10dif_pclmul ghash_clmulni_intel sha512_ssse3
sha256_ssse3 sha1_ssse3 scsi_transport_iscsi ib_cm aesni_intel
gf128mul crypto_simd cryptd rapl pcspkr hv_utils hv_balloon sg evdev
joydev mpls_router ip_tunnel ramoops pstore_blk pstore_zone efi_pstore
configfs nfnetlink vsock_loopback vmw_vsock_virtio_transport_common
hv_sock vmw_vsock_vmci_transport vsock vmw_vmci efivarfs ip_tables
x_tables autofs4 overlay squashfs dm_verity dm_bufio reed_solomon
dm_mod loop ext4 crc16 mbcache jbd2 crc32c_generic mlx5_ib ib_uverbs
ib_core mlx5_core mlxfw pci_hyperv pci_hyperv_intf hyperv_drm sr_mod
drm_shmem_helper drm_kms_helper cdrom sd_mod hv_storvsc
scsi_transport_fc drm scsi_mod hid_generic hid_hyperv serio_raw hid
hv_netvsc hyperv_keyboard scsi_common hv_vmbus
[ 30.372857] crc32_pclmul crc32c_intel
[ 30.385686] CR2: 00000000000000a0
[ 30.386562] ---[ end trace 0000000000000000 ]---
[ 31.285814] BUG: kernel NULL pointer dereference, address: 00000000000000a0
[ 31.287264] #PF: supervisor read access in kernel mode
[ 31.288392] #PF: error_code(0x0000) - not-present page
[ 31.289550] PGD 80000003075ef067 P4D 80000003075ef067 PUD 0
[ 31.290704] Oops: Oops: 0000 [#2] PREEMPT SMP PTI
[ 31.291711] CPU: 2 UID: 0 PID: 1469 Comm: vpp_wk_1 Tainted: G
D 6.12.57+deb13-amd64 #1 Debian 6.12.57-1
[ 31.293647] Tainted: [D]=DIE
[ 31.294427] Hardware name: Microsoft Corporation Virtual
Machine/Virtual Machine, BIOS Hyper-V UEFI Release v4.1 05/13/2024
[ 31.296231] RIP: 0010:hv_uio_channel_cb+0xd/0x20 [uio_hv_generic]
[ 31.297428] Code: 02 00 00 5b 5d e9 53 d8 ef d9 0f 1f 00 90 90 90
90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 0f 1f 44 00 00 48
8b 47 10 <48> 8b b8 a0 00 00 00 f0 83 44 24 fc 00 e9 51 3f ff ff 90 90
90 90
[ 31.300539] RSP: 0000:ffffa9221364fef0 EFLAGS: 00010046
[ 31.301656] RAX: 0000000000000000 RBX: 0000000000000014 RCX: 0000000000000014
[ 31.302999] RDX: 0000000000000001 RSI: ffffffffffffffff RDI: ffff998fdbe1c400
[ 31.304360] RBP: ffff998fc1b2d200 R08: ffff998fc1b2d200 R09: 0000000000000000
[ 31.305658] R10: 0000000000000000 R11: 0000000000000000 R12: ffff9992f1d41460
[ 31.306980] R13: ffff998fdbe1c400 R14: ffff998fdbe1c6a0 R15: ffffffffc100a160
[ 31.308293] FS: 00007fcea0d296c0(0000) GS:ffff9992f1d00000(0000)
knlGS:0000000000000000
[ 31.309787] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 31.310955] CR2: 00000000000000a0 CR3: 0000000150a4e005 CR4: 00000000003706f0
[ 31.312277] Call Trace:
[ 31.313024] <TASK>
[ 31.313711] vmbus_isr+0x1a5/0x210 [hv_vmbus]
[ 31.314712] __sysvec_hyperv_callback+0x32/0x60
[ 31.315725] sysvec_hyperv_callback+0x38/0x90
[ 31.316715] asm_sysvec_hyperv_callback+0x1a/0x20
[ 31.317739] RIP: 0033:0x7fcfedf3cbf6
[ 31.318728] Code: 54 24 18 90 49 8b 04 24 4a 8b 44 e8 40 49 8b 0e
48 85 c9 74 0f 8b 49 f8 eb 0c 66 2e 0f 1f 84 00 00 00 00 00 31 c9 31
d2 f7 f1 <39> c5 75 86 f3 90 eb d2 0f 31 41 8b 76 70 48 8b 4c 24 10 41
3b 76
[ 31.321967] RSP: 002b:00007fcea0d28980 EFLAGS: 00000246
[ 31.323101] RAX: 0000000000000000 RBX: 00007fcf1e079800 RCX: 0000000000000004
[ 31.324480] RDX: 0000000000000002 RSI: 0000000000000137 RDI: 00007fcf13400740
[ 31.325810] RBP: 0000000000000000 R08: 00007fcf144df828 R09: 00007fcf1e0cdc10
[ 31.327196] R10: 0000000000000100 R11: 00007fcf13400030 R12: 00007fcfee0d3f28
[ 31.328618] R13: 0000000000000004 R14: 00007fcfee0d1000 R15: 00007fcfee0ce240
[ 31.330010] </TASK>
[ 31.330721] Modules linked in: uio_hv_generic uio binfmt_misc
dm_crypt intel_rapl_msr intel_rapl_common rpcrdma
intel_uncore_frequency_common isst_if_mbox_msr sunrpc isst_if_common
rdma_ucm ib_iser rdma_cm skx_edac_common nfit ib_umad iw_cm libnvdimm
ib_ipoib libiscsi crct10dif_pclmul ghash_clmulni_intel sha512_ssse3
sha256_ssse3 sha1_ssse3 scsi_transport_iscsi ib_cm aesni_intel
gf128mul crypto_simd cryptd rapl pcspkr hv_utils hv_balloon sg evdev
joydev mpls_router ip_tunnel ramoops pstore_blk pstore_zone efi_pstore
configfs nfnetlink vsock_loopback vmw_vsock_virtio_transport_common
hv_sock vmw_vsock_vmci_transport vsock vmw_vmci efivarfs ip_tables
x_tables autofs4 overlay squashfs dm_verity dm_bufio reed_solomon
dm_mod loop ext4 crc16 mbcache jbd2 crc32c_generic mlx5_ib ib_uverbs
ib_core mlx5_core mlxfw pci_hyperv pci_hyperv_intf hyperv_drm sr_mod
drm_shmem_helper drm_kms_helper cdrom sd_mod hv_storvsc
scsi_transport_fc drm scsi_mod hid_generic hid_hyperv serio_raw hid
hv_netvsc hyperv_keyboard scsi_common hv_vmbus
[ 31.330809] crc32_pclmul crc32c_intel
[ 31.347331] CR2: 00000000000000a0
[ 31.348178] ---[ end trace 0000000000000000 ]---
[ 35.274759] BUG: kernel NULL pointer dereference, address: 00000000000000a0
[ 35.276406] #PF: supervisor read access in kernel mode
[ 35.277688] #PF: error_code(0x0000) - not-present page
[ 35.278974] PGD 80000003075ef067 P4D 80000003075ef067 PUD 0
[ 35.280288] Oops: Oops: 0000 [#3] PREEMPT SMP PTI
[ 35.281501] CPU: 0 UID: 0 PID: 1008 Comm: vpp_main Tainted: G
D 6.12.57+deb13-amd64 #1 Debian 6.12.57-1
[ 35.283706] Tainted: [D]=DIE
[ 35.284643] Hardware name: Microsoft Corporation Virtual
Machine/Virtual Machine, BIOS Hyper-V UEFI Release v4.1 05/13/2024
[ 35.286771] RIP: 0010:hv_uio_channel_cb+0xd/0x20 [uio_hv_generic]
[ 35.288201] Code: 02 00 00 5b 5d e9 53 d8 ef d9 0f 1f 00 90 90 90
90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 0f 1f 44 00 00 48
8b 47 10 <48> 8b b8 a0 00 00 00 f0 83 44 24 fc 00 e9 51 3f ff ff 90 90
90 90
[ 35.291920] RSP: 0000:ffffa922071cfef0 EFLAGS: 00010046
[ 35.293179] RAX: 0000000000000000 RBX: 0000000000000017 RCX: 0000000000000017
[ 35.294753] RDX: 0000000000000001 RSI: ffffffffffffffff RDI: ffff998fdbdf8000
[ 35.296292] RBP: ffff998fc1b31200 R08: ffff998fc1b31200 R09: 0000000000000000
[ 35.297830] R10: 0000000000000000 R11: 0000000000000000 R12: ffff9992f1c41460
[ 35.299480] R13: ffff998fdbdf8000 R14: ffff998fdbdf82a0 R15: ffffffffc100a160
[ 35.301039] FS: 00007fcfedc5ab00(0000) GS:ffff9992f1c00000(0000)
knlGS:0000000000000000
[ 35.302758] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 35.304064] CR2: 00000000000000a0 CR3: 0000000150a4e004 CR4: 00000000003706f0
[ 35.305642] Call Trace:
[ 35.306470] <TASK>
[ 35.308091] vmbus_isr+0x1a5/0x210 [hv_vmbus]
[ 35.309235] __sysvec_hyperv_callback+0x32/0x60
[ 35.310384] sysvec_hyperv_callback+0x38/0x90
[ 35.311556] asm_sysvec_hyperv_callback+0x1a/0x20
[ 35.312689] RIP: 0033:0x7fcfedf426fc
[ 35.313702] Code: 49 2b 56 30 41 0f b6 4e 44 48 d3 ea 48 85 d2 75
68 66 49 0f 6e d4 66 0f 62 d3 66 0f 5c d4 66 0f 28 ca 66 0f 15 ca f2
0f 58 ca <f2> 0f 59 e9 f2 41 0f 58 6e 50 66 0f 2e e8 76 94 48 8b 05 6d
c7 18
[ 35.317465] RSP: 002b:00007fcf1285fc80 EFLAGS: 00000246
[ 35.318710] RAX: 00000000477f033e RBX: 0000000000000003 RCX: 0000000000000022
[ 35.320334] RDX: 0000000000000000 RSI: 0000000000000004 RDI: 00007fcf13400740
[ 35.321894] RBP: 0000000000000004 R08: 0000000000000007 R09: 00007fcf1413ea28
[ 35.323385] R10: 0000000000001965 R11: 0000000000000000 R12: 00000006969940b8
[ 35.324878] R13: 0000000000000004 R14: 00007fcf13400740 R15: 00007fcfee0d3f28
[ 35.326423] </TASK>
[ 35.327186] Modules linked in: uio_hv_generic uio binfmt_misc
dm_crypt intel_rapl_msr intel_rapl_common rpcrdma
intel_uncore_frequency_common isst_if_mbox_msr sunrpc isst_if_common
rdma_ucm ib_iser rdma_cm skx_edac_common nfit ib_umad iw_cm libnvdimm
ib_ipoib libiscsi crct10dif_pclmul ghash_clmulni_intel sha512_ssse3
sha256_ssse3 sha1_ssse3 scsi_transport_iscsi ib_cm aesni_intel
gf128mul crypto_simd cryptd rapl pcspkr hv_utils hv_balloon sg evdev
joydev mpls_router ip_tunnel ramoops pstore_blk pstore_zone efi_pstore
configfs nfnetlink vsock_loopback vmw_vsock_virtio_transport_common
hv_sock vmw_vsock_vmci_transport vsock vmw_vmci efivarfs ip_tables
x_tables autofs4 overlay squashfs dm_verity dm_bufio reed_solomon
dm_mod loop ext4 crc16 mbcache jbd2 crc32c_generic mlx5_ib ib_uverbs
ib_core mlx5_core mlxfw pci_hyperv pci_hyperv_intf hyperv_drm sr_mod
drm_shmem_helper drm_kms_helper cdrom sd_mod hv_storvsc
scsi_transport_fc drm scsi_mod hid_generic hid_hyperv serio_raw hid
hv_netvsc hyperv_keyboard scsi_common hv_vmbus
[ 35.327278] crc32_pclmul crc32c_intel
[ 35.347826] CR2: 00000000000000a0
[ 35.348819] ---[ end trace 0000000000000000 ]---
[ 45.629814] RIP: 0010:hv_uio_channel_cb+0xd/0x20 [uio_hv_generic]
[ 45.631166] Code: 02 00 00 5b 5d e9 53 d8 ef d9 0f 1f 00 90 90 90
90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 0f 1f 44 00 00 48
8b 47 10 <48> 8b b8 a0 00 00 00 f0 83 44 24 fc 00 e9 51 3f ff ff 90 90
90 90
[ 45.634659] RSP: 0000:ffffa92213647ef0 EFLAGS: 00010046
[ 45.635762] RAX: 0000000000000000 RBX: 0000000000000018 RCX: 0000000000000018
[ 45.637168] RDX: 0000000000000001 RSI: ffffffffffffffff RDI: ffff998fdbdfb000
[ 45.638559] RBP: ffff998fc1b2f200 R08: ffff998fc1b2f200 R09: 0000000000000000
[ 45.639911] R10: 0000000000000000 R11: 0000000000000000 R12: ffff9992f1cc1460
[ 45.641269] R13: ffff998fdbdfb000 R14: ffff998fdbdfb2a0 R15: ffffffffc100a160
[ 45.642633] FS: 00007fcea0f2a6c0(0000) GS:ffff9992f1c80000(0000)
knlGS:0000000000000000
[ 45.644064] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 45.645346] CR2: 00000000000000a0 CR3: 0000000150a4e001 CR4: 00000000003706f0
[ 45.646697] Kernel panic - not syncing: Fatal exception in interrupt
[ 46.640112] Kernel Offset: 0x19200000 from 0xffffffff81000000
(relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[ 46.672487] pstore: dump skipped in Panic path because of concurrent dump
[ 46.674016] ---[ end Kernel panic - not syncing: Fatal exception in
interrupt ]---
Thanks,
Peter.
>
> I will preapre the report upstream, and upstream might have questions
> back to you.
>
> Regards,
> Salvatore
Reply to: