[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#1010073: marked as done (nvme read overhead sometimes, system hangs)

Your message dated Sat, 10 Aug 2024 14:10:25 +0200 (CEST)
with message-id <20240810121025.84F37BE2DE0@eldamar.lan>
and subject line Closing this bug (BTS maintenance for src:linux bugs)
has caused the Debian Bug report #1010073,
regarding nvme read overhead sometimes, system hangs
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@bugs.debian.org
immediately.)


-- 
1010073: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1010073
Debian Bug Tracking System
Contact owner@bugs.debian.org with problems
--- Begin Message --- 
Package: linux-image-4.19.0-20-amd64
Version: 4.19.235-1
Hello!
Sometimes in the evenings I see such a picture in atop (see attachment). Some nvme disks busy more then 100%. In /var/log/messages: 

Apr 23 20:59:17 nl104 kernel: [158003.256965] Modules linked in: binfmt_misc tcp_bbr sch_fq mst_pciconf(OE) msr aufs(OE) amd64_edac_mod edac_mce_amd kvm_amd kvm irqbypass crct10dif_pclmul crc32_pclmul efi_pstore g
hash_clmulni_intel efivars pcspkr ipmi_ssif nls_ascii nls_cp437 vfat fat ast ttm drm_kms_helper joydev drm ccp evdev i2c_algo_bit ipmi_si rng_core sp5100_tco ipmi_devintf ipmi_msghandler pcc_cpufreq acpi_cpufreq b
utton efivarfs ip_tables x_tables autofs4 ext4 crc16 mbcache jbd2 crc32c_generic fscrypto ecb hid_generic usbhid hid crc32c_intel aesni_intel mlx5_core(OE) aes_x86_64 mlxfw(OE) crypto_simd psample cryptd xhci_pci
ahci mlxdevm(OE) glue_helper xhci_hcd libahci auxiliary(OE) libata mlx_compat(OE) usbcore nvme devlink scsi_mod nvme_core i2c_piix4 usb_common [last unloaded: mst_pci]
Apr 23 20:59:17 nl104 kernel: [158003.257004] CPU: 42 PID: 0 Comm: swapper/42 Tainted: G           OE     4.19.0-20-amd64 #1 Debian 4.19.235-1
Apr 23 20:59:17 nl104 kernel: [158003.257004] Hardware name: Supermicro AS -1124US-TNRP/H12DSU-iN, BIOS 2.3a 03/03/2022
Apr 23 20:59:17 nl104 kernel: [158003.257010] RIP: 0010:_raw_spin_unlock_irqrestore+0x11/0x20
Apr 23 20:59:17 nl104 kernel: [158003.257012] Code: d8 48 3d 90 d0 03 00 76 cc 80 4d 00 08 eb 98 90 90 90 90 90 90 90 90 90 90 0f 1f 44 00 00 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 c3 66 0f 1f 84 00
00 00 00 00 0f 1f 44 00 00 8b 07
Apr 23 20:59:17 nl104 kernel: [158003.257013] RSP: 0018:ffff97d24d683e88 EFLAGS: 00000282 ORIG_RAX: ffffffffffffff13
Apr 23 20:59:17 nl104 kernel: [158003.257014] RAX: 00000000000000c1 RBX: ffffd8500dfc4020 RCX: 000000008040003e
Apr 23 20:59:17 nl104 kernel: [158003.257015] RDX: 000000008040003f RSI: 0000000000000282 RDI: 0000000000000282
Apr 23 20:59:17 nl104 kernel: [158003.257015] RBP: 0000000000000037 R08: 0000000000000000 R09: ffffffff83cf6000
Apr 23 20:59:17 nl104 kernel: [158003.257016] R10: ffff97b5373cc0c0 R11: 0000000000000001 R12: ffff98d1e8839858
Apr 23 20:59:17 nl104 kernel: [158003.257016] R13: 0000000000000282 R14: ffff98d1e8839140 R15: ffffd8500dfc6028
Apr 23 20:59:17 nl104 kernel: [158003.257017] FS:  0000000000000000(0000) GS:ffff97d24d680000(0000) knlGS:0000000000000000
Apr 23 20:59:17 nl104 kernel: [158003.257018] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Apr 23 20:59:17 nl104 kernel: [158003.257018] CR2: 0000562c3a963f29 CR3: 000000721580a000 CR4: 0000000000340ee0
Apr 23 20:59:17 nl104 kernel: [158003.257019] Call Trace:
Apr 23 20:59:17 nl104 kernel: [158003.257020]  <IRQ>
Apr 23 20:59:17 nl104 kernel: [158003.257023]  fq_flush_timeout+0x6a/0x90
Apr 23 20:59:17 nl104 kernel: [158003.257026]  ? fq_ring_free+0xd0/0xd0
Apr 23 20:59:17 nl104 kernel: [158003.257028]  call_timer_fn+0x2b/0x130
Apr 23 20:59:17 nl104 kernel: [158003.257030]  run_timer_softirq+0x1c7/0x3e0
Apr 23 20:59:17 nl104 kernel: [158003.257033]  ? recalibrate_cpu_khz+0x10/0x10
Apr 23 20:59:17 nl104 kernel: [158003.257034]  ? ktime_get+0x3a/0xa0
Apr 23 20:59:17 nl104 kernel: [158003.257036]  __do_softirq+0xde/0x2d8
Apr 23 20:59:17 nl104 kernel: [158003.257039]  irq_exit+0xba/0xc0
Apr 23 20:59:17 nl104 kernel: [158003.257040]  smp_apic_timer_interrupt+0x74/0x140
Apr 23 20:59:17 nl104 kernel: [158003.257041]  apic_timer_interrupt+0xf/0x20
Apr 23 20:59:17 nl104 kernel: [158003.257042]  </IRQ>
Apr 23 20:59:17 nl104 kernel: [158003.257045] RIP: 0010:cpuidle_enter_state+0xb9/0x320
Apr 23 20:59:17 nl104 kernel: [158003.257046] Code: e8 ec 3f b2 ff 80 7c 24 0b 00 74 17 9c 58 0f 1f 44 00 00 f6 c4 02 0f 85 3b 02 00 00 31 ff e8 1e cd b7 ff fb 66 0f 1f 44 00 00 <48> b8 ff ff ff ff f3 01 00 00 48
2b 1c 24 ba ff ff ff 7f 48 39 c3
Apr 23 20:59:17 nl104 kernel: [158003.257046] RSP: 0018:ffffb94fc05a7e90 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13
Apr 23 20:59:17 nl104 kernel: [158003.257047] RAX: ffff97d24d6a7140 RBX: 00008faf279e9658 RCX: 000000000000001f
Apr 23 20:59:17 nl104 kernel: [158003.257047] RDX: 00008faf279e9658 RSI: 0000000038e39189 RDI: 0000000000000000
Apr 23 20:59:17 nl104 kernel: [158003.257048] RBP: ffff98d1c417a400 R08: 0000000000000002 R09: 0000000000026a00
Apr 23 20:59:17 nl104 kernel: [158003.257048] R10: 00014381056cf8f5 R11: ffff97d24d6a6128 R12: 0000000000000001
Apr 23 20:59:17 nl104 kernel: [158003.257049] R13: ffffffff848ba558 R14: 0000000000000001 R15: 0000000000000000
Apr 23 20:59:17 nl104 kernel: [158003.257052]  do_idle+0x228/0x270
Apr 23 20:59:17 nl104 kernel: [158003.257054]  cpu_startup_entry+0x6f/0x80
Apr 23 20:59:17 nl104 kernel: [158003.257056]  start_secondary+0x1a4/0x200
Apr 23 20:59:17 nl104 kernel: [158003.257058]  secondary_startup_64+0xa4/0xb0
Apr 23 20:59:17 nl104 kernel: [158003.575053] Sending NMI from CPU 61 to CPUs 42:
Apr 23 20:59:17 nl104 kernel: [158003.576064] NMI backtrace for cpu 42
Apr 23 20:59:17 nl104 kernel: [158003.576064] CPU: 42 PID: 0 Comm: swapper/42 Tainted: G           OEL    4.19.0-20-amd64 #1 Debian 4.19.235-1
Apr 23 20:59:17 nl104 kernel: [158003.576065] Hardware name: Supermicro AS -1124US-TNRP/H12DSU-iN, BIOS 2.3a 03/03/2022
Apr 23 20:59:17 nl104 kernel: [158003.576065] RIP: 0010:native_queued_spin_lock_slowpath+0x52/0x190
Apr 23 20:59:17 nl104 kernel: [158003.576066] Code: 74 37 81 e6 00 ff ff ff 75 5f f0 0f ba 2f 08 8b 07 72 56 89 c2 30 e6 a9 00 00 ff ff 75 47 85 d2 74 0e 8b 07 84 c0 74 08 f3 90 <8b> 07 84 c0 75 f8 b8 01 00 00 00
66 89 07 c3 8b 37 81 fe 00 01 00
Apr 23 20:59:17 nl104 kernel: [158003.576067] RSP: 0018:ffff97d24d683e78 EFLAGS: 00000002
Apr 23 20:59:17 nl104 kernel: [158003.576067] RAX: 0000000000000101 RBX: 0000000000000282 RCX: 000000000000003a
Apr 23 20:59:17 nl104 kernel: [158003.576068] RDX: 0000000000000001 RSI: 0000000000000000 RDI: ffffd8500e086028
Apr 23 20:59:17 nl104 kernel: [158003.576068] RBP: 000000000000003a R08: 0000000000000000 R09: ffffffff83cf6000
Apr 23 20:59:17 nl104 kernel: [158003.576069] R10: ffff97d13d229280 R11: 0000000000000001 R12: ffff98d1e8839858
Apr 23 20:59:17 nl104 kernel: [158003.576069] R13: 0000000000000282 R14: ffff98d1e8839140 R15: ffffd8500e086028
Apr 23 20:59:17 nl104 kernel: [158003.576070] FS:  0000000000000000(0000) GS:ffff97d24d680000(0000) knlGS:0000000000000000
Apr 23 20:59:17 nl104 kernel: [158003.576070] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Apr 23 20:59:17 nl104 kernel: [158003.576071] CR2: 0000562c3a963f29 CR3: 000000721580a000 CR4: 0000000000340ee0
Apr 23 20:59:17 nl104 kernel: [158003.576071] Call Trace:
Apr 23 20:59:17 nl104 kernel: [158003.576071]  <IRQ>
Apr 23 20:59:17 nl104 kernel: [158003.576071]  _raw_spin_lock_irqsave+0x32/0x40
Apr 23 20:59:17 nl104 kernel: [158003.576072]  fq_flush_timeout+0x51/0x90
Apr 23 20:59:17 nl104 kernel: [158003.576072]  ? fq_ring_free+0xd0/0xd0
Apr 23 20:59:17 nl104 kernel: [158003.576072]  call_timer_fn+0x2b/0x130
Apr 23 20:59:17 nl104 kernel: [158003.576072]  run_timer_softirq+0x1c7/0x3e0
Apr 23 20:59:17 nl104 kernel: [158003.576072]  ? recalibrate_cpu_khz+0x10/0x10
Apr 23 20:59:17 nl104 kernel: [158003.576073]  ? ktime_get+0x3a/0xa0
Apr 23 20:59:17 nl104 kernel: [158003.576073]  __do_softirq+0xde/0x2d8
Apr 23 20:59:17 nl104 kernel: [158003.576073]  irq_exit+0xba/0xc0
Apr 23 20:59:17 nl104 kernel: [158003.576073]  smp_apic_timer_interrupt+0x74/0x140
Apr 23 20:59:17 nl104 kernel: [158003.576073]  apic_timer_interrupt+0xf/0x20
Apr 23 20:59:17 nl104 kernel: [158003.576073]  </IRQ>
Apr 23 20:59:17 nl104 kernel: [158003.576074] RIP: 0010:cpuidle_enter_state+0xb9/0x320
Apr 23 20:59:17 nl104 kernel: [158003.576074] Code: e8 ec 3f b2 ff 80 7c 24 0b 00 74 17 9c 58 0f 1f 44 00 00 f6 c4 02 0f 85 3b 02 00 00 31 ff e8 1e cd b7 ff fb 66 0f 1f 44 00 00 <48> b8 ff ff ff ff f3 01 00 00 48 2b 1c 24 ba ff ff ff 7f 48 39 c3
Apr 23 20:59:17 nl104 kernel: [158003.576074] RSP: 0018:ffffb94fc05a7e90 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13
Apr 23 20:59:17 nl104 kernel: [158003.576075] RAX: ffff97d24d6a7140 RBX: 00008faf279e9658 RCX: 000000000000001f
Apr 23 20:59:17 nl104 kernel: [158003.576075] RDX: 00008faf279e9658 RSI: 0000000038e39189 RDI: 0000000000000000
Apr 23 20:59:17 nl104 kernel: [158003.576076] RBP: ffff98d1c417a400 R08: 0000000000000002 R09: 0000000000026a00
Apr 23 20:59:17 nl104 kernel: [158003.576076] R10: 00014381056cf8f5 R11: ffff97d24d6a6128 R12: 0000000000000001
Apr 23 20:59:17 nl104 kernel: [158003.576076] R13: ffffffff848ba558 R14: 0000000000000001 R15: 0000000000000000
Apr 23 20:59:17 nl104 kernel: [158003.576076]  do_idle+0x228/0x270
Apr 23 20:59:17 nl104 kernel: [158003.576076]  cpu_startup_entry+0x6f/0x80
Apr 23 20:59:17 nl104 kernel: [158003.576077]  start_secondary+0x1a4/0x200
Apr 23 20:59:17 nl104 kernel: [158003.576077]  secondary_startup_64+0xa4/0xb0
--
WBR, Andrey Vasilishin

Attachment: nvme_bug.png
Description: PNG image


--- End Message ---
--- Begin Message --- 
Hi

This bug was filed for a very old kernel or the bug is old itself
without resolution.

If you can reproduce it with

- the current version in unstable/testing
- the latest kernel from backports

please reopen the bug, see https://www.debian.org/Bugs/server-control
for details.

Regards,
Salvatore

--- End Message ---
Reply to: