Package: src:linux Version: 3.2.46-1 Severity: normal Hello, I have several hosts running wheezy that randomly panic and reboot anywhere between a day and a few months of uptime. Kernel package versions: linux-image-3.2.0-4-amd64 3.2.46-1 linux-image-3.9-0.bpo.1-amd64 3.9.6-1~bpo70+1 The panic output in the log below points at the ccm crypto module. I have an exotic software configuration of which I may be the only user encountering this bug, and I will be re-configuring to work-around the issue (assuming that use of the AES-CCM encryption module is the problem), therefore this bug report may be irrelevant. Most hosts are running Xen (dom0/domU are all wheezy, fortunately dom0 hasn't crashed yet) and these hosts have AES-NI CPU instructions (aesni_intel crypto module). I've only seen a single non-Xen (and non-AES-NI) host panic once, but didn't capture the kernel output, so I'm not sure it's the same problem (however it was a week after activating AES-CCM configuration, so my gut feeling is that it is the same problem). Each host is using IPsec to secure traffic to a backup system. I've correlated at least two panics to have happend minutes after the backup process was completed (after a heavy spike in encrypted traffic perhaps?). For the IPsec configuration, my goal was to gain speed by using AES-NI CPU instructions, and make use of the AES-GCM (RFC 4106) or AES-CCM (RFC 4309) ciphers (efficient AES-CTR mode with authentication), and still have strong crypto (AES 256 key length). I'd prefer AES-GCM, but the netlink XFRM interface returns an error because 256 key length isn't implemented in the aesni_intel module¹, so I switched to [likely slower] AES-CCM. I'll will be changing IPsec configurations to the more classic AES-CBC modes and report back whether the panics stopped. Thanks Debian Kernel Team! ¹ https://wiki.strongswan.org/issues/341 -- Package-specific info: ** Version: Linux version 3.2.0-4-amd64 (debian-kernel@lists.debian.org) (gcc version 4.6.3 (Debian 4.6.3-14) ) #1 SMP Debian 3.2.46-1 ** Command line: root=/dev/xvda1 ro ** Not tainted ** Kernel log: [3176120.307128] BUG: unable to handle kernel NULL pointer dereference at 000000000000000c [3176120.307150] IP: [<ffffffff8118c422>] scatterwalk_start+0x3/0x12 [3176120.307167] PGD 0 [3176120.307172] Oops: 0000 [#1] SMP [3176120.307178] CPU 0 [3176120.307181] Modules linked in: ccm ipcomp6 xfrm6_tunnel tunnel6 deflate zlib_deflate nfnetlink_log nfnetlink twofish_generic twofish_x86_64_3way twofish_x86_64 twofish_common serpent blowfish_generic blowfish_x86_64 blowfish_common cast5 ctr ecb des_generic cbc algif_skcipher camellia xcbc hmac md4 sha512_generic sha256_generic sha1_ssse3 sha1_generic algif_hash af_alg xfrm6_mode_tunnel xfrm4_mode_tunnel seqiv esp6 xfrm6_mode_transport ipt_REJECT xfrm_user xfrm4_tunnel tunnel4 ipcomp xfrm_ipcomp esp4 ah4 af_key xt_multiport ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack iptable_filter ip_tables x_tables coretemp crc32c_intel ghash_clmulni_intel xen_netfront aesni_intel aes_x86_64 aes_generic cryptd ext4 crc16 mbcache jbd2 xen_blkfront [3176120.307277] [3176120.307282] Pid: 110, comm: kworker/0:1 Not tainted 3.2.0-4-amd64 #1 Debian 3.2.46-1 [3176120.307290] RIP: e030:[<ffffffff8118c422>] [<ffffffff8118c422>] scatterwalk_start+0x3/0x12 [3176120.307299] RSP: e02b:ffff88007b933d08 EFLAGS: 00010282 [3176120.307304] RAX: 0000000000000002 RBX: ffff88007bcfb8b8 RCX: 0000000000000000 [3176120.307314] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff88007b933d40 [3176120.307324] RBP: 0000000000000000 R08: ffff88007bcfb8b8 R09: ffff88007bcfb860 [3176120.307334] R10: ffff880070267cd0 R11: ffff88007bcfb8b8 R12: ffff880070267c00 [3176120.307342] R13: ffff880070267c00 R14: ffff88007b933d40 R15: ffff88007bcfbaf0 [3176120.307352] FS: 00007f3af218d7c0(0000) GS:ffff88007fc00000(0000) knlGS:0000000000000000 [3176120.307359] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b [3176120.307364] CR2: 000000000000000c CR3: 0000000005c99000 CR4: 0000000000002660 [3176120.307371] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [3176120.307377] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [3176120.307384] Process kworker/0:1 (pid: 110, threadinfo ffff88007b932000, task ffff88007bcdc300) [3176120.307390] Stack: [3176120.307393] ffffffffa022e29a ffff88007bcfba68 ffff880070013d86 ffffffff81003bae [3176120.307512] ffffffff81003007 ffffffffa00a93e0 ffffffffa00aa0e5 0000000000000000 [3176120.307521] ffff880070013d86 ffff88007bcdc6c0 ffff88007bcfb8b8 ffff88007bcfb860 [3176120.307530] Call Trace: [3176120.307538] [<ffffffffa022e29a>] ? get_data_to_compute+0x26/0x231 [ccm] [3176120.307547] [<ffffffff81003bae>] ? xen_mc_issue.constprop.23+0x3a/0x49 [3176120.307553] [<ffffffff81003007>] ? arch_local_irq_restore+0x7/0x8 [3176120.307564] [<ffffffffa00a93e0>] ? kernel_fpu_end+0x11/0x14 [aesni_intel] [3176120.307574] [<ffffffffa00aa0e5>] ? ctr_crypt+0x10d/0x13a [aesni_intel] [3176120.307580] [<ffffffffa022e5bf>] ? crypto_ccm_auth+0x11a/0x145 [ccm] [3176120.307587] [<ffffffffa022e621>] ? crypto_ccm_decrypt_done+0x37/0x69 [ccm] [3176120.307597] [<ffffffffa00201ea>] ? cryptd_blkcipher_crypt+0x53/0x5f [cryptd] [3176120.307606] [<ffffffff8104b552>] ? arch_local_save_flags+0x7/0x8 [3176120.307615] [<ffffffffa002077c>] ? cryptd_queue_worker+0x4c/0x6a [cryptd] [3176120.307623] [<ffffffff8105b389>] ? process_one_work+0x161/0x264 [3176120.307629] [<ffffffff8105a935>] ? cwq_activate_delayed_work+0x3c/0x48 [3176120.307635] [<ffffffff8105c34d>] ? worker_thread+0xc2/0x145 [3176120.307641] [<ffffffff8105c28b>] ? manage_workers.isra.25+0x15b/0x15b [3176120.307648] [<ffffffff8105f48d>] ? kthread+0x76/0x7e [3176120.307658] [<ffffffff81355cb4>] ? kernel_thread_helper+0x4/0x10 [3176120.307669] [<ffffffff81353d73>] ? int_ret_from_sys_call+0x7/0x1b [3176120.307680] [<ffffffff8134ee7c>] ? retint_restore_args+0x5/0x6 [3176120.307689] [<ffffffff81355cb0>] ? gs_change+0x13/0x13 [3176120.307693] Code: 01 c8 48 c1 f8 03 48 0f af c2 48 ba 00 00 00 00 00 88 ff ff 48 c1 e0 0c 48 01 d0 8b 57 08 81 e2 ff 0f 00 00 48 01 d0 c3 48 89 37 <83> 7e 0c 00 75 02 0f 0b 8b 46 08 89 47 08 c3 85 f6 74 1c 8b 4f [3176120.307745] RIP [<ffffffff8118c422>] scatterwalk_start+0x3/0x12 [3176120.307752] RSP <ffff88007b933d08> [3176120.307755] CR2: 000000000000000c [3176120.307768] ---[ end trace deea02a26eb3538a ]--- [3176120.307774] Kernel panic - not syncing: Fatal exception in interrupt [3176120.307780] Pid: 110, comm: kworker/0:1 Tainted: G D 3.2.0-4-amd64 #1 Debian 3.2.46-1 [3176120.307787] Call Trace: [3176120.307793] [<ffffffff813481ed>] ? panic+0x95/0x1a2 [3176120.307799] [<ffffffff81070df3>] ? arch_local_irq_disable+0x7/0x8 [3176120.307806] [<ffffffff8134eb77>] ? _raw_spin_unlock_irqrestore+0xe/0xf [3176120.307813] [<ffffffff8134f986>] ? oops_end+0xa9/0xb6 [3176120.307819] [<ffffffff81347b36>] ? no_context+0x1ff/0x20e [3176120.307824] [<ffffffff8104b1f4>] ? ns_to_timeval+0x9/0x27 [3176120.307835] [<ffffffff81351984>] ? do_page_fault+0x1b6/0x345 [3176120.307848] [<ffffffff810ea427>] ? arch_local_irq_restore+0x7/0x8 [3176120.307859] [<ffffffff810ea427>] ? arch_local_irq_restore+0x7/0x8 [3176120.307866] [<ffffffff810ead50>] ? kmem_cache_free+0x2d/0x69 [3176120.307874] [<ffffffff812c6d05>] ? tcp_data_snd_check+0x38/0xfb [3176120.307880] [<ffffffff812ca21c>] ? tcp_rcv_established+0xe4/0x562 [3176120.307889] [<ffffffff81003f47>] ? arch_local_irq_restore+0x7/0x8 [3176120.307896] [<ffffffff8134f0f5>] ? page_fault+0x25/0x30 [3176120.307901] [<ffffffff8118c422>] ? scatterwalk_start+0x3/0x12 [3176120.307908] [<ffffffffa022e29a>] ? get_data_to_compute+0x26/0x231 [ccm] [3176120.307915] [<ffffffff81003bae>] ? xen_mc_issue.constprop.23+0x3a/0x49 [3176120.307920] [<ffffffff81003007>] ? arch_local_irq_restore+0x7/0x8 [3176120.307929] [<ffffffffa00a93e0>] ? kernel_fpu_end+0x11/0x14 [aesni_intel] [3176120.307939] [<ffffffffa00aa0e5>] ? ctr_crypt+0x10d/0x13a [aesni_intel] [3176120.307946] [<ffffffffa022e5bf>] ? crypto_ccm_auth+0x11a/0x145 [ccm] [3176120.307952] [<ffffffffa022e621>] ? crypto_ccm_decrypt_done+0x37/0x69 [ccm] [3176120.307962] [<ffffffffa00201ea>] ? cryptd_blkcipher_crypt+0x53/0x5f [cryptd] [3176120.307969] [<ffffffff8104b552>] ? arch_local_save_flags+0x7/0x8 [3176120.307978] [<ffffffffa002077c>] ? cryptd_queue_worker+0x4c/0x6a [cryptd] [3176120.308040] [<ffffffff8105b389>] ? process_one_work+0x161/0x264 [3176120.308047] [<ffffffff8105a935>] ? cwq_activate_delayed_work+0x3c/0x48 [3176120.308067] [<ffffffff8105c34d>] ? worker_thread+0xc2/0x145 [3176120.308073] [<ffffffff8105c28b>] ? manage_workers.isra.25+0x15b/0x15b [3176120.308081] [<ffffffff8105f48d>] ? kthread+0x76/0x7e [3176120.308089] [<ffffffff81355cb4>] ? kernel_thread_helper+0x4/0x10 [3176120.308097] [<ffffffff81353d73>] ? int_ret_from_sys_call+0x7/0x1b [3176120.308105] [<ffffffff8134ee7c>] ? retint_restore_args+0x5/0x6 [3176120.308113] [<ffffffff81355cb0>] ? gs_change+0x13/0x13 ** Model information not available ** Loaded modules: esp6 ccm ipcomp6 xfrm6_tunnel tunnel6 xfrm6_mode_transport xt_multiport seqiv ecb algif_skcipher md4 algif_hash af_alg xfrm_user xfrm4_tunnel tunnel4 ipcomp xfrm_ipcomp esp4 ah4 deflate zlib_deflate ctr twofish_generic twofish_x86_64_3way twofish_x86_64 twofish_common camellia serpent blowfish_generic blowfish_x86_64 blowfish_common cast5 des_generic cbc xcbc rmd160 sha512_generic ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables ipt_REJECT xt_tcpudp sha256_generic nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack iptable_filter ip_tables x_tables sha1_ssse3 sha1_generic hmac crypto_null af_key coretemp crc32c_intel ghash_clmulni_intel aesni_intel xen_netfront aes_x86_64 aes_generic cryptd ext4 crc16 mbcache jbd2 xen_blkfront ** PCI devices: not available ** USB devices: not available -- System Information: Debian Release: 7.1 APT prefers stable APT policy: (500, 'stable') Architecture: amd64 (x86_64) Kernel: Linux 3.2.0-4-amd64 (SMP w/4 CPU cores) Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8) (ignored: LC_ALL set to en_US.UTF-8) Shell: /bin/sh linked to /bin/dash Versions of packages linux-image-3.2.0-4-amd64 depends on: ii debconf [debconf-2.0] 1.5.49 ii initramfs-tools [linux-initramfs-tool] 0.109.1 ii kmod 9-3 ii linux-base 3.5 ii module-init-tools 9-3 Versions of packages linux-image-3.2.0-4-amd64 recommends: pn firmware-linux-free <none> Versions of packages linux-image-3.2.0-4-amd64 suggests: pn debian-kernel-handbook <none> ii grub-pc 1.99-27+deb7u1 pn linux-doc-3.2 <none> Versions of packages linux-image-3.2.0-4-amd64 is related to: pn firmware-atheros <none> pn firmware-bnx2 <none> pn firmware-bnx2x <none> pn firmware-brcm80211 <none> pn firmware-intelwimax <none> pn firmware-ipw2x00 <none> pn firmware-ivtv <none> pn firmware-iwlwifi <none> pn firmware-libertas <none> pn firmware-linux <none> pn firmware-linux-nonfree <none> pn firmware-myricom <none> pn firmware-netxen <none> pn firmware-qlogic <none> pn firmware-ralink <none> pn firmware-realtek <none> pn xen-hypervisor <none> -- debconf information: linux-image-3.2.0-4-amd64/postinst/depmod-error-initrd-3.2.0-4-amd64: false linux-image-3.2.0-4-amd64/prerm/removing-running-kernel-3.2.0-4-amd64: true linux-image-3.2.0-4-amd64/postinst/ignoring-ramdisk: linux-image-3.2.0-4-amd64/postinst/missing-firmware-3.2.0-4-amd64: -- Gerald Turner Email: gturner@unzane.com JID: gturner@unzane.com GPG: 0xFA8CD6D5 21D9 B2E8 7FE7 F19E 5F7D 4D0C 3FA0 810F FA8C D6D5
Attachment:
pgplLkEC9d6Ja.pgp
Description: PGP signature