Bug#562815: Bug in kernel audit code makes system crash
Package: linux-image-2.6
Version: 2.6.26-2-686, 2.6.26-2-amd64, 2.6.26-2-openvz-amd64, 2.6.26-2-xen-amd64
My system crashes when I do auditctl -R audit.rules. I have figured out that the problem occurs after unloading and then loading some certain rules that include "-F dir=/dev" or "-F dir=/lib", or some other directory watch. For example:
# auditctl -a exit,always -F arch=b64 -S creat -F dir=/lib
# auditctl -d exit,always -F arch=b64 -S creat -F dir=/lib
# auditctl -a exit,always -F arch=b64 -S creat -F dir=/lib
result in
[ 154.715581] ------------[ cut here ]------------
[ 154.715581] kernel BUG at fs/inode.c:1152!
[ 154.715581] invalid opcode: 0000 [1] SMP
[ 154.715581] CPU 0
[ 154.715581] Modules linked in: ipv6 evdev ext3 jbd mbcache dm_mirror dm_log dm_snapshot dm_mod thermal_sys
[ 154.715581] Pid: 1256, comm: audit_prune_tre Not tainted 2.6.26-2-xen-amd64 #1
[ 154.715581] RIP: e030:[<ffffffff8029ceb0>] [<ffffffff8029ceb0>] iput+0x13/0x60
[ 154.715581] RSP: e02b:ffff88007dd4fe80 EFLAGS: 00010246
[ 154.715581] RAX: ffff88007da63701 RBX: ffff88007da63748 RCX: 0000000000000001
[ 154.715581] RDX: 0000000000001414 RSI: 0000000000000001 RDI: ffff88007da63748
[ 154.715581] RBP: ffff88007fd9e440 R08: 0000000300000000 R09: ffff88007e94f220
[ 154.715581] R10: ffff88007e2c9840 R11: ffffffff80285a7a R12: 0000000000000002
[ 154.715581] R13: ffff88007e94f180 R14: ffff88007e94f190 R15: 0000000000000000
[ 154.715581] FS: 00007f5a80f176e0(0000) GS:ffffffff8053a000(0000) knlGS:0000000000000000
[ 154.715581] CS: e033 DS: 0000 ES: 0000
[ 154.715581] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 154.715581] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 154.715581] Process audit_prune_tre (pid: 1256, threadinfo ffff88007dd4e000, task ffff88007fd39040)
[ 154.715581] Stack: ffff88007e94f190 ffffffff802b43ff ffff88007e94f190 ffff88007ddc4000
[ 154.715581] ffff88007e94f220 ffffffff802b4604 ffff88007e2c9840 ffffffff8025d69c
[ 154.715581] 0000000300000000 ffff88007e2c9840 ffffffff80255bdd ffff88007eb898a8
[ 154.715581] Call Trace:
[ 154.715581] [<ffffffff802b43ff>] ? put_inotify_watch+0x21/0x4d
[ 154.715581] [<ffffffff802b4604>] ? unpin_inotify_watch+0x11/0x1a
[ 154.715581] [<ffffffff8025d69c>] ? untag_chunk+0x3e6/0x40a
[ 154.715581] [<ffffffff80255bdd>] ? prune_tree_thread+0x0/0x26
[ 154.715581] [<ffffffff8025d6db>] ? prune_one+0x1b/0x66
[ 154.715581] [<ffffffff8025d756>] ? audit_prune_trees+0x30/0x59
[ 154.715581] [<ffffffff80255bf2>] ? prune_tree_thread+0x15/0x26
[ 154.715581] [<ffffffff80255bdd>] ? prune_tree_thread+0x0/0x26
[ 154.715581] [<ffffffff8023f593>] ? kthread+0x47/0x74
[ 154.715581] [<ffffffff8020be28>] ? child_rip+0xa/0x12
[ 154.715581] [<ffffffff8023f54c>] ? kthread+0x0/0x74
[ 154.715581] [<ffffffff8020be1e>] ? child_rip+0x0/0x12
[ 154.715581]
[ 154.715581]
[ 154.715581] Code: 08 02 00 00 77 ff ff ff eb df 48 83 ec 08 e8 d5 76 19 00 41 59 31 c0 c3 48 85 ff 53 48 89 fb 74 55 48 83 bf 08 02 00 00 40 75 04 <0f> 0b eb fe 48 8d 7f 48 48 c7 c6 20 41 5f 80 e8 a4 0f 07 00 85
[ 154.715581] RIP [<ffffffff8029ceb0>] iput+0x13/0x60
[ 154.715581] RSP <ffff88007dd4fe80>
[ 154.715581] ---[ end trace f46939304c84baf0 ]---
I run Debian 5.0.3 on HP Proliant DL180 G6 (Intel Xeon CPU). Audit package version is 1.7.4-1. The problem was first discovered in Xen domU (2.6.26-2-xen-amd64). Then I was able to reproduce it in dom0 and even in non-xen-enabled kernel 2.6.26-2-amd64 and 2.6.26-2-686, openvz version is also affected.
I wrote a letter to kernel audit code developers Steve Grubb, Miloslav Trmac and Alexander Viro. They asked me to try to reproduce this error with a more recent kernel to check if this crash was already fixed. So here follows what I did.
1. Installed linux-image-2.6.30-bpo.2-686 from Debian backports repository and booted it. Not using Xen for this time.
2. On one console I ran
# while [ 1 == 1 ]; do /bin/ls /bin; done
to produce some system load.
3. On another console I ran
# while [ 1 == 1 ]; do pidof auditctl || auditctl -R /etc/audit/audit.rules; done
to reproduce the error. I use pidof auditctl to ensure that each next auditctl -R runs after the previous one has finished.
The test runs well until I add the following rule:
# auditctl -w /var/log -p a -k ACCESS_LOGS
After adding this rule system hangs.
Finally, Alexander Viro had found two bugs in kernel audit code. He suggests to apply these two patches:
http://www.gossamer-threads.com/lists/linux/kernel/1169784
http://www.gossamer-threads.com/lists/linux/kernel/1169785
on top of the following patch commits:
def57543418a5f47debae28a0a9dea2effc11692
8f7b0ba1c853919b85b54774775f567f30006107
Without these patches Debian kernels are completely useless in cases when Linux auditing is enabled and audit rules are configured. Please, include the above patches into Debian stable packages.
Regards,
Sergey Sireskin
Reply to: