+nicholas piggin for the C interrupt stuff Le 06/08/2021 à 03:06, Finn Thain a écrit :
(Christophe, you've seen some of this before, however there are new results added at the end. I've Cc'd the mailing lists this time.) On Wed, 4 Aug 2021, Stan Johnson wrote:On 8/4/21 8:41 PM, Finn Thain wrote:$ curl https://lore.kernel.org/lkml/9b64dde3-6ebd-b446-41d9-61e8cb0d8c39@csgroup.eu/raw ../message.mboxok $ sha1 ../message.mbox SHA1 (../message.mbox) = 436ce0adf893c46c84c54607f73c838897caeeeaOn Wed, 4 Aug 2021, Christophe Leroy wrote:Can you check if they happen at commit c16728835$ git checkout c16728835eec Checking out files: 100% (20728/20728), done. Note: checking out 'c16728835eec'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by performing another checkout. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -b with the checkout command again. Example: git checkout -b <new-branch-name> HEAD is now at c16728835eec powerpc/32: Manage KUAP in C $ git am ../message.mbox warning: Patch sent with format=flowed; space at the end of lines might be lost. Applying: powerpc/32: Dismantle EXC_XFER_STD/LITE/TEMPLATE $ cp ../dot-config-powermac-5.13 .config $ make ARCH=powerpc CROSS_COMPILE=powerpc-linux-gnu- -j4 clean olddefconfig vmlinux $ strings vmlinux | fgrep 'Linux version' Linux version 5.12.0-rc3-pmac-00078-geb51c431b81 (johnson@ThinkPad) (powerpc-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0, GNU ld (GNU Binutils for Debian) 2.31.1) #1 SMP Wed Aug 4 21:50:47 MDT 2021 1) PB 3400c Hangs at boot (Mac OS screen), no serial console output 2) Wallstreet X fails, errors ("Kernel attempted to write user page", "BUG: Unable to handle kernel instruction fetch"), see Wallstreet_console-1.txt.The log shows that the error "Kernel attempted to write user page (b3399774) - exploit attempt?" happens after commit c16728835eec ("powerpc/32: Manage KUAP in C").
I think I found a possible cause for this. After the above patch, locking KUAP on interrupt is done in interrupt_enter_prepare(). But in case of NMI interrupt, that function is not called. That means that when leaving interrupt through interrupt_exit_kernel_prepare(), the supposedly saved previous KUAP status is garbage.
An easy way to fix that is to add missing stuff in interrupt_nmi_enter_prepare(), I'll do that at least for testing, but at the end it is not so easy, because of booke32 and 40x.
The problem on booke32 and 40x is that the "critical interrupts" exit goes through interrupt_return when they happened in user mode and bypass interrupt_return when they happened in kernel mode. So it is not easy to manage.
Can you check if they DO NOT happen at preceding commit c16728835~$ git checkout c16728835~ Previous HEAD position was c16728835eec powerpc/32: Manage KUAP in C HEAD is now at 0b45359aa2df powerpc/8xx: Create C version of kuap save/restore/check helpers $ git am ../message.mbox warning: Patch sent with format=flowed; space at the end of lines might be lost. Applying: powerpc/32: Dismantle EXC_XFER_STD/LITE/TEMPLATE $ cp ../dot-config-powermac-5.13 .config $ make ARCH=powerpc CROSS_COMPILE=powerpc-linux-gnu- -j4 clean olddefconfig vmlinux Linux version 5.12.0-rc3-pmac-00077-gc9f6e8dd045 3) PB 3400c Hangs at boot (Mac OS screen) 4) Wallstreet X fails, errors in console log (different than test 2), see Wallstreet_console-2.txt.This log shows that the errors "xfce4-session[1775]: bus error (7)" and "kernel BUG at arch/powerpc/kernel/interrupt.c:49!" happen prior to commit c16728835eec ("powerpc/32: Manage KUAP in C").
As mentionned by Nic, this is due to r11 being cloberred. For the time being the only r11 clobber identified is the one I have provided a fix for. I'm wondering whether it was applied for all further tests or not.
$ git checkout 0b45359aa2df ... HEAD is now at 0b45359aa2df powerpc/8xx: Create C version of kuap save/restore/check helpers $ git am ../message.mbox warning: Patch sent with format=flowed; space at the end of lines might be lost. Applying: powerpc/32: Dismantle EXC_XFER_STD/LITE/TEMPLATE $ cp ../dot-config-powermac-5.13 .config $ make ARCH=powerpc CROSS_COMPILE=powerpc-linux-gnu- -j4 clean olddefconfig vmlinux Linux version 5.12.0-rc3-pmac-00077-ge06b29ce146 5) PB 3400c Hangs at boot (Mac OS screen) 6) Wallstreet X failed (X login succeeded, but setting up desktop failed), errors in console log, see Wallstreet_console-3.txt.(No need for those two tests: it's exactly the same code and almost the same failure modes: "kernel BUG at arch/powerpc/kernel/interrupt.c:50".) On Thu, 5 Aug 2021, Stan Johnson wrote:On 8/5/21 12:47 AM, Finn Thain wrote:On Wed, 4 Aug 2021, Christophe Leroy wrote:Could you test without CONFIG_PPC_KUAP... $ git checkout c16728835eec ... HEAD is now at c16728835eec powerpc/32: Manage KUAP in C $ git am ../message.mbox warning: Patch sent with format=flowed; space at the end of lines might be lost. Applying: powerpc/32: Dismantle EXC_XFER_STD/LITE/TEMPLATE $ cp ../dot-config-powermac-5.13 .config $ scripts/config -d CONFIG_PPC_KUAP $ make ARCH=powerpc CROSS_COMPILE=powerpc-linux-gnu- -j4 clean olddefconfig vmlinux $ grep CONFIG_PPC_KUAP .config # CONFIG_PPC_KUAP is not set Linux version 5.12.0-rc3-pmac-00078-g5cac2bc3752 7) PB 3400c Hangs at boot (Mac OS screen) 8) Wallstreet Everything works, no errors (see Wallstreet_console-4.txt).That would seem to implicate CONFIG_PPC_KUAP itself. (Note that all builds up until this one have CONFIG_PPC_KUAP=y.)
Yes I believe so, see at the begining of this mail.
Could you test with CONFIG_PPC_KUAP and CONFIG_PPC_KUAP_DEBUG... $scripts/config -e CONFIG_PPC_KUAP $ scripts/config -e CONFIG_PPC_KUAP_DEBUG $ make ARCH=powerpc CROSS_COMPILE=powerpc-linux-gnu- -j4 clean olddefconfig vmlinux $ grep CONFIG_PPC_KUAP .config CONFIG_PPC_KUAP=y CONFIG_PPC_KUAP_DEBUG=y Linux version 5.12.0-rc3-pmac-00078-g5cac2bc3752 9) PB 3400c Hangs at boot (Mac OS screen) 10) Wallstreet X failed at first login, worked at second login, one error in console log ("BUG: Unable to handle kernel instruction fetch"), see Wallstreet_console-5.txt.One might expect to see "Kernel attempted to write user page (b3399774) - exploit attempt?" again here (see c16728835eec build above) but instead this log says "Oops: Kernel access of bad area, sig: 11".
Maybe the test should be done a second time. As r11 is garbage it may or may not be a user address. If it is a user address the we get "Kernel attempted to write user page". If it is a random kernel address, we likely get "Kernel access of bad area" instead.
BTW, this procedure could be made simpler and easier if I pushed git branches to a public repo for Stan to build, which included Christophe's fix plus hard-wired Kconfig changes. That way, the .config file could be held constant and the commit hash in the serial console log would be more meaningful.
I like the idea, I think I'm going to provide testing fixes through a git repo, that will for sure make things easier.
Thanks Christophe