[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Debian SID kernel doesn't boot on PowerBook 3400c





Le 07/08/2021 à 18:26, Stan Johnson a écrit :
On 8/7/21 8:35 AM, Christophe Leroy wrote:


Le 07/08/2021 à 15:09, Stan Johnson a écrit :
On 8/6/21 10:08 PM, Finn Thain wrote:

On Fri, 6 Aug 2021, Stan Johnson wrote:

$ egrep '(CONFIG_PPC_KUAP|CONFIG_VMAP_STACK)' .config
CONFIG_PPC_KUAP=y
CONFIG_PPC_KUAP_DEBUG=y
CONFIG_VMAP_STACK=y
$ strings vmlinux | fgrep "Linux version"
Linux version 5.13.0-pmac-00004-g63e3756d1bd ...
$ cp vmlinux ../vmlinux-5.13.0-pmac-00004-g63e3756d1bd-1

1) PB 3400c
vmlinux-5.13.0-pmac-00004-g63e3756d1bd-1
Boots, no errors logging in at (text) fb console. Logging in via ssh
and
running "ls -Rail /usr/include" generated errors (and a hung ssh
session). Once errors started, they repeated for almost every command.
See pb3400c-63e3756d1bdf-1.txt.

2) Wallstreet
vmlinux-5.13.0-pmac-00004-g63e3756d1bd-1
X login failed, there were errors ("Oops: Kernel access of bad area",
"Oops: Exception in kernel mode"). Logging in via SSH, there were no
additional errors after running "ls -Rail /usr/include" -- the errors
did not escalate as they did on the PB 3400.
See Wallstreet-63e3756d1bdf-1.txt.

...
$ egrep '(CONFIG_PPC_KUAP|CONFIG_VMAP_STACK)' .config
CONFIG_PPC_KUAP=y
CONFIG_PPC_KUAP_DEBUG=y
# CONFIG_VMAP_STACK is not set
$ strings vmlinux | fgrep "Linux version"
Linux version 5.13.0-pmac-00004-g63e3756d1bd ...
$ cp vmlinux ../vmlinux-5.13.0-pmac-00004-g63e3756d1bd-2

3) PB 3400c
vmlinux-5.13.0-pmac-00004-g63e3756d1bd-2
Filesystem was corrupt from the previous test (probably from all the
errors during shutdown). After fixing the filesystem:
Boots, no errors logging in at (text) fb console. Logging in via ssh
and
running "ls -Rail /usr/include" generated a few errors. There didn't
seem to be as many errors as in the previous test, there were a few
errors during shutdown but the shutdown was otherwise normal.
See pb3400c-63e3756d1bdf-2.txt.

4) Wallstreet
vmlinux-5.13.0-pmac-00004-g63e3756d1bd-2
X login worked, and there were no errors. There were no errors during
ssh access.
See Wallstreet-63e3756d1bdf-2.txt.


Thanks for collecting these results, Stan. Do you think that the
successful result from test 4) could have been just chance?

No. I repeated Test 4 above two more times on the Wallstreet. After
stomping on it as hard as I could, I didn't see any errors. I ran the
following tests simultaneously, with no errors:

a) Ping flood the Wallstreet
862132 packets transmitted, 862117 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.316/0.418/12.163/0.143 ms

b) "ls -Rail /usr" in an ssh window.

c) "find /usr/include -type f -exec sha1sum {} \;" in a second ssh
window.

d) With a, b and c running, I logged in at the X console (slow but it
worked). Load average was 7.0 as reported by uptime.

So the success seems to be repeatable (or at least the errors are so
unlikely to happen that I'm not seeing anything).


It appears that the bug affecting the Powerbook 3400 is unaffected by
CONFIG_VMAP_STACK.

Whereas the bug affecting the Powerbook G3 disappears when
CONFIG_VMAP_STACK is disabled (assuming the result from 4 is reliable).

Either way, these results reiterate that "Oops: Kernel access of bad
area,
sig: 11" was not entirely resolved by "powerpc/32s: Fix napping
restore in
data storage interrupt (DSI)".


That sounds right. Thanks for investigating this.



Thanks a lot for your patience and for the tests.

I'm still having hard time understanding what the problem is.

Could you try the new change I pushed into the git repo ? It shouldn't
have any effect, but I prefer to eliminate all possibilities. The
documentation says that SRR1 upper bit are 0 on DSI and the code relies
on that. But if the doc is wrong then that can explain the problem. So
now I'm forcing it to 0 regardless.

To get the change, you just have to do 'git pull -r' inside the
directory where you checked out the sources and build.

Thanks again
Christophe


Thanks, Christophe.

In the same directory as previous builds:

$ git checkout chleroy-linux/bugtest
HEAD is now at 63e3756d1bdf powerpc/interrupts: Also perform KUAP/KUEP
lock and usertime accounting on NMI
$ git pull -r
You are not currently on a branch.
Please specify which branch you want to rebase against.
...
$ git pull -r chleroy-linux
remote: Enumerating objects: 6, done.
remote: Counting objects: 100% (6/6), done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 6 (delta 0), reused 6 (delta 0), pack-reused 0
Unpacking objects: 100% (6/6), done.
 From https://github.com/chleroy/linux
    63e3756d1bdf..9023760b1361  bugtest    -> chleroy-linux/bugtest
Updating 63e3756d1bdf..9023760b1361
Fast-forward
  arch/powerpc/kernel/head_book3s_32.S | 1 +
  1 file changed, 1 insertion(+)
HEAD is up to date.

Hopefully I did that right and ended up at the right spot.

For tests 5 and 6:

$ cp ../dot-config-powermac-5.13 .config
$ scripts/config -e CONFIG_PPC_KUAP -e CONFIG_PPC_KUAP_DEBUG -e
CONFIG_VMAP_STACK
$ make ARCH=powerpc CROSS_COMPILE=powerpc-linux-gnu- -j4 clean
olddefconfig vmlinux
$ egrep '(CONFIG_PPC_KUAP|CONFIG_VMAP_STACK)' .config
CONFIG_PPC_KUAP=y
CONFIG_PPC_KUAP_DEBUG=y
CONFIG_VMAP_STACK=y
$ strings vmlinux | grep "Linux version"
Linux version 5.13.0-pmac-00005-g9023760b136 (johnson@ThinkPad)
(powerpc-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0, GNU ld (GNU Binutils for
Debian) 2.31.1) #3 SMP Sat Aug 7 09:29:11 MDT 2021
$ cp vmlinux ../vmlinux-5.13.0-pmac-00005-g9023760b136-1


5) PB 3400c
vmlinux-5.13.0-pmac-00005-g9023760b136-1
Boots, no errors logging in at (text) fb console. Logging in via ssh and
running "ls -Rail /usr/include" generated errors. As before, once errors
started, they seemed to escalate, including errors during "shutdown -r now".
See pb3400c-g9023760b136-1.txt.

6) Wallstreet
vmlinux-5.13.0-pmac-00005-g9023760b136-1
X login failed, and there were errors. Logging in via SSH, there were no
additional errors after running "ls -Rail /usr/include" -- as before,
the errors did not escalate as they did on the PB 3400.
See Wallstreet-g9023760b136-1.txt.

For tests 7 and 8:

$ cp ../dot-config-powermac-5.13 .config
$ scripts/config -e CONFIG_PPC_KUAP -e CONFIG_PPC_KUAP_DEBUG -d
CONFIG_VMAP_STACK
$ make ARCH=powerpc CROSS_COMPILE=powerpc-linux-gnu- -j4 clean
olddefconfig vmlinux
$ egrep '(CONFIG_PPC_KUAP|CONFIG_VMAP_STACK)' .config
CONFIG_PPC_KUAP=y
CONFIG_PPC_KUAP_DEBUG=y
# CONFIG_VMAP_STACK is not set
$ strings vmlinux | grep "Linux version"
Linux version 5.13.0-pmac-00005-g9023760b136 (johnson@ThinkPad)
(powerpc-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0, GNU ld (GNU Binutils for
Debian) 2.31.1) #4 SMP Sat Aug 7 09:49:03 MDT 2021
$ cp vmlinux ../vmlinux-5.13.0-pmac-00005-g9023760b136-2


7) PB 3400c
vmlinux-5.13.0-pmac-00005-g9023760b136-2
As before, the filesystem was corrupt from the previous test. After
fixing that, this kernel boots, and there were no errors from logging in
at the (text) fb console. Logging in via ssh and running "ls -Rail
/usr/include" generated errors. There were a few errors logging in at
the serial console and during shutdown, but the shutdown was otherwise
normal.
See pb3400c-g9023760b136-2.txt.

8) Wallstreet
vmlinux-5.13.0-pmac-00005-g9023760b136-2
X login worked, and there were no errors. There were also no errors
during ssh access.
Simultaneous stress test, also no errors:
a) Login at X console.
b) Ping flood the Wallstreet
359695 packets transmitted, 359688 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.322/0.428/16.857/0.165 ms
c) "ls -Rail /usr" in an ssh window.
d) "find /usr/include -type f -exec sha1sum {} \;" in a second ssh window.
See Wallstreet-g9023760b136-2.txt.

As far as I could tell, there were no significant changes from the
previous four tests.


Ok, that was expected, but I wanted to be 100% sure to avoid looking into the wrong direction.

To be honnest, I'm running out of ideas.

We have two remaining independant problems as far as I understand:

PB3400C (603ev core = No hash table)
- A KUAP fault, regardless of CONFIG_VMAP_STACK, due to a clobber of r11 registers apparently.

Wallstreet (Hash table)
- Random faults, only with CONFIG_VMAP_STACK


One thing I am wondering, could there be a link with SMP ?

Would you mind trying with a kernel built without CONFIG_SMP ?

Thanks
Christophe


Reply to: