I am seeing very similar problems to this since switching from Buster to Bullseye a couple of weeks ago. I similarly suspected the change in Kernel to be the culprit as whilst using Buster I had no issues. Is there some relatively simple way to test the 5.5 Kernel or any idea when it might be available for Bullseye?