[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#1107521: ath12k_pci errors and loss of connectivity in 6.12.y branch

+Vasant

On 2025-06-27 6:39 am, Baochen Qiang wrote:

[+ IOMMU list]

On 6/27/2025 12:21 AM, Matt Mower wrote:

Dear maintainer,

I have been experiencing lost network connection with the ath12k_pci driver
in the linux-6.12.y kernel branch. Often, when the issue occurs, the
network does not recover until I reboot the computer. A full report of the
errors I encounter, the symptoms that arise, and several dmesg attachments
are in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1107521 . I have
attached a dmesg from 6.12.34 for convenience. The short summary is:

1. I started noticing log lines like the following soon after boot when I
updated from 6.12.22 to 6.12.27. After these events occur, the network goes
down and often does not come back up.
    ath12k_pci 0000:c2:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT
domain=0x0010 address=0xfea00000 flags=0x0020]
2. I was able to reproduce this issue very rarely in 6.12.12 and 6.12.22.
The issue always occurs soon after boot in 6.12.27, 6.12.30, 6.12.33, and
6.12.34.
3. I have not reproduced the issue in 6.15.2 or 6.15.3.
4. In some cases, when shutting down the computer, a kernel bug caused my
computer to hang. I haven't determined whether this is related to the issue
above or an independent issue. Search the bug report
for PXL_20250611_140820085.jpg to see a picture of the kernel bug on my
laptop screen.
5. I have tested two firmware versions:
    a. fw_version 0x1108811c fw_build_timestamp 2025-05-17 00:21 fw_build_id
QC_IMAGE_VERSION_STRING=WLAN.HMT.1.1.c5-00284.1-QCAHMTSWPL_V1.0_V2.0_SILICONZ-3
    b. fw_version 0x100301e1 fw_build_timestamp 2023-12-06 04:05 fw_build_id
QC_IMAGE_VERSION_STRING=WLAN.HMT.1.0.c5-00481-QCAHMTSWPL_V1.0_V2.0_SILICONZ-3

Thanks,
Matt



I had a quick test with 6.12.27 kernel on both my Intel desktop and AMD RD but didn't hit
the issue. And I am using WLAN.HMT.1.1.c5-00284.1-QCAHMTSWPL_V1.0_V2.0_SILICONZ-3.

As mentioned in the Debian bug report, since reverting ath12k patches does not fix this
issue, maybe it comes from the IOMMU subsystem?
Faults are usually still indicative of the client driver/subsystem doing something not quite right - racily performing dma_unmap before the device has actually finished making accesses; mapping the wrong size such that the device accesses off the end of the mapping (this can often run into another valid mapping so not necessarily fault); mapping the wrong DMA direction such that the device then tries to write to a read-only page. However I suppose it's not impossible that some fix to amd-iommu in that period might have changed its behaviour in a way that exacerbates things - Vasant, does this strike a chord with anything you're aware of?
A couple more things I'd try on the ath12k side: firstly, boot with "iommu.strict=1" and see if that makes the faults any more frequent/reproducible; if a fault is fairly easily reproducible, then use the DMA API and/or IOMMU API tracepoints to compare the fault address to prior DMA mapping activity - that can usually reveal the nature of the bug enough to then know what to go looking for.
I wouldn't put much significance in whatever happens *after* the fault - presumably the driver is assuming the blocked DMA write has completed, so then goes on to read some incomplete descriptor as if it were valid, and thus may fall over in all manner of entertaining ways on bogus data. 

Thanks,
Robin.
Reply to: