On 07/08/2025 13:41, Samuel Thibault wrote:
Michael Kelly, le jeu. 07 août 2025 13:07:07 +0100, a ecrit:On 07/08/2025 12:13, Samuel Thibault wrote:TLS code is really difficult to debug indeed. But I'd doubt that's the issue, as it is largely used for many other cases, and the code is actually very trivial: just reading/writing %gs:0x38. And it's probably very easy for calls to __mig_get_reply_port() / __mig_dealloc_reply_port () to get wrong with mismatched cleanup code, e.g. port = __mig_get_reply_port() __mig_dealloc_reply_port(port); port2 = __mig_get_reply_port() __mig_dealloc_reply_port(port);I do hope that you are right although I hadn't expected mismatches from generated code.I wouldn't indeed, but there are also signal management and a couple explicit calls.
I have a greater understanding of how the 'assert (port == arg)' [mig-reply.c:__mig_dealloc_reply_port()] occurs. The assertion has occurred within tasks associated with storeio, ext2fs and stress-ng illustrating that it is a general problem.
In the case of ext2fs, an example occurs when it makes the RPC call to __fsys_getroot(). The kernel call to mach_msg_trap() returns MACH_SEND_INTERRUPTED in the first attempt at sending the message. The code within _hurd_intr_rpc_mach_msg() then attempts to resend the message but the message header has been modified by the first call into gnumach. A subsequent call to mach_msg_trap() returns EINTR and it drops into the error handling code of __fsys_getroot() to find that the port in the message header is unexpected and triggers the assertion.
The message header is modified within gnumach by the 'slow_send:' case within mach_msg_trap when ipc_mqueue_send() fails (presumed to be MACH_SEND_INTERRUPTED) to deliver the message. It is the call to ipc_kmsg_copyout_pseudo() that makes the modification to the kmsg header and ultimately the user space msg header.
I haven't determined the origin of the new port number that ipc_kmsg_copyout_pseudo() inserts into the message header. It is determined from the task's IPC space but it isn't instantly obvious to me why it translates into a seemingly unrelated port number. Perhaps I've just been staring at the code too long? I can continue staring a while longer if no one can advise. If the port number translation is expected then I can just concentrate on fixing the user end of the code within _hurd_intr_rpc_mach_msg().
I can solve my test case by 'correcting' the message header within the MACH_SEND_INTERRUPTED case of _hurd_intr_rpc_mach_msg(). There are other places in this code that realise that the header is modified by the system call and make a correction to it before attempting to resend the message. Adopting that approach and resetting 'm->header.msgh_local_port = rcv_name' before the 'goto message' resend attempt does appear to stop the occurrence of the assertion. I did actually restore other components of the message header (like it does in the other cases) but some of those might not be necessary (to be investigated/tested). Is this an appropriate solution or might there be other unexpected side effects?
I have also experienced a new 'lockup' involving process termination which I am investigating. It is related to task_terminate() again and I cannot yet tell if it is a consequence of my patch or whether it is simply similar.
I am still aiming to be able to achieve a 24 hour successful test run before making any patch suggestions but I'd welcome any thoughts on what I have determined so far.
Regards, Mike.