vm_object shadow pages and task termination

[Michael Kelly <mike@weatherwax.co.uk>]

Hello,

I began to investigate use of the stress-ng program to test the virtual 
memory system and in particular the behaviour under swap conditions. I 
didn't get to that goal just yet because I encountered an unrelated bug 
in the mach kernel exposed by stress-ng:

# stress-ng --vm 1 --vm-bytes 128K --vm-keep

This results in 1 stress-ng process, forking a child stress-ng which 
forks a 3rd stress-ng and begins what would normally be quite a long 
test run. After a few seconds I terminate all 3 processes and repeat the 
same stress-ng invocation. In most cases the kernel crashes immediately 
on the 2nd run but occasionally I might need to terminate and run a few 
more times. Various assertions originating in pmap.c occur.

I've spent a lot of time narrowing and debugging the fault, and I can 
provide more details if required, but to keep this message brief, here 
is a broad summary of what happens:

There  is a vm_map region of memory created by the parent process and 
inherited by the 2 child processes using VM_INHERIT_SHARE.

I added some validation code within pmap that I could call from the 
kernel debugger. After termination of the stress-ng processes this 
showed that there were some remaining references within the 
pv_head_table (i386/intel/pmap.c) to pmap_t objects that had already 
been destroyed. Eventually I found that the shared memory in stress-ng 
is managed by a vm_object having a 'shadow' vm_object and it was the 
pages within the shadow object that were not being unmapped before the 
vm_objects were terminated.

vm_object_pmap_remove() is the function called from within vm_map to 
clear pmap references for a shared map entry. This is the only place in 
mach where this function is called from. Currently it only removes pmap 
protection for all pages associated with the map_entry vm_object and 
disregards the shadow chain. I noticed that in the more general function 
vm_object_pmap_protect() there is code to recurse down the shadow chain 
and repeat the pmap deallocation performed on the main vm_object.

So in essence, I added the same shadow chain recursion functionality to 
vm_object_pmap_remove(). This works successfully within my test case. I 
am however not experienced enough with the mach code to claim this as a 
certain fix but offer it here for consideration. In particular I found 
it hard to get my head around the offset manipulations for the shadow 
chain. My patch includes that aspect but would need particular checking.

Regards,

Mike.

diff --git a/vm/vm_object.c b/vm/vm_object.c
index 3e6b28c4..efb169da 100644
--- a/vm/vm_object.c
+++ b/vm/vm_object.c
@@ -980,10 +980,11 @@ void vm_object_pmap_remove(
        vm_offset_t     start,
        vm_offset_t     end)
 {
-       vm_page_t       p;
+  vm_page_t    p;

-       if (object == VM_OBJECT_NULL)
-               return;
+  while (object != VM_OBJECT_NULL)
+    {
+        vm_object_t next_object;

        vm_object_lock(object);
        queue_iterate(&object->memq, p, vm_page_t, listq) {
@@ -992,7 +993,12 @@ void vm_object_pmap_remove(
                    (p->offset < end))
                        pmap_page_protect(p->phys_addr, VM_PROT_NONE);
        }
+       next_object = object->shadow;
+       start += object->shadow_offset;
+       end   += object->shadow_offset;
        vm_object_unlock(object);
+       object = next_object;
+    }
 }

 /*