That's expected: no firmware is deployed at first, the kernel complains,
d-i notices, and deploys stuff if relevant firmware packages are found,
and reload the relevant modules. This reloading is what breaks the
kernel. See upstream bug report:
https://bugzilla.kernel.org/show_bug.cgi?id=216844