I recently installed Buster and ran into this issue. The prompt for my cryptsetup password was almost unreadable because of the font size, the debian-installer was very very painful to use too.
I tried Sam's console-setup hook that relies on udev but I was unable to get the font set. However, the two-script solution by Jörg did work for me.
I was not able to reproduce the "double" font jump that Sam observed in between the crypt prompt and the system hand-over. I did see a difference in the "weight" (normal to thin) of the font and the color (white to grey), even though I was using the same in both contexts (initramfs, proper system).