Can you check fwupd.service for any errors that coincide with the higher memory usage?
Also; any bad USB devices or kernel errors? There are some bug reports that talked about descriptors getting left open when there was a bad USB device.
We might need to try to use valgrind to find this problem. You can try to run valgrind like this:
sudo valgrind --leak-check=full
--show-leak-kinds=definite /usr/libexec/fwupd/fwupd | tee
fwupd.log
You can leave that running a few hours and then ctrl-c and share
fwupd.log and we'll see if anything stands out.