Well I wouldn't worry too much about the number of loaded modules. That,
of course, assuming you are running a "modern" machine with several
hundreds megabytes of RAM. Modules use just a tiny fraction of RAM: 30,
40k ... the largest one for me being Reiserfs with 200k. But assuming an
average of 40k size, you'd be freeing up about 1MB of ram.
Of course if you're trying to run an embedded machine (for which you
wouldn't be running debian) or an old machine with 24MB ram (the minimum
for Debian), well in that case it would be an issue.
I wouldn't worry too much about it with a standard "modern" computer.
I wonder what happens to modules when the system needs RAM? I know
Windows stupidly swaps the System memory by default (unless you enable
the DisablePagingExecutive option in the registry). Does linux force the
modules to stay in RAM or does it try to swap them when it's RAM-hungry?
Cheers,
Hernan