For xfce, you might try this:
Settings Manager > Session and Startup > Application Autostart
Scroll down and uncheck Screensaver.
There may be additional things you need to do to make sure session stuff isn't loading screensavers through some other mechanism (i.e, squirreled away in a saved session or something). For that you might need to explicitly exit the screensaver (either through Settings Manager > Screensaver or a command line like xscreensaver-command -exit)
And you likely need the xset stuff you already did on top of that.