Close kmail, and open ~/.kde/share/config/kmailrc in your favorite editor. Replace true to false in every line checkmail=true for each IMAP folder you don't want to poll - this was much quicker than the GUI in my case.