You might be using keyboard-interactive authentication instead of password (they act pretty much the same by default). Make sure you have PAMAuthenticationViaKbdInt set to "no" in /etc/ssh/sshd_config.