auth [success=2 default=ignore] pam_p11.so
/usr/local/lib/libcvP11.so
[...]
This works nearly exactly as desired, "nearly" because though the
login with unix password works, the application shows "Login failed"
for a short time. Is there something I can change in the above file to
avoid this message?
I don't know what local library it is you used, but I encourage you to
consider the use of Debian packages libpam-p11 libpam-pkcs11 and
libpam-poldi - or if you already considered those then share why you
rejected them.