I have found that I am unable to connect to an ssh host if I have both my user's ssh config set to use a PCKS11 library and my yubikey based keys loaded into my ssh agent. I have tried both the opensc and yubico pcks11 libraries for accessing the card. The results differ slightly, but both ultimately fail to authenticate if my user's ssh config is set to use the PCKS11 library and the keys have been added to my ssh agent. ** using libykcs11.so from yubico-piv-tool 1.4.2 $ ssh-add -s /usr/lib/libykcs11.so Enter passphrase for PKCS#11: Card added: /usr/lib/libykcs11.so $ ssh-add -L | awk {'print $1,$3}' ssh-rsa /usr/lib/libykcs11.so ssh-rsa /usr/lib/libykcs11.so ssh-rsa /usr/lib/libykcs11.so ssh-rsa /usr/lib/libykcs11.so $ ssh -vv $REMOTEHOST OpenSSH_7.3p1, OpenSSL 1.0.2j 26 Sep 2016 ... debug1: Connecting to $REMOTEHOST [$REMOTEIP] port 22. debug1: Connection established. At this point the connection attempt simply hangs. Hoever if I remove the libykcs11.so library (and keys) from the ssh agent with the following: $ ssh-add -e /usr/lib/libykcs11.so Card removed: /usr/lib/libykcs11.so $ ssh-add -l The agent has no identities. The connection attempt proceeds and I get prompted for my pin: Enter PIN for 'YubiKey PIV': ** using opensc-pkcs11.so from opensc 0.16.0 $ ssh-add -s /lib/pkcs11/opensc-pkcs11.so Enter passphrase for PKCS#11: Card added: /lib/pkcs11/opensc-pkcs11.so $ ssh-add -L | awk {'print $1,$3}' ssh-rsa /lib/pkcs11/opensc-pkcs11.so ssh-rsa /lib/pkcs11/opensc-pkcs11.so ssh-rsa /lib/pkcs11/opensc-pkcs11.so ssh-rsa /lib/pkcs11/opensc-pkcs11.so $ ssh -vv $REMOTEHOST OpenSSH_7.3p1, OpenSSL 1.0.2j 26 Sep 2016 ... debug1: Offering RSA public key: /usr/lib/libykcs11.so debug2: we sent a publickey packet, wait for reply debug1: Server accepts key: pkalg ssh-rsa blen 279 debug2: input_userauth_pk_ok: fp SHA256:... sign_and_send_pubkey: signing failed: agent refused operation ... debug1: Next authentication method: password $USER@$REMOTEHOST's password: If I remove the library (and keys) and try the connection again: $ ssh-add -e /lib/pkcs11/opensc-pkcs11.so Card removed: /lib/pkcs11/opensc-pkcs11.so $ ssh-add -l The agent has no identities. $ ssh -vv $REMOTEHOST OpenSSH_7.3p1, OpenSSL 1.0.2j 26 Sep 2016 ... debug1: Offering RSA public key: /usr/lib/libykcs11.so debug2: we sent a publickey packet, wait for reply debug1: Server accepts key: pkalg ssh-rsa blen 279 debug2: input_userauth_pk_ok: fp SHA256:... Enter PIN for 'PIV_II (PIV Card Holder pin)': If I remove the PKCS11Provider directive from my user's ssh config, the keys loaded in the agent are used and everything works fine. However, if I then attempt to connect to the host without first loading the keys into the agent, I am not prompted for my yubikey pin. Ideally, I should be able to have both the user level PKCS11Provider directive and my keys loaded in the ssh agent. However, it appears that the user level directive is being attempted before trying to use the keys from the agent. I have found that I am unable to connect to an ssh host if I have both my user's ssh config set to use a PCKS11 library and my yubikey based keys loaded into my ssh agent. I have tried both the opensc and yubico pcks11 libraries for accessing the card. The results differ slightly, but both ultimately fail to authenticate if my user's ssh config is set to use the PCKS11 library and the keys have been added to my ssh agent. ** using libykcs11.so from yubico-piv-tool 1.4.2 $ ssh-add -s /usr/lib/libykcs11.so Enter passphrase for PKCS#11: Card added: /usr/lib/libykcs11.so $ ssh-add -L | awk {'print $1,$3}' ssh-rsa /usr/lib/libykcs11.so ssh-rsa /usr/lib/libykcs11.so ssh-rsa /usr/lib/libykcs11.so ssh-rsa /usr/lib/libykcs11.so $ ssh -vv $REMOTEHOST OpenSSH_7.3p1, OpenSSL 1.0.2j 26 Sep 2016 ... debug1: Connecting to $REMOTEHOST [$REMOTEIP] port 22. debug1: Connection established. At this point the connection attempt simply hangs. Hoever if I remove the libykcs11.so library (and keys) from the ssh agent with the following: $ ssh-add -e /usr/lib/libykcs11.so Card removed: /usr/lib/libykcs11.so $ ssh-add -l The agent has no identities. The connection attempt proceeds and I get prompted for my pin: Enter PIN for 'YubiKey PIV': ** using opensc-pkcs11.so from opensc 0.16.0 $ ssh-add -s /lib/pkcs11/opensc-pkcs11.so Enter passphrase for PKCS#11: Card added: /lib/pkcs11/opensc-pkcs11.so $ ssh-add -L | awk {'print $1,$3}' ssh-rsa /lib/pkcs11/opensc-pkcs11.so ssh-rsa /lib/pkcs11/opensc-pkcs11.so ssh-rsa /lib/pkcs11/opensc-pkcs11.so ssh-rsa /lib/pkcs11/opensc-pkcs11.so $ ssh -vv $REMOTEHOST OpenSSH_7.3p1, OpenSSL 1.0.2j 26 Sep 2016 ... debug1: Offering RSA public key: /usr/lib/libykcs11.so debug2: we sent a publickey packet, wait for reply debug1: Server accepts key: pkalg ssh-rsa blen 279 debug2: input_userauth_pk_ok: fp SHA256:... sign_and_send_pubkey: signing failed: agent refused operation ... debug1: Next authentication method: password $USER@$REMOTEHOST's password: If I remove the library (and keys) and try the connection again: $ ssh-add -e /lib/pkcs11/opensc-pkcs11.so Card removed: /lib/pkcs11/opensc-pkcs11.so $ ssh-add -l The agent has no identities. $ ssh -vv $REMOTEHOST OpenSSH_7.3p1, OpenSSL 1.0.2j 26 Sep 2016 ... debug1: Offering RSA public key: /usr/lib/libykcs11.so debug2: we sent a publickey packet, wait for reply debug1: Server accepts key: pkalg ssh-rsa blen 279 debug2: input_userauth_pk_ok: fp SHA256:... Enter PIN for 'PIV_II (PIV Card Holder pin)': If I remove the PKCS11Provider directive from my user's ssh config, the keys loaded in the agent are used and everything works fine. However, if I then attempt to connect to the host without first loading the keys into the agent, I am not prompted for my yubikey pin. Ideally, I should be able to have both the user level PKCS11Provider directive and my keys loaded in the ssh agent. However, it appears that the user level directive is being attempted before trying to use the keys from the agent.
It's been quite a while since this was reported and there has been no update. Is any further information needed?
Any update?
Sorry, but there isn't enough information to figure out what is going wrong. Please attach a full debug log from the client (ssh -vvv ...) as well as your ~/.ssh/config
Created attachment 3034 [details] the requested full debug log
Created attachment 3035 [details] the requested ssh config Attached you should find both the requested full debug log and ssh config. Please let me know if there is any additional information I can provide.
I have exactly the same issue, on Debian unstable, using OpenSSH 7.5p1 from the Debian packages, and a yubikey 4 Nano. My ssh -vvvv output is the same as Jamin's. I can provide additional information: (1) My second Yubikey, a Yubikey Neo, works fine even with the agent loaded and the PKCS11Provider option in the config. (2) When using the agent without the PKCS11Provider option, the ssh -vvv output is the identical same until: debug3: sign_and_send_pubkey: RSA <deleted> sign_and_send_pubkey: signing failed: agent refused operation debug1: Offering RSA public key: /usr/lib/x86_64-linux-gnu/opensc-pkcs11.so On the working client, things are: debug3: sign_and_send_pubkey: RSA <same-deleted-as-above> debug3: send packet: type 50 debug3: receive packet: type 52 debug1: Authentication succeeded (publickey). Authenticated to localhost ([127.0.0.1]:10022). debug1: channel 0: new [client-session] debug3: ssh_session2_open: channel_new: 0 debug2: channel 0: send open
And, after trying with PKCS11Provider option in place, and the agent refusing operation for the first time, I need to do the ssh-add -D ssh-add -e, ssh-add -s routine, or the agent will refuse operation even after removing the PKCS11Provider option: ssh -F config-with-PKSCS11Provider => agent refused operation ssh -F config-without-PKCS11Provider => agent refused operation ssh-add -D ssh-add -e ssh-add -s ssh -F config-without-PKCS11Provider => works Whenever ssh says "agent refused operations", ssh-agent started with -d logs "process_sign_request2: sshkey_sign: error in libcrypto". Hope this helps.
Created attachment 3126 [details] Tail of openSC debug log I believe this is not a problem of OpenSSH, but of the PKCS#11 module, which is not correctly handling the concurrent access from two separate processes (ssh and ssh-pkcs11-helper of ssh-agent). I can reproduce the same issue with latest OpenSC and OpenSSH. Running the current OpenSC in debug mode, shows similar errors as in the attachment, while running the ssh-agent in debug mode and adding the latest OpenSC pkcs11 module: OPENSC_DEBUG=9 ssh-agent -d I just tested the same case with the patch proposed in OpenSC upstream PR [1] and it seems to resolving the problem. This is also related to the recent change in OpenSC upstream, which is setting disconnect_action=leave by default (previously, it was "reset", which was also breaking long-running sessions such as ssh-agent). You can try if this will help you to resolve your problems. If not, please, provide also the debug logs from OpenSC as shown above. [1] https://github.com/OpenSC/OpenSC/pull/1256 [2] https://github.com/OpenSC/OpenSC/pull/1242
Looks like this is an OpenSC bug. If it still persists with the patch/version that Jakub recommended then please reopen this bug.
close bugs that were resolved in OpenSSH 8.5 release cycle