Created attachment 3385 [details] RFC patch to start the discussion Hi, in a recent bug report to Ubuntu [1] I was wondering that this issue seems to exist for so long and affect so many people [2][3][4][5][6][7][8]. It was also discussed within openssh in [9][10]. But in all of those cases all that was done was to suggest workarounds like "PubkeyAuthentication=no" or "IdentitiesOnly=yes", but IMHO those are all just exactly that - workarounds. Also there are many "related but not entirely the same" upstream bugs like [11][12], but it seems no one has yet discussed the approach we had in mind. If a usual user calls ssh like ssh -i <mykey> ... And gets: "Too many authentication failures" He'd not even think about <mykey> not even being tried. The problem is that the current order in pubkey_prepare will order those directly specified keys too late. * try keys in the following order: * 1. certificates listed in the config file * 2. other input certificates * 3. agent keys that are found in the config file * 4. other agent keys * 5. keys that are only listed in the config file Yes, if <mykey> is in the agent it will come at #3 which usually is early enough. But if <mykey> isn't in the agent which is usual for any new key that a user created/tries it will come at #5. If now the number of keys in #4 exceed the servers "i" the user gets rejected without even trying <mykey> that he has specified. Config may be config, but if a user specifies something directly on invocation I'd expect it to supersede the configuration. I'm clear that there might be more needed: - has it to be ordered behind certs? - do user-specified certs also need to get up in the list? - does a test need to be added? - discussion here about potential drawbacks - ... But I wanted to get things started and therefore I have attached a sugegsted initial patch as well as a Ubuntu PPA [13] with it applied. [1]: https://bugs.launchpad.net/debian/+source/openssh/+bug/1872145 [2]: https://serverfault.com/questions/36291/how-to-recover-from-too-many-authentication-failures-for-user-root [3]: https://www.tecmint.com/fix-ssh-too-many-authentication-failures-error/ [4]: https://superuser.com/questions/187779/too-many-authentication-failures-for-username [5]: https://blog.jasonmeridth.com/posts/ssh-too-many-authentication-failures/ [6]: https://security.stackexchange.com/questions/65120/ssh-always-too-many-authentication-failures [7]: https://www.unixtutorial.org/ssh-too-many-authentication-failures [8]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=203700 [9]: https://lists.mindrot.org/pipermail/openssh-unix-dev/2003-December/019931.html [10]: https://lists.mindrot.org/pipermail/openssh-unix-dev/2016-April/035029.html [11]: https://bugzilla.mindrot.org/show_bug.cgi?id=1499 [12]: https://bugzilla.mindrot.org/show_bug.cgi?id=1937 [13]: https://launchpad.net/~paelzer/+archive/ubuntu/bug-872145-ssh-prefer-user-configured-key
Example effect of the patch: Former behavior on a server with MaxAuthTries 4 the explicitly specified key would not have been tried: $ ssh -i /tmp/testkey -v horsea "echo 1" |& grep "Will attempt" debug1: Will attempt key: /home/paelzer/.ssh/id_rsa RSA ... agent debug1: Will attempt key: ubuntu@cpaelzer-bastion RSA ... agent debug1: Will attempt key: paelzer@lap RSA ... agent debug1: Will attempt key: paelzer@swarm.n RSA ... agent debug1: Will attempt key: /tmp/testkey RSA ... explicit With the change becomes this and works: $ ssh -i /tmp/testkey -v horsea "echo 1" |& grep "Will attempt" debug1: Will attempt key: /tmp/testkey RSA ... explicit debug1: Will attempt key: /home/paelzer/.ssh/id_rsa RSA ... agent debug1: Will attempt key: ubuntu@cpaelzer-bastion RSA ... agent debug1: Will attempt key: paelzer@lap RSA ... agent debug1: Will attempt key: paelzer@swarm.n RSA ... agent
Created attachment 3387 [details] identitiesOnly=explicit maybe we could do something like this: allow IdentitiesOnly=explicit to disable adding agent keys that aren't explicitly listed as IdentityFiles/-i
Hi Damien, the suggested IdentitiesOnly=explicit is interesting, but it won't cover the part of the users that need the fix the most. I was mostly thinking about the less experienced users - those who'd not understand why things are failing and not know how to hunt for the existing workarounds. The IdentitiesOnly=explicit option would only fix it for those people that know what is going on (as they need to set it) unless if it would be the default config value. But as default it would break plenty of other use cases. But thinking about configs, maybe we'd want/need to go a step further. Today the preference order is in the code, maybe we'd want to expose that as a config. With my patch applied we have 6 classes of Auth to offer. We might apply my patch, but then revamp it completely to have the order configurable. The following would represent the order with my patch applied: IdentitiesOrder=key-explicit,cert-configured,cert-other,key-agent-configured,key-agent,key-other Everyone is welcome to bikeshed on the terms, but it actually was more about discussing the idea :-)
I cannot understand what is issue with agent keys. User start agent and adds some keys(identities). It is expected those keys to take precedence over all other keys as they are loaded first! Then when is started client it could add other identities. Directive IdentitiesOnly set to yes is intended to minimize used agent keys. Sample: agent with keys agent1 agent2 agent3 To simplify let assume that configuration does no add other identities. a) client .. -i no_agent -i agent2 .. If IdentitiesOnly is set to yes client should try "agent2" and "no_agent". b) client .. -i no_agent .. If IdentitiesOnly is set to yes client should try only "no_agent". So I cannot see why IdentitiesOnly=yes is not solution. Reading OpenSSH manual page I partially agree with first report: ---- -i identity_file Selects a file from which the identity (private key) for public key authentication is read. The default is .... Identity files may also be specified on a per-host basis in the configuration file. It is possible to have multiple -i options (and multiple identities specified in configuration files). ---- The only things missing is that ssh(1) does not suggest for more details user to see directive IdentityFile ssh_config(5) where: ---- IdentityFile ... Additionally, any identities represented by the authentication agent will be used for authentication unless IdentitiesOnly is set. ... ---- "Additionally" is not appropriate word as agent keys are loaded first and is expected to be used first. It seems to me this report is just documentation issue.
Hi Roumen, I can absolutely see your POV that I'd like to summarize "if you read/know all of the documentation you see what happens". And I can follow your argument that from there the obvious improvement would be to enhance the docs to be more obvious. But if I turn it around to the users perspective I'd rather convinced of the proposed behavior: user-Example A) If we describe 100 admins the following scenario: 1. ssh agent has 5 keys loaded 2. you run ssh -i ExplicitKey foo@bar And we then ask them "Do you expect that ExplicitKey will be tried?" I'm pretty sure the majority will answer "yes it will try ExplicitKey". And even if you then hint at MaxAuthTries limiting the amount that can be tried I assume that most would expect "what I specified explicitly would go first, since after all I specified it explicitly". user-Example B) What currently happens to users is something like: 1. `ssh -i ExplicitKey foo@bar` works fine 2. .. N. some other actions which eventually make ssh-agent hold >= MaxAuthTries other keys 3. `ssh -i ExplicitKey foo@bar` suddenly fails now 4. Puzzled ?!?, after a long time finding the subtle details of Agent/MaxAuthTries and wishes that at least what he specified explicitly would have been tried. Improved-Messaging example C) Turning the case around again (no offense please, this example is phrased slightly provocative to show my point). If the behavior isn't changed, then I'd suggest instead of a doc change that people first have to fail, then find the doc then understand it all ... Instead if ssh gives up failing before the key on the commandline was even tried ssh could emit a slightly different error. Instead of "Too many authentication failures" It could say: "Too many authentication failures, But just so you know, the key you thought you use wasn't even tried" I hope that helps to clarify why I think IdentitiesOnly and/or the documentation thereof isn't enough. Thanks in advance, Christian