Having the ssh-agent to store the keys is nice and all, but I still like having my passphrase protected keys be actually protected from use by my having to enter a passphrase to use them. This should, of course, be optional as ssh-agent does also enable a lot of use-cases where the current situation is good too. Scenario: I'm now forced to use ssh-agent to access some systems as the remote system (out of my control) requires the ssh-agent to receive a signed key; however, for security reasons I much prefer being prompted for the passphrase whenever a key is used. There is zero ability to use ssh-agent at present to do this. Option #1: Add "AgentPromptForPassphrase" to the ssh_config file; values could be (yes, no) or (disabled, load, always); if 'enabled' (anything other than 'no', 'disabled' in the previous suggestions) then: - Enable ssh-agent to prompt when loading the keys when it starts up like Putty's SSH Agent does. (Annoying but okay) - Require tools to prompt for the passphrase before interacting with ssh-agent Option #2: Add an option to ssh_config to be able to set the key lifetime (e.g ssh-add -t) so users can set a short lifetime (e.g 30 seconds), and enable auto-loading of the key prior to its use so that ssh-agent will properly pick it up and use it for the connection. The user can then set the default lifetime of the key to a very low number and essentially have the key auto-reload on each new connection. Per Option #2, AFAIK the setting to autoload a key (AddKeysToAgent) only works after a key has been successfully used. I haven't been able to get it to work in my scenario b/c the remote side only talks to the agent; the key, however, is specified via the IdentityFile keyword in the config for the host. Note: I'm using OpenSSH 7.2p2, Ubuntu 16.04 I'm guessing Option #2 would be easier to implement given since it most likely requires few changes to the overall architecture.