Our company is developing HSM and library that implements the PKCS#11 standard. Our clients use these solutions, including work with OpenSSH. Our PKCS#11 library is distributed in the usual way for Linux-based OS: they are installed in /opt/<company name>/ and symbolic links are created in /usr/lib/. Paths to libraries are registered by using ldconfig. We encountered difficulties with the need of having loadable providers in /usr/lib/ or /usr/local/lib/ by their real so-libraries (not symbolic links). These settings can be changed at the building stage of OpenSSH, but OpenSSH packages contain default values in various repositories. In addition, developers of opensc-pkcs11 faced the same problem and they now are duplicating installed libraries in several places as a solution. Was the introduction of allowed_providers a solution of CVE-2016-1009? We would like to suggest you to allow changing allowed_providers via config file in /etc/. This change would add flexibility to the product configuration and still prevent an untrustred provider from running (if modifying the configuration file requires the same rights as adding the provider to the predefined directories). Also, this change will allow you to install providers in isolated directories and register them to work with OpenSSH without unnecessary copies or rebuilds of OpenSSH.
ssh-agent doesn't have a configuration file and we're not inclined to add one for a single option. If the default allowlist is missing customary paths for loadable modules then we're willing to add them though.