Currently configuration for the ssh client program is stored in /etc/ssh/ssh_config or in ~/.ssh/config. The latter one however can grow rather large if you have a lot of servers to manage. For the sake of manageability and clarity it makes sense to store configurations for example for different clusters in different config files. This is in a way comparable with most system services which have their own /etc/$SERVICENAME/conf.d/ directories where you can drop various configuration files. The ssh-client configuration is not capable of such a behaviour. You can store the config in the single configuration file only. There is no way of including further files or a directory with more configuration files. A workaround is to create files like ~/.ssh/config.d/cluster1.conf ~/.ssh/config.d/cluster2.conf ~/.ssh/config.d/cluster3.conf and merge them at every change via cat ~/.ssh/config.d/*.conf > ~/.ssh/config This is however sub-optimal since you might forget changes you made in config, or might forget to merge at all. The best would be to add an "include" function to ssh to auto-search for configuration in the file ~/.ssh/config as well as for files of the type ~/.ssh/config.d/*.conf. That would make the management of configurations much easier. Security: Adding additional config files means adding further points where the client can be mis-configured. However, if the merge is implemented correctly there should be no difference for ssh-client between several conf files or one single file. Also, several distributed files can be managed and reviewed much easier, so the security might even be improved.