The ssh ignores the "IdentityFile" config in ~/.ssh/config and instead uses the keys from Seahorse in alphabetical order. There is no way how to force the order the keys. This may cause the unability to login on sites with limited numbers of login attempts.
Doesn't "IdentitiesOnly yes" do what you want?
from "man ssh_config" IdentitiesOnly Specifies that ssh(1) should only use the authentication identity files configured in the ssh_config files, even if ssh-agent(1) offers more identities. The argument to this keyword must be ``yes'' or ``no''. This option is intended for situations where ssh-agent offers many different identities. The default is ``no''. If you don't want ssh to touch the agent at all, then unset the SSH_AUTH_SOCK environment variable.
Move resolved bugs to CLOSED after 5.7 release