Solar Designer noticed indirectly that sshd child processes (and probably other parts of OpenSSH) can block if ~/.ssh/authorized_keys is a named pipe with an open writer. Perhaps it would be worth checking S_ISREG before trying to open a key file? (I'm hoping nobody is actually relying on this as a feature. I can't imagine how you'd do so reliably.)
Created attachment 1517 [details] open authorized_keys and shosts in non-blocking mode, check st_mode .shosts is another file that the server can be made to open. This diff should fix it and authorized_keys.
that patch has a small bug (don't use it), but a similar one has been submitted and will be in openssh-5.1. Thanks!
I'd add O_NOCTTY. On some systems it is a no-op, but on others it makes a difference. Also, a maximum size check both before and during reads could make sense, but it is not clear what the limit should be (1 MB maybe? or would anyone want to put thousands of keys on an account?) I suppose O_NOFOLLOW would break some existing setups and it does not buy all that much (at least not when arbitrary hard links are allowed by the kernel and there are interesting things on the same device).
Mass update RESOLVED->CLOSED after release of openssh-5.1