Bug 3022 - "dup() in/out/err failed" is output when stdin, stdout, or stderr is a TCP/IP socket
Summary: "dup() in/out/err failed" is output when stdin, stdout, or stderr is a TCP/IP...
Status: CLOSED WORKSFORME
Alias: None
Product: Portable OpenSSH
Classification: Unclassified
Component: ssh (show other bugs)
Version: 8.0p1
Hardware: ix86 Windows 10
: P5 normal
Assignee: Assigned to nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-06-26 02:08 AEST by minidino
Modified: 2022-02-25 13:59 AEDT (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description minidino 2019-06-26 02:08:58 AEST
Due to limitations in Windows, the CreateProcess() API accepts anonymous pipes for the stdin, stdout, and stderr handles.  However, buried in the MSDN docs is information that states that anonymous pipes cannot be made non-blocking.  This creates a situation where two processes can deadlock (e.g. one process is blocking on stdin while another process is blocking on stdout or stderr).

There are two known fixes for this on Windows:  Use multiple threads (one thread for each pipe) or use a little-known mode of CreateProcess() and pass connected TCP/IP socket handles for each of the pipes to the target process instead of anonymous pipes.  Not all languages support multithreading and using a thread per pipe is kind of wasteful of system resources.  Of course, establishing a TCP/IP connection per pipe is also wasteful but enables single-threaded, non-blocking code to work without deadlocking both processes.  Most processes work fine with TCP/IP stdout/stderr (TCP/IP stdin is a bit more iffy and depends on how the underlying C library works).

OpenSSH calls dup() in ssh.c and then bails out with the error message "dup() in/out/err failed" when it can't duplicate socket handle(s).  The connection then terminates.
Comment 1 Damien Miller 2019-09-05 22:11:00 AEST
Is this issue with the Cygwin support in portable OpenSSH or the Microsoft OpenSSH port?
Comment 2 Damien Miller 2022-01-14 15:44:06 AEDT
closing for lack of followup
Comment 3 Damien Miller 2022-02-25 13:59:56 AEDT
closing bugs resolved before openssh-8.9