Bug 536

This is probably a variant of the setsid/controlling tty wierdness that 
Solaris had (bug #245).  Try adding "#define STREAMS_PUSH_ACQUIRES_CTTY 1" to 
config.h and recompiling.

Also reported by Frank Adelstein (frank at MyCause dot com):

[quote]
Perhaps this is a known problem, but I only found one instance of someone
describing it on the net with no follow-up.  I just installed openssh 
(3.6p1) on a linux system (running an old 2.0.34 kernel) with privilege 
separation disabled and get the following messages in /var/log/messages:

Apr  2 15:48:34 ernestine sshd[6153]: error: ioctl(TIOCSCTTY): Operation not 
permitted
Apr  2 15:48:34 ernestine sshd[6153]: error: open /dev/tty failed - could not 
set controlling tty: Device not configured

The effect is that ^Z's are mostly ignored (but not by vi) and ^C kills
the session, rather than a running program.

After poking around the net and the code a bit, I found that changing line
318 in sshpty.c from:

        if (ioctl(*ttyfd, TIOCSCTTY, NULL) < 0)
to
        if (ioctl(*ttyfd, TIOCSCTTY, 1) < 0)

seems to fix things.  Is this a known bug or something peculiar to my
installation?  Let me know if there is any further information I can 
provide.  Thanks for any input.
[/quote]

Adding the STREAMS_PUSH_ACQUIRES_CTTY define works. Thanks for the help.

Don't close the bug yet, we still need to fix the source.

I don't think it's exactly the same as Solaris but a different problem with the 
same symptoms.  I don't have a Linux 2.0 box handy to go digging into the root 
cause.

Any volunteers, or do we rename STREAMS_PUSH_ACQUIRES_CTTY to something more 
generic and define it for Linux 2.0?

*** Bug 540 has been marked as a duplicate of this bug. ***

Adding that this is not 2.0 specific, I see the same behavior on 2.4 with libc5.

* This error also happens on Linux Kernel 2.2.22, glibc 2.3.1 with openssh-3.6p1  
  and openssh-3.6.1p1. openssh-3.5p1 is the latest version running on this 
  system.

* It happens with users with "bash" as default shell.

* sshd -d => works

* #define STREAMS_PUSH_ACQUIRES_CTTY 1 => works

* The workaround from "Frank Adelstein" does not work.

Very similar problem on LRP due to an error that says 
(approximately, since I did a roll back to 3.4p1 and do not have access to a
compiler to recompile 3.6p1) "insufficient privilage to create tty", 
Will update the thread when I get the machine back.

Compile environment: Debian Slink

Compile environment: Debian Slink, kernel 2.2.12
Target environment: LRP, kernel 2.2.19, glibc 2.0

This works:
#define STREAMS_PUSH_ACQUIRES_CTTY 1 

This works:
modify line 318 of sshpty.c
if (ioctl(*ttyfd, TIOCSCTTY, 1) < 0)

Tested with 3.6p1 and 3.6.1.p1

Created attachment 282 [details]
Don't call setsid() on Linux 2.0 and 2.2

Please try the the attached patch.  It renames STREAMS_PUSH_ACQUIRES_CTTY ->
SSHD_ACQUIRES_CTTY and defines it for Linux 2.0 and 2.0.

The patch is against the CVS tree and you'll need to run "autoreconf".	If you
don't have a CVS tree or autoconf handy you can try this snapshot instead:
http://www.zip.com.au/~dtucker/openssh/test/openssh-linux20_ctty.tar.gz

Make that "2.0 and 2.2" and does anyone know a good way of detecting libc5?

Created attachment 283 [details]
cttytest.c: Test for broken Linux/glibc/openpty controlling terminal behaviour

OK, the data points we've got are:
broken: 2.0.34 kernel, unknown libc
broken: 2.4.? kernel, libc5
broken: 2.0.38 kernel, glibc 2.0.7 (Debian Slink)
working: kernel 2.4.18, glibc 2.3.2 (Redhat 8)

So it looks like a libc thing.	Searching the glibc ChangeLog:

1999-05-24  Ulrich Drepper  <drepper@cygnus.com>
	* login/openpty.c (openpty): Make sure pty does not because
	controlling TTY.

So it looks like openpty is the culprit.  I have attached a test program that
should detect this.  You made need to link with -lutil.

On Debian Slink:
$ gcc cttytest.c -lutil
$ ./a.out
Test failed: reacquired controlling tty

And on Redhat 8:
$ ./a.out 
Test passed.

I'll wrap it into a configure test and attach a patch.

Unfortunately I have migrated the 2.4.18/libc5 system to glibc 2.2.3 since my
comment, your test passes, as expected. I believe this test is a winner.

Created attachment 284 [details]
Add configure test for broken openpty() on Linux.

Please test.  You will need to run "autoreconf".  I have updated the snapshot
mentioned in comment #10.

I had to move the -lutil stuff to before the platform-specific tests since the
openpty test needs libutil, I'm not sure about that from a style point of view.

Created attachment 285 [details]
Set SSHD_ACQUIRES_CTTY unless openpty tests OK.

Similar to previous patch but inverts the sense of the test and adds a bit of
error checking.

Defining SSHD_ACQUIRES_CTTY is the safe option, so it's defined *unless* we're
on a known-good configuration.	(The previous test would not set it if the
compilation of the test failed and would produce a partially-broken sshd in
that case).

Created attachment 286 [details]
Set SSHD_ACQUIRES_CTTY unless openpty tests OK (clean up)

Add a Linux-specific openpty/libutil test and leave the original login/libutil
tests alone.  Fix some spaces -> tabs.

Tests OK on Redhat 8 and Debian Slink.

This is the final patch unless someone points out a problem.

Fix applied.

Mass change of RESOLVED bugs to CLOSED