Bug 1598

Description of problem:
Sometimes ssh-session hangs up after remote command already completed.

How reproducible:
Intermittent, no clear understanding on how to reproduce the issue.
 
Steps to Reproduce:
No clear understanding on how to get this issue.

Actual results:
Ssh-sesson hangs up. Client side can not complete it’s work.

Expected results:
Ssh-session completed successfully and connection Successfully closed.


Additional info:
[root@ibm3-e var]# rpm -aq | grep openssh
openssh-4.3p2-26
openssh-server-4.3p2-26
openssh-askpass-4.3p2-26
openssh-clients-4.3p2-26

uname  -a
Linux ibm3-e.asa.merann.ru 2.6.18-53.el5 #1 SMP Wed Oct 10 16:34:02 EDT 2007 i686 i686 i386 GNU/Linux

Some debug information were collected:
1.	Strace logs:
root      2953     1  0 Apr02 ?        00:00:00 /usr/sbin/sshd
root     20073  2953  0 Apr07 ?        00:00:00  \_ sshd: nortel [priv]
nortel   20075 20073  0 Apr07 ?        00:00:16  |   \_ sshd: nortel@notty
root     30714  2953  0 10:53 ?        00:00:00  \_ sshd: nortel [priv]
nortel   30728 30714  0 10:53 ?        00:00:00      \_ sshd: nortel@pts/1
nortel   30729 30728  0 10:53 pts/1    00:00:00          \_ -bash
 root     31057 30729  0 10:56 pts/1    00:00:00              \_ su -
…
strace -p 20075
Process 20075 attached - interrupt to quit select(12, [3 6 9 11], [], NULL, NULL)  = 1 (in [3]) rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 read(3, "%:\5\256\247\245W~\241jP/\270=o\264\213\252\2164\366\377\261\264g\303KfQI\226\353"..., 16384) = 64 write(3, "\204S\326\265\"\31\3256rr_\361\372zQ\317\325~v|YI\223\265\1\241\32V\224\347\221\250", 32) = 32 select(12, [3 6 9 11], [], NULL, NULL)  = 1 (in [3]) rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 ….

Collected debug information from sshd log:
info Apr 15 08:36:48 Apr 15 08:36:48 marcello  pam_unix(system-auth:session): session closed for user root debug Apr 15 08:36:48 Apr 15 08:36:48 marcello  debug1: Received SIGCHLD.
debug Apr 15 08:36:48 Apr 15 08:36:48 marcello  debug1: session_by_pid: pid 14897 debug Apr 15 08:36:48 Apr 15 08:36:48 marcello  debug1: session_exit_message: session 0 channel 0 pid 14897 debug Apr 15 08:36:48 Apr 15 08:36:48 marcello  debug1: session_exit_message: release channel 0 <no more messages in the log>

Open descriptors information:
lsof -p 20075 -p 20073
COMMAND   PID   USER   FD   TYPE     DEVICE    SIZE    NODE NAME
sshd    20073   root  cwd    DIR       3,10    4096       2 /
sshd    20073   root  rtd    DIR       3,10    4096       2 /
sshd    20073   root  txt    REG       3,10  391436  372641 /usr/sbin/sshd
sshd    20073   root  mem    REG       3,10   13420  327725 /lib/libutil-2.5.so
<some libs here>
sshd    20073   root  DEL    REG        0,8         4389874 /dev/zero
sshd    20073   root  DEL    REG        0,8         4389866 /dev/zero
sshd    20073   root    0u   CHR        1,3            1069 /dev/null
sshd    20073   root    1u   CHR        1,3            1069 /dev/null
sshd    20073   root    2u   CHR        1,3            1069 /dev/null
sshd    20073   root    3u  IPv4    4389854             TCP vrf2a.bvwpitlab.ca:ssh->em-pit.bvwpitlab.ca:43178 (ESTABLISHED)
sshd    20073   root    4u  unix 0xed384740         4389877 socket
sshd    20073   root    6u  unix 0xed3843c0         4389882 socket
sshd    20075 nortel  cwd    DIR       3,10    4096       2 /
sshd    20075 nortel  rtd    DIR       3,10    4096       2 /
<some libs here>

sshd    20075 nortel  DEL    REG        0,8         4389874 /dev/zero
sshd    20075 nortel  DEL    REG        0,8         4389866 /dev/zero
sshd    20075 nortel    0u   CHR        1,3            1069 /dev/null
 sshd    20075 nortel    1u   CHR        1,3            1069 /dev/null
sshd    20075 nortel    2u   CHR        1,3            1069 /dev/null
sshd    20075 nortel    3u  IPv4    4389854             TCP vrf2a.bvwpitlab.ca:ssh->em-pit.bvwpitlab.ca:43178 (ESTABLISHED)
sshd    20075 nortel    4u  unix 0xed384740         4389877 socket
sshd    20075 nortel    5u  unix 0xed384580         4389881 socket
sshd    20075 nortel    6r  FIFO        0,6         4389887 pipe
sshd    20075 nortel    7w  FIFO        0,6         4389887 pipe
sshd    20075 nortel    9u  unix 0xed384900         4389889 socket
sshd    20075 nortel   11u  unix 0xed384c80         4389891 socket

select(12, [3 6 9 11], [], NULL, NULL)  = 1 (in [3])

sshd    20075 nortel    3u  IPv4    4389854             TCP vrf2a.bvwpitlab.ca:ssh->em-pit.bvwpitlab.ca:43178 (ESTABLISHED)
sshd    20073   root    3u  IPv4    4389854             TCP vrf2a.bvwpitlab.ca:ssh->em-pit.bvwpitlab.ca:43178 (ESTABLISHED)


During another round of testing new kernel with some fix of another ssh issue were used and this issue still can be observed even with a newer kernel:
“kernel-2.6.18-128.el5.it265105.i686.rpm” 

https://bugzilla.redhat.com/show_bug.cgi?id=448328
 
 
Red Hat Bugzilla – Bug 448328 


I believe that is could be know openssh issue: 
http://brneurosci.org/linuxsetup54.html

My initial investigation shows that it is looks like the issue that could be fixed with suggested patch.

I the web-page above claims that this issue already fixed I tried to check session.c in CSV but did not find this fix in. Were it actually fixed?