Bug 502

Summary: sshd fails when "Compression yes" set on HPUX
Product: Portable OpenSSH Reporter: David Brownell <dbrownell>
Component: sshdAssignee: OpenSSH Bugzilla mailing list <openssh-bugs>
Status: CLOSED WORKSFORME    
Severity: normal    
Priority: P2    
Version: -current   
Hardware: HPPA   
OS: HP-UX   

Description David Brownell 2003-02-25 09:16:33 AEDT 
sshd fails each time when I set "Compression yes" in /usr/local/etc/sshd_config
on an HPUX 10.20.
OpenSSH version is OpenSSH_3.5p1
zlib version is 1.1.4
OpenSSL version is OpenSSL 0.9.6g 9 Aug 2002
The compiler is the HP C/ANSI C B.10.20.03

syslog on the sshd systems reports:
Feb 24 12:46:32 als3 sshd[15645]: fatal: buffer_append_space: alloc 10506240 not
supported

I've tried recompiling zlib and openssh with different options to no effect.
sshd -ddd shows:
debug3: Seeding PRNG from /usr/local/libexec/ssh-rand-helper
debug1: sshd version OpenSSH_3.5p1
debug1: private host key: #0 type 0 RSA1
debug3: Not a RSA1 key file /usr/local/etc/ssh_host_rsa_key.
debug1: read PEM private key done: type RSA
debug1: private host key: #1 type 1 RSA
debug3: Not a RSA1 key file /usr/local/etc/ssh_host_dsa_key.
debug1: read PEM private key done: type DSA
debug1: private host key: #2 type 2 DSA
debug1: Bind to port 22 on 0.0.0.0.
Server listening on 0.0.0.0 port 22.
Generating 768 bit RSA key.
RSA key generation complete.
debug1: Server will not fork when running in debugging mode.
Connection from 12.168.109.200 port 35213
debug1: Client protocol version 2.0; client software version OpenSSH_3.4p1
debug1: match: OpenSSH_3.4p1 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-1.99-OpenSSH_3.5p1
debug2: Network child is on pid 22200
debug3: preauth child monitor started
debug3:  entering
debug3: privsep user:group 74:74
debug1: permanently_set_uid: 74/74
debug1: list_hostkey_types: ssh-rsa,ssh-dss
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug2: kex_parse_kexinit:
diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa,ssh-dss
debug2: kex_parse_kexinit:
aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour,aes192-cbc,aes256-cbc,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit:
aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour,aes192-cbc,aes256-cbc,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit:
hmac-md5,hmac-sha1,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit:
hmac-md5,hmac-sha1,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,zlib
debug2: kex_parse_kexinit: none,zlib
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: kex_parse_kexinit:
diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa,ssh-dss
debug2: kex_parse_kexinit:
aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour,aes192-cbc,aes256-cbc,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit:
aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour,aes192-cbc,aes256-cbc,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit:
hmac-md5,hmac-sha1,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit:
hmac-md5,hmac-sha1,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: zlib
debug2: kex_parse_kexinit: zlib
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: mac_init: found hmac-md5
debug1: kex: client->server aes128-cbc hmac-md5 zlib
debug2: mac_init: found hmac-md5
debug1: kex: server->client aes128-cbc hmac-md5 zlib
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST received
debug3:  entering: type 0
debug3: : waiting for MONITOR_ANS_MODULI
debug3: : checking request 0debug3:  entering: type 1

debug3:  entering
debug3: : got parameters: 1024 2048 8192
debug3:  entering: type 1
debug2: : 0 used once, disabling now
debug3: : remaining 0
debug3:  entering
debug1: SSH2_MSG_KEX_DH_GEX_GROUP sent
debug1: dh_gen_key: priv key bits set: 128/256
debug1: bits set: 1575/3191
debug1: expecting SSH2_MSG_KEX_DH_GEX_INIT
debug1: bits set: 1553/3191
debug3:  entering
debug3:  entering: type 4
debug3: : waiting for MONITOR_ANS_SIGN
debug3:  entering: type 5
debug3:  entering
debug3: : checking request 4
debug3:
debug3: : signature 4002f878(143)
debug3:  entering: type 5
debug2: : 4 used once, disabling now
debug1: SSH2_MSG_KEX_DH_GEX_REPLY sentdebug3:  entering

debug1: kex_derive_keys
debug1: newkeys: mode 1
debug1: Enabling compression at level 6.
debug1: SSH2_MSG_NEWKEYS sent
debug1: waiting for SSH2_MSG_NEWKEYS
debug1: newkeys: mode 0
debug1: SSH2_MSG_NEWKEYS received
debug1: KEX done
debug1: userauth-request for user root service ssh-connection method none
debug1: attempt 0 failures 0
debug3:  entering
debug3:  entering: type 6
debug3: : waiting for MONITOR_ANS_PWNAM
debug3: : checking request 6debug3:  entering: type 7

debug3:  entering
debug3:
debug3: : sending MONITOR_ANS_PWNAM: 1
debug3:  entering: type 7
debug2: : 6 used once, disabling nowdebug2: input_userauth_request: setting up
authctxt for root

debug3:  entering
debug3:  entering
debug3:  entering: type 3
debug2: input_userauth_request: try method nonedebug3: : checking request 3

debug3: : service=ssh-connection, style=
debug3:  entering
debug2: : 3 used once, disabling now
debug3:  entering: type 10
debug3:  entering
debug3: : waiting for MONITOR_ANS_AUTHPASSWORD
debug3:  entering: type 11
debug3: : checking request 10
debug3:  enteringdebug3: : sending result 0

debug3:  entering: type 11
debug3: : user not authenticatedFailed none for root from 12.168.109.200 port
35213 ssh2

debug3:  entering
Failed none for root from 12.168.109.200 port 35213 ssh2
debug1: userauth-request for user root service ssh-connection method publickey
debug1: attempt 1 failures 1
debug2: input_userauth_request: try method publickey
debug1: test whether pkalg/pkblob are acceptable
debug3:  entering
debug3:  entering: type 20
debug3: : waiting for MONITOR_ANS_KEYALLOWED
debug3: : checking request 20debug3:  entering: type 21

debug3:  entering
debug3:  entering
debug3: : key_from_blob: 4002f8a0
debug1: temporarily_use_uid: 0/3 (e=0/3)
debug1: trying public key file //.ssh/authorized_keys
debug1: restore_uid: 0/3
debug1: temporarily_use_uid: 0/3 (e=0/3)
debug1: trying public key file //.ssh/authorized_keys2
debug3: secure_filename: checking '/.ssh'
debug3: secure_filename: checking '/'
debug3: secure_filename: terminating check at '/'
debug1: matching key found: file //.ssh/authorized_keys2, line 1
Found matching DSA key: be:ca:c4:c5:ad:b3:4a:7c:42:c1:2d:3e:7e:30:91:e5
debug1: restore_uid: 0/3
debug3: : key 4002f8a0 is allowed
debug3:  entering: type 21
debug3:  entering
debug2: userauth_pubkey: authenticated 0 pkalg ssh-dss
Postponed publickey for root from 12.168.109.200 port 35213 ssh2
debug1: userauth-request for user root service ssh-connection method publickey
debug1: attempt 2 failures 1
debug2: input_userauth_request: try method publickey
debug3:  entering
debug3:  entering: type 20
debug3: : waiting for MONITOR_ANS_KEYALLOWED
debug3:  entering: type 21
debug3: : checking request 20debug3:  entering

debug3:  entering
debug3: : key_from_blob: 4002f8c0
debug1: temporarily_use_uid: 0/3 (e=0/3)
debug1: trying public key file //.ssh/authorized_keys
debug1: restore_uid: 0/3
debug1: temporarily_use_uid: 0/3 (e=0/3)
debug1: trying public key file //.ssh/authorized_keys2
debug3: secure_filename: checking '/.ssh'
debug3: secure_filename: checking '/'
debug3: secure_filename: terminating check at '/'
debug1: matching key found: file //.ssh/authorized_keys2, line 1
Found matching DSA key: be:ca:c4:c5:ad:b3:4a:7c:42:c1:2d:3e:7e:30:91:e5
debug1: restore_uid: 0/3
debug3: : key 4002f8c0 is allowed
debug3:  entering: type 21
debug3:  entering
debug3:  entering
debug3:  entering: type 22
debug3: : waiting for MONITOR_ANS_KEYVERIFY
debug3: : checking request 22debug3:  entering: type 23

debug3:  entering
debug1: ssh_dss_verify: signature correct
debug3: : key 4002f8f0 signature verified
debug3:  entering: type 23
debug2: userauth_pubkey: authenticated 1 pkalg ssh-dss
Accepted publickey for root from 12.168.109.200 port 35213 ssh2
debug3: : Sending new keys: 40030e08 40030d88
debug3: : converting 40030e08
debug3: : converting 40030d88
debug3: : New keys have been sent
debug3: : Sending compression state
debug3:  entering: type 24
debug3: : Finished sending state
Accepted publickey for root from 12.168.109.200 port 35213 ssh2
debug1: : root has been authenticated by privileged process
debug3: : Waiting for new keys
debug3:  entering: type 24
debug3:  entering
debug3: : 4002fba0(118)
debug2: mac_init: found hmac-md5
debug3: : Waiting for second key
debug3: : 4002fba0(118)
debug2: mac_init: found hmac-md5
debug3: : Getting compression state
debug3: : Getting Network I/O buffers
debug3: : Share sync
debug3: : Share sync end
debug1: newkeys: mode 0
debug1: newkeys: mode 1
debug1: Entering interactive session for SSH2.
debug1: fd 3 setting O_NONBLOCK
debug1: fd 7 setting O_NONBLOCK
debug1: server_init_dispatch_20
buffer_append_space: alloc 10506240 not supported
debug1: Calling cleanup 0x4001f72a(0x0)


David Brownell
Comment 1 Darren Tucker 2003-05-12 19:12:58 AEST 
Is there an older libz in you library path?  Mark Janssen reported an identical 
problem (all the way down to the number after "alloc") which was solved by 
recompiling zlib.  See:
http://marc.theaimsgroup.com/?l=openssh-unix-dev&m=104487220204506
http://marc.theaimsgroup.com/?l=openssh-unix-dev&m=104613235206211
Comment 2 Darren Tucker 2003-08-22 12:52:20 AEST 
No answer == no bug.

FWIW this works for me on HP-UX 11.00.
Comment 3 Damien Miller 2004-04-14 12:24:18 AEST 
Mass change of RESOLVED bugs to CLOSED