Hi. Nagios command "check_ssh" sometimes cause the problem on server side in auth.log: Dec 20 22:43:24 ns1 sshd[15957]: fatal: Read from socket failed: Connection reset by peer [preauth] Dec 20 22:43:31 ns1 sshd[93749]: fatal: Read from socket failed: Connection reset by peer [preauth] Dec 20 22:43:33 ns1 sshd[8780]: fatal: Read from socket failed: Connection reset by peer [preauth] Dec 20 22:43:33 ns1 sshd[32834]: fatal: Read from socket failed: Connection reset by peer [preauth] sometimes no: Dec 20 22:43:24 ns1 sshd[50110]: Connection closed by 1.1.1.1 [preauth] Dec 20 22:43:32 ns1 sshd[96172]: Connection closed by 1.1.1.1 [preauth] Dec 20 22:43:32 ns1 sshd[98599]: Connection closed by 1.1.1.1 [preauth] Help.
Right, so check_ssh is opening a connection to a sshd and closing it ungracefully. What's the actual problem?
Why sometimes good, sometimes bad?
You'll probably need to provide some more information to tell that, either from the server (LogLevel debug3) or the client (via whatever mechanism it has). Long shot guess: the server is hitting the MaxStartups limit?
MaxStartups 10 (i set default), only I and Nagios connected to server. It happened after upgrade from 5 to 6. And now servers send me periodic with this failed message... Nagios command 'check_ssh' don't have debug parameter (.
the server can still provide some logs, but you haven't provided them, and without anything to go on we can't help.
Server side: root@ns1:/home/kvarnakov # /usr/local/sbin/sshd -ddd debug2: load_server_config: filename /usr/local/etc/ssh/sshd_config debug2: load_server_config: done config len = 1235 debug2: parse_server_config: config /usr/local/etc/ssh/sshd_config len 1235 debug3: /usr/local/etc/ssh/sshd_config:13 setting Port 22 debug3: /usr/local/etc/ssh/sshd_config:16 setting ListenAddress 1.1.1.1 debug3: /usr/local/etc/ssh/sshd_config:21 setting Protocol 2 debug3: /usr/local/etc/ssh/sshd_config:26 setting HostKey /usr/local/etc/ssh/ssh_host_rsa_key debug3: /usr/local/etc/ssh/sshd_config:27 setting HostKey /usr/local/etc/ssh/ssh_host_dsa_key debug3: /usr/local/etc/ssh/sshd_config:30 setting KeyRegenerationInterval 1h debug3: /usr/local/etc/ssh/sshd_config:31 setting ServerKeyBits 1024 debug3: /usr/local/etc/ssh/sshd_config:35 setting SyslogFacility AUTH debug3: /usr/local/etc/ssh/sshd_config:36 setting LogLevel debug3 debug3: /usr/local/etc/ssh/sshd_config:40 setting LoginGraceTime 1m debug3: /usr/local/etc/ssh/sshd_config:41 setting PermitRootLogin no debug3: /usr/local/etc/ssh/sshd_config:42 setting StrictModes yes debug3: /usr/local/etc/ssh/sshd_config:43 setting MaxAuthTries 3 debug3: /usr/local/etc/ssh/sshd_config:44 setting MaxSessions 10 debug3: /usr/local/etc/ssh/sshd_config:46 setting RSAAuthentication no debug3: /usr/local/etc/ssh/sshd_config:47 setting PubkeyAuthentication yes debug3: /usr/local/etc/ssh/sshd_config:48 setting AuthorizedKeysFile .ssh/authorized_keys debug3: /usr/local/etc/ssh/sshd_config:51 setting RhostsRSAAuthentication no debug3: /usr/local/etc/ssh/sshd_config:53 setting HostbasedAuthentication no debug3: /usr/local/etc/ssh/sshd_config:61 setting PasswordAuthentication no debug3: /usr/local/etc/ssh/sshd_config:62 setting PermitEmptyPasswords no debug3: /usr/local/etc/ssh/sshd_config:65 setting ChallengeResponseAuthentication yes debug3: /usr/local/etc/ssh/sshd_config:86 setting UsePAM yes debug3: /usr/local/etc/ssh/sshd_config:88 setting AllowAgentForwarding no debug3: /usr/local/etc/ssh/sshd_config:89 setting AllowTcpForwarding no debug3: /usr/local/etc/ssh/sshd_config:90 setting GatewayPorts no debug3: /usr/local/etc/ssh/sshd_config:91 setting X11Forwarding no debug3: /usr/local/etc/ssh/sshd_config:95 setting PrintMotd yes debug3: /usr/local/etc/ssh/sshd_config:96 setting PrintLastLog yes debug3: /usr/local/etc/ssh/sshd_config:97 setting TCPKeepAlive yes debug3: /usr/local/etc/ssh/sshd_config:99 setting UseLogin no debug3: /usr/local/etc/ssh/sshd_config:100 setting UsePrivilegeSeparation yes debug3: /usr/local/etc/ssh/sshd_config:101 setting PermitUserEnvironment no debug3: /usr/local/etc/ssh/sshd_config:103 setting Compression delayed debug3: /usr/local/etc/ssh/sshd_config:104 setting ClientAliveInterval 0 debug3: /usr/local/etc/ssh/sshd_config:105 setting ClientAliveCountMax 3 debug3: /usr/local/etc/ssh/sshd_config:106 setting UseDNS yes debug3: /usr/local/etc/ssh/sshd_config:108 setting PidFile /var/run/sshd.pid debug3: /usr/local/etc/ssh/sshd_config:109 setting MaxStartups 10 debug3: /usr/local/etc/ssh/sshd_config:111 setting PermitTunnel no debug3: /usr/local/etc/ssh/sshd_config:112 setting ChrootDirectory none debug3: /usr/local/etc/ssh/sshd_config:115 setting Banner none debug3: /usr/local/etc/ssh/sshd_config:119 setting UseLPK yes debug3: /usr/local/etc/ssh/sshd_config:121 setting LpkServers ldap://srv01.ldap.ru ldap://srv02.ldap.ru debug3: /usr/local/etc/ssh/sshd_config:122 setting LpkUserDN ou=users,ou=sys,o=ldap,c=ru debug3: /usr/local/etc/ssh/sshd_config:123 setting LpkGroupDN ou=groups,ou=sys,o=ldap,c=ru debug3: /usr/local/etc/ssh/sshd_config:124 setting LpkForceTLS no debug3: /usr/local/etc/ssh/sshd_config:125 setting LpkSearchTimelimit 3 debug3: /usr/local/etc/ssh/sshd_config:126 setting LpkBindTimelimit 3 debug3: /usr/local/etc/ssh/sshd_config:129 setting Subsystem sftp /usr/local/libexec/sftp-server debug1: sshd version OpenSSH_6.2p2 FreeBSD-openssh-portable-6.2.p2_3,1, OpenSSL 0.9.8y 5 Feb 2013 debug3: Incorrect RSA1 identifier debug1: read PEM private key done: type RSA debug1: private host key: #0 type 1 RSA debug3: Incorrect RSA1 identifier debug1: read PEM private key done: type DSA debug1: private host key: #1 type 2 DSA debug1: rexec_argv[0]='/usr/local/sbin/sshd' debug1: rexec_argv[1]='-ddd' debug1: madvise(): Operation not permitted debug2: fd 4 setting O_NONBLOCK debug1: Bind to port 22 on 1.1.1.1 Server listening on 1.1.1.1 port 22. debug1: fd 5 clearing O_NONBLOCK debug1: Server will not fork when running in debugging mode. debug3: send_rexec_state: entering fd = 8 config len 1235 debug3: ssh_msg_send: type 0 debug3: send_rexec_state: done debug1: rexec start in 5 out 5 newsock 5 pipe -1 sock 8 debug1: inetd sockets after dupping: 4, 4 Connection from 2.2.2.2 port 27871 debug1: Client protocol version 2.0; client software version check_ssh_1.4.16 debug1: no match: check_ssh_1.4.16 debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_6.2p2 FreeBSD-openssh-portable-6.2.p2_3,1 debug2: fd 4 setting O_NONBLOCK debug2: Network child is on pid 49597 debug3: preauth child monitor started debug3: privsep user:group 22:22 [preauth] debug1: permanently_set_uid: 22/22 [preauth] debug1: list_hostkey_types: ssh-rsa,ssh-dss [preauth] debug1: SSH2_MSG_KEXINIT sent [preauth] Read from socket failed: Connection reset by peer [preauth] debug1: do_cleanup [preauth] debug3: PAM: sshpam_thread_cleanup entering [preauth] debug1: monitor_read_log: child log fd closed debug3: mm_request_receive entering debug1: do_cleanup debug3: PAM: sshpam_thread_cleanup entering debug1: Killing privsep child 49597
I use nss-pam-ldapd, may be it problem...
A couple of things: (In reply to Kiril Varnakov from comment #6) [...] > debug1: sshd version OpenSSH_6.2p2 > FreeBSD-openssh-portable-6.2.p2_3,1, OpenSSL 0.9.8y 5 Feb 2013 This is a modified version of sshd. Can you reproduce the problem with the stock version compiled from the source on openssh.com? > debug1: SSH2_MSG_KEXINIT sent [preauth] > Read from socket failed: Connection reset by peer [preauth] this looks like the client is crashing during key exchange. A number of methods and ciphers were added in recent versions and some clients have had trouble with the size of the list. you could try disabling some of them in the server config: KexAlgorithms diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1 Ciphers aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour
With ssh from base system: -------------------------------------------------------- root@ns1:/home/kvarnakov # /usr/sbin/sshd -ddd debug2: load_server_config: filename /etc/ssh/sshd_config debug2: load_server_config: done config len = 883 debug2: parse_server_config: config /etc/ssh/sshd_config len 883 debug3: /etc/ssh/sshd_config:17 setting VersionAddendum ??? debug3: /etc/ssh/sshd_config:19 setting Port 22 debug3: /etc/ssh/sshd_config:20 setting Protocol 2 debug3: /etc/ssh/sshd_config:21 setting AddressFamily inet debug3: /etc/ssh/sshd_config:22 setting ListenAddress 1.1.1.1 debug3: /etc/ssh/sshd_config:28 setting HostKey /etc/ssh/ssh_host_dsa_key debug3: /etc/ssh/sshd_config:31 setting KeyRegenerationInterval 1h debug3: /etc/ssh/sshd_config:32 setting ServerKeyBits 768 debug3: /etc/ssh/sshd_config:36 setting SyslogFacility AUTH debug3: /etc/ssh/sshd_config:37 setting LogLevel INFO debug3: /etc/ssh/sshd_config:41 setting LoginGraceTime 2m debug3: /etc/ssh/sshd_config:42 setting PermitRootLogin no debug3: /etc/ssh/sshd_config:43 setting StrictModes yes debug3: /etc/ssh/sshd_config:44 setting MaxAuthTries 3 debug3: /etc/ssh/sshd_config:46 setting RSAAuthentication no debug3: /etc/ssh/sshd_config:47 setting PubkeyAuthentication yes debug3: /etc/ssh/sshd_config:48 setting AuthorizedKeysFile .ssh/authorized_keys debug3: /etc/ssh/sshd_config:51 setting RhostsRSAAuthentication no debug3: /etc/ssh/sshd_config:53 setting HostbasedAuthentication no debug3: /etc/ssh/sshd_config:56 setting IgnoreUserKnownHosts yes debug3: /etc/ssh/sshd_config:58 setting IgnoreRhosts yes debug3: /etc/ssh/sshd_config:61 setting PasswordAuthentication yes debug3: /etc/ssh/sshd_config:62 setting PermitEmptyPasswords no debug3: /etc/ssh/sshd_config:65 setting ChallengeResponseAuthentication no debug3: /etc/ssh/sshd_config:85 setting UsePAM no debug3: /etc/ssh/sshd_config:87 setting AllowTcpForwarding no debug3: /etc/ssh/sshd_config:88 setting GatewayPorts no debug3: /etc/ssh/sshd_config:89 setting X11Forwarding no debug3: /etc/ssh/sshd_config:92 setting PrintMotd yes debug3: /etc/ssh/sshd_config:93 setting PrintLastLog yes debug3: /etc/ssh/sshd_config:94 setting TCPKeepAlive yes debug3: /etc/ssh/sshd_config:95 setting UseLogin no debug3: /etc/ssh/sshd_config:96 setting UsePrivilegeSeparation yes debug3: /etc/ssh/sshd_config:97 setting PermitUserEnvironment no debug3: /etc/ssh/sshd_config:98 setting Compression delayed debug3: /etc/ssh/sshd_config:99 setting ClientAliveInterval 0 debug3: /etc/ssh/sshd_config:100 setting ClientAliveCountMax 3 debug3: /etc/ssh/sshd_config:101 setting UseDNS yes debug3: /etc/ssh/sshd_config:102 setting PidFile /var/run/sshd.pid debug1: HPN Buffer Size: 65536 debug1: sshd version OpenSSH_5.8p2_hpn13v11 ??? debug3: Incorrect RSA1 identifier debug1: read PEM private key done: type DSA debug1: private host key: #0 type 2 DSA debug1: rexec_argv[0]='/usr/sbin/sshd' debug1: rexec_argv[1]='-ddd' debug1: madvise(): Operation not permitted debug2: fd 3 setting O_NONBLOCK debug1: Bind to port 22 on 1.1.1.1. debug1: Server TCP RWIN socket size: 65536 debug1: HPN Buffer Size: 65536 Server listening on 81.176.72.17 port 22. debug1: fd 4 clearing O_NONBLOCK debug1: Server will not fork when running in debugging mode. debug3: send_rexec_state: entering fd = 7 config len 883 debug3: ssh_msg_send: type 0 debug3: send_rexec_state: done debug1: rexec start in 4 out 4 newsock 4 pipe -1 sock 7 debug1: inetd sockets after dupping: 3, 3 debug1: res_init() Connection from 2.2.2.2 port 37109 debug1: HPN Disabled: 0, HPN Buffer Size: 65536 debug1: Client protocol version 2.0; client software version check_ssh_1.4.16 debug1: no match: check_ssh_1.4.16 debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_5.8p2_hpn13v11 ??? debug2: fd 3 setting O_NONBLOCK debug2: Network child is on pid 34321 debug3: preauth child monitor started debug3: mm_request_receive entering debug3: privsep user:group 22:22 debug1: permanently_set_uid: 22/22 debug1: list_hostkey_types: ssh-dss debug1: SSH2_MSG_KEXINIT sent Read from socket failed: Connection reset by peer debug1: do_cleanup debug1: do_cleanup ------------------------------------------- but if i start in demon mode, i don't see this error in log. PS: With custom KexAlgorithms and Ciphers error repeated.
(In reply to Kiril Varnakov from comment #9) > With ssh from base system: > debug1: sshd version OpenSSH_5.8p2_hpn13v11 ??? this is also not the code supplied by us. If you can reproduce the problem with the stock code from openssh.com then we may be able to help, otherwise you need to seek help from the people who supplied the modified sshd.
Ok, thank you.
Close all resolved bugs after 7.3p1 release