problem was solved by adding LIBS='-pthread' to ./configure

guess that this had to be made in ./autoconf for all users

> Preprocessor flags: [...] -Iyes/include
>       Linker flags: [...] -Lyes/lib

That looks wrong.  What flags did you give to configure?

> guess that this had to be made in ./autoconf for all users

In general OpenSSH doesn't need and should not be linked with pthreads.  Generally the only time it's needed is if libcrypto needs it (which appears to be the case here; it looks like you have a static libcrypto which is dynamically linked against pthreads).

(In reply to Darren Tucker from comment #2)
> > Preprocessor flags: [...] -Iyes/include
> >       Linker flags: [...] -Lyes/lib
> 
> That looks wrong.  What flags did you give to configure?
> 
> > guess that this had to be made in ./autoconf for all users
> 
> In general OpenSSH doesn't need and should not be linked with
> pthreads.  Generally the only time it's needed is if libcrypto needs
> it (which appears to be the case here; it looks like you have a
> static libcrypto which is dynamically linked against pthreads).

./configure --with-pie --with-ssl-dir=/usr/local/lib/ --with-libedit --with-ldns --with-zlib --with-selinux LIBS='-pthread'

dunno how to check if libcrypto is static or not:

$ whereis libcrypto
libcrypto: /usr/local/lib/libcrypto.a
$ file /usr/local/lib/libcrypto.a
/usr/local/lib/libcrypto.a: current ar archive

libcrypto is from LibreSSL 2.9.1 and was compiled with --with-pie too, if it does matter.

> > Preprocessor flags: [...] -Iyes/include
> >       Linker flags: [...] -Lyes/lib
> 
> That looks wrong.

certainly, but where from could it jump up?

# find / -name 'yes'
find: ‘/run/user/1000/gvfs’: Permission denied
/home/admz/build/coreutils-8.31/src/yes
/usr/bin/yes

(In reply to admzzz from comment #4)
[...]
> certainly, but where from could it jump up?

The default for --with-$something flags (ie $withval in configure.ac) is "yes".  In OpenSSH's case, many of the --with-$something take arguments that are paths and I could imagine "yes" ending up being interpreted as a path.

That said, so far I haven't been able to reproduce it so I'm not entirely sure.

(In reply to admzzz from comment #3)
[...]
> --with-ssl-dir=/usr/local/lib/

That should be /usr/local (it's the top-level directory containing lib/ and include/).

> [...] --with-zlib

--with-zlib is the default, normally you'd only use that to specify an alternative directory.

> --with-selinux LIBS='-pthread'
> 
> dunno how to check if libcrypto is static or not:
> 
> $ whereis libcrypto
> libcrypto: /usr/local/lib/libcrypto.a

On most platforms including Linux, .a files are static libraries, .so are shared libraries.

> $ file /usr/local/lib/libcrypto.a
> /usr/local/lib/libcrypto.a: current ar archive

Static library.  Compare:

$ file /usr/local/lib/libcrypto.so.45.0.4 
/usr/local/lib/libcrypto.so.45.0.4: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=c2bd54fa0ddba657fd1031822948f7d57c01ba13, with debug_info, not stripped

Anyway, I don't see anything that we could change.

halfsolved ;-)

i'd deleted previous directory, and redid all steps again:

$ cd build
$ git clone https://github.com/openssh/openssh-portable.git
$ cd openssh-portable
$ autoreconf
#######$ ./configure --with-pie --with-ssl-dir=/usr/local/lib/ --with-libedit --with-ldns --with-zlib --with-selinux LIBS='-pthread'


adding key to ./configure one by one the mistake was catched:
 -Iyes/include  & -Lyes/lib are caused by --with-ldns key (without it all ok, rechecked this too).

dunno what to check next and how.

thanks for all corrections.


./configure --with-pie --with-ssl-dir=/usr/local --with-libedit --with-selinux LIBS='-pthread

OpenSSH has been configured with the following options:
                     User binaries: /usr/local/bin
                   System binaries: /usr/local/sbin
               Configuration files: /usr/local/etc
                   Askpass program: /usr/local/libexec/ssh-askpass
                      Manual pages: /usr/local/share/man/manX
                          PID file: /var/run
  Privilege separation chroot path: /var/empty
            sshd default user PATH: /usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
                    Manpage format: doc
                       PAM support: no
                   OSF SIA support: no
                 KerberosV support: no
                   SELinux support: yes
              MD5 password support: no
                   libedit support: yes
                   libldns support: no
  Solaris process contract support: no
           Solaris project support: no
         Solaris privilege support: no
       IP address in $DISPLAY hack: no
           Translate v4 in v6 hack: yes
                  BSD Auth support: no
              Random number source: OpenSSL internal ONLY
             Privsep sandbox style: seccomp_filter

              Host: x86_64-pc-linux-gnu
          Compiler: cc
    Compiler flags: -g -O2 -pipe -Wall -Wpointer-arith -Wuninitialized -Wsign-compare -Wformat-security -Wsizeof-pointer-memaccess -Wno-pointer-sign -Wno-unused-result -fno-strict-aliasing -D_FORTIFY_SOURCE=2 -ftrapv -fno-builtin-memset -fstack-protector-strong -fPIE  
Preprocessor flags: -I/usr/local/include  -D_XOPEN_SOURCE=600 -D_BSD_SOURCE -D_DEFAULT_SOURCE -I/usr/include/editline
      Linker flags: -L/usr/local/lib -Wl,--gc-sections -Wl,-s -Wl,-v -Wl,-z,relro -Wl,-z,now -Wl,-z,noexecstack -fstack-protector-strong -pie 
         Libraries: -lcrypto -ldl -lutil -lz -pthread -lcrypt -lresolv
         +for sshd:  -lselinux
          +for ssh:  -lselinux


how to check what's wrong with --with-ldns ?

(In reply to admzzz from comment #8)
[...]
> how to check what's wrong with --with-ldns ?

Find the output from the ldns tests in config.log. My guess libldns is linked against a different libcrypto than the one you are trying to use.

configure:10176: checking for ldns-config
configure:10209: result: no
configure:10251: checking for ldns support
configure:10264: cc -o conftest -g -O2 -pipe -Wall -Wpointer-arith -Wuninitialized -Wsign-compare -Wformat-security -Wsizeof-pointer-memaccess -Wno-pointer-sign -Wno-unused-result -fno-strict-aliasing -D_FORTIFY_SOURCE=2 -ftrapv -fno-builtin-memset -fstack-protector-strong  -D_XOPEN_SOURCE=600 -D_BSD_SOURCE -D_DEFAULT_SOURCE -Iyes/include -Wl,--gc-sections -Wl,-s -Wl,-v -Wl,-z,relro -Wl,-z,now -Wl,-z,noexecstack -fstack-protector-strong -Lyes/lib conftest.c -lldns -lutil -lz -pthread >&5
GNU ld (GNU Binutils for Ubuntu) 2.26.1
conftest.c: In function 'main':
conftest.c:116:26: warning: variable 'status' set but not used [-Wunused-but-set-variable]
 int main() { ldns_status status = ldns_verify_trusted(NULL, NULL, NULL, NULL); status=LDNS_STATUS_OK; exit(0); }
                          ^~~~~~
collect2 version 8.1.0
/usr/bin/ld -plugin /usr/lib/gcc/x86_64-linux-gnu/8/liblto_plugin.so -plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/8/lto-wrapper -plugin-opt=-fresolution=/tmp/ccOQZTSB.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lpthread -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s --sysroot=/ --build-id --eh-frame-hdr -m elf_x86_64 --hash-style=gnu --as-needed -dynamic-linker /lib64/ld-linux-x86-64.so.2 -z relro -o conftest /usr/lib/x86_64-linux-gnu/crt1.o /usr/lib/x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/8/crtbegin.o -Lyes/lib -L/opt/qt511/lib/x86_64-linux-gnu -L/opt/qt511/lib/../lib -L/usr/local/lib/x86_64-linux-gnu -L/usr/local/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/jvm/java-8-oracle/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/8 -L/usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/8/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib -L/opt/qt511/lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/x86_64-linux-gnu/qt5 -L/lib/x86_64-linux-gnu -L/usr/local/lib -L/usr/lib/jvm/java-8-oracle/lib -L/usr/lib/gcc/x86_64-linux-gnu/8/../../.. --gc-sections -s -v -z relro -z now -z noexecstack /tmp/ccJG0oJF.o -lldns -lutil -lz -lgcc --push-state --as-needed -lgcc_s --pop-state -lpthread -lc -lgcc --push-state --as-needed -lgcc_s --pop-state /usr/lib/gcc/x86_64-linux-gnu/8/crtend.o /usr/lib/x86_64-linux-gnu/crtn.o
configure:10264: $? = 0
configure:10265: result: yes


hmmm.... nothing about libcrypto.

if libldns is linked against a different libcrypto - how to check it? and what is the full_name (or url) of its (libldns) project? to recompile with recent libressl is not a problem, i hope...

https://www.nlnetlabs.nl/projects/ldns/about/
LDNS 1.7.0 (Current version)
Date:    20 Dec, 2016

is it one that need to be recompiled?
am i the first and the only one who had got in this caveat? ;-)

you were right: with ldns 1.7.0 compiled against latest libressl 2.9.1 these  -Iyes/include  & -Lyes/lib disappeared and even openssh compiled fine, but sudo checkinstall failed:

Installing with make install...

========================= Installation results ===========================
(cd openbsd-compat && make)
make[1]: Entering directory '/home/admz/build/openssh-portable/openbsd-compat'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/home/admz/build/openssh-portable/openbsd-compat'
/bin/mkdir -p /usr/local/bin
/bin/mkdir -p /usr/local/sbin
/bin/mkdir -p /usr/local/share/man/man1
/bin/mkdir -p /usr/local/share/man/man5
/bin/mkdir -p /usr/local/share/man/man8
/bin/mkdir -p /usr/local/libexec
/bin/mkdir -p -m 0755 /var/empty
/usr/bin/install -c -m 0755 -s ssh /usr/local/bin/ssh
/usr/bin/install -c -m 0755 -s scp /usr/local/bin/scp
/usr/bin/install -c -m 0755 -s ssh-add /usr/local/bin/ssh-add
/usr/bin/install -c -m 0755 -s ssh-agent /usr/local/bin/ssh-agent
/usr/bin/install -c -m 0755 -s ssh-keygen /usr/local/bin/ssh-keygen
/usr/bin/install -c -m 0755 -s ssh-keyscan /usr/local/bin/ssh-keyscan
/usr/bin/install -c -m 0755 -s sshd /usr/local/sbin/sshd
/usr/bin/install -c -m 4711 -s ssh-keysign /usr/local/libexec/ssh-keysign
/usr/bin/install -c -m 0755 -s ssh-pkcs11-helper /usr/local/libexec/ssh-pkcs11-helper
/usr/bin/install -c -m 0755 -s sftp /usr/local/bin/sftp
/usr/bin/install -c -m 0755 -s sftp-server /usr/local/libexec/sftp-server
/usr/bin/install -c -m 644 ssh.1.out /usr/local/share/man/man1/ssh.1
/usr/bin/install -c -m 644 scp.1.out /usr/local/share/man/man1/scp.1
/usr/bin/install -c -m 644 ssh-add.1.out /usr/local/share/man/man1/ssh-add.1
/usr/bin/install -c -m 644 ssh-agent.1.out /usr/local/share/man/man1/ssh-agent.1
/usr/bin/install -c -m 644 ssh-keygen.1.out /usr/local/share/man/man1/ssh-keygen.1
/usr/bin/install -c -m 644 ssh-keyscan.1.out /usr/local/share/man/man1/ssh-keyscan.1
/usr/bin/install -c -m 644 moduli.5.out /usr/local/share/man/man5/moduli.5
/usr/bin/install -c -m 644 sshd_config.5.out /usr/local/share/man/man5/sshd_config.5
/usr/bin/install -c -m 644 ssh_config.5.out /usr/local/share/man/man5/ssh_config.5
/usr/bin/install -c -m 644 sshd.8.out /usr/local/share/man/man8/sshd.8
/usr/bin/install -c -m 644 sftp.1.out /usr/local/share/man/man1/sftp.1
/usr/bin/install -c -m 644 sftp-server.8.out /usr/local/share/man/man8/sftp-server.8
/usr/bin/install -c -m 644 ssh-keysign.8.out /usr/local/share/man/man8/ssh-keysign.8
/usr/bin/install -c -m 644 ssh-pkcs11-helper.8.out /usr/local/share/man/man8/ssh-pkcs11-helper.8
/bin/mkdir -p /usr/local/etc
/usr/local/etc/ssh_config already exists, install will not overwrite
/usr/local/etc/sshd_config already exists, install will not overwrite
/usr/local/etc/moduli already exists, install will not overwrite
./ssh-keygen: error while loading shared libraries: libldns.so.2: cannot open shared object file: No such file or directory
Makefile:388: recipe for target 'host-key' failed
make: *** [host-key] Error 127

****  Installation failed. Aborting package creation.


any suggestions?

before this attempt i had
# dpkg -l|grep ldns
ii  libldns-dev:amd64                                           1.6.17-8ubuntu0.1                                           amd64        ldns library for DNS programming
ii  libldns1:amd64                                              1.6.17-8ubuntu0.1                                           amd64        ldns library for DNS programming

you were right: with ldns 1.7.0 compiled against latest libressl 2.9.1 these  -Iyes/include  & -Lyes/lib disappeared and even openssh compiled fine, but sudo checkinstall failed:

Installing with make install...

========================= Installation results ===========================
(cd openbsd-compat && make)
make[1]: Entering directory '/home/admz/build/openssh-portable/openbsd-compat'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/home/admz/build/openssh-portable/openbsd-compat'
/bin/mkdir -p /usr/local/bin
/bin/mkdir -p /usr/local/sbin
/bin/mkdir -p /usr/local/share/man/man1
/bin/mkdir -p /usr/local/share/man/man5
/bin/mkdir -p /usr/local/share/man/man8
/bin/mkdir -p /usr/local/libexec
/bin/mkdir -p -m 0755 /var/empty
/usr/bin/install -c -m 0755 -s ssh /usr/local/bin/ssh
/usr/bin/install -c -m 0755 -s scp /usr/local/bin/scp
/usr/bin/install -c -m 0755 -s ssh-add /usr/local/bin/ssh-add
/usr/bin/install -c -m 0755 -s ssh-agent /usr/local/bin/ssh-agent
/usr/bin/install -c -m 0755 -s ssh-keygen /usr/local/bin/ssh-keygen
/usr/bin/install -c -m 0755 -s ssh-keyscan /usr/local/bin/ssh-keyscan
/usr/bin/install -c -m 0755 -s sshd /usr/local/sbin/sshd
/usr/bin/install -c -m 4711 -s ssh-keysign /usr/local/libexec/ssh-keysign
/usr/bin/install -c -m 0755 -s ssh-pkcs11-helper /usr/local/libexec/ssh-pkcs11-helper
/usr/bin/install -c -m 0755 -s sftp /usr/local/bin/sftp
/usr/bin/install -c -m 0755 -s sftp-server /usr/local/libexec/sftp-server
/usr/bin/install -c -m 644 ssh.1.out /usr/local/share/man/man1/ssh.1
/usr/bin/install -c -m 644 scp.1.out /usr/local/share/man/man1/scp.1
/usr/bin/install -c -m 644 ssh-add.1.out /usr/local/share/man/man1/ssh-add.1
/usr/bin/install -c -m 644 ssh-agent.1.out /usr/local/share/man/man1/ssh-agent.1
/usr/bin/install -c -m 644 ssh-keygen.1.out /usr/local/share/man/man1/ssh-keygen.1
/usr/bin/install -c -m 644 ssh-keyscan.1.out /usr/local/share/man/man1/ssh-keyscan.1
/usr/bin/install -c -m 644 moduli.5.out /usr/local/share/man/man5/moduli.5
/usr/bin/install -c -m 644 sshd_config.5.out /usr/local/share/man/man5/sshd_config.5
/usr/bin/install -c -m 644 ssh_config.5.out /usr/local/share/man/man5/ssh_config.5
/usr/bin/install -c -m 644 sshd.8.out /usr/local/share/man/man8/sshd.8
/usr/bin/install -c -m 644 sftp.1.out /usr/local/share/man/man1/sftp.1
/usr/bin/install -c -m 644 sftp-server.8.out /usr/local/share/man/man8/sftp-server.8
/usr/bin/install -c -m 644 ssh-keysign.8.out /usr/local/share/man/man8/ssh-keysign.8
/usr/bin/install -c -m 644 ssh-pkcs11-helper.8.out /usr/local/share/man/man8/ssh-pkcs11-helper.8
/bin/mkdir -p /usr/local/etc
/usr/local/etc/ssh_config already exists, install will not overwrite
/usr/local/etc/sshd_config already exists, install will not overwrite
/usr/local/etc/moduli already exists, install will not overwrite
./ssh-keygen: error while loading shared libraries: libldns.so.2: cannot open shared object file: No such file or directory
Makefile:388: recipe for target 'host-key' failed
make: *** [host-key] Error 127

****  Installation failed. Aborting package creation.


any suggestions?

before this attempt i had
# dpkg -l|grep ldns
ii  libldns-dev:amd64                                           1.6.17-8ubuntu0.1                                           amd64        ldns library for DNS programming
ii  libldns1:amd64                                              1.6.17-8ubuntu0.1                                           amd64        ldns library for DNS programming




thanks a lot for your help, the last problem was solved by: 
1) --with-ldns=/usr/local
2) apt purge libdns1 libdns-dev -y $-)

(In reply to admzzz from comment #13)
> ./ssh-keygen: error while loading shared libraries: libldns.so.2:
> cannot open shared object file: No such file or directory
[...]
> any suggestions?

You probably need to add /usr/local/lib (or wherever you put libldns.so.2) to ld.so.conf so ld.so knows where to find it (see the man page for ld.so(8)).

Darren Tucker,
thanks a lot again for your help, one more (and the last, i hope!) problem:

$ ssh localhost
sign_and_send_pubkey: signing failed: agent refused operation
sign_and_send_pubkey: signing failed: agent refused operation
admz@localhost: Permission denied (publickey,keyboard-interactive).

solution was googled:

$ eval `ssh-agent -s`
Agent pid 18664
$ ssh localhost
Last login: Wed Apr 10 08:49:52 2019 from 127.0.0.1

how to avoid entering this "eval `ssh-agent -s`" before every connection attempt?

...this misbehaviour is new on my mint 18.02, with previous openssh 7.8p1 all works fine... and it does not depends from the used key (i'd specially create a new one (without password, of course)- "ssh-keygen -t rsa -f ~/.ssh/idrsa -b 4096 -v -C admz -a 100" - same shit :-( )...

detailed log of refusions (both are the same):

debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering public key: /home/admz/.ssh/idrsa RSA SHA256:KWPtbHQnNDqwR4B+uyORLQ21c56uTHPaqzdcntvJ4QA explicit agent
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 60
debug1: Server accepts key: /home/admz/.ssh/idrsa RSA SHA256:KWPtbHQnNDqwR4B+uyORLQ21c56uTHPaqzdcntvJ4QA explicit agent
debug3: sign_and_send_pubkey: RSA SHA256:KWPtbHQnNDqwR4B+uyORLQ21c56uTHPaqzdcntvJ4QA
debug3: sign_and_send_pubkey: signing using rsa-sha2-512
sign_and_send_pubkey: signing failed: agent refused operation

(In reply to admzzz from comment #15)
[...]
> $ ssh localhost
> sign_and_send_pubkey: signing failed: agent refused operation
> sign_and_send_pubkey: signing failed: agent refused operation
> admz@localhost: Permission denied (publickey,keyboard-interactive).

Something is offering to be an ssh-agent but is then refusing the requests when ssh sends them for some reason.  My guess is it's a third-party tool like gnome-keyring.  In the failure state, try:

$ echo $SSH_AUTH_SOCK
$ lsof $SSH_AUTH_SOCK

> how to avoid entering this "eval `ssh-agent -s`" before every
> connection attempt?

Assuming my guess is correct, unset SSH_AUTH_SOCK.

> ...this misbehaviour is new on my mint 18.02, with previous openssh
> 7.8p1 all works fine... [..]

compare the sign_and_send_pubkey line from the working version.

(In reply to admzzz from comment #16)
> debug3: sign_and_send_pubkey: signing using rsa-sha2-512
> sign_and_send_pubkey: signing failed: agent refused operation

My guess is that the third-party agent does not support rsa-sha2-512.

thanks a lot again, 
'unset SSH_AUTH_SOCK' in ~/.bashrc works fine.

it was a pleasure to speak with you, sincerely.

Darren Tucker,
excuse me, one more question: is it possible to cross-compile openssh statically on x86-64 for armv7-a (i.e. without use of glibc and password's authentication)? what severities and ambushes could happen on this way?

or maybe you know about precompiled bunaries (i'd googled them, but found nothing)?

...i know about dropbear (and hate it for key's conversion and for impossibility of cross-compiling it on x86-64 accordingly to musl's restrictions)...

(In reply to admzzz from comment #19)
> Darren Tucker,
> excuse me, one more question: is it possible to cross-compile
> openssh statically on x86-64 for armv7-a (i.e. without use of glibc
> and password's authentication)?

It should be possible since OpenSSH does support cross-compilation via the standard --build and --target arguments to configure.  That said it is exercised much less frequently than the usual self-hosted builds. 

> what severities and ambushes could happen on this way?

There's a lot of extra things that can go wrong with a cross-compilation environment so if it breaks you get to keep both pieces.

> or maybe you know about precompiled binaries (i'd googled them, but
> found nothing)?

Not something I usually take much notice of, but if your platform is compatible with openwrt binaries I know they at least ship openssh packages.

Anyway this has drifted a long way from the original bug report so I'm closing.

closing resolved bugs as of 8.6p1 release