Bug 3530

Summary:

AC_CHECK_FUNCS of libfido2 functions generate wrong results if not using pkg-config

Product:

Portable OpenSSH

Reporter:

yzhangad

Component:

Build system

Assignee:

Damien Miller <djm>

Status:

CLOSED FIXED

Severity:

normal

CC:

djm, dtucker

Priority:

Version:

9.1p1

Hardware:

All

OS:

Linux

Bug Depends on:

Bug Blocks:

3480

Attachments:

Description	Flags
Implement James' fix	dtucker: ok+

Description yzhangad 2023-02-01 06:43:57 AEDT

When autoconf uses `AC_CHECK_FUNCS` to test the existence of necessary libfido2 functions, it appends `$LIBFIDO2` to `$LIBS`: https://github.com/openssh/openssh-portable/blob/a20b7e999773e6333c8aa9b0a7fa41966e63b037/configure.ac#L3248. In an environment without `pkg-config`, this makes `-lfido2 -lcbor` appear at the end of the linker flags.

The problem of this: `libcbor` is not the only dependency of `libfido2`: it also depends on, e.g. `libcrypto`, which is part of OpenSSL, and that can be configured via the `--with-ssl-dir` option. When OpenSSL and `libfido2` are built statically, the existing `configure.ac` makes `-lcrypto` go before `-lfido2`, making the linkage break, thus causing false negative results of `AC_CHECK_FUNCS`.

The fix should consist just of replacing `LIBS="$LIBS $LIBFIDO2"` with `LIBS="$LIBFIDO2 $LIBS"`.

Comment 1 Damien Miller 2023-02-01 09:23:59 AEDT

Created attachment 3654 [details]
Implement James' fix

Comment 2 Damien Miller 2023-02-01 12:18:21 AEDT

Applied and will be in the OpenSSH 9.2 release due shortly.

Comment 3 Damien Miller 2023-03-17 13:38:51 AEDT

OpenSSH 9.3 has been released. Close resolved bugs