The skeychallenge() call has four arguments in NetBSD: int skeychallenge(struct skey *mp, const char *name, char *ss, size_t sslen); OpenSSH does not know about the 4th argument, so when you build with --with-skey, the compile fails. If I put this into config.h: #define skeychallenge(a,b,c) skeychallenge(a,b,c,sizeof(c)) then compilation is successful.
Perhaps configure could test for a 4-arg version the same as it does for loginfailed() on AIX, then add the define if required?
Is there an easy way to detect how many args a #define has from autoconf? It may need a test prog.
There's an example in the AIX section of configure.ac (look for LOGINFAILED_4ARG). It just does AC_TRY_COMPILE.
If someone could write (and test!) a patch against configure.ac to check for a 4-arg skeychallenge we will review and probably apply it. The test should not be NetBSD-specific, in case other platforms have the same issue. See LOGINFAILED_4ARG for an example of how it can be done.
Created attachment 602 [details] Test if skeychallenge() takes 4 args.
Patch applied and will be in 3.8.1p1. Thanks for the report.
Closed with release of portable OpenSSH 3.8.1p1