Bug 3032 - (Fixable) MIPSPro compilation errors, failing regression test in test_sshbuf_misc.c & sshd host key loading issues on IRIX 6.5
Summary: (Fixable) MIPSPro compilation errors, failing regression test in test_sshbuf_...
Status: CLOSED FIXED
Alias: None
Product: Portable OpenSSH
Classification: Unclassified
Component: sshd (show other bugs)
Version: 8.0p1
Hardware: MIPS64 IRIX
: P5 normal
Assignee: Assigned to nobody
URL:
Keywords:
Depends on:
Blocks: V_8_2
  Show dependency treegraph
 
Reported: 2019-07-06 04:35 AEST by Ole Weidner
Modified: 2021-04-23 15:02 AEST (History)
2 users (show)

See Also:


Attachments
./configure output (27.23 KB, text/plain)
2019-07-06 20:53 AEST, Ole Weidner
no flags Details
make output (239.17 KB, text/plain)
2019-07-06 20:53 AEST, Ole Weidner
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ole Weidner 2019-07-06 04:35:44 AEST
I am trying to build OpenSSH with ./configure --with-prngd-socket=/dev/urandom:

              Host: mips-sgi-irix6.5
          Compiler: c99
    Compiler flags: -64 -mips4 -c99 -O2 -LANG:anonymous_unions=ON -I/opt/vntg/include:/usr/include -L/opt/vntg/lib64 -L/usr/lib64  
Preprocessor flags: -64 -c99 -O2 -LANG:anonymous_unions=ON -I/opt/vntg/include -L/opt/vntg/lib64 -L/usr/lib64
      Linker flags: -64 -L/opt/vntg/lib64 -L/usr/lib64 
         Libraries: -lcrypto -lz  -lgen

Two fixable errors occur during compilation:

  cc-1515 c99: ERROR File = sshkey.c, Line = 1270
    A value of type "int" cannot be assigned to an entity of type "char *".

          if ((blobcopy = strndup(cp, space)) == NULL) {

This one I managed to 'fix' by casting the strndup() return value to (char*). I naively assume that is a valid solution? 

The second error is only triggered when the --with-prngd-socket= configure option is set:

  cc-1126 c99: ERROR File = entropy.c, Line = 213
    Expected a statement.

This error is trivial to fix by adding the missing closing bracket in entropy.c:208

After successfully building OpenSSH with these two 'fixes', make tests fails as follows:

regress/unittests/sshbuf/test_sshbuf_misc.c:44 test #77 "sshbuf_dump"
ASSERT_INT_NE(feof(out), 0) failed:
   feof(out) = 0
           0 = 0
/bin/sh[5]: 10908 Abort(coredump)
make[1]: *** [Makefile:230: unit] Error 134
make[1]: Leaving directory '/opt/vntg/src/openssh-8.0p1/regress'
make: *** [Makefile:591: tests] Error 2

In addition, sshd won't start up because it is "unable to load host key", even though the key is valid:

$> ssh-keygen -f hostkey_ecdsa -t ecdsa -N '' -b 521
Your identification has been saved in hostkey_ecdsa.
Your public key has been saved in hostkey_ecdsa.pub.

$> `pwd`/sshd -ddd -D -h hostkey_ecdsa -f sshd_config
debug2: load_server_config: filename sshd_config
debug2: load_server_config: done config len = 195
debug2: parse_server_config: config sshd_config len 195
debug3: sshd_config:41 setting AuthorizedKeysFile .ssh/authorized_keys
debug3: sshd_config:109 setting Subsystem sftp  /usr/libexec/sftp-server
debug1: sshd version OpenSSH_8.0, OpenSSL 1.1.0j  20 Nov 2018
Unable to load host key: /opt/vntg/src/openssh-8.0p1/hostkey_ecdsa
sshd: no hostkeys available -- exiting.
Comment 1 Darren Tucker 2019-07-06 12:11:30 AEST
>   cc-1515 c99: ERROR File = sshkey.c, Line = 1270
>     A value of type "int" cannot be assigned to an entity of type
> "char *".
> 
>           if ((blobcopy = strndup(cp, space)) == NULL) {
> 
> This one I managed to 'fix' by casting the strndup() return value to
> (char*). I naively assume that is a valid solution? 

It's not a proper fix.  It'll probably work in some case where ints and pointers are the same size, but in others (eg "LP64" systems where only longs and pointers are 64 bit) it'll truncate the pointer.

This one is probably because it doesn't have the prototype for strndup.  In C (well, up to c89 at least) functions are assumed to return int.

Is it using the system's (ie does config.h define HAVE_STRNDUP)?  If not we need to add the prototype to the compat library headers.    You want to add the prototype "char *strndup(const char *s, size_t n);"

> The second error is only triggered when the --with-prngd-socket=
> configure option is set:
> 
>   cc-1126 c99: ERROR File = entropy.c, Line = 213
>     Expected a statement.

Thanks, I've applied the fix for that:
https://github.com/openssh/openssh-portable/commit/01a1e21cd55d99293c8ff8ed7c590f2ee440da43

> After successfully building OpenSSH with these two 'fixes', make
> tests fails as follows:

I'd try again after fixing the strndup thing.
Comment 2 Darren Tucker 2019-07-06 19:42:48 AEST
I've applied fixes for both.  Please try a snapshot (https://www.mindrot.org/openssh_snap/) dated 20190707 or later and see if the problem has been resolved.
Comment 3 Ole Weidner 2019-07-06 19:55:41 AEST
(In reply to Darren Tucker from comment #2)
> I've applied fixes for both.  Please try a snapshot
> (https://www.mindrot.org/openssh_snap/) dated 20190707 or later and
> see if the problem has been resolved.

HAVE_STRNDUP is not defined in config.h Adding the strndup prototype "char *strndup(const char *s, size_t n);" fixes the compiler error. However, the regression test #77 "sshbuf_dump" still fails.

I'll try the 20190707 snapshot (I don't see it online yet) later today. 
(In reply to Darren Tucker from comment #2)
> I've applied fixes for both.  Please try a snapshot
> (https://www.mindrot.org/openssh_snap/) dated 20190707 or later and
> see if the problem has been resolved.
Comment 4 Darren Tucker 2019-07-06 20:03:02 AEST
You can also use the github mirror: https://github.com/openssh/openssh-portable but you will need to run autoconf's "autoreconf" to rebuild configure and related files first.
Comment 5 Darren Tucker 2019-07-06 20:29:04 AEST
I've had a look at the test code and it's not obvious what's going wrong.  Could you please attach to this bug (please use "Add attachment" the output of "./configure && make >/dev/null".  Some of the other warnings might give some clues.
Comment 6 Ole Weidner 2019-07-06 20:53:13 AEST
Created attachment 3297 [details]
./configure output
Comment 7 Ole Weidner 2019-07-06 20:53:37 AEST
Created attachment 3298 [details]
make output
Comment 8 Ole Weidner 2019-07-06 20:55:26 AEST
./configure and make output fron the latest git master. Compilation issues are fixed, regression test #77 still fails.
Comment 9 Darren Tucker 2019-07-06 23:45:54 AEST
Comment on attachment 3297 [details]
./configure output

[...]
>    Compiler flags: -64 -mips4 -c99 -O2 -LANG:anonymous_unions=ON -I/opt/vntg/include:/usr/include -L/opt/vntg/lib64 -L/usr/lib64  
>Preprocessor flags: -64 -c99 -O2 -LANG:anonymous_unions=ON -I/opt/vntg/include -L/opt/vntg/lib64 -L/usr/lib64

Did you explicitly give it 64bit compiler flags?  If so does omitting them make any difference?

> cc-3968 c99: WARNING File =cc-3968 c99: WARNING File = sshbuf-getput-basic.c, Line = 307
  implicit conversion of a 64-bit integral type to a smaller integral type
          (potential portability problem)

  	POKE_U64(p, val);
  	^, Line = 307
>  implicit conversion of a 64-bit integral type to a smaller integral type
>          (potential portability problem)
>
>  	POKE_U64(p, val);
>  	^

These ones look suspicious but looking at the code and macro I don't see what it's complaining about since the code:

sshbuf_put_u64(struct sshbuf *buf, u_int64_t val)
{
        u_char *p;
[...]
        POKE_U64(p, val);

and macro:

#define POKE_U64(p, v) \
        do { \
                const u_int64_t __v = (v); \
                ((u_char *)(p))[0] = (__v >> 56) & 0xff; \
                ((u_char *)(p))[1] = (__v >> 48) & 0xff; \
                ((u_char *)(p))[2] = (__v >> 40) & 0xff; \
                ((u_char *)(p))[3] = (__v >> 32) & 0xff; \
                ((u_char *)(p))[4] = (__v >> 24) & 0xff; \
                ((u_char *)(p))[5] = (__v >> 16) & 0xff; \
                ((u_char *)(p))[6] = (__v >> 8) & 0xff; \
                ((u_char *)(p))[7] = __v & 0xff; \
        } while (0)

only seem to refer to 64 bit integral types.
Comment 10 Ole Weidner 2019-07-07 21:54:30 AEST
(In reply to Darren Tucker from comment #9)
> Comment on attachment 3297 [details]
> ./configure output
> 
> [...]
> >    Compiler flags: -64 -mips4 -c99 -O2 -LANG:anonymous_unions=ON -I/opt/vntg/include:/usr/include -L/opt/vntg/lib64 -L/usr/lib64  
> >Preprocessor flags: -64 -c99 -O2 -LANG:anonymous_unions=ON -I/opt/vntg/include -L/opt/vntg/lib64 -L/usr/lib64
> 
> Did you explicitly give it 64bit compiler flags?  If so does
> omitting them make any difference?

I did, yes. Those are my standard compiler flags for everything I build on IRIX64. I'll try to rebuild without to. I'll also try a 32-bit build to see if that makes any difference...

I did a bit of naive debugging in sshd and it seems that the host key loading error is rooted somewhere in the atomicio function. I'll try to dig a bit deeper as soon as I find some time.
Comment 11 Darren Tucker 2019-07-08 12:58:19 AEST
(In reply to Ole Weidner from comment #10)
> I did a bit of naive debugging in sshd and it seems that the host
> key loading error is rooted somewhere in the atomicio function. I'll
> try to dig a bit deeper as soon as I find some time.

You might want to try reading a key produced with the problematic build on another system.  That'll tell you if the problem is in the writing of the key (if it doesn't work on the other system) or the reading of the key (if it does work on the other system).
Comment 12 Damien Miller 2019-10-09 15:07:27 AEDT
Retarget these bugs to 8.2 release
Comment 13 Ole Weidner 2019-12-02 01:14:14 AEDT
I finally found some time to do a clean 32-bit build with openssl-1.1.1d and openssh-8.1p1. Unfortunately it yields the same issue - "Unable to load host key":

[root@octane /]% file /opt/vntg32/sbin/sshd
/opt/vntg32/sbin/sshd:  ELF N32 MSB mips-4 dynamic executable MIPS - version 1

[root@octane /]% /opt/vntg32/sbin/sshd -ddd -t -f /opt/vntg32/etc/sshd_config
debug2: load_server_config: filename /opt/vntg32/etc/sshd_config
debug2: load_server_config: done config len = 202
debug2: parse_server_config: config /opt/vntg32/etc/sshd_config len 202
debug3: /opt/vntg32/etc/sshd_config:41 setting AuthorizedKeysFile .ssh/authorized_keys
debug3: /opt/vntg32/etc/sshd_config:109 setting Subsystem sftp  /opt/vntg32/libexec/sftp-server
debug1: sshd version OpenSSH_8.1, OpenSSL 1.1.1d  10 Sep 2019
debug1: Unable to load host key: /opt/vntg32/etc/ssh_host_rsa_key
debug1: Unable to load host key: /opt/vntg32/etc/ssh_host_ecdsa_key
debug1: Unable to load host key: /opt/vntg32/etc/ssh_host_ed25519_key
sshd: no hostkeys available -- exiting.

The keys exist and I can load them with an older version (OpenSSH_6.2p1) without any problems.

[root@octane /]% ls -l /opt/vntg32/etc/
total 1155
-rw-r--r--    1 root     sys       577388 Nov 30 23:28 moduli
-rw-r--r--    1 root     sys         1484 Nov 30 23:28 ssh_config
-rw-------    1 root     sys         1381 Nov 30 23:28 ssh_host_dsa_key
-rw-r--r--    1 root     sys          601 Nov 30 23:28 ssh_host_dsa_key.pub
-rw-------    1 root     sys          505 Nov 30 23:28 ssh_host_ecdsa_key
-rw-r--r--    1 root     sys          173 Nov 30 23:28 ssh_host_ecdsa_key.pub
-rw-------    1 root     sys          399 Nov 30 23:28 ssh_host_ed25519_key
-rw-r--r--    1 root     sys           93 Nov 30 23:28 ssh_host_ed25519_key.pub
-rw-------    1 root     sys         2602 Nov 30 23:28 ssh_host_rsa_key
-rw-r--r--    1 root     sys          565 Nov 30 23:28 ssh_host_rsa_key.pub
-rw-r--r--    1 root     sys         3191 Nov 30 23:28 sshd_config
drwxr-xr-x    5 root     sys          133 Nov 30 11:40 ssl
Comment 14 Darren Tucker 2019-12-02 17:11:53 AEDT
Did you try loading these keys on another system and loading keys generated on other system on this one as suggested in comment #11?  That'll narrow it down to whether the problem is on the generation or reading side.

("It works with the older version on the same system" is not conclusive since in some cases problems with openssl or openssh can cause it to interop with itself but not other implementations).

One other thing to try if you have not already: openssl's self-tests.  In the openssl build directory, run "make tests".
Comment 15 Ole Weidner 2019-12-07 00:11:47 AEDT
I have tested the host keys generated with OpenSSL 1.1.1d on IRIX with OpenSSH-7.9 on MacOS. SSH loads the keys witout any issues.


The OpenSSL 1.1.1d test suite on IRIX reports quite a few issues though:

Test Summary Report
-------------------
../test/recipes/01-test_symbol_presence.t        (Wstat: 512 Tests: 4 Failed: 2)
  Failed tests:  2, 4
  Non-zero exit status: 2
../test/recipes/01-test_test.t                   (Wstat: 256 Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
../test/recipes/30-test_evp.t                    (Wstat: 256 Tests: 10 Failed: 1)
  Failed test:  7
  Non-zero exit status: 1
../test/recipes/70-test_asyncio.t                (Wstat: 256 Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
../test/recipes/70-test_recordlen.t              (Wstat: 256 Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
../test/recipes/70-test_servername.t             (Wstat: 256 Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
../test/recipes/80-test_ct.t                     (Wstat: 256 Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
../test/recipes/80-test_dtls.t                   (Wstat: 256 Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
../test/recipes/80-test_ssl_new.t                (Wstat: 6400 Tests: 29 Failed: 25)
  Failed tests:  1-18, 20-21, 24-28
  Non-zero exit status: 25
../test/recipes/80-test_ssl_old.t                (Wstat: 1280 Tests: 6 Failed: 5)
  Failed tests:  2-6
  Non-zero exit status: 5
../test/recipes/80-test_sslcorrupt.t             (Wstat: 256 Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
../test/recipes/90-test_sslapi.t                 (Wstat: 256 Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
../test/recipes/90-test_sslbuffers.t             (Wstat: 256 Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
../test/recipes/90-test_tls13ccs.t               (Wstat: 256 Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
Files=155, Tests=1451, 1528 wallclock secs (18.69 usr  1.07 sys + 1151.89 cusr 317.98 csys = 1489.63 CPU)
Result: FAIL
Comment 16 Darren Tucker 2019-12-09 17:07:48 AEDT
(In reply to Ole Weidner from comment #15)
>   Failed test:  1
>   Non-zero exit status: 1
> ../test/recipes/30-test_evp.t     (Wstat: 256 Tests: 10 Failed: 1)
>   Failed test:  7

This one makes me very suspicious.  The tests in that file are:

my @files = ( "evpciph.txt", "evpdigest.txt", "evpencod.txt", "evpkdf.txt",
    "evpmac.txt", "evppbe.txt", "evppkey.txt", "evppkey_ecc.txt",
    "evpcase.txt", "evpccmcavs.txt" );

and #7 is "evppkey.txt".  If we look at that file (test/recipes/30-test_evp_data/evppkey.txt) it's "Public key algorithm tests" including parsing private and public keys.

I think your OpenSSL is busted.
Comment 17 Ole Weidner 2019-12-18 08:16:39 AEDT
Busted indeed... I rebuilt openssh against an older version, openssl-1.0.2t, and the host key loading issue disappeared. 

OpenSSH-8.1p1 now works fine on IRIX.

Thanks for all the debugging help! I guess I'll file a bug with the openssl team then... ;-)
Comment 18 Damien Miller 2021-04-23 15:02:04 AEST
closing resolved bugs as of 8.6p1 release