I have been doing some work with OpenSSH 5.8p1, attempting to use an alternative engine for crypto. In mac.c, the function mac_init() calls HMAC_Init() without previously having called HMAC_CTX_init(). However, OpenSSL documentation states that HMAC_CTX_init() is mandatory. See http://www.openssl.org/docs/crypto/hmac.html HMAC_CTX_init() initialises a HMAC_CTX before first use. It must be called. HMAC_CTX_init() must have been called before the first use of an HMAC_CTX in this function. N.B. HMAC_Init() had this undocumented behaviour in previous versions of OpenSSL - failure to switch to HMAC_Init_ex() in programs that expect it will cause them to stop working. While this appears to currently cause no issues with OpenSSH 5.8p1 using OpenSSL 1.0.0d and the default crypto engine, it may cause problems (such as segfaults in my case) when attempting to use an alternative engine.
Created attachment 2110 [details] macinit.diff insert HMAC_CTX_init
patch applied - will be in openssh-6.0. Thanks!
Set all RESOLVED bugs to CLOSED with release of OpenSSH 7.1