Bug 3511

Summary: KbdInteractiveAuthentication and Golang goroutines scheduler
Product: Portable OpenSSH Reporter: krasnovu
Component: PAM supportAssignee: Assigned to nobody <unassigned-bugs>
Status: CLOSED INVALID    
Severity: normal CC: djm
Priority: P5    
Version: 9.1p1   
Hardware: amd64   
OS: Linux   

Description krasnovu 2022-12-20 20:41:36 AEDT 
Hi! I don't know how to solve my problem and I don't know where the problem is, in OpenSSH or Golang runtime.

I am writing a multi-factor PAM module in Golang. The module asks questions to the user and waits for answers from him. So to make the dialog work, I switch `KbdInteractiveAuthentication yes`. Everything works, but as soon as the golang goroutine is launched, the module freezes (stuck, hang). Hangs up so that no code in the module is running anymore. The description for `KbdInteractiveAuthentication` says `Change to yes to enable challenge-response passwords (beware issues with some PAM modules and threads)`. Can you describe how this option works, why does it break the golang goroutines scheduler?

If this is a known issue and a problem in golang, then I will open a new issue on github, since mine (https://github.com/golang/go/issues/57394) was closed due to the fact that I could not explain the cause of the problem.

If this is an OpenSSH issue, then I can provide the necessary logs and a minimal project to reproduce the issue.

ps. Unfortunately, even if I write in one thread, the http package in golang itself launches goroutines and the module freezes.

pss. I've tried rebuilding the OpenSSH server with the UNSUPPORTED_POSIX_THREADS_HACK declaration and then everything works as it should, but I can't ask all users to rebuild the OpenSSH server. And the description says there is no support for enabling this.

Thank you for your attention!
Comment 1 Damien Miller 2022-12-21 15:55:41 AEDT 
I don't know if Go is ideal for interfacing with a C ABI running in another process - the golang runtime makes many assumptions about the system that may not be true when loaded into another process' address space. In the case of sshd, some of these assumptions likely relate to threads (OpenSSH isn't, Golang is), fork/exec (which is tricky around threads) and sshd clobbering file descriptors using closefrom().

The way I have seen other people attempt similar things is to use a small C shim that compiles to the loadable module that talks via a socket to the application logic that is written in Go. This keeps most of the code in Go but provides a clean and minimal ABI that isn't likely to make assumptions that cause problems.

The other approach is to write the whole thing in C, C++ or Rust, each of which are more explicit about doing things that could cause problems elsewhere.
Comment 2 krasnovu 2022-12-21 23:47:43 AEDT 
(In reply to Damien Miller from comment #1)

Thanks for the recommendations, Damien!
Comment 3 Damien Miller 2023-03-17 13:39:13 AEDT 
OpenSSH 9.3 has been released. Close resolved bugs