OpenSSH isn't setting the AIX capabilities correctly, so I had a look into fixing this. It looks to me like port-aix.c could be simplified by removing all setrlimit() calls and instead use the AIX functions setpcred()/setpenv() to set up the user environment. They are documented in http://tre.ii.uib.no/doc_link/en_US/a_doc_lib/libs/basetrf2/setpcred.htm http://tre.ii.uib.no/doc_link/en_US/a_doc_lib/libs/basetrf2/setpenv.htm Please consider applying the following patches so that we can use OpenSSH to run jobs on large page enabled AIX systems. The patches have been tested on AIX 5.1D, but according to my IBM contacts also AIX 4 uses these functions for setting up the login environment. http://marc.theaimsgroup.com/?l=openssh-unix-dev&m=102266568505721&q=p4 http://marc.theaimsgroup.com/?l=openssh-unix-dev&m=102266568505721&q=p3 -jf
I finally got a chance to try this. I got compile errors with gcc on AIX 4.2.1 and 4.3.3. gcc -g -O2 -Wall -Wpointer-arith -Wno-uninitialized -I. -I.. -I/usr/local/include -DHAVE_CONFIG_H -c port-aix.c port-aix.c: In function `set_limits_from_userattr': port-aix.c:35: too few arguments to function `setpcred' port-aix.c:36: too few arguments to function `setpenv' The following patch works for me.
Created attachment 107 [details] Add extra params to setpcred and setpenv
Created attachment 109 [details] Merge three previous patches.
Created attachment 110 [details] removed setpenv call
Two things... We should probably fail if it fails: if (setpcred (user, NULL)) fatal("Failed to set AIX process credentials."); and I'm a bit uncertain about the call to setpenv(). I think it does a bit too much, and with the PENV_INIT it breaks sftp and it doesn't manage to set the TERM correctly . The effect of calling setpenv() is that various user environment variables are set up _and_ the user is given his shell. I think we should just remove the call to setpenv(). Fixed patch attachment above this message.. -jf
I'd like to make a single commit to resolve this. Can someone at IBM or someone verify this is the correct way to resolve the code? - Ben
I investigated the use of setpcred(). This call is what should be used in port-aix.c. The setpenv() should not be used.
Created attachment 114 [details] Would the following be acceptable version of this patch?
Worked on my test box (AIX 4.3.3). Patch had HTML LT/GT mangling and missing line continuation in configure.ac which required fixing by hand. + setrlimit setsid setpcred setvbuf sigaction sigvec snprintf \ + socketpair strerror strlcat strlcpy strmode strsep sysconf tcgetpgrp \ + truncate utimes vhangup vsnprintf waitpid __b64_ntop _getpty)
Commited fix.
Mass change of RESOLVED bugs to CLOSED