if the platform does not support getgrouplist directly then sshd uses getgrent to enumerate group membership (for group allow / deny). The AIX version of this API only supports /etc/group and NIS and so the group features do not work with other naming providers (nss_ldap say). The AIX port should use getuserattr(user, S_GROUPS, ..), this uses NSS. We can provide a candidate fix for this if needed.
(In reply to comment #0) > The AIX port should use getuserattr(user, S_GROUPS, ..), this uses NSS. > > We can provide a candidate fix for this if needed. Sure, go ahead. I've not looked into AIX's NIS support (I don't use it) but it sounds like the right fix would be to implement a getgrouplist() in port-aix.c
Created attachment 1144 [details] Implement getgrouplist() on AIX via getgrset(). Attached is an implementation of getgrouplist() while looking at another problem but it may also help here.
Created attachment 1430 [details] Implement getgroupslist via getgrset (diff against 4.7p1). It appears that my previous patch did not get the #ifdefs right. Updated patch attached.
If this works correctly then I would like to target 4.8. I encourage anyone who is interested in this change, especially if you yous LDAP or NSS on your AIX boxes, to please try the patch and report success or failure. (Preferably in this bug, but feel free to email me privately if you're allergic to bugzilla.) Thanks.
Success! We tested this patch on AIX 5.3 using "match" configuration entries too, everything is working fine, AllowGroups, DenyGroups and others now work with LDAP.
This patch has been applied and will be in the 4.8 release. Thanks.
Fix shipped in 4.9/4.9p1 release.