Bug 1495

Summary: sftp performance of "ls -l" cmd is slow on network-authed servers
Product: Portable OpenSSH Reporter: mark smith <masmitty>
Component: sftp-serverAssignee: Assigned to nobody <unassigned-bugs>
Status: CLOSED FIXED    
Severity: normal CC: djm, dtucker, michael.rhyner
Priority: P2    
Version: -current   
Hardware: All   
OS: All   
Bug Depends on:    
Bug Blocks: 1626    
Attachments:
Description Flags
sftp-namecache.patch none

Description mark smith 2008-07-30 06:49:57 AEST 
when sftp client connects to sftp-server and does "ls -l" command on a large directory (many files), performance is unacceptably slow when the sftp-server node used ldap/kerberos/NIS/...(etc) to look up EVERY uid/gid on every file.

a simple fix is to cache/buffer the uid/gid of the previous file from the stat()/fstat() & chk to see if they have changed before issuing the lookup of the same uid/gid (ergo inhibiting a wait-for-network to get info that we already have)

same issue true for v4.1p1 through current.
Comment 1 Damien Miller 2009-01-21 21:27:35 AEDT 
Isn't this the role of nscd (or similar)?
Comment 2 Michael Rhyner 2009-07-06 18:52:28 AEST 
nscd helped on this issue, issuing e.g. 'ls -l' is now as fast as expected. Cache times within nscd.conf could be quite small, e.g. between 20 and 60 seconds, it you want to avoid that services rely on outdated information.
Comment 3 Darren Tucker 2010-01-14 16:24:08 AEDT 
Created attachment 1779 [details]
sftp-namecache.patch

Add a small LRU to cache user and group names.
Comment 4 Darren Tucker 2010-01-15 10:46:31 AEDT 
deraadt points out that similar functions (user_from_uid and group_from_gid) exist in OpenBSD's libc ( both keep a 64-entry hash table as a cache)  We have just commited a change with that.

We'll add these functions to the compat library for any platform that doesn't have them.
Comment 5 Darren Tucker 2010-03-26 10:50:53 AEDT 
With the release of 5.4p1, this bug is now considered closed.