Bug 1935 - ls -l with wildcards produces incorrect output
Summary: ls -l with wildcards produces incorrect output
Status: CLOSED FIXED
Alias: None
Product: Portable OpenSSH
Classification: Unclassified
Component: sftp (show other bugs)
Version: 5.8p1
Hardware: All Linux
: P2 normal
Assignee: Assigned to nobody
URL:
Keywords:
Depends on:
Blocks: V_6_0
  Show dependency treegraph
 
Reported: 2011-09-10 23:28 AEST by Joe
Modified: 2015-08-11 23:03 AEST (History)
1 user (show)

See Also:


Attachments
glob.diff (2.15 KB, patch)
2011-09-21 12:50 AEST, Damien Miller
no flags Details | Diff
sftp.diff (605 bytes, patch)
2011-09-21 12:53 AEST, Damien Miller
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Joe 2011-09-10 23:28:28 AEST
This is probably best demonstrated with some sample output.  Issuing the command:

sftp> ls -l localhost.access.log.?.gz

gives the following (abbreviated) output:

29787 Aug 31 02:24 localhost.access.log.2.gz
43664 Sep  1 02:23 localhost.access.log.3.gz
37122 Aug 29 02:23 localhost.access.log.4.gz
31291 Aug 26 02:24 localhost.access.log.5.gz
33117 Aug 27 02:22 localhost.access.log.6.gz
30870 Aug 30 02:24 localhost.access.log.7.gz
26787 Sep  2 02:23 localhost.access.log.8.gz
38128 Aug 28 02:24 localhost.access.log.9.gz

whereas ls -l reports the following (correct) data:

26787 Sep  2 06:23 localhost.access.log.2.gz
43664 Sep  1 06:23 localhost.access.log.3.gz
29787 Aug 31 06:24 localhost.access.log.4.gz
30870 Aug 30 06:24 localhost.access.log.5.gz
37122 Aug 29 06:23 localhost.access.log.6.gz
38128 Aug 28 06:24 localhost.access.log.7.gz
33117 Aug 27 06:22 localhost.access.log.8.gz
31291 Aug 26 06:24 localhost.access.log.9.gz

So it seems that the wildcard is munging the file size and timestamp in
some random manner.  The bad output also happens with an asterisk instead of a question mark.

Client is 5.8p1 on Debian-7 x86.  Server is 5.5p1 on Debian-6 amd64.
Comment 1 Damien Miller 2011-09-11 08:13:02 AEST
Can you replicate this with a more recent server version? It might be a bug in 5.5 that has been fixed since.
Comment 2 Joe 2011-09-11 08:40:04 AEST
Yes, I tried against my localhost, i.e., server also running 5.8p1, and got similar results:

ls -l (on localhost)

43690 Sep  3 11:35 localhost.access.log.0901.gz
26813 Sep  3 11:35 localhost.access.log.0902.gz
24932 Sep 10 08:44 localhost.access.log.0903.gz

sftp> ls -l localhost.access.log.*.gz
26193 Sep 10 08:45 localhost.access.log.0901.gz
43690 Sep  3 11:35 localhost.access.log.0902.gz
35457 Sep 10 08:46 localhost.access.log.0903.gz
Comment 3 Damien Miller 2011-09-11 08:49:05 AEST
One more thing to try: could you please try to recreate on localhost after cutting ssh/sshd out of the picture. Can you replicate using "sftp -D /usr/lib/openssh/sftp-server" to directly connect sftp to sftp-server (you might need a different path - AFAIK this one is correct for Debian/Ubuntu).

Also, what filesystem are these logs hosted on?
Comment 4 Joe 2011-09-11 09:09:41 AEST
Same results using "sftp -D /usr/lib/openssh/sftp-server".  On my local host, the files are on ext3.  BTW, if I'm not mistaken this started happening when I upgraded to 5.8p1 on the client.
Comment 5 Damien Miller 2011-09-12 10:43:10 AEST
okay, I can replicate it now if I copy your filenames. Using single-character wildcards doesn't seem enough, there has to be a preamble.

Also, the screwup persists when sorting is turned off (ls -lf) so it might be in the server or glob code.
Comment 6 Damien Miller 2011-09-21 12:50:10 AEST
Created attachment 2093 [details]
glob.diff

fix glob GLOB_KEEPSTAT vs sorting
Comment 7 Damien Miller 2011-09-21 12:53:39 AEST
Created attachment 2094 [details]
sftp.diff

don't bother sorting remote ls glob output
Comment 8 Damien Miller 2011-09-21 12:57:19 AEST
okay, either of these diffs will fix your bug. I intend to commit both.

The problem is that glob()'s GLOB_KEEPSTAT support (retain the results of stat() operations in an array) didn't take into account that glob() sorts its results by default. The first patch implements a sort that takes the separate array of stat information into account so it doesn't get out of sync with the pathnames.

Furthermore, "ls" shouldn't bother with glob's internal sorting anyway because it implements its own sort which might be different. The second patch disables glob's sort for remote ls operations.

If you are applying the first patch, then please "cd openbsd-compat" first.
Comment 9 Damien Miller 2011-09-22 21:53:52 AEST
These patches have been applied and will be in OpenSSH-6.0. Thanks!
Comment 10 Damien Miller 2015-08-11 23:03:26 AEST
Set all RESOLVED bugs to CLOSED with release of OpenSSH 7.1