Bug 1173 - scp reports lost connection for very large files
Summary: scp reports lost connection for very large files
Status: CLOSED FIXED
Alias: None
Product: Portable OpenSSH
Classification: Unclassified
Component: scp (show other bugs)
Version: 4.3p2
Hardware: HPPA HP-UX
: P2 normal
Assignee: Assigned to nobody
URL:
Keywords:
: 1174 (view as bug list)
Depends on:
Blocks: V_4_4
  Show dependency treegraph
 
Reported: 2006-03-17 17:12 AEDT by Ermin Borovac
Modified: 2006-09-28 19:26 AEST (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ermin Borovac 2006-03-17 17:12:23 AEDT
As the summary says scp (4.3p2) reports 'lost connection' for very large files (e.g. 10GB) and terminates transfer around 1.8G mark.

We have verified that file transfer works fine with 4.2p1.

I'll attempt to describe where I think the problem is.

The size of the file being transferred is around 10G (10435459928), however with 'scp -v' it is reported (see line below) as merely 1.8G or so (1845525336).

Sending file modes: C0644 1845525336 onedrain_opsncc_pg_dump.out

Further investigation points to scp.c line 607

                snprintf(buf, sizeof buf, "C%04o %lld %s\n",
                    (u_int) (stb.st_mode & FILEMODEMASK),
                    (long long)stb.st_size, last);
                if (verbose_mode) {
                        fprintf(stderr, "Sending file modes: %s", buf);
                }

Value in stb.st_size is correct (10435459928), however after snprintf() variable buf holds incorrect value (1845525336).

Now, snprintf() that gets used is the one in openbsd-compat/bsd-snprintf.c not the native HP-UX one. Output from 'configure' shows the following

checking whether vsnprintf returns correct values on overflow... no
configure: WARNING: ****** Your vsnprintf() function is broken, complain to your vendor

Consequently BROKEN_SNPRINTF gets defined in config.h and snprintf() definition comes from openbsd-compat/bsd-snprintf.c.

Here are relevant lines from bsd-snprintf.c line 470

static void fmtint(char *buffer, size_t *currlen, size_t maxlen,
                    long value, int base, int min, int max, int flags)
{
        int signvalue = 0;
        unsigned long uvalue;

It seems that type for 'value' should be 'LLONG' instead of 'long' and similarly type for 'uvalue' should be 'unsigned LLONG' instead of 'unsigned long'.

In dopr() 'value' is defined as 'LLONG' (long long) but when it's passed to fmtint() it's cast from 'LLONG' (long long) to 'long'. This seems to be root cause for snprintf() anomaly.

After making these changes to fmtint, scp reports correct size for the file and transfers it completely.

--- bsd-snprintf.c.orig 2005-12-17 22:32:04.000000000 +1100
+++ bsd-snprintf.c      2006-03-17 17:08:29.000000000 +1100
@@ -161,7 +161,7 @@
 static void fmtstr(char *buffer, size_t *currlen, size_t maxlen,
                    char *value, int flags, int min, int max);
 static void fmtint(char *buffer, size_t *currlen, size_t maxlen,
-                   long value, int base, int min, int max, int flags);
+                   LLONG value, int base, int min, int max, int flags);
 static void fmtfp(char *buffer, size_t *currlen, size_t maxlen,
                   LDOUBLE fvalue, int min, int max, int flags);
 static void dopr_outch(char *buffer, size_t *currlen, size_t maxlen, char c);
@@ -468,10 +468,10 @@
 /* Have to handle DP_F_NUM (ie 0x and 0 alternates) */

 static void fmtint(char *buffer, size_t *currlen, size_t maxlen,
-                   long value, int base, int min, int max, int flags)
+                   LLONG value, int base, int min, int max, int flags)
 {
        int signvalue = 0;
-       unsigned long uvalue;
+       unsigned LLONG uvalue;
        char convert[20];
        int place = 0;
        int spadlen = 0; /* amount to space pad */
Comment 1 Darren Tucker 2006-03-19 00:10:38 AEDT
That seems right.  Patch applied, it will be in the next release.  Thanks for the report.
Comment 2 Darren Tucker 2006-03-21 20:00:16 AEDT
*** Bug 1174 has been marked as a duplicate of this bug. ***
Comment 3 Olaf Flebbe 2006-03-21 20:16:22 AEDT
(In reply to comment #2)
> *** Bug 1174 has been marked as a duplicate of this bug. ***

That's funny ... 

Comment 4 Darren Tucker 2006-09-28 19:26:03 AEST
With the release of 4.4, we believe that this bug is now closed.  For information about the release please see http://www.openssh.com/txt/release-4.4 .