Bug 1141

Summary:	Is there a sound reason for MAX_MSG_LENGTH being 256KB?
Product:	Portable OpenSSH	Reporter:	Malcolm Baldridge <openssh-bugzill>
Component:	sftp	Assignee:	Assigned to nobody <unassigned-bugs>
Status:	CLOSED WORKSFORME
Severity:	enhancement
Priority:	P4
Version:	4.2p1
Hardware:	ix86
OS:	All

Description Malcolm Baldridge 2006-01-02 11:41:38 AEDT

This entry is entirely motivated by a curious entry in http://winscp.net/eng/docs/history specifically, the following entry:

----->
Workaround for OpenSSH limit of 256 kB for size of SFTP packet. Consequence was occasional interruption of transfer with error âConnection has been unexpectedly closed. Server sent command exit status 11.â
<-----

I see the code in sftp-[client|server].c (the server's version doesn't reference MAX_MSG_LENGTH, but rather uses a hard-coded 256 * 1024 -- perhaps this should be patched in any event), but there was no explanatory text on the part of the author.  I imagine it's probably set conservatively to avoid too much memory pressure on the server-side.

I'm quite puzzled though that the maximum message isn't negotiated somehow, though, as it seems unusually harsh to cause transfer faults if the message size is too large.  Is WinSCP simply badly programmed, i.e., clients are meant to double the message sizes until a supervisory error occurs and return to the last successful message size?  

Would there be any significant risk impact to raising the maximum message size, other than increased memory usage of course.  If I were to raise the maximum message size in both the server/client, would unmodified OpenSSH clients have connection problems when performing file transfers with these modified servers?

Thanks,
MB

Comment 1 Damien Miller 2006-01-02 12:25:51 AEDT

The protocol doesn't have any mechanism to support negotiation of packet sizes and only requires that clients and servers accept a minimum of 32k. It doesn't gave any means to report "this message was discarded because it was too big" either.

256kB is a pretty large packet size for sftp and if the peer is properly designed (i.e. maintains a window of pending requests, which WinSCP does) then increasing the packet size doesn't improve performance. For example, increasing the packet size from 256kB to 512kB would reduce overhead by less than .006%. 

On the other hand, as sftp is a packetised protocol, increasing the packet size has a negative effect on application responsiveness - e.g. in OpenSSH ^C is only processed on packet boundaries.

In answer to your last question, if you modify the server to support a larger maximum size then you will still be able to connect with any client (though I don't see what the point is) because it is the client that determines the length of requests and therefore the size of the server's replies. If you modify the client to use larger messages by default, then you will break when connecting to servers that don't support the larger messages.

I have made SFTP_MAX_MSG_LENGTH common to both sftp-server and sftp, but given the above, its actual value is not going to change.

Comment 2 Malcolm Baldridge 2006-01-02 13:08:25 AEDT

This is all fine, then.  I just thought to ask about it.  I appreciate the quick and courteous reply detailing the issues behind the choice of the message size limits.

Thanks,
MB

Comment 3 Darren Tucker 2006-10-07 11:43:57 AEST

Change all RESOLVED bug to CLOSED with the exception of the ones fixed post-4.4.