Hi, I'm experiencing some bandwidth issue when using SCP on SunOS and using SUN hardware platform n240. These n240 server are using SUN IPMP feature. The SCP bandwidth issue is seen when files are transfered between two n240 servers using the active IP Interface (bge0). When the inactive IP Interface (bge1) is used to execute the transfer, no bandwidth issue is seen. Bandwidth issue is only seen when SCP "get" is executed. "Pushing" a file using SCP do not causing any bandwidth issue in any way. Here is a summay of the issue: - SCP "push" - No Bandwidth issue - Transfer rate = 10MB/s - SCP "get" - Inactive IP Interface - No Bandwidth issue => 10 MB/s - SCP "get" - Active IP Interface - Bandwidth issue => ~1.0MB/s - 500 kB/s Testing was accomplish using SFTP and FTP. No Bandwidth issue seen. Testing was also accomplish using SCP -1....No Bandwidth issue seen. Versions of OpenSSH used for testing: 3.7.1p2 and 4.0p1 Problem seen with both OpenSSH version. Below are more detail regarding the transfer issue. Any ideas, comments or solutions would be appreciated. Thanks, Chris ====================================================================== N240/SunOS Version where issue was reproduced. ====================================================================== SunOS version: SunOS 5.8 Generic 108528-29 Jan 2004 OR SunOS TLWIPC01 5.8 Generic_117350-13 sun4u sparc SUNW,Netra-240 ssh -V OpenSSH_3.7.1p2-pwexp24, SSH protocols 1.5/2.0, OpenSSL 0.9.7c 30 Sep 2003 OR OpenSSH_4.0p1, OpenSSL 0.9.7f 22 Mar 2005 ====================================================================== Test Executed ====================================================================== We created a 50MB file on a webserver. n240-A> mkfile 50m /tmp/testfile n240-A> chmod u+w /tmp/testfile We send it to n240-B like this n240-A> scp /tmp/testfile sysadmin@n240-B:/tmp We retrieve the file like this n240-A> scp sysadmin@n240-B:/tmp/testfile /tmp ====================================================================== File Transfer results. ====================================================================== [SERVER1]/tmp:=> ifconfig -a lo0: flags=1000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 bge0: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 2 inet 47.135.87.18 netmask ffffffe0 broadcast 47.135.87.31 groupname imspub bge0:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2 inet 47.135.87.20 netmask ffffffe0 broadcast 47.135.87.31 bge1: flags=69040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,STANDBY ,INACTIVE> mtu 1500 index 3 inet 47.135.87.19 netmask ffffffe0 broadcast 47.135.87.31 groupname imspub Active IP Interface (bge0): [SERVER0]/tmp:=> scp nortel@47.135.87.18:/tmp/testfile /tmp/testfile The authenticity of host '47.135.87.18 (47.135.87.18)' can't be established. RSA key fingerprint is 39:2d:49:3c:96:16:7f:6c:00:c1:e5:2b:6f:7b:14:ea. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '47.135.87.18' (RSA) to the list of known hosts. nortel@47.135.87.18's password: testfile 100% 50MB 774.2KB/s 01:01 [SERVER0]/tmp:=> scp /tmp/testfile nortel@47.135.87.18:/tmp/testfile nortel@47.135.87.18's password: testfile 100% 50MB 10.2MB/s 00:04 [SERVER0]/tmp:=> ftp 47.135.87.18 ftp> get testfile 52428800 bytes received in 4.5 seconds (11379.72 Kbytes/s) ftp> put testfile 52428800 bytes sent in 4.5 seconds (11295.58 Kbytes/s) Inactive IP Interface (bge1): [SERVER0]/tmp:=> scp nortel@47.135.87.19:/tmp/testfile /tmp/testfile The authenticity of host '47.135.87.19 (47.135.87.19)' can't be established. RSA key fingerprint is 39:2d:49:3c:96:16:7f:6c:00:c1:e5:2b:6f:7b:14:ea. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '47.135.87.19' (RSA) to the list of known hosts. nortel@47.135.87.19's password: testfile 100% 50MB 10.1MB/s 00:04 [SERVER0]/tmp:=> scp /tmp/testfile nortel@47.135.87.19:/tmp/testfile nortel@47.135.87.19's password: testfile 100% 50MB 10.1MB/s 00:05 [SERVER0]/tmp:=> ftp 47.135.87.19 ftp> get testfile 52428800 bytes received in 4.5 seconds (11296.81 Kbytes/s) ftp> put testfile 52428800 bytes sent in 4.6 seconds (11251.99 Kbytes/s)
Created attachment 874 [details] disable nagle algorithm Wild guess: does this patch make any difference?
I'm trying to test your solution but I'm having all sort of issues. The Solaris 8 we currently use doesn't contain any development tools/packages what so ever. I'm trying to compile the OpenSSH code (with your patch) but it's not working too good. I had to install gcc and a lot of development/tools packages and also modify a few of my OS settings in order for the ./configure to work. I have that working properly now but the "make" and "make install" don't seem to be executing properly. I'm working on it. In order to install OpenSSH and OpenSSL recent version, I downloaded pre- builded Solaris8 packages. Would it be possible to build one with the code changes suggested? Thanks, Chris
Created attachment 884 [details] Modified code and scp results Finally was able to test out the code modification suggested. No improvement regarding the scp bandwith issue. Code modified (misc.c) does not seem to get hit. When executing "scp -v", no debug output was not displayed.
Created attachment 1013 [details] Increase SSH2 channel window, based on HPN-SSH patch I think I've figured out what's going on here. My guess is that when you failover interface is in active mode, there's some kind of delay passing received packets back up the stack. (Maybe this would show up in pings of the active interface?) Currently, OpenSSH doesn not handle SSHv2 connections with a significant bandwidth-delay product very efficiently, since they are throttled by the size of the SSH2 channel windows. This would explain why it doesn't happen with sshv1 (there's no channels, so no channel buffer limits) and doesn't happen when you 'push" data (the acks would be delayed, but would have minimal effect on the throughput). This would also mean that other things are be affected too, however your TCP stack is covering up the problem pretty well. If my back-of-the-envelope calculations are correct, the delay is in the order of 50ms. If you are still experiencing the problem, please try this patch. It's a stripped-down variant of PSC's high-performance networking patch (http://www.psc.edu/networking/projects/hpn-ssh/) that should only affect the area causing you the problems.
Note that for full effect this mod should be running at both ends of the connection.
This problem isn't Solaris-specific - it happens on Linux platforms, too. I'd like to know if there are any plans to include the PSC HPN patches (http://www.psc.edu/networking/projects/hpn-ssh/) into openssh.
Hi. OpenSSH 4.7 increased the default channel window sizes to 2MB which, If my guess about the cause is correct, should resolve this issue. Could you please test with 4.7 and let use know whether or not the problem still occurs? Thanks.
Since this is believed fixed and there has been no response in the bug, I am closing this bug. Please reopen if this is not the case.
Mass update RESOLVED->CLOSED after release of openssh-5.1