Created attachment 3218 [details] PROTOCOL v1 i haven't been able to find any prior art in this space. if anyone is familiar with a server/client implementing a relevant extension, feel free to highlight it. `sftp` has a -B option to set the transfer buffer size. this applies to both read & write SFTP packets. when doing a read of a really large size, OpenSSH will respond with short reads. e.g. if you request 1MiB, the server will just respond with 64KiB (sftp-server.c:process_read hardcodes buf[64*1024]). this leaves the client with expecting to chunk things up at one size, but ends up having to backfill things dynamically. the client is able to recover though, so that's good. when doing a write of a really large size, OpenSSH will just close the connection as soon as it sees the header with the large write. the server hardcodes 256KiB (SFTP_MAX_MSG_LENGTH) and any attempt to write anything larger than that is immediately rejected. this is not friendly and clients basically have to default to 32KiB all the time and force users to manually select a size that they happen to know the server they're connecting to supports. if the server supports an extension to dynamically query the exact limits the server imposes, the client can start with the 32KiB default, and then automatically increase to something with better throughput.
That seems reasonable, though some of these are only implicitly defined. E.g. there is no internal limit to the max number of file handles, sftp-server will just allocate up to the fd/heap rlimits. So IMO there should be some way to express "unspecified" or "unlimited" (internally at least) limits. Maybe 0 for a limit? Also, the values might as well be 64 bits.
i tried to account for that here: > If the server doesn't enforce a specific limit, then the field may be set to 0. > This implies the server relies on the OS to enforce limits (e.g. available > memory or file handles), and such limits might be dynamic. The client SHOULD > take care to not try to exceed reasonable limits. i think that covers it ? wrt 32bit vs 64bit, i couldn't imagine a situation where a single packet would hit that, but i guess if we're laying groundwork for a protocol that will stick around for decades, might as well plan for then rather than regret it.
Created attachment 3238 [details] PROTOCOL v1 change all the fields from uint32 to uint64
was there anything other feedback ?
sorry for stalling. This looks fine to me
i've posted a patch series then implementing things: https://lists.mindrot.org/pipermail/openssh-unix-dev/2020-November/038977.html hopefully it'll be easy to get through since the proposal is simple :).
anything i can do to help with reviewing the posted patches ?
looks like the PROTOCOL & server support have been merged so far. whoo!
yes, I'll try to do the rest shortly after the 8.5 release (this week)
everything should be merged now for the next release (8.6). thanks Damien!
closing resolved bugs as of 8.6p1 release