Bug 3096

Summary:	sftp requires multiple round trips to do a write with a fsync()
Product:	Portable OpenSSH	Reporter:	Shawn Landden <shawn>
Component:	sftp	Assignee:	Assigned to nobody <unassigned-bugs>
Status:	CLOSED WORKSFORME
Severity:	enhancement	CC:	djm, dtucker
Priority:	P5
Version:	8.1p1
Hardware:	Other
OS:	Linux

Description Shawn Landden 2019-11-20 05:13:21 AEDT

https://libssh2.org/libssh2_sftp_fsync.html

Correct me if I am wrong, but the sftp protocol seems to require multiple round trips to do a write followed by a fsync. Perhaps O_SYNC is a solution here. The issue is that sftp can be over very high latency links.

I edit source code over sftp. Very few editors do their IO on a separate thread (vscode is an exception) and thus can be used for this. But even if they can, the latency is horrible, because of multiple round trips. And I *do* want fsync behavior, and while sync() calls are notorious for being expensive, in this case the round trips are more expensive.

Comment 1 Shawn Landden 2019-11-20 05:14:30 AEDT

This bug is really just for fixing the protocol, but consideration for how applications can use that is of course important. O_SYNC is probably the most straight-forward way, perhaps with opening the same file twice so that it doesn't limit the program's options regarding sync behavior.

Comment 2 Darren Tucker 2019-11-20 12:06:34 AEDT

Why would the protocol require a round-trip?

The fsync extension operates on an open file handle (https://cvsweb.openbsd.org/src/usr.bin/ssh/PROTOCOL section 3.6), there can be multiple requests in flight from client to server (https://tools.ietf.org/html/draft-ietf-secsh-filexfer-02 section 3) and the server must process the operations on a given file handle in order  (filexfer-02 section 6.1), so a client should be able to send open,write,write,write,fsync,close, then process all of the replies.

Comment 3 Damien Miller 2019-11-22 13:08:04 AEDT

Darren is correct - the sftp protocol doesn't require an additional round-trip for a fsync operation. It can be pipelined after (or during) writes. Our sftp-server should deal with this just fine.

Comment 4 Damien Miller 2020-01-25 16:38:06 AEDT

I think the problem here is that libssh2 doesn't offer a way to pipeline the request, not with sftp-server itself.

Comment 5 Damien Miller 2021-04-23 14:58:49 AEST

closing resolved bugs as of 8.6p1 release

Comment 6 Shawn Landden 2021-04-24 10:01:02 AEST

You didn't read your own comment when you closed this bug. You said it was possible, and then contradicted yourself. I know there might not be the interest or resources to do this, but that is not a reason to close he bug.

Comment 7 Damien Miller 2021-04-25 16:00:31 AEST

Pretty sure it's not my reading comprehension that is at fault here. 

As Darren pointed out in comment #2, nothing in the sftp protocol or sftp-server requires a roundtrip in this case.

As I pointed out in comment #4, the problem likely lies in your (non-OpenSSH) client software.