Bug 3096 - sftp requires multiple round trips to do a write with a fsync()
Summary: sftp requires multiple round trips to do a write with a fsync()
Status: CLOSED WORKSFORME
Alias: None
Product: Portable OpenSSH
Classification: Unclassified
Component: sftp (show other bugs)
Version: 8.1p1
Hardware: Other Linux
: P5 enhancement
Assignee: Assigned to nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-11-20 05:13 AEDT by Shawn Landden
Modified: 2021-04-25 16:00 AEST (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Shawn Landden 2019-11-20 05:13:21 AEDT
https://libssh2.org/libssh2_sftp_fsync.html

Correct me if I am wrong, but the sftp protocol seems to require multiple round trips to do a write followed by a fsync. Perhaps O_SYNC is a solution here. The issue is that sftp can be over very high latency links.

I edit source code over sftp. Very few editors do their IO on a separate thread (vscode is an exception) and thus can be used for this. But even if they can, the latency is horrible, because of multiple round trips. And I *do* want fsync behavior, and while sync() calls are notorious for being expensive, in this case the round trips are more expensive.
Comment 1 Shawn Landden 2019-11-20 05:14:30 AEDT
This bug is really just for fixing the protocol, but consideration for how applications can use that is of course important. O_SYNC is probably the most straight-forward way, perhaps with opening the same file twice so that it doesn't limit the program's options regarding sync behavior.
Comment 2 Darren Tucker 2019-11-20 12:06:34 AEDT
Why would the protocol require a round-trip?

The fsync extension operates on an open file handle (https://cvsweb.openbsd.org/src/usr.bin/ssh/PROTOCOL section 3.6), there can be multiple requests in flight from client to server (https://tools.ietf.org/html/draft-ietf-secsh-filexfer-02 section 3) and the server must process the operations on a given file handle in order  (filexfer-02 section 6.1), so a client should be able to send open,write,write,write,fsync,close, then process all of the replies.
Comment 3 Damien Miller 2019-11-22 13:08:04 AEDT
Darren is correct - the sftp protocol doesn't require an additional round-trip for a fsync operation. It can be pipelined after (or during) writes. Our sftp-server should deal with this just fine.
Comment 4 Damien Miller 2020-01-25 16:38:06 AEDT
I think the problem here is that libssh2 doesn't offer a way to pipeline the request, not with sftp-server itself.
Comment 5 Damien Miller 2021-04-23 14:58:49 AEST
closing resolved bugs as of 8.6p1 release
Comment 6 Shawn Landden 2021-04-24 10:01:02 AEST
You didn't read your own comment when you closed this bug. You said it was possible, and then contradicted yourself. I know there might not be the interest or resources to do this, but that is not a reason to close he bug.
Comment 7 Damien Miller 2021-04-25 16:00:31 AEST
Pretty sure it's not my reading comprehension that is at fault here. 

As Darren pointed out in comment #2, nothing in the sftp protocol or sftp-server requires a roundtrip in this case.

As I pointed out in comment #4, the problem likely lies in your (non-OpenSSH) client software.