Bug 3504 - file content lost after scp to same file at localhost
Summary: file content lost after scp to same file at localhost
Status: CLOSED FIXED
Alias: None
Product: Portable OpenSSH
Classification: Unclassified
Component: scp (show other bugs)
Version: 8.7p1
Hardware: amd64 Linux
: P5 major
Assignee: Assigned to nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-11-20 10:23 AEDT by Stephane Thiell
Modified: 2023-03-17 13:38 AEDT (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Stephane Thiell 2022-11-20 10:23:12 AEDT
It looks like that when scp is using the SFTP protocol (the default now), the following use case doesn't work anymore.

Reproducer:

  $ echo bar > /tmp/foo
  $ cat /tmp/foo
  bar
  $ scp /tmp/foo localhost:/tmp/foo
  foo                                                                                                                                                                                                                       
  0%    0     0.0KB/s   --:-- ETA

Then, data in /tmp/foo is lost:

  $ cat /tmp/foo
  $ 

It looks like sftp opens the destination file with O_TRUNC which is different than with the legacy SCP, which works as expected:

  $ echo bar > /tmp/foo
  $ scp -O /tmp/foo localhost:/tmp/foo
  foo                                                                                                                                                                                                                     
  100%    4     2.6KB/s   00:00    
  $ cat /tmp/foo
  bar
Comment 1 Dominique Martinet 2022-11-20 11:38:31 AEDT
A couple of notes:

 - that might look like a weird usecase (Don't do this!), but this can happen with clustershell copy mode (clush --copy) where it's very easy to send a file to all configured machines with the same path, and if the source machine is included in that list we will run something like this scp command (that's the reason Stephane found that out -- I just think it's better to explain where we come from)

 - regardless of the localhost case, in case a transfer is interrupted in the middle both the old scp protocol and sftp will leave the target file clobbered. O_TRUNC is objectively better than leaving part of the original content behind here, but a potential better alternative would be to do like rsync and write to a temporary file (rsync uses `.<originalfilename>.<randomchars>`)
That file can be removed safely if the connection is interrupted, and the original file would be left untouched.


(perhaps it'd be better to ask this not to openssh portable but to the bugs@openbsd list?)

Thank you!
Comment 2 Damien Miller 2022-11-21 09:19:57 AEDT
This was fixed in OpenSSH 9.1

commit 56a0697fe079ff3e1ba30a2d5c26b5e45f7b71f8
Author: djm@openbsd.org <djm@openbsd.org>
Date:   Fri May 13 06:31:50 2022 +0000

    upstream: arrange for scp, when in sftp mode, to not ftruncate(3) files
    
    early
    
    previous behavious of unconditionally truncating the destination file
    would cause "scp ~/foo localhost:" and "scp localhost:foo ~/" to
    delete all the contents of their destination.
    
    spotted by solene@ sthen@, also bz3431; ok dtucker@
    
    OpenBSD-Commit-ID: ca39fdd39e0ec1466b9666f15cbcfddea6aaa179
Comment 3 Damien Miller 2023-03-17 13:38:33 AEDT
OpenSSH 9.3 has been released. Close resolved bugs