Fix hang when receiving a large file into a proxied special remote

Only indicate that we're done with the bytestring once it all gets written.
Otherwise, the end of it may get garbage collected before we can process
it, leading to a hang.

This seems to have been introduced in commit
cdc4bd7443a4921dd0573a0babedc51c1d62f918. Which oddly was trying to fix a
very similar problem, but specific to a cluster node. In that commit,
things got out of order, with it signaling it was done with the bytestring
before it has written all of it to the file.

My test case for this bug is a directory special remote
with a file being sent to it via a proxy accessed via ssh or http.
The file was 10 mb, and it hung on the last few kb of it not being
received.

I've also tested this fix in the case of proxying to a cluster node
directory special remote over http, which was the case
cdc4bd7443a4921dd0573a0babedc51c1d62f918 was dealing with.

diff --git a/Annex/Proxy.hs b/Annex/Proxy.hs
index ac4a9c4e9bd688de9414b196d3c2bf3a2a92ed03..546c61dd0ca4c908dd6040649dc01f998a3347da 100644 (file)
--- a/Annex/Proxy.hs
+++ b/Annex/Proxy.hs
@@ -249,11 +249,11 @@ proxySpecialRemote protoversion r ihdl ohdl owaitv oclosedv mexportdb = go
 
        receivetofile iv h n = liftIO receivebytestring >>= \case
                Just b -> do
+                       n' <- storetofile iv h n (L.toChunks b)
                        liftIO $ atomically $ 
                                putTMVar owaitv ()
                                        `orElse`
                                readTMVar oclosedv
-                       n' <- storetofile iv h n (L.toChunks b)
                        -- Normally all the data is sent in a single
                        -- lazy bytestring. However, when the special
                        -- remote is a node in a cluster, a PUT is

diff --git a/CHANGELOG b/CHANGELOG
index 07c9df1aeaee2367f3d5e13aea204a15b7a3806b..14dc3bf9cad4d4dc8fc729c6cc495bda50a7691d 100644 (file)
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -23,6 +23,7 @@ git-annex (10.20241031) UNRELEASED; urgency=medium
     and then use the P2P protocol to inform the proxy that the content has
     been stored there, which will result in the same git-annex branch state
     updates as sending DATA via the proxy.
+  * Fix hang when receiving a large file into a proxied special remote.
 
  -- Joey Hess <id@joeyh.name>  Thu, 17 Oct 2024 11:02:17 -0400