fix hang at end of PUT to proxied p2p http remote

sendExactly will now be sure to evaluate the whole lazy ByteString.

In this case, the lazy ByteString was exactly the right lenth.
But, it seems that L.take caused it to not actually be fully evaluated.

In servePut, this manifested as gather never being fully evaluated,
which caused the hang.

Very, very subtle, and horrible bug. Clearly the use of lazy ByteString
(or really just laziness) is at fault, and it would be very worth moving
to conduit or whatever to avoid this.

diff --git a/P2P/IO.hs b/P2P/IO.hs
index 9158c7b6d17d7f4ab54d26700767cadf913ba489..025c52da9f222a29e9e6c2615bcbb4a8f6aa0b59 100644 (file)
--- a/P2P/IO.hs
+++ b/P2P/IO.hs
@@ -335,12 +335,16 @@ debugMessage conn prefix m = do
 -- Must avoid sending too many bytes as it would confuse the other end.
 -- This is easily dealt with by truncating it.
 --
+-- However, the whole ByteString will be evaluated here, even if
+-- the end of it does not get sent.
+--
 -- If too few bytes are sent, the only option is to give up on this
 -- connection. False is returned to indicate this problem.
 sendExactly :: Len -> L.ByteString -> Handle -> MeterUpdate -> IO Bool
 sendExactly (Len n) b h p = do
-       sent <- meteredWrite' p (B.hPut h) (L.take (fromIntegral n) b)
-       return (fromBytesProcessed sent == n)
+       let (x, y) = L.splitAt (fromIntegral n) b
+       sent <- meteredWrite' p (B.hPut h) x
+       L.length y `seq` return (fromBytesProcessed sent == n)
 
 receiveExactly :: Len -> Handle -> MeterUpdate -> IO L.ByteString
 receiveExactly (Len n) h p = hGetMetered h (Just n) p

diff --git a/doc/design/p2p_protocol.mdwn b/doc/design/p2p_protocol.mdwn
index 0102bb495ae23616e8ac39a17bb473ebef781540..5e1629957ed67631f28d333b871fe3536b84aad8 100644 (file)
--- a/doc/design/p2p_protocol.mdwn
+++ b/doc/design/p2p_protocol.mdwn
@@ -115,7 +115,7 @@ the client sends:
 
 The server responds with either SUCCESS or FAILURE.
 The former indicates the content is locked. It will remain
-locked until the client sends:
+locked until the client sends its next message, which must be:
 
        UNLOCKCONTENT Key
 

diff --git a/doc/todo/git-annex_proxies.mdwn b/doc/todo/git-annex_proxies.mdwn
index f03624d7a31dd950f7aedfc3211da03de822f2de..7e2eda6be04ffe8db1b5c96dd71b03ad9a7d16f4 100644 (file)
--- a/doc/todo/git-annex_proxies.mdwn
+++ b/doc/todo/git-annex_proxies.mdwn
@@ -28,10 +28,19 @@ Planned schedule of work:
 
 ## work notes
 
-* http server proxying hangs on git-annex copy --to it
+* This against a http proxied remote leads to a protocol error:
 
-  All the data gets sent first. Suggests the hang is at connection teardown
-  time.
+    git-annex move foo --to origin-c
+    git-annex get foo --from origin-c
+
+    ERROR expected UNLOCKCONTENT
+
+  May need to run the commands a few times before it happens.
+
+  I think it's because proxyRequest treats LOCKCONTENT as a single
+  command+reponse, with UNLOCKCONTENT separately. So it's possible for
+  there to be two different connections to the proxied remote,
+  with LOCKCONTENT being sent to one, and UNLOCKCONTENT to the other one.
 
 * test http server proxying with special remotes