improve p2p protocol handling of requested object not available

Avoid spurious "verification of content failed" message when downloading
content from a ssh or tor remote fails due to the remote no longer having a
copy of the content.

The P2P protocol already handled this case by sending DATA 0, followed by
VALID. But VALID was not really right, because the data is not the
requested data. So, send DATA 0, followed by INVALID. Old versions of
git-annex handle INVALID the same as VALID in this case. Now new versions
avoid displaying an incorrect message.

It would be better for the P2P protocol to have a different way to indicate
this, like perhaps sending INVALID without DATA. But that would be a
breaking change and need a new protocol verison. Since INVALID already is
part of the protocol and already needs to be handled, using it for this
special case too seems ok, and avoids the complication of another protocol
version.

This commit was sponsored by Jochen Bartl on Patreon.

diff --git a/CHANGELOG b/CHANGELOG
index 18890f78f039ed481e3c7972aa20e9510033c3ab..85a60e66d3eb6c776e75f5ef5d39f3c40d069f3e 100644 (file)
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -4,6 +4,9 @@ git-annex (8.20201128) UNRELEASED; urgency=medium
     extension. (Reversion introduced in version 8.20201007.)
   * Fix bug that made the next download after an empty file from a ssh
     or tor remote fail.
+  * Avoid spurious "verification of content failed" message when downloading
+    content from a ssh or tor remote fails due to the remote no longer
+    having a copy of the content.
 
  -- Joey Hess <id@joeyh.name>  Mon, 30 Nov 2020 12:55:49 -0400
 

diff --git a/P2P/Annex.hs b/P2P/Annex.hs
index 41f5f3b5dd72606eeb889479c844ebc78d37d5da..d107f6ef3b2e194186e70d9bbdec1dbfc3fdd979 100644 (file)
--- a/P2P/Annex.hs
+++ b/P2P/Annex.hs
@@ -172,6 +172,14 @@ runLocal runst runner a = case a of
                                runner validitycheck >>= \case
                                        Right (Just Valid) ->
                                                return (rightsize, UnVerified)
+                                       Right (Just Invalid) | l == 0 ->
+                                               -- Special case, for when
+                                               -- content was not
+                                               -- available to send, 
+                                               -- which is indicated by
+                                               -- sending 0 bytes and 
+                                               -- Invalid.
+                                               return (False, UnVerified)
                                        _ -> do
                                                -- Invalid, or old protocol
                                                -- version. Validity is not

diff --git a/P2P/Protocol.hs b/P2P/Protocol.hs
index e9895d3de4e6a1bab59e5ce8f53e59a939aacca6..bc340e1c769004ee09ad415a62081306a9a75837 100644 (file)
--- a/P2P/Protocol.hs
+++ b/P2P/Protocol.hs
@@ -508,13 +508,15 @@ serveAuthed servermode myuuid = void $ serverLoop handler
 sendContent :: Key -> AssociatedFile -> Offset -> MeterUpdate -> Proto Bool
 sendContent key af offset@(Offset n) p = go =<< local (contentSize key)
   where
-       go Nothing = sender (Len 0) L.empty (return Valid)
        go (Just (Len totallen)) = do
                let len = totallen - n
                if len <= 0
                        then sender (Len 0) L.empty (return Valid)
                        else local $ readContent key af offset $
                                sender (Len len)
+       -- Content not available to send. Indicate this by sending
+       -- empty data and indlicate it's invalid.
+       go Nothing = sender (Len 0) L.empty (return Invalid)
        sender len content validitycheck = do
                let p' = offsetMeterUpdate p (toBytesProcessed n)
                net $ sendMessage (DATA len)

diff --git a/doc/bugs/p2p_protocol_misbehavior_when_location_log_out_of_date.mdwn b/doc/bugs/p2p_protocol_misbehavior_when_location_log_out_of_date.mdwn
index 9e967a20007b3dfeb95ea52f989c0bffc2ade11b..5e5aa747212bc0d311ed14cdd527c7eff082b3aa 100644 (file)
--- a/doc/bugs/p2p_protocol_misbehavior_when_location_log_out_of_date.mdwn
+++ b/doc/bugs/p2p_protocol_misbehavior_when_location_log_out_of_date.mdwn
@@ -35,9 +35,16 @@ closes the connection, the next move fails when it should not need to.
 > to avoid needing to add to the protocol. That should avoid
 > the spurious "verification of content failed".
 > 
+> > Done and it did.
+> 
 > But what causes the connection to get closed? It seems that
 > while the server sends VALID, the client never debugs that it received
 > it. Indeeed, the receiveMessage call that should receive it
 > fails because the handle is closed at that point. Seems that
 > this is caused by trying to receive 0 bytes as indicated by DATA
-> ending up closing the handle.
+> ending up closing the handle. Another case of it involved getting
+> an empty file followed by a second file.
+> 
+> > This bug is fixed.
+
+[[done]] --[[Joey]]