drain transferrer read handle when shutting it down

Fixes updating git index file after getting an unlocked file when
annex.stalldetection is set.

The transferrer may want to send additional protocol messages when it's
shut down. Closing the read handle prevented it from doing that, and caused
it to crash rather than cleanly shutting down.

Draining the handle without processing the protocol seemed ok to do,
because anything it outputs is going to be some side message displayed
at shutdown. Displaying those once per transferrer process that is running
seems unncessary.

Sponsored-by: Dartmouth College's DANDI project

diff --git a/Annex/TransferrerPool.hs b/Annex/TransferrerPool.hs
index 513dc1f28e6b49b7181a84b086e453f72158104f..b1046a959006a693eff70cb82068bbb25d63ca8f 100644 (file)
--- a/Annex/TransferrerPool.hs
+++ b/Annex/TransferrerPool.hs
@@ -217,9 +217,15 @@ mkTransferrer signalactonsvar (RunTransferrer program params batchmaker) = do
                , transferrerWrite = writeh
                , transferrerHandle = ph
                , transferrerShutdown = do
-                       hClose readh
+                       -- The transferrer may write to stdout
+                       -- as it's shutting down, so don't close
+                       -- the readh right away. Instead, drain
+                       -- anything sent to it.
+                       drainer <- async $ void $ hGetContents readh
                        hClose writeh
                        void $ waitForProcess ph
+                       wait drainer
+                       hClose readh
                        unregistersignalprop
                }
 

diff --git a/CHANGELOG b/CHANGELOG
index eebd73c927ab90d33e6b0b0a39d7fa65d0337b66..e74887764c501864996d9be072011ba09094bdbf 100644 (file)
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -29,6 +29,8 @@ git-annex (10.20220823) UNRELEASED; urgency=medium
     directory to somewhere else.
     (Introduced in version 10.20220525)
   * Improved handling of --time-limit when combined with -J
+  * Fix updating git index file after getting an unlocked file 
+    when annex.stalldetection is set.
 
  -- Joey Hess <id@joeyh.name>  Mon, 29 Aug 2022 15:03:04 -0400
 

diff --git a/doc/bugs/reports_file___34__modified__34___whenever_it_is_not/comment_13_5c03731700b2a025755fb06c9a96f331._comment b/doc/bugs/reports_file___34__modified__34___whenever_it_is_not/comment_13_5c03731700b2a025755fb06c9a96f331._comment
new file mode 100644 (file)
index 0000000..7574901
--- /dev/null
+++ b/doc/bugs/reports_file___34__modified__34___whenever_it_is_not/comment_13_5c03731700b2a025755fb06c9a96f331._comment
@@ -0,0 +1,30 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 13"""
+ date="2022-09-22T18:16:22Z"
+ content="""
+Avoided the early stdout handle close, and that did fix this bug as
+reported.
+
+The related problems I identified in comment #12 are still unfixed, so
+leaving this open for now.
+
+I think what ought to be done to wrap this up is make restagePointerFile
+record the files that need to be restaged in a log file. Then at shutdown,
+git-annex can read the log file, and restage everything listed in it.
+This will solve multiple problems:
+
+* When a previous git-annex process was interrupted after a get/drop of an
+  unlocked file, the file will be in the log, so git-annex can notice
+  that and handle the restaging.
+* When a stalled `git-annex transferrer` is killed, the parent git-annex
+  will read the log and handle the restaging that it was not able to do.
+* When multiple processes are trying to restage files at the same time,
+  an exclusive lock can be used to make only one of them run, and it can
+  handle restaging the files that the others have recorded in the log too.
+* As a bonus, in the situations where git-annex is legitimately unable to
+  restage files, it can still record them to be restaged later. And the
+  "only a cosmetic problem" message can tell the user to run a single
+  simple git-annex command, rather than a complicated 
+  `git update-index` command per file.
+"""]]