fix replaceFile makeAnnexLink race

replaceFile created a temp file, which was guaranteed to not overlap with
another temp file. However, makeAnnexLink then deleted that file, in
preparation for making the symlink in its place. This caused a race, since
some other replaceFile could create a temp file, using the same name!

I was able to reproduce the race easily running git-annex add -J10 in a
directory with 100 files (all with different contents). Some files would
get ingested into the annex, but their annex links would fail to be added.

There could be other situations where this same problem could occur.
Perhaps when the assistant is adding a file, if the user manually also ran
git-annex add. Perhaps in cases not involving adding a file.

The new replaceFile makes a temprary directory, which is guaranteed to be
unique, and doesn't make a temp file in there. makeAnnexLink can thus
create the symlink without problem and the race is avoided.

Audited all calls to replaceFile to make sure that the old behavior of
providing an empty temp file was not relied on.

The general problem of asking for a temp file and deleting it as part of
the process of using it could reach beyond replaceFile. Did some quick
audits and didn't find other cases of it. Probably only symlink creation
stuff would tend to make that mistake, mostly.

diff --git a/Annex/ReplaceFile.hs b/Annex/ReplaceFile.hs
index 1144ba08397da9fb4e889bf89531b368b670373e..b090da9057372b93c1ba23fa7e74fa755871092c 100644 (file)
--- a/Annex/ReplaceFile.hs
+++ b/Annex/ReplaceFile.hs
@@ -1,6 +1,6 @@
 {- git-annex file replacing
  -
- - Copyright 2013 Joey Hess <id@joeyh.name>
+ - Copyright 2013-2015 Joey Hess <id@joeyh.name>
  -
  - Licensed under the GNU GPL version 3 or higher.
  -}
@@ -9,35 +9,27 @@ module Annex.ReplaceFile where
 
 import Common.Annex
 import Annex.Perms
+import Utility.Tmp
 
 {- Replaces a possibly already existing file with a new version, 
  - atomically, by running an action.
  - 
- - The action is passed a temp file, which it can write to, and once
- - done the temp file is moved into place.
+ - The action is passed the name of temp file, in a temp directory, 
+ - which it can write to, and once done the temp file is moved into place
+ - and anything else in the temp directory is deleted.
  -
- - The action can throw an IO exception, in which case the temp file
+ - The action can throw an IO exception, in which case the temp directory
  - will be deleted, and the existing file will be preserved.
  -
  - Throws an IO exception when it was unable to replace the file.
  -}
 replaceFile :: FilePath -> (FilePath -> Annex ()) -> Annex ()
-replaceFile file action = replaceFileOr file action (liftIO . nukeFile)
-
-{- If unable to replace the file with the temp file, runs the
- - rollback action, which is responsible for cleaning up the temp file. -}
-replaceFileOr :: FilePath -> (FilePath -> Annex ()) -> (FilePath -> Annex ()) -> Annex ()
-replaceFileOr file action rollback = do
-       tmpdir <- fromRepo gitAnnexTmpMiscDir
-       void $ createAnnexDirectory tmpdir
-       tmpfile <- liftIO $ setup tmpdir
-       go tmpfile `catchNonAsync` (const $ rollback tmpfile)
-  where
-       setup tmpdir = do
-               (tmpfile, h) <- openTempFileWithDefaultPermissions tmpdir "tmp"
-               hClose h
-               return tmpfile
-       go tmpfile = do
+replaceFile file action = do
+       misctmpdir <- fromRepo gitAnnexTmpMiscDir
+       void $ createAnnexDirectory misctmpdir
+       let basetmp = takeFileName file
+       withTmpDirIn misctmpdir basetmp $ \tmpdir -> do
+               let tmpfile = tmpdir <> basetmp
                action tmpfile
                liftIO $ replaceFileFrom tmpfile file
 

diff --git a/debian/changelog b/debian/changelog
index 2a05e8b0b8c911231b08410c7c7193c9a19d6b21..4f10ff57d968537a321ccabb5c718f3fc2af209c 100644 (file)
--- a/debian/changelog
+++ b/debian/changelog
@@ -8,6 +8,8 @@ git-annex (5.20151102.2) UNRELEASED; urgency=medium
     are imported to is not a directort, but perhaps an annexed file.
   * Concurrent progress bars are now displayed when using -J with a command
     that moves file contents around.
+  * Fix race that could result in an annexed file's symlink not being
+    created, when eg, running concurrent git-annex adds.
 
  -- Joey Hess <id@joeyh.name>  Wed, 04 Nov 2015 12:50:20 -0400