sped up git-annex smudge --clean by 25%

Disabling git-annex branch update for this command is
ok, because it does not use any information from the branch,
but only logs the location when it adds a key.

Sponsored-by: Dartmouth College's Datalad project

diff --git a/CHANGELOG b/CHANGELOG
index 7a1d4066e5f8a265571076275fa01d98336916a8..32ba1e069821dd1d565c2432cc8d9d1bfa21f627 100644 (file)
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -10,6 +10,7 @@ git-annex (8.20210904) UNRELEASED; urgency=medium
     retrieving from a borg repository.
   * Resume where it left off when copying a file to/from a local git remote
     was interrupted.
+  * Sped up git-annex smudge --clean by 25%.
 
  -- Joey Hess <id@joeyh.name>  Fri, 03 Sep 2021 12:02:55 -0400
 

diff --git a/Command/Smudge.hs b/Command/Smudge.hs
index eb7b8b49c0c43736d9495acbea27ab150fe89026..bfb5916c01219244c696da5ade389b0ec5d53e41 100644 (file)
--- a/Command/Smudge.hs
+++ b/Command/Smudge.hs
@@ -30,6 +30,7 @@ import Annex.InodeSentinal
 import Utility.InodeCache
 import Config.GitConfig
 import qualified Types.Backend
+import qualified Annex.BranchState
 
 import qualified Data.ByteString as S
 import qualified Data.ByteString.Lazy as L
@@ -87,6 +88,7 @@ smudge file = do
 -- injested content if so. Otherwise, the original content.
 clean :: RawFilePath -> CommandStart
 clean file = do
+       Annex.BranchState.disableUpdate -- optimisation
        b <- liftIO $ L.hGetContents stdin
        ifM fileoutsiderepo
                ( liftIO $ L.hPut stdout b

diff --git a/doc/bugs/Windows__58___substantial_per-file_cost_for___96__add__96__/comment_3_85f07d906b836ac968ab877258180c0c._comment b/doc/bugs/Windows__58___substantial_per-file_cost_for___96__add__96__/comment_3_85f07d906b836ac968ab877258180c0c._comment
index ce7bb4013328386819aa9587f352207642a64d5a..0eb1184d9137c5b8afa2f8988e90a0b07cb0e2d6 100644 (file)
--- a/doc/bugs/Windows__58___substantial_per-file_cost_for___96__add__96__/comment_3_85f07d906b836ac968ab877258180c0c._comment
+++ b/doc/bugs/Windows__58___substantial_per-file_cost_for___96__add__96__/comment_3_85f07d906b836ac968ab877258180c0c._comment
@@ -13,8 +13,8 @@ The middle is slightly an outlier, and it would be better to have more data
 points, but what this says to me is it's probably around 38x more expensive
 on windows than on linux for git-annex smudge --clean to run.
 
-git-annex smudge --clean makes on the order of 3000 syscalls, including
-opening 200 files, execing git 30 times, and statting 400 files. That's
+git-annex smudge --clean makes on the order of 4000 syscalls, including
+opening 200 files, execing git 8 times, and statting 500 files. That's
 around 10x as many syscalls as git add makes. And it's run once per file. So
 relatively small differences in syscall performance between windows and
 linux can add up.

diff --git a/doc/bugs/Windows__58___substantial_per-file_cost_for___96__add__96__/comment_6_3947b44c88820da49d8a2d1afd3595ff._comment b/doc/bugs/Windows__58___substantial_per-file_cost_for___96__add__96__/comment_6_3947b44c88820da49d8a2d1afd3595ff._comment
new file mode 100644 (file)
index 0000000..7753eab
--- /dev/null
+++ b/doc/bugs/Windows__58___substantial_per-file_cost_for___96__add__96__/comment_6_3947b44c88820da49d8a2d1afd3595ff._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 6"""
+ date="2021-09-23T17:48:19Z"
+ content="""
+I noticed in the strace that smudge --clean ran git cat-file 2
+more times than necessary. Also was able to avoid updating the git-annex
+branch, which eliminates several calls to git (depending on the number of
+remotes). On Linux, this made it 25% faster. Might be more on Windows.
+
+Rest of the strace looks clean, nothing else stands out as unncessary.
+"""]]