init: Avoid scanning for annexed files, which can be lengthy in a
large repository. Instead that scan is done on demand. This lets git-annex
init be run and some query commands be used in a repository without
waiting.
Note that autoinit already behaved this way, so while this will mean some
commands like git-annex get/unlock/add will do the scan the first time run,
that is not really a significant behavior change.
And, it's really better to have a consistent behavior. The reason for
the inconsistency was a strange bug discussed in
b3c4579c7907147a496bdf2c73b42238d8b239d6. Avoiding reconcileStaged in
init will keep avoiding whatever that was.
Sponsored-by: Dartmouth College's DANDI project
import Annex.Version
import Annex.Difference
import Annex.UUID
-import Annex.WorkTree
import Annex.Fixup
import Annex.Path
import Config
Right username -> [username, at, hostname, ":", reldir]
Left _ -> [hostname, ":", reldir]
-initialize :: Bool -> Maybe String -> Maybe RepoVersion -> Annex ()
-initialize autoinit mdescription mversion = checkInitializeAllowed $ \initallowed -> do
+initialize :: Maybe String -> Maybe RepoVersion -> Annex ()
+initialize mdescription mversion = checkInitializeAllowed $ \initallowed -> do
{- Has to come before any commits are made as the shared
- clone heuristic expects no local objects. -}
sharedclone <- checkSharedClone
ensureCommit $ Annex.Branch.create
prepUUID
- initialize' autoinit mversion initallowed
+ initialize' mversion initallowed
initSharedClone sharedclone
-- Everything except for uuid setup, shared clone setup, and initial
-- description.
-initialize' :: Bool -> Maybe RepoVersion -> InitializeAllowed -> Annex ()
-initialize' autoinit mversion _initallowed = do
+initialize' :: Maybe RepoVersion -> InitializeAllowed -> Annex ()
+initialize' mversion _initallowed = do
checkLockSupport
checkFifoSupport
checkCrippledFileSystem
unlessM isBareRepo $ do
hookWrite postCheckoutHook
hookWrite postMergeHook
- unless autoinit $
- scanAnnexedFiles
AdjustedBranch.checkAdjustedClone >>= \case
AdjustedBranch.InAdjustedClone -> return ()
where
needsinit = ifM autoInitializeAllowed
( do
- tryNonAsync (initialize True Nothing Nothing) >>= \case
+ tryNonAsync (initialize Nothing Nothing) >>= \case
Right () -> noop
Left e -> giveup $ show e ++ "\n" ++
"git-annex: automatic initialization failed due to above problems"
where
needsinit =
whenM (initializeAllowed <&&> autoInitializeAllowed) $ do
- initialize True Nothing Nothing
+ initialize Nothing Nothing
autoEnableSpecialRemotes remotelist
{- Checks if a repository is initialized. Does not check version for ugrade. -}
initRepo' :: Maybe String -> Maybe StandardGroup -> Annex ()
initRepo' desc mgroup = unlessM isInitialized $ do
- initialize False desc Nothing
+ initialize desc Nothing
u <- getUUID
maybe noop (defaultStandardGroup u) mgroup
{- Ensure branch gets committed right away so it is
git-annex (10.20221105) UNRELEASED; urgency=medium
* Support quettabyte and yottabyte.
- * Sped up the initial scanning for annexed files by 21%.
+ * Sped up the initial scan for annexed files by 21%.
+ * init: Avoid scanning for annexed files, which can be lengthy in a
+ large repository. Instead that scan is done on demand.
-- Joey Hess <id@joeyh.name> Fri, 18 Nov 2022 12:58:06 -0400
else ifM (Annex.Branch.hasSibling <||> (isJust <$> Fields.getField Fields.autoInit))
( do
liftIO checkNotReadOnly
- initialize True Nothing Nothing
+ initialize Nothing Nothing
getUUID
, return NoUUID
)
Just v | v /= wantversion ->
giveup $ "This repository is already a initialized with version " ++ show (fromRepoVersion v) ++ ", not changing to requested version."
_ -> noop
- initialize False
+ initialize
(if null (initDesc os) then Nothing else Just (initDesc os))
(initVersion os)
unless (noAutoEnable os)
then return $ toUUID s
else Remote.nameToUUID s
storeUUID u
- checkInitializeAllowed $ initialize' False Nothing
+ checkInitializeAllowed $ initialize' Nothing
Annex.SpecialRemote.autoEnable
next $ return True
start _ =
starting "upgrade" (ActionItemOther Nothing) (SeekInput []) $ do
whenM (isNothing <$> getVersion) $ do
- initialize False Nothing Nothing
+ initialize Nothing Nothing
r <- upgrade False latestVersion
next $ return r
content="""
Implemented the two optimisations discussed above, and init in that
repository dropped from 24 seconds to 19 seconds, a 21% speedup.
+
+I think that's as fast as reconcileStaged is likely to get without
+some deep optimisation of the persistent library.
+
+Then I realized that `git-annex init` does not really need to scan for
+associated files. That can be done later, when running a command that needs
+to access the keys database. Indeed, when git-annex is used in a clone of
+an annexed repo without explicitly running `git-annex init`, that's what
+it already did. I've implemented that, so now `git-annex init` takes 3
+seconds or so. The price will be paid later, the first time running a
+`git-annex add` or `git-annex unlock` or `git-annex get`.
"""]]
projects / git-annex.git / commitdiff