fix problems with upgrade of local remotes

Upgrade other repos than the current one by running git-annex upgrade
inside them, which avoids problems with upgrade code making assumptions
that the cwd will be inside the repo being upgraded.

In particular, this fixes a problem where upgrading a v7 repo to v8 caused
an ugly git error message.

I actually could not find a way to make Upgrade.V7 work properly
without changing directory to the remote. Once I got git ls-files to work,
the git cat-file failed because :path can only be used in the current git
repo.

diff --git a/CHANGELOG b/CHANGELOG
index 1a5a12dc945749a1ca3bde09371f59913a84c3c7..031b7460974b03a1e7f98afeff79b24e53931d3c 100644 (file)
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -4,6 +4,11 @@ git-annex (8.20200227) UNRELEASED; urgency=medium
     GETCONFIG to query values like "name". (Introduced in version 7.20200202.7.)
   * Fix bug that caused unlocked annexed dotfiles to be added to git by the
     smudge filter when annex.dotfiles was not set.
+  * Upgrade other repos than the current one by running git-annex upgrade
+    inside them, which avoids problems with upgrade code making assumptions
+    that the cwd will be inside the repo being upgraded. In particular,
+    this fixes a problem where upgrading a v7 repo to v8 caused an ugly
+    git error message.
   * Improve behavior when a directory git-annex is writing to gets
     unmounted. Previously it could in some cases re-create the mount point
     and directory tree, and even write object contents to the wrong disk.

diff --git a/Command/Upgrade.hs b/Command/Upgrade.hs
index ead0c4b8673fc0f36fc3f00c01e28eeffd6784b6..2b343d9188dfe19e8eb418e2986d55031243ebc5 100644 (file)
--- a/Command/Upgrade.hs
+++ b/Command/Upgrade.hs
@@ -1,6 +1,6 @@
 {- git-annex command
  -
- - Copyright 2011 Joey Hess <id@joeyh.name>
+ - Copyright 2011-2020 Joey Hess <id@joeyh.name>
  -
  - Licensed under the GNU AGPL version 3 or higher.
  -}
@@ -20,13 +20,28 @@ cmd = dontCheck repoExists $
        noDaemonRunning $
        -- ^ avoid upgrading repo out from under daemon
        command "upgrade" SectionMaintenance "upgrade repository"
-               paramNothing (withParams seek)
+               paramNothing (seek <$$> optParser)
 
-seek :: CmdParams -> CommandSeek
-seek = withNothing (commandAction start)
+data UpgradeOptions = UpgradeOptions
+       { autoOnly :: Bool
+       }
 
-start :: CommandStart
-start = starting "upgrade" (ActionItemOther Nothing) $ do
+optParser :: CmdParamsDesc -> Parser UpgradeOptions
+optParser _ = UpgradeOptions
+       <$> switch
+               ( long "autoonly"
+               <> help "only do automatic upgrades"
+               )
+
+seek :: UpgradeOptions -> CommandSeek
+seek o = commandAction (start o)
+
+start :: UpgradeOptions -> CommandStart
+start (UpgradeOptions { autoOnly = True }) = do
+       starting "upgrade" (ActionItemOther Nothing) $ do
+       getVersion >>= maybe noop checkUpgrade
+       next $ return True
+start _ = starting "upgrade" (ActionItemOther Nothing) $ do
        whenM (isNothing <$> getVersion) $ do
                initialize Nothing Nothing
        r <- upgrade False latestVersion

diff --git a/Upgrade.hs b/Upgrade.hs
index 1a16da214d2e2f9c47d596dfeea2d92d9ff622d9..13e55f8cde15a49b0dda9951e644a74f821f70de 100644 (file)
--- a/Upgrade.hs
+++ b/Upgrade.hs
@@ -1,6 +1,6 @@
 {- git-annex upgrade support
  -
- - Copyright 2010-2019 Joey Hess <id@joeyh.name>
+ - Copyright 2010-2020 Joey Hess <id@joeyh.name>
  -
  - Licensed under the GNU AGPL version 3 or higher.
  -}
@@ -12,6 +12,8 @@ module Upgrade where
 import Annex.Common
 import qualified Annex
 import qualified Git
+import Config
+import Config.Files
 import Annex.Version
 import Types.RepoVersion
 #ifndef mingw32_HOST_OS
@@ -61,18 +63,26 @@ needsUpgrade v
 upgrade :: Bool -> RepoVersion -> Annex Bool
 upgrade automatic destversion = do
        upgraded <- go =<< getVersion
-       when upgraded $
-               setVersion destversion
+       when upgraded
+               postupgrade
        return upgraded
   where
        go (Just v)
                | v >= destversion = return True
-               | otherwise = ifM (up v)
-                       ( go (Just (RepoVersion (fromRepoVersion v + 1)))
-                       , return False
+               | otherwise = ifM upgradingRemote
+                       ( upgraderemote
+                       , ifM (up v)
+                               ( go (Just (RepoVersion (fromRepoVersion v + 1)))
+                               , return False
+                               )
                        )
        go _ = return True
 
+       postupgrade = ifM upgradingRemote
+               ( reloadConfig
+               , setVersion destversion
+               )
+
 #ifndef mingw32_HOST_OS
        up (RepoVersion 0) = Upgrade.V0.upgrade
        up (RepoVersion 1) = Upgrade.V1.upgrade
@@ -87,3 +97,19 @@ upgrade automatic destversion = do
        up (RepoVersion 6) = Upgrade.V6.upgrade automatic
        up (RepoVersion 7) = Upgrade.V7.upgrade automatic
        up _ = return True
+
+       -- Upgrade local remotes by running git-annex upgrade in them.
+       -- This avoids complicating the upgrade code by needing to handle
+       -- upgrading a git repo other than the current repo.
+       upgraderemote = do
+               rp <- fromRawFilePath <$> fromRepo Git.repoPath
+               cmd <- liftIO readProgramFile
+               liftIO $ boolSystem' cmd
+                       [ Param "upgrade"
+                       , Param "--quiet"
+                       , Param "--autoonly"
+                       ]
+                       (\p -> p { cwd = Just rp })
+
+upgradingRemote :: Annex Bool
+upgradingRemote = isJust <$> fromRepo Git.remoteName

diff --git a/doc/bugs/v7_upgrade_of_local_clone_bug.mdwn b/doc/bugs/v7_upgrade_of_local_clone_bug.mdwn
index da21118bcfd634c44e141c577ac099041d9710c4..1192f9bedfec67437b06ab09e407daa437d2e872 100644 (file)
--- a/doc/bugs/v7_upgrade_of_local_clone_bug.mdwn
+++ b/doc/bugs/v7_upgrade_of_local_clone_bug.mdwn
@@ -5,10 +5,17 @@ a repo that has it as a remote, this is displayed:
 
 This happens because git ls-files is run to list the files of the clone.
 But, it has some strange behavior when relative paths are used. Result is
-it always fails. This also causes the keys database of the clone to not get
-repopulated after being deleted for the upgrade. That's not a fatal problem
-because git-annex is always prepared for the keys database being out of
-date, but it could result in considerably more work being done later.
+it always fails. 
+
+This also causes the keys database of the clone to not get
+repopulated after being deleted for the upgrade. 
+That's not a fatal problem because git-annex is always prepared for the
+keys database being out of date, but it could result in considerably more
+work being done later.
+
+Also, the associated files does not get repopulated when it fails like
+that. That could cause worse problems. I have not tried to see what happens
+when the repo that fails to be upgraded has unlocked files.
 
 I also found some v1 upgrade code that does the same thing, and presumably
 also has the problem, although there are probably no v1 repos left. 
@@ -22,3 +29,11 @@ that looks like it would have then problem. That would affect upgrading
 from a V5 direct mode repo to V6.
 
 --[[Joey]
+
+> Fixed all such upgrade bugs by changing how local remotes are upgraded,
+> running git-annex upgrade inside the remote.
+> 
+> Also, guarded all git ls-files calls with a check that it's not being
+> run on a local remote, just in case there are other such bugs.
+> 
+> [[done]] --[[Joey]]

diff --git a/doc/git-annex-upgrade.mdwn b/doc/git-annex-upgrade.mdwn
index c27fedd6429006b2d972cdb0a2b44c310da96bf7..d9657a6d924ef003d2f31046808a53ac20084e02 100644 (file)
--- a/doc/git-annex-upgrade.mdwn
+++ b/doc/git-annex-upgrade.mdwn
@@ -8,7 +8,7 @@ git annex upgrade
 
 # DESCRIPTION
 
-Upgrades the repository.
+Upgrades the repository to the latest version.
 
 Each git-annex repository has an annex.version in its git configuration,
 that indicates the repository version. When an old repository version
@@ -24,6 +24,13 @@ was only used by its author. It's expected that git-annex will always
 support upgrading from all past repository versions -- this is necessary to
 allow archives to be taken offline for years and later used.
 
+# OPTIONS
+
+* --autoonly
+
+  Only do whatever automatic upgrade can be done, don't necessarily
+  upgrade to the latest version. This is used internally by git-annex.
+
 # SEE ALSO
 
 [[git-annex]](1)