fix git-annex sync --content with proxied remotes

Loading the remote list a second time was removing all proxied remotes.
That happened because setting up the proxied remote added some config
fields to the in-memory git config, and on the second load, it saw those
configs and decided not to overwrite them with the proxy.

Now on the second load, that still happens. But now, the proxied
git configs are used to generate a remote same as if those configs were
all set. The reason that didn't happen before was twofold,
the gitremotes cache was not dropped, and the remote's url field was not
set correctly.

The problem with the remote's url field is that while it was marked as
proxy inherited, all other proxy inherited fields are annex- configs.
And the code to inherit didn't work for the url field.

Now it all works, but git-annex sync is left running git push/pull on
the proxied remote, which doesn't work. That still needs to be fixed.

diff --git a/Remote.hs b/Remote.hs
index fc27ea206ffb0a9fa875e56aa2852feca0131c9b..a57a929114184b3d97240a821abb90d0b780abbc 100644 (file)
--- a/Remote.hs
+++ b/Remote.hs
@@ -448,8 +448,8 @@ claimingUrl' remotefilter url = do
   where
        checkclaim = maybe (pure False) (`id` url) . claimUrl
 
-{- Is this a remote of a type we can sync with, or a special remote
- - with an annex:: url configured? -}
+{- Is this a remote of a type that git pull and push work with?
+ - That includes special remotes with an annex:: url configured. -}
 gitSyncableRemote :: Remote -> Bool
 gitSyncableRemote r
        | gitSyncableRemoteType (remotetype r) 

diff --git a/Remote/Git.hs b/Remote/Git.hs
index 8cbb04abaa03f03722d988f8a3fca35c396247ac..6f7d4265c2b6baf35b7768a22f7cc4b5997422fa 100644 (file)
--- a/Remote/Git.hs
+++ b/Remote/Git.hs
@@ -811,14 +811,14 @@ listProxied proxies rs = concat <$> mapM go rs
                | any isconfig (M.keys (Git.config g)) = pure Nothing
                | otherwise = do
                        -- Not using addGitConfigOverride for inherited
-                       -- configs other than the uuid, because child
-                       -- git processes do not need them to be provided
-                       -- with -c.
+                       -- configs, because child git processes do not
+                       -- need them to be provided with -c.
                        Annex.adjustGitRepo (pure . annexconfigadjuster)
                        return $ Just $ renamedr
          where
                renamedr = 
-                       let c = adduuid configkeyUUID $ Git.fullconfig r
+                       let c = adduuid configkeyUUID $
+                               Git.fullconfig r
                        in r 
                                { Git.remoteName = Just proxyname
                                , Git.config = M.map Prelude.head c
@@ -827,6 +827,7 @@ listProxied proxies rs = concat <$> mapM go rs
                
                annexconfigadjuster r' = 
                        let c = adduuid (configRepoUUID renamedr) $
+                               addurl (remoteConfig renamedr "url") $
                                inheritconfigs $ Git.fullconfig r'
                        in r'
                                { Git.config = M.map Prelude.head c
@@ -836,6 +837,9 @@ listProxied proxies rs = concat <$> mapM go rs
                adduuid ck = M.insert ck
                        [Git.ConfigValue $ fromUUID $ proxyRemoteUUID p]
 
+               addurl ck = M.insert ck
+                       [Git.ConfigValue $ encodeBS $ Git.repoLocation r]
+
                inheritconfigs c = foldl' inheritconfig c proxyInheritedFields
                
                inheritconfig c k = case (M.lookup dest c, M.lookup src c) of

diff --git a/Types/GitConfig.hs b/Types/GitConfig.hs
index c4a739e454e2164a25886204d84f7a206af3df22..2f8a4e3875d89ac5301a4e47ec2ccf43ee0241c6 100644 (file)
--- a/Types/GitConfig.hs
+++ b/Types/GitConfig.hs
@@ -620,7 +620,7 @@ remoteGitConfigField = \case
        -- Allow proxy chains.
        ProxyField -> inherited "proxy"
        ClusterNodeField -> uninherited "cluster-node"
-       UrlField -> inherited "url"
+       UrlField -> uninherited "url"
        ShellField -> inherited "shell"
        SshOptionsField -> inherited "ssh-options"
        RsyncOptionsField -> inherited "rsync-options"

diff --git a/doc/todo/git-annex_proxies.mdwn b/doc/todo/git-annex_proxies.mdwn
index 0ad17a00b4c78d6bd1fa8155c2be9b5b7b6709cb..57f7ca5186362a4226e983613a7414a809edf666 100644 (file)
--- a/doc/todo/git-annex_proxies.mdwn
+++ b/doc/todo/git-annex_proxies.mdwn
@@ -26,15 +26,14 @@ In development on the `proxy` branch.
 
 For June's work on [[design/passthrough_proxy]], remaining todos:
 
-* `git-annex sync --content` does not send content to clusters, or to
-  proxied remotes. Seems that the second call to Remote.list somehow
-  is failing to listProxies. 
+* `git-annex sync` etc should not treat clusters as git syncable remotes.
 
 * `git-annex sync` etc, when operating on clusters, should first 
   operate on the cluster as a whole, to take advantages of fanout on upload
   and mass drop. Only operate on individual cluster nodes afterwards,
   to handle cases such as a cluster containing a key, but some node
-  wanting and lacking the key.
+  wanting and lacking the key. Perhaps just setting cost for nodes slightly
+  higher than the cluster cost will be enough?
 
 * On upload to cluster, send to nodes where it's preferred content, and not
   to other nodes.