prevent appends except when annex.alwayscompact=false

I would like for a new repo version to enable appends, but to do so
safely would need a v11 followed by a 1 year delay followed by a v12
that does it. Since a similar v9 and v10 transition is currently
happening, and is less than 6 months along in most repos, it does not
feel wise to stack up another year-long transition behind that. What if
I need to hurry up a new repo version for some other change?

Added todo so I remember to make this change at some time when a v11
and probably v12 repo version do make sense.

Sponsored-by: Dartmouth College's DANDI project

diff --git a/Annex/Branch.hs b/Annex/Branch.hs
index 456d9d6619309a0417cf4a35f5eef004744cdeda..e98ce121679fd52c9ccefb3cf1889e229fe5b830 100644 (file)
--- a/Annex/Branch.hs
+++ b/Annex/Branch.hs
@@ -418,6 +418,12 @@ data ChangeOrAppend t = Change t | Append t
  - reading the journal file, and so can be faster when many lines are being
  - written to it. The information that is recorded will be effectively the
  - same, only obsolate log lines will not get compacted.
+ -
+ - Currently, only appends when annex.alwayscompact=false. That is to
+ - avoid appending when an older version of git-annex is also in use in the
+ - same repository. An interrupted append could leave the journal file in a
+ - state that would confuse the older version. This is planned to be
+ - changed in a future repository version.
  -}
 changeOrAppend :: Journalable content => RegardingUUID -> RawFilePath -> (L.ByteString -> ChangeOrAppend content) -> Annex ()
 changeOrAppend ru file f = lockJournal $ \jl ->
@@ -427,7 +433,12 @@ changeOrAppend ru file f = lockJournal $ \jl ->
                                oldc <- getToChange ru file
                                case f oldc of
                                        Change newc -> set jl ru file newc
-                                       Append toappend -> append jl file appendable toappend
+                                       Append toappend -> 
+                                               set jl ru file $
+                                                       oldc <> journalableByteString toappend
+                                               -- Use this instead in v11
+                                               -- or whatever.
+                                               -- append jl file appendable toappend
                        , case f mempty of
                                -- Append even though a change was
                                -- requested; since mempty was passed in,

diff --git a/CHANGELOG b/CHANGELOG
index 9f6d22fcd710e60c951d11990ab0c112d982df60..d8d07dd65e1ff6921006bca4b7c6891f2acc7acd 100644 (file)
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -1,4 +1,4 @@
-git-annex (10.20220625) UNRELEASED; urgency=medium
+git-annex (10.20220724) UNRELEASED; urgency=medium
 
   * Improve handling of parallelization with -J when copying content
     from/to a git remote that is a local path.
@@ -20,7 +20,8 @@ git-annex (10.20220625) UNRELEASED; urgency=medium
   * Fix a reversion that prevented --batch commands (and the assistant)
     from noticing data written to the journal by other commands.
   * Added annex.alwayscompact setting which can be unset to speed up
-    writes to the git-annex branch in some cases.
+    writes to the git-annex branch in some cases. See its documentation
+    for important notes on when it's appropariate to use.
 
  -- Joey Hess <id@joeyh.name>  Tue, 28 Jun 2022 14:49:17 -0400
 

diff --git a/doc/git-annex.mdwn b/doc/git-annex.mdwn
index 1c679861605e10d021bbb067ce99d302697b7be1..34c3ba7f60577f23a873e13395ed4214c3443b2f 100644 (file)
--- a/doc/git-annex.mdwn
+++ b/doc/git-annex.mdwn
@@ -1051,6 +1051,11 @@ repository, using [[git-annex-config]]. See its man page for a list.)
   `-c annex.alwayscompact=false` is `git-annex registerurl --batch`,
   when adding a large number of urls to the same key.
 
+  This option was first supported by git-annex version 10.20220724.
+  It is not entirely safe to set this option in a repository that may also
+  be used by an older version of git-annex at the same time as a version
+  that supports this option.
+
 * `annex.allowsign`
 
   By default git-annex avoids gpg signing commits that it makes when

diff --git a/doc/todo/v11_changes.mdwn b/doc/todo/v11_changes.mdwn
new file mode 100644 (file)
index 0000000..2e40f0c
--- /dev/null
+++ b/doc/todo/v11_changes.mdwn
@@ -0,0 +1,12 @@
+This is a todo for collecting changes that could lead to a v11 repository
+version.
+
+* Append to journal files even when annex.alwayscompact=true.
+  This can make it a lot faster in some cases.
+  See note in Annex.Branch.changeOrAppend.
+
+  It's important that this only happen when no git-annex version
+  older than 10.20220724 can plausibly be running in a repository
+  after upgrading to the repo version that enables this. Depending on the
+  timing of v11, this may need to be put in a v12 upgrade that is delayed
+  some amount of time (eg 1 year) after v11.