Joey Hess [Mon, 26 Dec 2022 16:42:45 +0000 (12:42 -0400)]
Removed the vendored git-lfs and the GitLfs build flag
AFAICS all git-annex builds are using the git-lfs library not the vendored
copy.
Debian stable does have a too old haskell-git-lfs package to be able to
build git-annex from source, but there is not currently a backport of a
recent git-annex to Debian stable. And if they update the backport at some
point, they should be able to backport the library too.
Joey Hess [Thu, 22 Dec 2022 19:15:25 +0000 (15:15 -0400)]
Added libgcc_s.so.1 to the linux standalone build so pthread_cancel will work
In Makefile, listed additional deps of Build/Standalone. Without that,
it does not get updated for the change to Utility/LinuxMkLibs.hs when
compiling incrementally.
Joey Hess [Thu, 22 Dec 2022 18:26:35 +0000 (14:26 -0400)]
switch to readMaybe to handle values with leading number followed by non-number
readish ignores a trailing string after a number, but to support values
like "YYYY:MM:DD" which it makes sense to compare lexographically,
require the whole string to be parsed as a number in order to enable
numeric comparison.
Joey Hess [Tue, 20 Dec 2022 20:02:42 +0000 (16:02 -0400)]
use status --ignore-submodules in configureSmudgeFilter
Speed up git-annex upgrade (from v5) and init in a repository that has
submodules. Setting the config does not affect the submodules, so avoid
the work of getting status in them, which may involve using the smudge
filter etc.
Sponsored-By: the NIH-funded NICEMAN (ReproNim TR&D3) project
Joey Hess [Tue, 20 Dec 2022 19:42:34 +0000 (15:42 -0400)]
--anything and --nothing
Added --anything (and --nothing). Eg, git-annex find --anything will list
all annexed files whether or not the content is present. This is slightly
faster and clearer than --include=* or --exclude=*
While I can't imagine how --nothing will be used, preferred content
expressions already had anything and nothing, so might as well support both
as matching options as well.
Joey Hess [Tue, 20 Dec 2022 19:17:50 +0000 (15:17 -0400)]
convert renameFile to moveFile to support cross-device moves
Improve handling of some .git/annex/ subdirectories being on other
filesystems, in the bittorrent special remote, and youtube-dl integration,
and git-annex addurl.
The only one of these that I've confirmed to be a problem is in the
bittorrent special remote when .git/annex/tmp and .git/annex/othertmp are
on different filesystems.
As well as auditing for renameFile, also audited for createLink, all of
those are ok as are the other remaining renameFile calls. Also audited all
code paths that use .git/annex/othertmp, and did not find any other
cross-device problems. So, removing mention of othertmp needing to be on
the same device.
The restage.log was kept locked while running a complex index refresh
action. In an unusual situation, that action could need to write to the
restage log, which caused a deadlock.
The solution is a two-stage process. First the restage.log is moved to a
work file, which is done with the lock held. Then the content of the work
file is read and processed, which happens without the lock being held.
This is all done in a crash-safe manner.
Note that streamRestageLog may not be fully safe to run concurrently
with itself. That's ok, because restagePointerFiles uses it with the
index lock held, so only one can be run at a time.
streamRestageLog does delete the restage.old file at the end without
locking. If a calcRestageLog is run concurrently, it will either see the
file content before it was deleted, or will see it's missing. Either is
ok, because at most this will cause calcRestageLog to report more
work remains to be done than there is.
Joey Hess [Wed, 7 Dec 2022 17:46:16 +0000 (13:46 -0400)]
speed up --pattern
The splitting of the tests into parts for parallelism made --pattern
do extra work, because init tests have to be run for each part, but
many of the parts are empty.
For example, git-annex test --pattern '/move (ssh remote)/'
took 12 seconds to run before. This improves the runtime to 4 seconds.
Joey Hess [Mon, 21 Nov 2022 18:39:26 +0000 (14:39 -0400)]
When youtube-dl is not available in PATH, use yt-dlp instead
Debian is going to drop youtube-dl which is not active upstream, and yt-dlp
is the replacement. This will make it be used if youtube-dl gets removed.
If an old version of youtube-dl remains installed, git-annex will still use
it. That might not be desirable, but changing git-annex to use yt-dlp in
preference to youtube-dl when both are installed risks breaking when
the user has annex.youtube-dl-options set to something that is supported
by youtube-dl, but not by yt-dlp.
Joey Hess [Fri, 18 Nov 2022 17:58:35 +0000 (13:58 -0400)]
don't frontload reconcileStaged in git-annex init
init: Avoid scanning for annexed files, which can be lengthy in a
large repository. Instead that scan is done on demand. This lets git-annex
init be run and some query commands be used in a repository without
waiting.
Note that autoinit already behaved this way, so while this will mean some
commands like git-annex get/unlock/add will do the scan the first time run,
that is not really a significant behavior change.
And, it's really better to have a consistent behavior. The reason for
the inconsistency was a strange bug discussed in b3c4579c7907147a496bdf2c73b42238d8b239d6. Avoiding reconcileStaged in
init will keep avoiding whatever that was.
projects / git-annex.git / log