Joey Hess [Wed, 23 Aug 2023 17:31:31 +0000 (13:31 -0400)]
oldkeys: check associated files by default and add --unchecked
Removed the prior code that checked for keys used by current versions of
the files being acted on. It is redundant with the associated files
check (so long as the associated files database is always up-to-date,
which reconcileStaged should accomplish).
Joey Hess [Tue, 22 Aug 2023 18:51:06 +0000 (14:51 -0400)]
oldkeys: New command that lists the keys used by old versions of a file
The tricky thing about this turned out to be handling renames and reverts.
For that, it has to make two passes over the git log, and to avoid
buffering a possibly huge amount of logs in memory (ie the whole git log of
an entire repository!), runs git log twice.
(It might be possible to speed this up by asking git log to show a diff,
and so avoid needing to use catKey.)
Joey Hess [Tue, 22 Aug 2023 15:58:13 +0000 (11:58 -0400)]
diffdriver: Added --get option
Removed the dontCheck repoExists, because running it in a repo that has not
been initialized yet would update location log with nouuid. And I guess
it's ok for it to only support running in git-annex repos.
Joey Hess [Tue, 22 Aug 2023 14:25:53 +0000 (10:25 -0400)]
avoid unncessary use of curl when conduit will do
Avoid using curl when annex.security.allowed-ip-addresses is set but
neither annex.web-options nor annex.security.allowed-url-schemes is set to
a value that needs curl.
Joey Hess [Wed, 16 Aug 2023 19:57:30 +0000 (15:57 -0400)]
convert checkAvailable to use availability rather than localpath
Every remote that sets localpath also implements an availability that
reutrns Unavailable when a local directory is not available.
This makes external remotes, and others that get support for
availability Unavailable to be used by checkAvailable. (Which is only
used by the assistant.)
Had to keep localpath though, since other parts of the assistant use it
to eg, sync with a remote when a removable drive is plugged in.
Joey Hess [Wed, 16 Aug 2023 18:31:31 +0000 (14:31 -0400)]
let Remote.availability return Unavilable
This is groundwork for making special remotes like borg be skipped by
sync when on an offline drive.
Added AVAILABILITY UNAVAILABLE reponse and the UNAVAILABLERESPONSE extension
to the external special remote protocol. The extension is needed because
old git-annex, if it sees that response, will display a warning
message. (It does continue as if the remote is globally available, which
is acceptable, and the warning is only displayed at initremote due to
remote.name.annex-availability caching, but still it seemed best to make
this a protocol extension.)
The remote.name.annex-availability git config is no longer used any
more, and is documented as such. It was only used by external special
remotes to cache the availability, to avoid needing to start the
external process every time. Now that availability is queried as an
Annex action, the external is only started by sync (and the assistant),
when they actually check availability.
Joey Hess [Tue, 15 Aug 2023 18:21:53 +0000 (14:21 -0400)]
Stop bundling curl in the OSX dmg
New curl binary links to libldap with a @loader_path that prevents using
the binary when the dmg is used elsewhere.
See https://github.com/datalad/git-annex/issues/170
git-annex doesn't use curl by default anyway, so it doesn't really need to
be included in the dmg.
Joey Hess [Tue, 15 Aug 2023 16:57:41 +0000 (12:57 -0400)]
fix empty tree import when directory does not exist
Fix behavior when importing a tree from a directory remote when the
directory does not exist. An empty tree was imported, rather than the
import failing. Merging that tree would delete every file in the
branch, if those files had been exported to the directory before.
The problem was that dirContentsRecursive returned [] when the directory
did not exist. Better for it to throw an exception. But in commit 74f0d67aa3988a71f3a53b88de4344272d924b95 back in 2012, I made it never
theow exceptions, because exceptions throw inside unsafeInterleaveIO become
untrappable when the list is being traversed.
So, changed it to list the contents of the directory before entering
unsafeInterleaveIO. So exceptions are thrown for the directory. But still
not if it's unable to list the contents of a subdirectory. That's less of a
problem, because the subdirectory does exist (or if not, it got removed
after being listed, and it's ok to not include it in the list). A
subdirectory that has permissions that don't allow listing it will have its
contents omitted from the list still.
(Might be better to have it return a type that includes indications of
errors listing contents of subdirectories?)
The rest of the changes are making callers of dirContentsRecursive
use emptyWhenDoesNotExist when they relied on the behavior of it not
throwing an exception when the directory does not exist. Note that
it's possible some callers of dirContentsRecursive that used to ignore
permissions problems listing a directory will now start throwing exceptions
on them.
The fix to the directory special remote consisted of not making its
call in listImportableContentsM use emptyWhenDoesNotExist. So it will
throw an exception as desired.
Joey Hess [Wed, 9 Aug 2023 17:30:43 +0000 (13:30 -0400)]
Add git-annex remote refs that are not newer to the merged refs list
Significant startup speed increase by avoiding repeatedly checking if some
remote git-annex branch refs need to be merged when it is not newer.
One way this could happen is when there are 2 remotes that are themselves
connected. The git-annex branch on the first remote gets updated. Then the
second remote pulls from the first, and merges in its git-annex branch.
Then the local repo pulls from the second remote, and merges its git-annex
branch. At this point, a pull from the first remote will get a git-annex
branch that is not newer, but is not on the merged refs list.
In my big repo, git-annex startup time dropped from 4 seconds to 0.1 seconds.
There were 5 to 10 such remote refs out of 18 remotes.
Joey Hess [Wed, 9 Aug 2023 16:43:48 +0000 (12:43 -0400)]
info: Added --dead-repositories option
I considered a more wide-ranging config option to make other commands
also show dead repositories. But it would be difficult to implement that
because Remote.keyLocations is used to get locations, filtering out dead
repos, and commands like get then try to use those locations. So a config
setting would make dead repos sometimes be acted on by commands.
Joey Hess [Wed, 2 Aug 2023 16:48:12 +0000 (12:48 -0400)]
windows back to lts-18.13 temporarily
I can't seem to get stack to resolve dependencies with Win32-2.13.4.0,
no matter what I try. Why it blows up, I don't know.
And allow-newer: true actually causes it to downgrade Win32 to the one
version that won't build. Unbelivable that allows downgrades.
So just gonna have to wait for that to get into stackage nightly, and
then stack.yaml can be updated to use that, and the changes in this
commit reverted.
Joey Hess [Wed, 2 Aug 2023 15:50:12 +0000 (11:50 -0400)]
fix stack build on windows
For whatever reason, putting Win32-2.13.4.0 in stack.yaml results in
stack blowing up with many unrelated dependency problems.
But making git-annex depend on that version lets stack resolve deps.
Joey Hess [Wed, 2 Aug 2023 14:08:00 +0000 (10:08 -0400)]
avoid cpp failure on windows
Seems that while the module is not imported by anything on windows, it
still gets cpped, and MIN_VERSION_unix is not defined so it failed to
preprocess.
Joey Hess [Wed, 2 Aug 2023 00:17:29 +0000 (20:17 -0400)]
fix hang when built with unix-2.8
git-annex test hang when running git-annex add in an adjusted unlocked
branch. I couldn't seem to reproduce the hang outside the test suite.
Seems that the code added in 26a9ea12d11f85b43987d863449c790f7320d9d1
was buggy, and as that commit was made without testing it, building with
unix-2.8 exposed the bug.
I don't fully understand the bug, which involves fdToHandle
and then closing the fd, vs closing the handle. May somehow involve
laziness or forcing around the S.hGet? Using hClose solved it
in any case.
(Also eliminated checkcontentfollowssymlinks to fix a build warning
when it's not used.)
Joey Hess [Tue, 1 Aug 2023 22:53:26 +0000 (18:53 -0400)]
stack.yaml: Update to build with ghc-9.6.2 and aws-0.24
This enables some new features that need the new aws.
Use http-client-restricted-0.1.0 because it uses the crypton side of the
cryptonite/crypton fork, which seems to be needed for ghc-9.6.2.
Dependency on connection removed because of the cryptonite/crypton fork.
This avoids needing a build flag. It was only used to throw a typed
exception in Utility.Url, which nothing depended on.
Used a fork of bloomfilter because it's not being maintained and no longer
builds as-of this ghc version. (I have been trying to contact its
maintainer about it, and emailed him today suggesting I take over the
package.)
Joey Hess [Tue, 1 Aug 2023 21:47:30 +0000 (17:47 -0400)]
fix build with unix-2.8.0
It made UserInfo into a pattern to discourage manually constructing
them, so just to use UserInfo in a type signature of a function that
consumes them, have to import the new ByteString module.
Joey Hess [Tue, 1 Aug 2023 19:08:56 +0000 (15:08 -0400)]
Setup.hs: Stop installing man pages, desktop files, and the git-annex-shell and git-remote-tor-annex symlinks
Anything still relying on that, eg via cabal v1-install will need to
change to using make install-home. Which was added back in 2019 in 6491b626140a9e5cc8e99fff4a1434d7c0e9bc39 because cabal new-build
(now the default) already didn't use Setup in a way that let its
installation of those things work.
Notably this means Setup does not need to depend on unix-compat, which is
useful because in 0.7 it removed System.PosixCompat.User, which Setup
needed to determine where to install the desktop files. See
https://github.com/haskell-pkg-janitors/unix-compat/issues/3
projects / git-annex.git / log