import: Sped up import from special remote when the imported tree is unchanged
authorJoey Hess <joeyh@joeyh.name>
Tue, 2 Jan 2024 17:56:50 +0000 (13:56 -0400)
committerJoey Hess <joeyh@joeyh.name>
Tue, 2 Jan 2024 17:57:31 +0000 (13:57 -0400)
commita5b9c2ca699228e82a99b672cb2796d449dcdf92
tree6a6a97b9f012fa2f56d00fe7e20fb6d9f9e21192
parent1e775ab83b2075b093e5a2803c337ce66a887780
import: Sped up import from special remote when the imported tree is unchanged

I saw a nearly 2 minute speed up from this, in a repo with 56000 files some
of which are preferred content of the special remote and others not. In
such a case, addBackExportExcluded has to do a lot of work, which is
unncessary when the tree is unchanged.

When using sync --content, preferred content checking of that many files
takes about 1 minute. So this speeds up sync --content by 3x.
When using git-annex import, the speed up is much larger.

Sponsored-by: Nicholas Golder-Manning on Patreon
Annex/Import.hs
CHANGELOG