Sped up view branch construction by 50%
authorJoey Hess <joeyh@joeyh.name>
Mon, 13 Feb 2023 17:29:57 +0000 (13:29 -0400)
committerJoey Hess <joeyh@joeyh.name>
Mon, 13 Feb 2023 17:29:57 +0000 (13:29 -0400)
commit826b225ca81d469640e51628865dbd64a1d37773
tree5eafefdb171cb2711d26974db260bb09e419efad
parentc733ccdf2181ea213eecff4f64c7f1f498cd1574
Sped up view branch construction by 50%

A benchmark in my sound repository with `git-annex view feedtitle=*`
took 2:52 wall clock time before and 1:58 after. Though it still only used
130% of CPU.

This is the same kind of optimisation that is in seekFilteredKeys, though
that precaches location logs while this streams the metadata logs direct
to parsing them.

seekFilteredKeys contains more streaming, to find the annexed files, and
this could be further sped up with similar streaming.

Sponsored-by: Nicholas Golder-Manning on Patreon
Annex/View.hs
CHANGELOG
doc/todo/faster_incremental_update_of_view_branch_by_git-annex_sync.mdwn