Added a comment: keys db optimization
authorIlya_Shlyakhter <Ilya_Shlyakhter@web>
Wed, 2 Jun 2021 16:53:03 +0000 (16:53 +0000)
committeradmin <admin@branchable.com>
Wed, 2 Jun 2021 16:53:03 +0000 (16:53 +0000)
doc/todo/speed_up_keys_db_update_with_git_streaming/comment_1_0afd62cbe52eba137fbfd1907337e273._comment [new file with mode: 0644]

diff --git a/doc/todo/speed_up_keys_db_update_with_git_streaming/comment_1_0afd62cbe52eba137fbfd1907337e273._comment b/doc/todo/speed_up_keys_db_update_with_git_streaming/comment_1_0afd62cbe52eba137fbfd1907337e273._comment
new file mode 100644 (file)
index 0000000..577480b
--- /dev/null
@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="keys db optimization"
+ date="2021-06-02T16:53:02Z"
+ content="""
+\"users who often switch between branches that have tens to hundreds of thousands of diverged files will find it slow\" -- that's my use case ;)  Could one keys-to-files db be kept per branch?
+
+Maybe, the keys db could be split, based e.g. on prefix of md5 of the key, into separate sqlite files, and the writing to them parallelized?
+It's common to be working on a many-core machine.
+
+Is the keys-to-locked-files db used for anything besides detecting keys used by more than one files?  For that one purpose there might be faster solutions.
+But, if it's implemented, maybe it also be used to remove the [[limitation|git-annex-preferred-content]] that \"when a command is run with the --all option, or in a bare repository, there is no filename associated with an annexed object, and so \"include=\" and \"exclude=\" will not match\"?
+"""]]