From 2e92cef13f676b8daf02367bafe1c04121229e07 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Thu, 1 Jun 2023 18:43:17 -0400 Subject: [PATCH] comment --- ..._b8b3eb5fff4ad3478dfbbd2120d4b00a._comment | 35 +++++++++++++++++++ 1 file changed, 35 insertions(+) create mode 100644 doc/bugs/importtree_spends_hours_reading_cidsdb/comment_6_b8b3eb5fff4ad3478dfbbd2120d4b00a._comment diff --git a/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_6_b8b3eb5fff4ad3478dfbbd2120d4b00a._comment b/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_6_b8b3eb5fff4ad3478dfbbd2120d4b00a._comment new file mode 100644 index 0000000000..3543623b2e --- /dev/null +++ b/doc/bugs/importtree_spends_hours_reading_cidsdb/comment_6_b8b3eb5fff4ad3478dfbbd2120d4b00a._comment @@ -0,0 +1,35 @@ +[[!comment format=mdwn + username="joey" + subject="""Results with 150000 files""" + date="2023-06-01T18:17:34Z" + content=""" +Initial sync: 39:01.37 + +Then `git-annex import master:testdata --from source` +with no changed files: 8:36.30 +(This is writing the cidsdb since the initial sync no +longer does, see comment #2) + +Subsequent imports with no changed files: 0:37.21 + +Subsequent syncs with no changed files: 3:18.61 + +Subsequent syncs with no changed files, when not on an adjusted branch: 0:38.40 + +That 37 second run time for an import with no changed files seems to +indicate that my optimisation so far was successful at eliminating all +cidsdb reading. It's bottlenecked on git operations now. + +I also tried an initial import of 150000 files to see how much of that +initial sync was other work than the import (updating adjusted branch, +checking out files, etc). + +Initial import: 38:24.36 + +So the initial sync is mostly spending time on imports, not other stuff. +It could perhaps be sped up some. Of course it does support -J2 which may +speed it up throuh parallism. + +Seems like a sync with no changed files on an adjusted branch is doing +unncessary work to update the adjusted branch when nothing has changed. +"""]] -- 2.30.2