From 5798013fb2fb16c3df203580ee84efc4cca4f192 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 31 Jan 2017 12:30:44 -0400 Subject: [PATCH] reproduced bug; started analysis This commit was sponsored by Peter Hogg on Patreon. --- ..._e7811b548054d3d6851facb8d3bf8153._comment | 25 +++++++++++++++++++ 1 file changed, 25 insertions(+) create mode 100644 doc/todo/more_efficient_memory_usage_with_git-annex_unused/comment_1_e7811b548054d3d6851facb8d3bf8153._comment diff --git a/doc/todo/more_efficient_memory_usage_with_git-annex_unused/comment_1_e7811b548054d3d6851facb8d3bf8153._comment b/doc/todo/more_efficient_memory_usage_with_git-annex_unused/comment_1_e7811b548054d3d6851facb8d3bf8153._comment new file mode 100644 index 0000000000..5247bb61cd --- /dev/null +++ b/doc/todo/more_efficient_memory_usage_with_git-annex_unused/comment_1_e7811b548054d3d6851facb8d3bf8153._comment @@ -0,0 +1,25 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2017-01-31T15:36:58Z" + content=""" +You need to provide at least a version number, and ideally enough +information to reproduce the bug when filing bug reports. + +Anyhow, I ran git-annex (HEAD) unused in my big repo, and its memory use +got up to over 1 gb which is much more than I would expect (should be +a couple hundred mb max). + +The memory growth happens in the stage when it's +constructing the bloom filter for the keys in the diff between the +index and other branches. In my big repo, those diffs are quite large; +eg I have a branch with 70k files and another with 0 files. + +I replaced insertMB with noop, so the bloom filters are not really +populated, and it still uses as much memory. So +the memory is not being leaked by the bloom filters themselves, but +is instead being leaked when processing the branch diffs, +or something like that. + +Need to profile to find what's leaking. +"""]] -- 2.30.2