blew a day investigating this and still don't understand it fully

diff --git a/doc/todo/memory_use_increase/comment_6_8329ffed8b4e84440ae5326d40ece517._comment b/doc/todo/memory_use_increase/comment_6_8329ffed8b4e84440ae5326d40ece517._comment
new file mode 100644 (file)
index 0000000..ab2c8ba
--- /dev/null
+++ b/doc/todo/memory_use_increase/comment_6_8329ffed8b4e84440ae5326d40ece517._comment
@@ -0,0 +1,13 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 6"""
+ date="2020-10-13T18:59:08Z"
+ content="""
+In the alternatepipenullsplit branch, I made what I think is the relevant
+part of LsFiles not use a lazy bytestring, but instead read chunks
+directly to strict bytestrings. It did not change the PINNED amount or reduce
+overall memory use.
+
+So the pinning must be happening somewhere other than where I had assumed
+it was happening.
+"""]]

diff --git a/doc/todo/memory_use_increase/comment_7_68f580f2df32ee9027f70b4407451937._comment b/doc/todo/memory_use_increase/comment_7_68f580f2df32ee9027f70b4407451937._comment
new file mode 100644 (file)
index 0000000..9b5a0db
--- /dev/null
+++ b/doc/todo/memory_use_increase/comment_7_68f580f2df32ee9027f70b4407451937._comment
@@ -0,0 +1,16 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 7"""
+ date="2020-10-13T19:15:41Z"
+ content="""
+The seekHelper version in comment 3 significantly changes the 
+shape of the profile. PINNED is still there, but only a couple mb,
+and it's a constant amount of memory, not an upward slope.
+
+So I think that identifies the main culprit for the mysterious PINNED.
+runSegmentPaths is supposed to stream, but it seems it somehow is hanging
+on to values longer than it should.
+
+Although the max memory use of that version is still around 90 mb
+according to `time`. The profile is flat at 3500 kb.
+"""]]