From 93d8585fafce398cc3419f973476feca99934de9 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 13 Jul 2021 13:17:27 -0400 Subject: [PATCH] comment --- ..._3_3cd814797e837fae3dba8d6d99f41ec1._comment | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100644 doc/todo/option_to_individually_hash_chunks/comment_3_3cd814797e837fae3dba8d6d99f41ec1._comment diff --git a/doc/todo/option_to_individually_hash_chunks/comment_3_3cd814797e837fae3dba8d6d99f41ec1._comment b/doc/todo/option_to_individually_hash_chunks/comment_3_3cd814797e837fae3dba8d6d99f41ec1._comment new file mode 100644 index 0000000000..3784170a81 --- /dev/null +++ b/doc/todo/option_to_individually_hash_chunks/comment_3_3cd814797e837fae3dba8d6d99f41ec1._comment @@ -0,0 +1,17 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 3""" + date="2021-07-13T17:09:11Z" + content=""" +Hashing fixed sized chunks does not get very far toward +deduplication, it really needs a rolling hash. + +The chunk design was designed to support this kind of thing from the +beginning, see [[design/assistant/chunks]]. Each chunk has its own Key, and +the chunk log only needs to somehow store data to get from an object's Key +to a list of chunk Keys. Which is currently just a chunk size and number of +chunks, but could be a list of Keys, or a pointer to where to find the +list. + +So it's doable, but I have never quite found the tuits to attempt it. +"""]] -- 2.30.2