From: Joey Hess Date: Tue, 19 Jul 2022 19:45:08 +0000 (-0400) Subject: thoughts X-Git-Tag: archive/raspbian/10.20250416-2+rpi1~1^2~72^2~24 X-Git-Url: https://dgit.raspbian.org/?a=commitdiff_plain;h=d832a7c211a7479ce5c18c5931f0451b08e7a35b;p=git-annex.git thoughts --- diff --git a/doc/todo/registerurl__58___do_changes_in_journal___34__in_place__34____63__/comment_18_6b4df9d81c523e090b83c56bfdce373a._comment b/doc/todo/registerurl__58___do_changes_in_journal___34__in_place__34____63__/comment_18_6b4df9d81c523e090b83c56bfdce373a._comment new file mode 100644 index 0000000000..556d08adec --- /dev/null +++ b/doc/todo/registerurl__58___do_changes_in_journal___34__in_place__34____63__/comment_18_6b4df9d81c523e090b83c56bfdce373a._comment @@ -0,0 +1,16 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 18""" + date="2022-07-19T18:28:15Z" + content=""" +This could be repository version 11, but there is unfortunately not a way to +guarantee that an old git-annex process from before the repository upgrade +is not still running and getting confused. The `v9-v10` upgrade waits an +entire year to ensure no such processes are left, and a similar wait would +be needed here. + +Still, you could force upgrade the repositories that you need to be fast right +away, or initialize new repositories at the new repository version. Since +most people are not hit by this kind of performance problem, waiting a +while before it's available everywhere is not a problem. +"""]] diff --git a/doc/todo/registerurl__58___do_changes_in_journal___34__in_place__34____63__/comment_19_b23a39f88879b3c770e112f9e82c4607._comment b/doc/todo/registerurl__58___do_changes_in_journal___34__in_place__34____63__/comment_19_b23a39f88879b3c770e112f9e82c4607._comment new file mode 100644 index 0000000000..b5d9b73418 --- /dev/null +++ b/doc/todo/registerurl__58___do_changes_in_journal___34__in_place__34____63__/comment_19_b23a39f88879b3c770e112f9e82c4607._comment @@ -0,0 +1,35 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 19""" + date="2022-07-19T18:38:33Z" + content=""" +If we don't need to worry about breaking an old git-annex process, thanks +to an upgrade process that makes sure there are not any, a simpler and +faster method for atomic appends is as follows: + +On upgrade, add a trailing NUL to all existing journal files. + +When writing to a journal file, use a temp file that is renamed into place +atomically like now, but add a trailing NUL to it. + +When appending to a journal file first read the last byte. +If it's a trailing NUL, write the current size of the journal file to the size +file. Then rewind back before the NUL and append to there, followed +by a trailing NUL. Then delete the size file. + +On append, if the last byte is not a trailing NUL, read the size file, +and seek to that size, so discarding a previous partial append. Then +proceed to append to it, write the trailing NUL, and delete the size file. + +When reading from a journal file, discard the trailing NUL. If there is no +trailing NUL, a partial append has been detected. Get the size from the +size file, and read only that much of the journal file, to discard the +partial append. + +Reading may not need the journal lock, if it can otherwise recover from +races with append. Treating an append that is still in progress as an +interrupted append is fine for the purposes of reading. And +when there's no trailing NUL and it reads the size +file, the append may have finished and so deleted the size file. If so, +retry the read of the journal file, and look again for a trailing NUL. +"""]]