From: Joey Hess Date: Thu, 4 Sep 2025 17:01:59 +0000 (-0400) Subject: avoid relatedTemplate ever returning "" X-Git-Tag: archive/raspbian/10.20251029-1+rpi1~1^2~3^2~157 X-Git-Url: https://dgit.raspbian.org/?a=commitdiff_plain;h=67f00027d1b326c979db8b81c973a61234c406d7;p=git-annex.git avoid relatedTemplate ever returning "" add: Fix crash adding filenames that are exactly 21 bytes long and begin with a utf-8 character. Also longer filenames that start with "....." would cause the same crash. I also audited for other calls to truncateFilePath that could truncate it to "". Most use pathmax so are not a problem. Backend.Utilities.genKeyName could possibly truncate it like that, but appends the md5 so would not be a problem either. Sponsored-by: Kevin Mueller --- diff --git a/CHANGELOG b/CHANGELOG index dfd780321a..978e86ec55 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -1,6 +1,8 @@ git-annex (10.20250829) UNRELEASED; urgency=medium * drop: --fast support when dropping from a remote. + * add: Fix crash adding filenames that are exactly 21 bytes long and + begin with a utf-8 character. -- Joey Hess Fri, 29 Aug 2025 12:34:06 -0400 diff --git a/Utility/FileSystemEncoding.hs b/Utility/FileSystemEncoding.hs index d66d8a008c..2fb726f1fc 100644 --- a/Utility/FileSystemEncoding.hs +++ b/Utility/FileSystemEncoding.hs @@ -119,11 +119,14 @@ fromRawFilePath = decodeBS toRawFilePath :: FilePath -> RawFilePath toRawFilePath = encodeBS -{- Truncates a FilePath to the given number of bytes (or less), +{- Truncates a path to the given number of bytes (or less), - as represented on disk. - - Avoids returning an invalid part of a unicode byte sequence, at the - cost of efficiency when running on a large FilePath. + - + - Note that this may return ""! That can happen if it is asked to truncate + - to eg 1 byte, but the input path starts with a unicode byte sequence. -} truncateFilePath :: Int -> RawFilePath -> RawFilePath #ifndef mingw32_HOST_OS diff --git a/Utility/Tmp.hs b/Utility/Tmp.hs index f373ca6c1c..c47cdfcb0b 100644 --- a/Utility/Tmp.hs +++ b/Utility/Tmp.hs @@ -120,8 +120,11 @@ relatedTemplate' f - ending in ".", and others like VFAT don't allow a - filename to end with trailing whitespace, so avoid - truncating a filename to end that way. -} - B.dropWhileEnd disallowed $ + let p = B.dropWhileEnd disallowed $ truncateFilePath (len - templateAddedLength) f + in if B.null p + then "t" + else p | otherwise = f where len = B.length f diff --git a/doc/bugs/git-annex_add__47__unlock_fails_for_some_names.mdwn b/doc/bugs/git-annex_add__47__unlock_fails_for_some_names.mdwn index 0dce8ad00c..59823cfbbd 100644 --- a/doc/bugs/git-annex_add__47__unlock_fails_for_some_names.mdwn +++ b/doc/bugs/git-annex_add__47__unlock_fails_for_some_names.mdwn @@ -93,3 +93,5 @@ Yes, git-annex has been fantastic for managing large datasets across multiple ma --- This issue appears to affect **all Cyrillic filenames**, not just the initially identified patterns, making the current version of git-annex barely usable for repositories containing non-Latin filenames. + +> [[fixed|done]] --[[Joey]] diff --git a/doc/bugs/git-annex_add__47__unlock_fails_for_some_names/comment_1_af33dfae3ccbc24f84c84612337b98bc._comment b/doc/bugs/git-annex_add__47__unlock_fails_for_some_names/comment_1_af33dfae3ccbc24f84c84612337b98bc._comment new file mode 100644 index 0000000000..cd36ba1158 --- /dev/null +++ b/doc/bugs/git-annex_add__47__unlock_fails_for_some_names/comment_1_af33dfae3ccbc24f84c84612337b98bc._comment @@ -0,0 +1,27 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2025-09-04T16:00:49Z" + content=""" +Reproduced. Thank you for an excellent bug report. + +And it is the temp filename generation causing the problem. + + mkdir(".git/annex/othertmp/.0", 0777) = 0 + unlink(".git/annex/othertmp/.0") = -1 EISDIR (Is a directory) + symlink(".git/annex/objects/k8/wf/SHA256E-s3--98ea6e4f216f2fb4b69fff9b3a44842c38686ca685f3f55dc48c5d3fb1107be4.md/SHA256E-s3--98ea6e4f216f2fb4b69fff9b3a44842c38686ca685f3f55dc48c5d3fb1107be4.md", ".git/annex/othertmp/.0") = -1 EEXIST (File exists) + +The cause is that relatedTemplate is returning "", which is not something the code +is prepared for. That results in the ".0" directory name, and `".0" "" == ".0"` +so it uses the same path for the temp file as for the subdirectory. + +Not all cyrllic names are affected though. Only ones that are exactly 21 +bytes long. Longer or shorter are both ok. + +The reason is that relatedTemplate wants to reserve 20 bytes for the random +part of the temp filename. With a 21 byte filename, that means it wants to +truncate it to 1 byte. But it that lands in the middle of the first unicode +character, which is not allowed, so it truncates it to 0 bytes instead. + +I've fixed this bug. +"""]]