importfeed: Fix a crash when used in a non-unicode locale
authorJoey Hess <joeyh@joeyh.name>
Mon, 15 Nov 2021 17:32:31 +0000 (13:32 -0400)
committerJoey Hess <joeyh@joeyh.name>
Mon, 15 Nov 2021 19:04:21 +0000 (15:04 -0400)
commit2bd778a46ed071c2dd534ebe3b7007b7ae60d1c1
treef33453ac62c49e1cf514a631721cc0692424aea8
parentb12f7f84f793d50d46a59c92295a8dadeb899544
importfeed: Fix a crash when used in a non-unicode locale

See comment for analysis.

At first I thought I'd need to convert all T.unpack in git-annex, but
luckily not -- so long as the Text is read from a file, the filesystem
encoding is applied and T.unpack is fine. It's only when using Feed
that the filesystem encoding is not applied.

While this fixes the crash, it does result in some mojibake, eg:
itemid=http://www.manager-tools.com/2014/01/choosing-a-company-work-chapter-7-���-questions/

Have not tracked that down, but it must be unrelated, because
I've verified that it roundtrips when using encodeUf8:

joey@darkstar:~/src/git-annex>LANG=C ghci  Utility/FileSystemEncoding.hs
ghci> useFileSystemEncoding
ghci> Just f <- Text.Feed.Import.parseFeedFromFile "/home/joey/tmp/career_tools_podcasts.xml"
ghci> Just (_, x) = Text.Feed.Query.getItemId (Text.Feed.Query.feedItems f !! 0)
ghci> decodeBS (Data.Text.Encoding.encodeUtf8 x)
"http://www.manager-tools.com/2014/01/choosing-a-company-work-chapter-7-\56546\56448\56467-questions/"
ghci> writeFile "foo" $ decodeBS (Data.Text.Encoding.encodeUtf8 x)
Writes a file containing the ENDASH character.

Sponsored-by: Jochen Bartl on Patreon
CHANGELOG
Command/ImportFeed.hs
doc/bugs/importfeed_failing_with_hPut__58___invalid_argument.mdwn
doc/bugs/importfeed_failing_with_hPut__58___invalid_argument/comment_1_a36feb24e627bba9c146fc791fca2a6b._comment [new file with mode: 0644]