From: Joey Hess <joeyh@joeyh.name>
Date: Fri, 2 Feb 2024 19:50:41 +0000 (-0400)
Subject: compare urls irrespective of downloader
X-Git-Tag: archive/raspbian/10.20250416-2+rpi1~1^2~25^2~204
X-Git-Url: https://dgit.raspbian.org/?a=commitdiff_plain;h=0c64cd30c246ee5fd0d24a837cafeb4d1ab3a3fb;p=git-annex.git

compare urls irrespective of downloader

importfeed --force: Avoid creating duplicates of existing already
downloaded files when yt-dlp or a special remote was used.
---

diff --git a/CHANGELOG b/CHANGELOG
index 6eca06eba1..d0f759cc6a 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -2,6 +2,8 @@ git-annex (10.20240130) UNRELEASED; urgency=medium
 
   * importfeed: Added --scrape option, which uses yt-dlp to screen scrape
     the equivilant of an RSS feed.
+  * importfeed --force: Avoid creating duplicates of existing
+    already downloaded files when yt-dlp or a special remote was used.
 
  -- Joey Hess <id@joeyh.name>  Mon, 29 Jan 2024 15:59:33 -0400
 
diff --git a/Command/ImportFeed.hs b/Command/ImportFeed.hs
index e4e78818ce..dbc8cb8bf0 100644
--- a/Command/ImportFeed.hs
+++ b/Command/ImportFeed.hs
@@ -462,7 +462,7 @@ runDownload todownload url extension cache cv getter = do
 				in d </> show n ++ "_" ++ base
 		tryanother = makeunique (n + 1) file
 		alreadyexists = liftIO $ isJust <$> catchMaybeIO (R.getSymbolicLinkStatus (toRawFilePath f))
-		checksameurl k = ifM (elem url <$> getUrls k)
+		checksameurl k = ifM (elem url . map fst . map getDownloader <$> getUrls k)
 			( return Nothing
 			, tryanother
 			)
diff --git a/doc/forum/importfeed_on_multiple___34__overlapping__34___playlists/comment_4_0bf6d5b03032c0b9fd35c8410aea832c._comment b/doc/forum/importfeed_on_multiple___34__overlapping__34___playlists/comment_4_0bf6d5b03032c0b9fd35c8410aea832c._comment
new file mode 100644
index 0000000000..2b79698131
--- /dev/null
+++ b/doc/forum/importfeed_on_multiple___34__overlapping__34___playlists/comment_4_0bf6d5b03032c0b9fd35c8410aea832c._comment
@@ -0,0 +1,16 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 4"""
+ date="2024-02-02T19:27:41Z"
+ content="""
+Oh, but it seems that `importfeed --force` already has code to handle this.
+It checks if the file already exists and has the same url recorded as the url
+being imported. For some reason in that case, it says it failed to import
+the url, which seems a bit odd behavior to me, but in any case it doesn't
+add a "2_" file.
+
+That works for regular rss feeds, but does not work for yt-dlp urls.
+
+The reason is the url mangling done for yt-dlp urls (and other special
+remote urls). Fixed this bug.
+"""]]