From daebd50ee3d47829d6b1202bd6f74f650dee432e Mon Sep 17 00:00:00 2001 From: "beryllium@5bc3c32eb8156390f96e363e4ba38976567425ec" Date: Tue, 17 Oct 2023 09:41:12 +0000 Subject: [PATCH] --- ..._with_suffixes_differing_only_in_case.mdwn | 27 +++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100644 doc/forum/duplicates_with_suffixes_differing_only_in_case.mdwn diff --git a/doc/forum/duplicates_with_suffixes_differing_only_in_case.mdwn b/doc/forum/duplicates_with_suffixes_differing_only_in_case.mdwn new file mode 100644 index 0000000000..4084c7e7b2 --- /dev/null +++ b/doc/forum/duplicates_with_suffixes_differing_only_in_case.mdwn @@ -0,0 +1,27 @@ +I am using Google Cloud as a special remote to backup photos in one annex. + +It's in my interests, due to storage costs, to keep duplicates to a minimum. I've noticed that the cloud bucket has a lot more files than I would have expected. + +I performed the following to get the number of files in the bucket: + +[[!format sh """ +$ gsutil ls gs://cloud-/ | wc -l + +"""]] + +and of course, in the git-annex: + +[[!format sh """ +$ find . -path .git -o \( -type f -print \) | wc -l + +"""]] + +When I took a sample of the list, I found may of the files that shared the exact key, but had, for example. both .jpg and .JPG suffixes. + +Some of these files, when checked with git-annex whereused --key=, some were files I had maybe re-arranged in folders. I'm running a script at the moment to get a wider sample of what's happening with these duplicates. + +One of the problems I speculate may be coming into play is that I did originally copy files to the cloud remote via an older, Windows git-annex version (repo version I think at 5?). + +I think over time interoperability issues have ironed out, but this might be a remnant of when these were more prolific.... though I do still have one that I will report separately once I've gather more data. + +Any advise on how I might deal with the duplicates, get them deleted out of the bucket? -- 2.30.2