metadata --batch: Avoid crashing when a non-annexed file is input

Turns out that CommandStart actions do not have their exceptions caught,
which is why the giveup was causing a crash. Mostly these actions
do not do very much work on their own, but it does seem possible there
are other commands whose CommandStart also throws an exception.

So, my first attempt at a fix was to catch those exceptions. But,
--json-error-messages then causes a difficulty, because in order to output
a json error message, an action needs to have been started; that sets up
the json object that the error message will be included in a field of.

While it would be possible to output an object with just an error field,
this would be json output of a format that the user has no reason to
expect, that happens only in an exceptional circumstance. That is something
I have always wanted to avoid with the json output; while git-annex man
pages don't document what the json looks like, the output has always
been made to be self-describing. Eg, it includes "error-messages":[]
even when there's no errors.

With that ruled out, it doesn't seem a good idea to catch CommandStart
exceptions and display the error to stderr when --json-error-messages
is set. And so I don't know if it makes sense to catch exceptions from that
at all. Maybe I'd have a different opinion if --json-error-messages did not
exist though.

So instead, output a blank line like other batch commands do.
This also leaves open the possibility of implementing support for matching
object with metadata --json, which would also want to output a blank line
when the input didn't match.

Sponsored-by: Dartmouth College's DANDI project

diff --git a/CHANGELOG b/CHANGELOG
index 1b89b6e94adc3c6970d33a4267707458d8597330..732bbc0baa23c0ba56fbaea79cc6518ef0e0b1c5 100644 (file)
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -1,3 +1,10 @@
+git-annex (8.20211029) UNRELEASED; urgency=medium
+
+  * metadata --batch: Avoid crashing when a non-annexed file is input,
+    instead output a blank line like other batch commands do.
+
+ -- Joey Hess <id@joeyh.name>  Mon, 01 Nov 2021 13:19:46 -0400
+
 git-annex (8.20211028) upstream; urgency=medium
 
   * Removed support for accessing git remotes that use versions of

diff --git a/Command/MetaData.hs b/Command/MetaData.hs
index f2fd87ffbfa4cec8c73b03c5ea6818122a7cd0b4..08359bf8252120cf9fde5844a8708ed260aedd7b 100644 (file)
--- a/Command/MetaData.hs
+++ b/Command/MetaData.hs
@@ -94,7 +94,7 @@ seek o = case batchOption o of
                JSONOutput _ -> ifM limited
                        ( giveup "combining --batch with file matching options is not currently supported"
                        , batchInput fmt parseJSONInput 
-                               (commandAction . startBatch)
+                               (commandAction . batchCommandStart . startBatch)
                        )
                _ -> giveup "--batch is currently only supported in --json mode"
 
@@ -176,7 +176,7 @@ startBatch (si, (i, (MetaData m))) = case i of
                mk <- lookupKey f
                case mk of
                        Just k -> go k (mkActionItem (k, AssociatedFile (Just f)))
-                       Nothing -> giveup $ "not an annexed file: " ++ fromRawFilePath f
+                       Nothing -> return Nothing
        Right k -> go k (mkActionItem k)
   where
        go k ai = starting "metadata" ai si $ do

diff --git a/doc/bugs/metadata_cmd._vs._--json-error-messages.mdwn b/doc/bugs/metadata_cmd._vs._--json-error-messages.mdwn
index 9f5524636c381cc4520424f2a559c7ee1c67834d..7e38c2f026d9397873a120e421f3d95125f3bef1 100644 (file)
--- a/doc/bugs/metadata_cmd._vs._--json-error-messages.mdwn
+++ b/doc/bugs/metadata_cmd._vs._--json-error-messages.mdwn
@@ -4,3 +4,5 @@ When invoking `git-annex metadata --batch --json --json-error-messages`, if an e
 
 [[!meta author=jwodder]]
 [[!tag projects/dandi]]
+
+> [[fixed|done]] --[[Joey]]

diff --git a/doc/bugs/metadata_cmd._vs._--json-error-messages/comment_1_198df45c72d0c9b4997d1342ffb666ca._comment b/doc/bugs/metadata_cmd._vs._--json-error-messages/comment_1_198df45c72d0c9b4997d1342ffb666ca._comment
new file mode 100644 (file)
index 0000000..faa240a
--- /dev/null
+++ b/doc/bugs/metadata_cmd._vs._--json-error-messages/comment_1_198df45c72d0c9b4997d1342ffb666ca._comment
@@ -0,0 +1,13 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2021-11-01T16:27:48Z"
+ content="""
+For consistency with other --batch, I've made it reply with a blank line
+when the input is not an annexed file.
+
+Do note that --json-error-messages cannot cram every possible kind of error
+message into a json object. In particular, errors that occur at startup,
+and not when acting on a particular file or key, do not fit into the json
+schema.
+"""]]

diff --git a/doc/git-annex-metadata.mdwn b/doc/git-annex-metadata.mdwn
index e5b8b1948ce2d1d90e34e67af042a92a38d85f1d..1f1877d33089745f685c6fcf03af243b263793cd 100644 (file)
--- a/doc/git-annex-metadata.mdwn
+++ b/doc/git-annex-metadata.mdwn
@@ -133,7 +133,9 @@ the modified file.
   enable `--json` along with `--batch`.
 
   In batch mode, git-annex reads lines from stdin, which contain
-  JSON objects. It replies to each input with an output JSON object.
+  JSON objects. It replies to each input annexed file
+  with an output JSON object. (But if the file is not an annexed file,
+  an empty line will be output.)
 
   The format of the JSON sent to git-annex can be the same as the JSON that
   it outputs. Or, a simplified version. Only the "file" (or "key") field
@@ -159,9 +161,6 @@ the modified file.
 
        {"file":"foo","fields":{"author":[]}}
 
-  Note that matching options do not affect the files that are
-  processed when in batch mode.
-
 * Also the [[git-annex-common-options]](1) can be used.
 
 # EXAMPLES