From 9ce207532e06a9b844f0d066629c81ff627bceb3 Mon Sep 17 00:00:00 2001 From: "benjamin.poldrack@d09ccff6d42dd20277610b59867cf7462927b8e3" Date: Thu, 11 Jul 2024 07:23:30 +0000 Subject: [PATCH] --- ...copy__95__file__95__range_for_get_and_copy.mdwn | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 doc/todo/use_copy__95__file__95__range_for_get_and_copy.mdwn diff --git a/doc/todo/use_copy__95__file__95__range_for_get_and_copy.mdwn b/doc/todo/use_copy__95__file__95__range_for_get_and_copy.mdwn new file mode 100644 index 0000000000..1e056135df --- /dev/null +++ b/doc/todo/use_copy__95__file__95__range_for_get_and_copy.mdwn @@ -0,0 +1,14 @@ +I was looking into why `annex get` and `annex copy` between local clones on NFS mounts don't utilize NFS4.2's server-side copy feature, +which would be pretty relevant for a setup like ours (institutional compute cluster; big datalad datasets). +This seems to boil down to not calling `copy_file_range`. However, `cp` generally does call `copy_file_range`, so that seemed confusing. +Turns out the culprit is `--reflink=always` which does not work as expected on ZFS. `--reflink=auto` does, though. +A summary of how that is, can be found here: https://github.com/openzfs/zfs/pull/13392#issuecomment-1742172842 + +I am not sure why annex insists on `always` rather than `auto`, so not sure whether the solution actually would be to change that. +Reading old issues it seems the reason was to let annex handle the fallback, which kinda is the problem in case of ZFS. +Changing (back?) to `auto` may be an issue in other cases - I don't know. If annex' fallback when `cp --reflink=always` fails would end up calling `copy_file_range`, that would still solve the issue, though, as NFS would then end up performing a server-side copy rather than transferring big files back and forth. + +Just to be clear: It's specifically ZFS via NFS on linux that's the issue here. + + +P.S.: Didn't want to call this a bug, mostly b/c the "real bug" isn't in annex exactly (see link above), so putting it here. -- 2.30.2