From: Joey Hess Date: Thu, 31 Jul 2025 19:20:28 +0000 (-0400) Subject: update design doc with changes from genericp2p branch X-Git-Tag: archive/raspbian/10.20251029-1+rpi1~1^2~3^2~236^2~1 X-Git-Url: https://dgit.raspbian.org/?a=commitdiff_plain;h=59bcfc838e25ae7de7fadf031d76385899860881;p=git-annex.git update design doc with changes from genericp2p branch That branch is basically ready to merge, but needs more testing in a chicken and egg situation. --- diff --git a/doc/design/generic_p2p_transport.mdwn b/doc/design/generic_p2p_transport.mdwn new file mode 100644 index 0000000000..18e966bff8 --- /dev/null +++ b/doc/design/generic_p2p_transport.mdwn @@ -0,0 +1,42 @@ +This is a generic interface that allows git-annex to use a P2P network. +The [[P2P_protocol]] is used, to allow accessing a peer's git-annex +repository as a git remote. + +Examples of such networks are tor, yggstack or fowl. (git-annex has a +built-in integration with tor which does not use this interface.) + +Such a P2P network has some form of address, which can be +used to connect to a given peer by address across the network. + +A git remote using the P2P network has an url of the form +`p2p-annex:::
` + +To connect to that remote, git-annex runs the command +`git-annex-p2p-`, giving it the P2P network address as its only +parameter. The command is responsible for connecting to that peer, and +relaying data to it. Data fed into the command on stdin should be sent to +the peer, and data received from the peer should be output to stdout. If it +is unable to connect, the command can exit nonzero. When the peer closes +connection, the command can exit zero. + +To configure `git-annex remotedaemon` to listen on a given P2P network, +the user runs `git-annex p2p --enable `. That also +runs `git-annex-p2p-`, this time with the parameter "address". +That should output one or more lines, the P2P network address (or addresses) +that can be used by peers to connect to the repository. It can first do +whatever it needs to do to set up the P2P network. + +To handle incoming connections from peers, `git-annex remotedaemon` runs +`git-annex-p2p-`, with two parameters. The first parameter is the +P2P address of the local repository, obtained earlier as described above. +The second parameter is the path to a unix socket file, which git-annex +will have already created. git-annex listens for connections from peers +that are made to the socket, and services them. + +Note that, if the P2P network does not natively use a unix socket file, +a command like `socat` can be run by `git-annex-p2p- socket` +to convert the P2P network's own equivilant into a unix socket file. + +The program [[git-remote-p2p-annex]] is included in git-annex as a git +remote helper program. git will use that program to handle `pull` and +`push` with git remotes that use the `p2p-annex::` url scheme. diff --git a/doc/design/p2p_socket_transport.mdwn b/doc/design/p2p_socket_transport.mdwn deleted file mode 100644 index 725acb8b5e..0000000000 --- a/doc/design/p2p_socket_transport.mdwn +++ /dev/null @@ -1,40 +0,0 @@ -This is a generic interface that allows git-annex to use a P2P network. - -Examples of such networks are tor, yggstack or fowl. (git-annex has a -built-in integration with tor which does not use this interface.) - -Such a P2P network has some form of address, which can be -used to connect to a given peer by address across the network. - -A git remote using the P2P network has an url of the form -`p2p-annex::+
` - -To connect to that remote, git-annex runs the command -`git-annex-p2p-`, giving it the P2P network address as its only -parameter. The command is responsible for connecting to that peer, and -relaying data to it. Data fed into the command on stdin should be sent to -the peer, and data received from the peer should be output to stdout. If it -is unable to connect, the command can exit nonzero. When the peer closes -connection, the command can exit zero. - -To handle incoming connections from peers, `git-annex remotedaemon` -runs `git-annex-p2p-` with the parameter "socket", followed -by the P2P address of the local repository. The command -should output the path of a unix socket file. When it does, `git-annex -remotedaemon` will use that socket file to listen for connections from -peers, and service them. (The [[P2P_protocol]] is spoken over these -connections.) - -Note that, if the P2P network does not natively use a unix socket file, -a command like `socat` can be run by `git-annex-p2p- socket` -to convert the P2P network's own equivilant into a unix socket file. - -To configure `git-annex remotedaemon` to listen on a given P2P network, -the user runs `git-annex p2p --enable `. That also -runs `git-annex-p2p-`, this time with the parameter "address". -That should output the P2P network address that can be used by peers -to connect to the repository. - -The program [[git-remote-p2p-annex]] is included in git-annex as a git -remote helper program. git will use that program to handle `pull` and -`push` with git remotes that use the `p2p-annex::` url scheme. diff --git a/doc/todo/generic_p2p_socket_transport/comment_12_c106b0a8011cc5f66e35894e9899c2ab._comment b/doc/todo/generic_p2p_socket_transport/comment_12_c106b0a8011cc5f66e35894e9899c2ab._comment index b6b2203482..941ebe62c1 100644 --- a/doc/todo/generic_p2p_socket_transport/comment_12_c106b0a8011cc5f66e35894e9899c2ab._comment +++ b/doc/todo/generic_p2p_socket_transport/comment_12_c106b0a8011cc5f66e35894e9899c2ab._comment @@ -3,7 +3,7 @@ subject="""comment 12""" date="2025-07-29T16:41:07Z" content=""" -I have started a design document at [[design/p2p_socket_transport]], +I have started a design document at [[design/generic_p2p_transport]], to collect all the scattered decisions here into a coherent document that can be used by someone implementing support for one of these networks. diff --git a/doc/todo/generic_p2p_socket_transport/comment_13_b415aa7cd50892c562eba9de2f4a47e9._comment b/doc/todo/generic_p2p_socket_transport/comment_13_b415aa7cd50892c562eba9de2f4a47e9._comment new file mode 100644 index 0000000000..38c9fbb62a --- /dev/null +++ b/doc/todo/generic_p2p_socket_transport/comment_13_b415aa7cd50892c562eba9de2f4a47e9._comment @@ -0,0 +1,14 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 13""" + date="2025-07-31T19:16:43Z" + content=""" +The `genericp2p` branch now has what seems to be a fully working +implementation of this. I have not fully tested it because I don't +have a real git-annex-p2p-foo command to test it with. Still I can see the +remotedaemon handling connections, and also making outgoing connections +seems to work. + +There have been some changes to the design, especially around how the +socket file is set up. +"""]]