© 2014 Sören Brunk
License: AGPL-3+
-Files: doc/special_remotes/external/*
+Files: doc/special_remotes/external/* doc/special_remotes/p2p/git-annex-p2p-unix-sockets
Copyright: © 2013 Joey Hess <id@joeyh.name>
License: GPL-3+
A git remote using the P2P network has an url of the form
`p2p-annex::<netname>:<address>`
-To connect to that remote, git-annex runs the command
+The program [[git-remote-p2p-annex]] is included in git-annex as a git
+remote helper program. git will use that program to handle `pull` and
+`push` with git remotes that use the `p2p-annex::` url scheme.
+
+## program interface
+
+To connect to a P2P remote, git-annex runs the command
`git-annex-p2p-<netname>`, giving it the P2P network address as its only
parameter. The command is responsible for connecting to that peer, and
relaying data to it. Data fed into the command on stdin should be sent to
a command like `socat` can be run by `git-annex-p2p-<netname> socket`
to convert the P2P network's own equivilant into a unix socket file.
-The program [[git-remote-p2p-annex]] is included in git-annex as a git
-remote helper program. git will use that program to handle `pull` and
-`push` with git remotes that use the `p2p-annex::` url scheme.
+## example
+
+Here's a simple shell script example. While this avoids using any real
+P2P network and dummys it up by symlinking unix socket files together,
+its skeleton should be a good starting point.
+
+[[!inline pages="special_remotes/p2p/git-annex-p2p-unix-sockets" feeds=no]]
* [[git]]
* [[hook]]
* [[httpalso]]
+* [[p2p]]
* [[rclone]]
* [[rsync]]
* [[S3]] (Amazon S3, and other compatible services)
-There are three ways to implement a new special remote:
+There are four ways to implement a new special remote:
1. Using the [[hook]] special remote to tell git-annex what commands
to run to store and retrieve data. This is the easiest way, and
--- /dev/null
+A P2P network can be used to connect together git-annex repositories. This
+lets a regular git remote have an url that points to another peer on the
+network. Both git fetch/push and git-annex can be used with that remote
+the same as any other remote.
+
+The [[tor]] support is a special case of this, which is built into
+git-annex.
+
+For other P2P networks, a fairly simple program is used to connect
+git-annex up with the network. Install one of these programs to use the P2P
+network of your choice:
+
+* [[git-annex-p2p-unix-sockets]] This is only a demo, using unix
+ sockets in `/tmp` rather than a real P2P network. Not for real world
+ use.
+
+To write your own program to use the P2P network of your choice,
+see [[design/generic_p2p_transport]]. Edit this page to add more programs!
+
+## setup
+
+Once you have the program installed, the next step is to run this in your
+git-annex repository to enable the P2P network:
+
+ git-annex p2p --enable <netname>
+
+Replace `<netname>` with the name of the program after the
+"git-annex-p2p-".
+
+Then [[git-annex remotedaemon|git-annex-remotedaemon]] can be used to serve
+incoming connections from peers, and [[git-annex p2p|git-annex-p2p]] can be
+used to set up connections to peers on the network. For example, you and a
+friend could run these commands in your repositories to pair them:
+
+ git-annex remotedaemon
+ git-annex p2p --pair
+
+Once a connection with a peer is set up, you have a git remote that can be
+used like any other remote. Including `git pull`, `git push`, and using
+git-annex commands to store content on it, etc.
+
+## security
+
+This is only as secure as the underlying P2P network, and it should only be
+used with P2P networks that at least encrypt all traffic sent over them.
+
+It's a good idea, but not mandatory, for the P2P network to
+cryptographically verify the identity of peers. Any modern encrypted P2P
+network should do that. That prevents you from connecting to an
+impersonator, and perhaps leaking data from your repository to them.
+However, even if the P2P network does not verify the identity of peers,
+git-annex only allows people you have paired with to connect to your
+repository.
+
+Anyone you give access to your git-annex repository using this can get any
+of the files stored in it, and can also drop the content of any annexed file
+from it.
--- /dev/null
+#!/bin/sh
+# Example P2P network transit program for git-annex.
+#
+# This simulates a multi-node P2P network using unix
+# socket files in /tmp.
+#
+# Copyright 2025 Joey Hess; icenced under the GNU GPL version 3 or higher.
+
+set -e
+
+if [ "$1" = address ]; then
+ # Output the local P2P network address.
+ #
+ # For the purposes of this example, a new address is made up each
+ # time this is run. Using the current unix second to get a fairly
+ # unique address.
+ myaddress=$(date +%s)
+ echo "$myaddress"
+else
+ socketfile="$2"
+ if [ -z "$socketfile" ]; then
+ # Connect to the peer's address and relay stdin and stdout.
+ peeraddress="$1"
+ # For the purposes of this demo, socat is used, and simply
+ # connects stdio to the unix socket file in /tmp.
+ socat - UNIX-CONNECT:"/tmp/$peeraddress"
+ else
+ # Arrange for incoming connections from peers to connect to
+ # the unix socket provided by git-annex. The local
+ # P2P network address is also available to use.
+ myaddress="$1"
+ # For the purposes of this demo, the socket file provided
+ # by git-annex is symlinked to the location in /tmp.
+ ln -sf $(realpath "$socketfile") "/tmp/$myaddress"
+ fi
+fi
What do you think?
[[!tag projects/INM7]]
+
+> [[done]] --[[Joey]]
--- /dev/null
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 14"""
+ date="2025-08-01T17:41:53Z"
+ content="""
+This is merged!
+
+See [[special_remotes/p2p]] for the end-user documentation,
+including a list where you can add whatever scripts you implement for
+different P2P networks.
+"""]]
projects / git-annex.git / commitdiff