p2p protocol version 4 for DATA-PRESENT

diff --git a/doc/design/p2p_protocol.mdwn b/doc/design/p2p_protocol.mdwn
index 5b15142a521f7a6277741d282fef612e6a39b7d4..d9cf75a02f3ff79adce65f47f9326d5705db0dc3 100644 (file)
--- a/doc/design/p2p_protocol.mdwn
+++ b/doc/design/p2p_protocol.mdwn
@@ -56,7 +56,7 @@ any authentication.
 
 The client sends the highest protocol version it supports:
 
-       VERSION 3
+       VERSION 4
 
 The server responds with the highest protocol version it supports
 that is less than or equal to the version the client sent:
@@ -190,7 +190,8 @@ ALREADY-HAVE-PLUS. The subsequent list of UUIDs are additional
 UUIDs where the content is stored, in addition to the UUID where
 the client was going to send it.
 
-Otherwise, it responds with:
+When the server wants the client to send the content to it, 
+it replies with:
 
        PUT-FROM Offset
 
@@ -200,7 +201,7 @@ the client to start. This allows resuming transfers.
 The client then sends a DATA message with content of the file from
 the offset to the end of file.
 
-In protocol version 1 and above, after the data, the client sends an
+In protocol version 1 and above, after the DATA, the client sends an
 additional message, to indicate if the content of the file has changed
 while it was being sent.
 
@@ -213,6 +214,18 @@ it replies with SUCCESS. Otherwise, FAILURE.
 In protocol version 2 and above, the server can optionally reply with
 SUCCESS-PLUS and a list of UUIDs where the content was stored.
 
+In protocol version 4 and above, rather than sending a DATA message,
+the client can send the content to the repository in some other way
+than using the protocol. To indicate that it has sent the content
+like that, the client sends:
+
+    DATA-PRESENT
+
+The server responds to DATA-PRESENT by checking if the content is
+present in the repository. If so it proceeds the same as if the client
+had used DATA to send the data and then sent VALID. Eg, it responds with
+SUCCESS (or FAILURE if it cannot verify that the data is present).
+
 ## Getting content from the server
 
 To get content from the server, the client sends:

diff --git a/doc/design/p2p_protocol_over_http.mdwn b/doc/design/p2p_protocol_over_http.mdwn
index 1c5ff28a61fa14354c7b57a48e7befa2efb591ec..0e49782fa0dc55ad5e46fb6d3b47bc74dade5e87 100644 (file)
--- a/doc/design/p2p_protocol_over_http.mdwn
+++ b/doc/design/p2p_protocol_over_http.mdwn
@@ -49,7 +49,7 @@ over HTTPS.
 
 Requests are versioned. The versions correspond to
 P2P protocol versions. The version is part of the request path,
-eg "v3"
+eg "v4"
 
 If the server does not support a particular version, the
 request will fail with a 404, and the client should fall
@@ -99,7 +99,7 @@ with 404 Not Found.
 Note that the common parameters bypass and clientuuid, while
 accepted, have no effect. Both are optional for this request.
 
-### GET /git-annex/$uuid/v3/key/$key
+### GET /git-annex/$uuid/v4/key/$key
 
 Get the content of a key from the repository with the specified uuid.
 
@@ -144,20 +144,24 @@ an unlocked annexed file, which got modified while it was being sent.
 When the key is not present on the server, it will respond
 with 404 Not Found.
 
+### GET /git-annex/$uuid/v3/key/$key
+
+Identical to v4.
+
 ### GET /git-annex/$uuid/v2/key/$key
 
-Identical to v3.
+Identical to v4.
 
 ### GET /git-annex/$uuid/v1/key/$key
 
-Identical to v3.
+Identical to v4.
 
 ### GET /git-annex/$uuid/v0/key/$key
 
-Same as v3, except the X-git-annex-data-length header is not used.
+Same as v4, except the X-git-annex-data-length header is not used.
 Additional checking client-side will be required to validate the data.
 
-### POST /git-annex/$uuid/v3/checkpresent
+### POST /git-annex/$uuid/v4/checkpresent
 
 Checks if a key is currently present on the server.
 
@@ -173,19 +177,23 @@ The body of the request is empty.
 The server responds with a JSON object with a "present" field that is true
 if the key is present, or false if it is not present.
 
+### POST /git-annex/$uuid/v3/checkpresent
+
+Identical to v4.
+
 ### POST /git-annex/$uuid/v2/checkpresent
 
-Identical to v3.
+Identical to v4.
 
 ### POST /git-annex/$uuid/v1/checkpresent
 
-Identical to v3.
+Identical to v4.
 
 ### POST /git-annex/$uuid/v0/checkpresent
 
-Identical to v3.
+Identical to v4.
 
-### POST /git-annex/$uuid/v3/lockcontent
+### POST /git-annex/$uuid/v4/lockcontent
 
 Locks the content of a key on the server, preventing it from being removed.
 
@@ -203,19 +211,23 @@ The key will remain locked for 10 minutes. But, usually `keeplocked`
 is used to control the lifetime of the lock, using the "lockid"
 parameter from the server's reply. (See below.)
 
+### POST /git-annex/$uuid/v3/lockcontent
+
+Identical to v4.
+
 ### POST /git-annex/$uuid/v2/lockcontent
 
-Identical to v3.
+Identical to v4.
 
 ### POST /git-annex/$uuid/v1/lockcontent
 
-Identical to v3.
+Identical to v4.
 
 ### POST /git-annex/$uuid/v0/lockcontent
 
-Identical to v3.
+Identical to v4.
 
-### POST /git-annex/$uuid/v3/keeplocked
+### POST /git-annex/$uuid/v4/keeplocked
 
 Controls the lifetime of a lock on a key that was earlier obtained
 with `lockcontent`.
@@ -252,21 +264,25 @@ locked for 10 minutes from the time it was first locked.
 Note that the common parameters bypass and clientuuid, while
 accepted, have no effect.
 
+### POST /git-annex/$uuid/v3/keeplocked
+
+Identical to v4.
+
 ### POST /git-annex/$uuid/v2/keeplocked
 
-Identical to v3.
+Identical to v4.
 
 ### POST /git-annex/$uuid/v1/keeplocked
 
-Identical to v3.
+Identical to v4.
 
 ### POST /git-annex/$uuid/v0/keeplocked
 
-Identical to v3.
+Identical to v4.
 
-### POST /git-annex/$uuid/v3/remove
+### POST /git-annex/$uuid/v4/remove
 
-Remove a key's content from the server.
+Remove a key's content from the repository.
 
 Example:
 
@@ -284,21 +300,25 @@ or false if the key was not able to be removed.
 The JSON object can have an additional field "plusuuids" that is a list of
 UUIDs of other repositories that the content was removed from.
 
+### POST /git-annex/$uuid/v3/remove
+
+Identical to v4.
+
 ### POST /git-annex/$uuid/v2/remove
 
-Identical to v3.
+Identical to v4.
 
 ### POST /git-annex/$uuid/v1/remove
 
-Same as v3, except the JSON will not include "plusuuids".
+Same as v4, except the JSON will not include "plusuuids".
 
 ### POST /git-annex/$uuid/v0/remove
 
 Identical to v1.
 
-### POST /git-annex/$uuid/v3/remove-before
+### POST /git-annex/$uuid/v4/remove-before
 
-Remove a key's content from the server, but only before a specified time.
+Remove a key's content from the repository, but only before a specified time.
 
 Example:
 
@@ -314,7 +334,11 @@ removal will fail and the server will respond with: `{"removed": false}`
 This is used to avoid removing content after a point in 
 time where it is no longer locked in other repostitories.
 
-### POST /git-annex/$uuid/v3/gettimestamp
+### POST /git-annex/$uuid/v3/remove-before
+
+Identical to v4.
+
+### POST /git-annex/$uuid/v4/gettimestamp
 
 Gets the current timestamp from the server.
 
@@ -331,9 +355,13 @@ current value of its monotonic clock, as a number of seconds.
 Important: If multiple servers are serving this API for the same
 repository, they MUST all use the same monotonic clock.
 
-### POST /git-annex/$uuid/v3/put
+### POST /git-annex/$uuid/v3/gettimestamp
 
-Store content on the server.
+Identical to v4.
+
+### POST /git-annex/$uuid/v4/put
+
+Store content in the repository.
 
 Example:
 
@@ -358,11 +386,18 @@ There are are also these optional parameters:
   Number of bytes that have been omitted from the beginning of the file. 
   Usually this will be determined by making a `putoffset` request.
 
+* `data-present`
+
+  When set to 1, this indicates that the data has been sent to the repository
+  in some other way. The body of the request will be empty. The server will
+  verify that the data is present in the repository and will proceed the
+  same as if the data was sent in the request.
+
 The `Content-Type` header should be `application/octet-stream`.
 
 The `X-git-annex-data-length` must be included. It indicates the number
-of bytes of content that are expected to be sent.
-Note that there is no need to send a Content-Length header.
+of bytes of content that are expected to be sent in the body of the
+request. Note that there is no need to send a Content-Length header.
 
 If the length of the body is different than what the the
 X-git-annex-data-length header indicated, then the data is invalid and
@@ -371,10 +406,16 @@ an unlocked annexed file, which got modified while it was being sent.
 
 The server responds with a JSON object with a field "stored"
 that is true if it received the data and stored the content.
+(Or when `data-out-of-band` is used, if it has verified that the key is
+present in the repository.)
 
 The JSON object can have an additional field "plusuuids" that is a list of
 UUIDs of other repositories that the content was stored to.
 
+### POST /git-annex/$uuid/v3/put
+
+Identical to v4 except the `data-out-of-band` parameter cannot be used.
+
 ### POST /git-annex/$uuid/v2/put
 
 Identical to v3.
@@ -387,7 +428,7 @@ Same as v3, except the JSON will not include "plusuuids".
 
 Same as v1, except additional checking is done to validate the data.
 
-### POST /git-annex/$uuid/v3/putoffset
+### POST /git-annex/$uuid/v4/putoffset
 
 Asks the server what `offset` can be used in a `put` of a key.
 
@@ -419,13 +460,17 @@ part way through a `PUT`, a synthetic empty `DATA` followed by `INVALID`
 will be used to get the P2P protocol back into a state where it will accept
 any request.]
 
+### POST /git-annex/$uuid/v3/putoffset
+
+Identical to v4.
+
 ### POST /git-annex/$uuid/v2/putoffset
 
-Identical to v3.
+Identical to v4.
 
 ### POST /git-annex/$uuid/v1/putoffset
 
-Same as v3, except the JSON will not include "plusuuids".
+Same as v4, except the JSON will not include "plusuuids".
 
 ## parts of P2P protocol that are not supported over HTTP