ii protocol documentation
=========================
This document describes the basic ii protocol, as implemented in tii.
It aims to be as clear and concise as possible.

Network structure
-----------------
Clients aka points can:
* post messages
* fetch echos (conferences) and their message ID lists
* fetch messages by their IDs

Nodes aka stations can:
* accept (or not accept) posted messages from points
* serve echo lists
* serve echos and their message ID lists
* serve message bundles
* fetch echos and messages from other stations

The main transport protocol is currently HTTP/HTTPS, although the spec doesn't
theoretically limit the ways of message transfer. E.g. fetching can be easily
implemented over Gopher/Finger/Nex/Spartan/Gemini etc.

The API spec below is given for the HTTP(S) protocol.

Echo and message naming convention
----------------------------------
Within the network, echo names must be from 3 to 120 characters long and have
at least one dot (.) character. Message IDs must be exactly 20 characters long
and depend on the message contents hash (see the exact algorithm below).

Station (node) HTTP API (as implemented in tii)
-----------------------------------------------
Every station must implement the following HTTP API calls.
In case of multi-line responses, the newline separator must be "\n" character
(line feed, ASCII 10).

- Fetching the public echo list (IDEC extension) -

Request: GET /list.txt
Response: newline-separated list of echo_name:msg_count:echo_description

- Fetching the echo message count (IDEC extension) -

Request: GET /x/c/echo.1.name/echo.2.name/..
Response: newline separated list of echo_name:msg_count

Note: msg_count must not decrease even if some messages are deleted.

- Listing messages in the echo (s) -

Request: GET /u/e/echo.1.name/echo.2.name/...
IDEC extended syntax: GET /u/e/echo.1.name/echo.2.name/.../offset:count
(where offset can be negative)
Response: newline-separated list of echo names and message IDs in the format:

echo.1.name
msgid1fromecho1
msgid2fromecho1
...
echo.2.name
msgid1fromecho2
msgid2fromecho2
...

When a new message is posted to the echo, it gets appended to the end of the
corresponding message ID list for this echo.

- Fetching the message bundles -

Request: GET /u/m/msgid1/msgid2/...
Response: newline-separated list of msgid:base64_msgtext

where base64_msgtext is a Base64-encoded Node-to-Point Message (see below).

- Posting a message -

Request: POST /u/point
Content-Type: application/x-www-form-urlencoded 
Data: pauth=auth_string&tmsg=base64_msgtext
Response: in case of success, must start with "msg ok"

where base64_msgtext is the Base64-encoded Point-to-Node Message (see below).
The maximum length of the tmsg field must be 87382 bytes.

- Listing IDEC features (IDEC extension) -

Request: GET /x/features
Response: newline-separated list of supported non-standard URL paths
(x/c, u/e, /list.txt etc)

Note: if the /x/features path is unavailable, the client must assume that no
IDEC extensions except /list.txt are supported by the server.

Node-to-Point Message format
----------------------------
The encoding must be UTF-8 and the newline separator must be "\n" (ASCII 10).
Every Node-to-Point message contains the following fields in this particular
order, all of them are mandatory and start on a new line each:

* Line 1: message tags. Must start with "ii/ok". If "ii/ok/repto/id" form is
  encountered, then the id refers to the message this message replies to.
* Line 2: echo name where the message was posted.
* Line 3: message Unix timestamp (integer, in seconds, UTC)
* Line 4: message sender name
* Line 5: message sender address (autofilled by the originating station)
* Line 6: message recipient name (or All if there's no particular recipient)
* Line 7: message subject
* Line 8: must be empty
* Line 9 and further: message body

Point-to-Node Message format
----------------------------
The encoding must be UTF-8 and the newline separator must be "\n" (ASCII 10).
Every Point-to-Node message contains the following fields in this particular
order, all of them are mandatory and start on a new line each:

* Line 1: echo name where the message is being posted.
* Line 2: message recipient name (or All if there's no particular recipient)
* Line 3: message subject
* Line 4: must be empty
* Line 5 and further: message body

If you are replying to a message [msgid], then message body must begin with:

@repto:msgid

and the message text itself must start on the next line.

Message ID generation algorithm
-------------------------------
This algorithm must be implemented by every station to generate message IDs:

1. Calculate SHA256 of the message in the Node-to-Point format as binary data.
2. Calculate Base64 of the resulting binary hash sum.
3. Truncate to the first 20 characters.
4. Replace all occurrences of + or - with A, and / or _ with Z.
5. The result of these operations is your ii message ID.

Implementation notes
--------------------
* Most HTTP servers are configured to reject long GET lines, so tii passes a
  limited amount of message IDs to the /u/m endpoints. This behaviour can be
  configured in the stations.txt file.
* Some of the crucial validations are for message IDs to be 20 lines and for
  all messages (Node-to-Point and Point-to-Node) to strictly have LF line
  endings, not CRLF.
* The message order in an echo does not always match the timestamp ordering;
  it is fully up to the client on how to sort the messages internally. The
  messages are only guaranteed to be saved by the server in the order they
  arrive onto the server.

--- Luxferre ---
