tii/README

tii: a Tcl-based suite for working with ii/IDEC protocol
========================================================
This suite implements the client and server sides of the ii/IDEC protocol of
distributed, cross-platform, text-based communication (a FIDOnet successor, so
to speak). See ii-doc.txt for the protocol documentation.

The tii suite requires at least Tcl 8.6 to run. Running it inside starpacks is
possible but not recommended.

It also depends upon SQlite 3.44 and above (3.47 recommended) with the proper
Tcl extension enabled for it..

The tii distribution consists of the following parts:

* tiifetch.tcl: the core ii/IDEC message fetching library and CLI utility
* tiipost.tcl: the core ii/IDEC message posting library and CLI utility
* tiiview.tcl: the CLI viewer of the fetched ii/IDEC messages and conferences
* tii.db: the SQLite3 database that contains all messages and echo lists
* config.txt: the set of parameters for all HTTP requests by the tii scripts
* stations.txt: the list of stations to be auto-fetched by tiifetch when none
  of its command-line parameters is passed (not included in the repo)
* auth.txt: the list of station/authstring mappings to be used by tiipost when
  posting messages to a particular station (not included in the repo)
* tiix.tcl: the GUI ii/IDEC viewer that also leverages tiifetch and tiipost to
  provide fetching and posting functionality
* tiid.tcl: the basic ii/IDEC node software that uses the same DB format
* tiid-user.tcl: user management utility for the tiid node

Readiness status
----------------
* tiifetch.tcl: ready/tested
* tiipost.tcl: ready/tested
* tiiview.tcl: ready/tested
* tii.db (format): ready/tested
* config.txt (format/fields): ready/tested
* stations.txt (format): ready/tested
* auth.txt (format): ready/tested
* tiix.tcl: ready/testing
* tiid.tcl: ready/testing
* tiid-user.tcl: ready/tested
* Overall status: basically ready, bugfixing in progress

Client usage
------------

### Fetching the messages (tiifetch.tcl): ###

tclsh tiifetch.tcl [station_url] [echos] [dbfile]

This command will fetch all messages into the dbfile ("tii.db" in the script
dir by default) from the station_url (can be empty, see below) based on the
echo conference names (can be delimited with slash /, comma (,) or semicolon
(;)) and create the corresponding file structure if it's missing.

Fetching is supported for the following station URL schemes and protocols:

* HTTP (http://)
* HTTPS (https://)
* Gopher (gopher://)
* Gopher over TLS (gophers://)
* Finger (finger://)
* Nex (nex://)
* Spartan (spartan://)
* Gemini (gemini://)

If the station_url parameter is empty or no parameters are passed at all,
tiifetch.tcl will look for a file called stations.txt that lists (each on a
new line) all the station URLs to sync from and the amount of IDs that can
be fetched from a particular station with a single GET request (this amount
can be 12 or more):

https://url1 389
gemini://url2 24
http://url3 12

Messages from all listed stations will be merged into the same echo database.
You also can comment out a station to temporarily stop fetching from it by
prepending the # sign.

### Viewing the messages from CLI (tiiview.tcl): ###

tclsh tiiview.tcl [echo_name] [filter_string] [line_width] [dbfile]

If the echo_name parameter is passed, this command will write all formatted
messages from the coresponding echo conference to the standard output.
You should pipe this message stream to your $PAGER terminal application, like
less or most. The messages will be formatted according to the filter_string
format (see below). If the filter string is empty or omitted, the messages
will appear in the exact order their IDs appear in the echo conference file.

If the echo_name parameter is empty or no parameters are passed at all, this
command will output the list of echo conferences registered in the dbdir.
The conference list will be alphabetically ordered and the filter_string will
have no effect at all if you pass it.

If the line_width parameter is omitted, the text reflows to 80 chars per line.

If the dbfile is omitted, it defaults to "tii.db" in the script directory.

This component is fully offline and can only work with a compatible message
database that tiifetch.tcl can generate.

The filter string can take one of the following basic forms:

* [number]: only take [number] messages from the head (start) of the list
* r[number]: same but output messages from newest to oldest in the list

If [number] is 0 then it means no message limit.
The reverse operation is always done after limiting the results.

e.g. r50 will output 50 newest messages in the conference, starting from the
most recent one. The default basic value is h0, so no filter applied will mean
outputting all messages from the oldest to the newest.

You can extend the basic form by appending a search regular expression to it,
like this: [form]/[regex]. Note that the regex always applies AFTER the basic
filtering has been done. Also note that the search is done within all fields
of the message. E.g. h100/retro will find the retro-themed messages among the
first 100 of them.

### Posting the messages from CLI (tiipost.tcl): ###

echo my_post_text | tiipost.tcl station_root_url echo_name msgto subj [repto]

The tiipost.tcl script accepts the message body from the standard input. The
body must be provided in UTF-8 encoding and all command line parameters except
[repto] (reply-to ID) are mandatory.

The script will look for the auth.txt file in its directory, with the format:

station_url_1 password1
station_url_2 password2
...

If the station root URL matches one of the lines in auth.txt, it automatically
will use the password as the authstring when posting to the station.

On success, tiipost.tcl will output a message that starts with "Success", in
case of an error the line will start with "Error". The server-side description
will follow this success or error status.

As of now, posting is only supported for HTTP/HTTPS station URLs.

HTTP client configuration
-------------------------
For both fetching and posting, you can configure several values in the config
file (config.txt in the script directory, the example is included in the repo)
that will allow you to set HTTP proxy and override the user agent. As of now,
the file is a Tcl dictionary (`key "value"`) and has the following fields:

* useragent: the user agent string to set for all HTTP requests
* proxyhost: the HTTP proxy hostname to use (if empty or omitted, the scripts
  will connect directly)
* proxyport: the HTTP port to use on the proxy (only effective if proxyhost is
  set to a non-empty string)
* net_timeout: the timeout (in milliseconds) for all network requests

Any of the fields can be omitted, as well as the file itself.
You can also use torsocks with any script invocation in order to fully cloak
your originating IP address.

Server usage
------------
This section is in progress.

### Starting the tiid server ###

The tiid.tcl server accepts the following parameters:

tiid.tcl [port] [nodename] [dbfile]

where:

* port (default 8080) is the TCP port to listen to,
* nodename (default "tiid") is the unique node name (needed to fill the fields
like message originating address),
* dbfile (default "tii.db" in the script directory) is your database path.

The tiid server is compatible with all existing HTTP-based clients and also
with the clients that utilize Gopher/Nex transport, like tiifetch/tiix. It has
automatic protocol detection, so both HTTP and Gopher/Nex clients can connect
to the same port.

Although the tii client only can post via HTTP, the tiid server also supports
direct TCP posting with GET-like queries via Gopher/Nex protocols.

### User management on the tiid server ###

As of now, only manual user management is supported. The simplest way is to
use the included tiid-user.tcl script to perform administrative tasks.

Adding a user or changing its password (auth string):

tiid-user.tcl dbfile user someusername auth SuperSecretAuth123 acl "*"

If you don't supply the acl parameter, it will default to "*".

Changing the list of echos the user can post ("*" means all, "" means none):

tiid-user.tcl dbfile user someusername acl "echo.1,echo.2..."

Note that you need to set up ACL every time you change a user's password.

In both cases, the database file and the user field are mandatory.


Credits
-------
Created by Luxferre in 2024, released into public domain with no warranties.