# StreamGoose: an open-source livestreaming chat aggregator and widget helper

StreamGoose (aka StreamGoOSE, Streamer's Go-based Open Source Enhancer) is an open tool to solve the problem of aggregating several live chats to display on a livestream.

As of now, most of the StreamGoose codebase is written in Go 1.23.1. The Web part is written in HTML5, CSS3 and JavaScript using the ES2015 standard.

## Current features

The status of StreamGoose is currently **alpha**. Use at your own risk.

Nevertheless, the following features are already implemented:

- Twitch and YouTube chat aggregation (see "Backend implementation status" section)
- Handling Twitch emotes
- Handling YouTube non-Unicode emotes (see "Backend implementation status" section)
- Message styling based on Twitch and YouTube roles
- Twitch username styling based on color preference
- Message highlighting based on regular expressions
- Third-party action (`/me`) command handling
- YouTube avatar icon support
- Twitch system message support (needs more testing)
- [Noto Color Emoji](https://fonts.google.com/noto/specimen/Noto+Color+Emoji) font bundled for universal Unicode emoji support

## Planned features

- Moving to OAuth 2.0 for YouTube
- "Deleted message" support for YouTube
- Automated Twitch chat badges output (global and channel-specific)
- Internal chat ranking system by username
- Backend-independent support for oldschool plaintext emoticons ;-)
- Starting window geometry customization via config (now doable via custom JS)
- ...

## Architecture

StreamGoose consists of three components: a set of backends, a message broker server and a Web-based frontend.

### Message broker server

This is the main StreamGoose component. It provides three main functions:

- accepting new messages from backends via external (`POST /new`) and internal (`message_transport` Go channel) ways;
- providing the unified message queue for the frontend side consumption;
- serving static files that make up the frontend (`index.html`, `script.js`, `style.css` and images) over the plain HTTP protocol.

Additionally, the server binary also launches the backends it knows about and opens a simple GUI window to show the Web content locally, feeding its unified output directly into the JS side. The latter part became possible thanks to the [webview_go](https://pkg.go.dev/centrifuge.hectabit.org/HectaBit/webview_go) library.

### Backends

StreamGoose can operate with two kinds of backends: internal and external.

An **internal** StreamGoose backend is a Go program embedded as a goroutine into the main binary. It communicates with the message broker by sending messages to the `message_transport` Go channel. The messages are defined using this Go structure:

```go
type ChatMessage struct {
  Id uint64        `json:"id"`        /* message id (auto-filled) */
  Type string      `json:"type"`      /* message type string */
  Timestamp string `json:"timestamp"` /* timestamp string (formatted) */
  Username string  `json:"username"`  /* username */
  Text string      `json:"text"`      /* message text */
  Icon string      `json:"icon"`      /* optional message icon URI */
}
```

The `Id` field is populated automatically whenever a message enters the broker. For others, see below for the detailed description of the corresponding JSON fields in the external backends.

An **external** StreamGoose backend is a program (written in any network-enabled programming language) that can make a POST request to the corresponding message broker URL (`http://[broker_host]:60053/new` endpoint). The request body shall be a JSON object with the following fields (all passed as strings):

- `type`: message type, consisting of hyphenated classes (e.g. `yt-moderator` or `tw-vip-color#A1B2C3` etc, see below in the "Customization" section of this README). This field defines how the message will be styled when being rendered on the frontend.
- `timestamp`: an ISO 8601 **UTC** timestamp string denoting when the message was sent. The only recommended format (in strftime syntax) is `%Y-%m-%dT%H:%M:%SZ`.
- `username`: the message author name that should be displayed in the chat. 
- `text`: the message text that should be displayed in the chat.
- `icon`: an optional icon URL passed to the chat if the message author has a personal user profile picture (avatar). This field can be left empty but should be included in the request anyway.

Whenever a new JSON message is shaped with the fields above, it shall be sent to the broker at the `/new` endpoint with the HTTP POST method. A `{"status":1}` response will be sent back if the request is successful. The message

As of now, StreamGoose provides two reference backends: for Twitch and for YouTube, both internal and written in Go. See "Backend implementation status" section for current notes and caveats.

### Frontend

The StreamGoose frontend is a Web-based (HTML5) application served by the message broker server. It is opened in the built-in WebView-based GUI and provides a constantly updated aggregated live chat display, message type dependent styling, emote handling, custom highlighting and so on. The users are free and even encouraged to modify the web assets located in the `/webroot` directory to fit the chat to their own visual preferences.

## Building and installation

### Linux

You need to have Go 1.23.1 and above, a POSIX-compatible "make" as well as the `webkit2gtk-4.1` dev package.

Just run `make` and get all the resulting files in the `out` directory.

### macOS

TBD

### Windows

TBD

## Usage

Enter the distribution directory and run the `./streamgoose` or `streamgoose.exe` binary. It should run with the default `config.json` file, start the server, spawn the internal backends and open the UI window. Alternatively, you can pass another JSON file as the first command line parameter.

### Configuration

StreamGoose is mostly configured through the `config.json` file distributed in the same directory as the main binary.

The following fields in the `config.json` are backend-agnostic:

- `server_port`: a local port to run the HTTP REST server on. Default value: `60053` (for "GOOSE").
- `server_listen_ip`: a local IP address to bind the server to. Default value: `127.0.0.1`. Recommended to leave at this value, only change it if you plan on connecting external backends from other machines in your LAN.
- `debug`: print out various debug information about incoming messages and events. Can be `true` or `false`.

The following fields in the `config.json` are specific to the Twitch internal backend:

- `tw_backend_enabled`: whether or not the Twitch internal backend is turned on.
- `tw_nickname`: **your own** Twitch nickname. You need to use the same nickname as the one you used for obtaining the token (see ("Connecting Twitch internal backend" section below).
- `tw_channel`: the Twitch channel that you want to receive messages from (needs to start with `#`).

The following fields in the `config.json` are specific to the YouTube internal backend:

- `yt_backend_enabled`: whether or not the YouTube internal backend is turned on.
- `yt_api_root_url`: the API proxy root URL. Recommended to leave at `https://yt.lemnoslife.com/noKey` for now. This is a temporary field, it is subject to removal once the backend switches to using OAuth 2.0.
- `yt_api_poll_timeout`: the interval (in seconds) to poll the YouTube chat with. For now, it's recommended to set to at least 3 seconds to not overload the API proxy.
- `yt_channel`: the YouTube channel that you want to receive messages from (needs to start with `@`). If only this field is specified and the channel has several live broadcasts running, the backend will fetch the first broadcast from the list.
- `yt_video_id`: an optional field that allows you to pass a particular broadcast video ID to get the live chat messages from. Overrides the `yt_channel` field.

Example `config.json` file:

```json
{
  "server_port": 60053,
  "server_listen_ip": "127.0.0.1",
  "debug": true,
  "tw_backend_enabled": true,
  "tw_nickname": "Suborg",
  "tw_channel": "#foxyshadow",
  "yt_backend_enabled": true,
  "yt_api_root_url": "https://yt.lemnoslife.com/noKey",
  "yt_api_poll_timeout": 3,
  "yt_channel": "@foxyshadow"
}
```

### Connecting Twitch internal backend

The following procedure needs to be only done once:

1. Go to the [Twitch Chat OAuth Password Generator](https://twitchapps.com/tmi/) website.
2. Login with your own Twitch account and give it the access.
3. Copy the token string that starts with `oauth:` and paste it into the `twitch-token.txt` file in the `auth` subdirectory in the application directory.
4. Save the file. The Twitch connection should now work.
5. If the connection doesn't work, make sure that the `tw_backend_enabled` field is set to `true` in `config.json` and that you used the same Twitch nickname to get the token as the one specified in the `tw_nickname` field.

### Connecting YouTube internal backend

As of now, nothing needs to be done except setting the `yt_backend_enabled` field to `true` in `config.json`, but this section will change soon once the backend moves to OAuth 2.0.

## Customization

Being a Web-based application, the StreamGoose frontend provides vast customization capabilities on both the layout and client-side logic sides.

### Styling

The `webroot/style.css` is the main point of customization. Here is a short (and vastly incomplete) reference on the CSS selectors and classes that you can operate on:

- `#msglist > div`: an individual message block
- `.icon`: user avatar (e.g. from YouTube backend)
- `.username`: username text
- `.msg`: message text
- `.timestamp`: message timestamp (hidden by default)
- `.tw`: a Twitch message (applies to the whole message block, as well as **all the subsequent classes** listed here)
- `.yt`: a YouTube message
- `.me`: third-party action
- `.broadcaster`: a message that comes from the Twitch/YouTube channel owner
- `.moderator`: a message that comes from a Twitch/YouTube chat moderator
- `.sponsor`: a message that comes from a YouTube chat sponsor
- `.verified`: a message that comes from a YouTube verified account

Additionally, every Twitch badge is converted into a CSS class that can be applied to a message block, e.g.

- `.vip`: a message that comes from a Twitch VIP chat user
- `.subscriber`: a message that comes from a Twitch subscriber
- `.admin`: a message that comes from a Twitch admin

and so on. You can find a full list of badges and their IDs by issuing a GET request to `https://api.twitchinsights.net/v1/badges/global` URL and seeing the `setID` field.

### Regex-based message highlighting

The `webroot/custom.js` allows you to pass custom JavaScript, including the predefined `messageHighlightRules` JS object. The keys in this object are regular expression strings, the values are the colors to apply to the messages where these expressions match. The matching is always case-insensitive and Unicode-aware. For the color values, you can use whatever CSS supports: `#hex` notation, symbolic names, `rgba()` or `hsla()` notation.

A canonical example is for matching alternative words in addition to the nickname:

```js
var messageHighlightRules = {
  "(^|\\s)(suborg|luxferre|lux)(\,|;|:|\\s|$)": "#ff0000",
  "(^|\\s)(foxy|foxyshadow|fox)(\,|;|:|\\s|$)": "orange",
  "(^|\\s)(bluepanda|bluebear|sinipandoo)(\,|;|:|\\s|$)": "blue"
}
```

Each regular expression captures: the string start or a whitespace character, then one of the nickname alternatives, then either a comma, a semicolon, a colon or another whitespace character or the end of the string. If the message text matches the first expression, it is colored in red, if it matches the second one, it is colored in orange, if it matches the third one, it is colored in blue. 

## Backend implementation status

### Twitch (internal)

Mostly stable. Some edge cases still could be found and further investigated.

Most chat badges are still ignored when rendering the messages.

### YouTube (internal)

Currently, the YouTube backend doesn't have OAuth 2.0 flow implemented as of yet. A single API key would consume the quota too quickly, so, right now, the backend uses [an unofficial Youtube API proxy](https://yt.lemnoslife.com/) which can and sometimes does have downtimes.

Rewriting the backend to support OAuth is one of the highest priorities.

Additionally, YouTube does not have its own chat emote API as of now, so the emnote set is hardcoded in the `/webroot/img` directory and can become outdated over time.

## FAQ

### What were the reasons for starting this project?

In the game livestreaming community, there is some demand for a saner and more flexible alternative to the commercial live chat aggregation solutions, such as RutonyChat. In fact, witnessing several streamers struggling with some RutonyChat issues was the main trigger to start the project. On top of that, once the main functionality is complete, there are some plans to extend StreamGoose to support more things beyond just chat aggregation. 

### Why is it called StreamGoose?

This name was chosen as a winner of a poll offering 10 various names suggested by the members of two Discord servers with a lot of small-scale game livestreamers. It was one of the draft names too, given that, at the very beginning of the project, the broker server was written in C and based on the [Mongoose](https://mongoose.ws) library.

Additionally, this name turned out to be pretty unique — at the time of publishing this repo, no other desktop application with such a name existed.

### How can I contribute? 

Everyone is free to send their suggestions and patches. Once you get a Codeberg account, you can participate here in various ways:

1. Discuss, report and propose things at the issue tracker.
2. Create feature branches and merge requests if you know how to implement a particular feature in the Go-based StreamGoose core.
3. Write entirely new external independent backends in the programming languages of your choice (please place them into the `/external` directory of this repo). As a bonus, if you write a backend in Go and observe the same internal `ChatMessage` structure, we can even consider turning it into a Goroutine and include internally into the next StreamGoose version, provided it is already tested and stable enough.

### The resulting binary size is around 6 megabytes or even more, is it normal?

Yes, due to the way Go links its networking client and server code and everything else statically (where possible). On Linux, you may try using GCC Go instead to achieve fully dynamic linking. Keep in mind that GCC Go is not fully supported yet, and if you come to our issue tracker with an issue that's only reproducible with GCC Go, it's less likely to be resolved.

### Beyond Twitch and YouTube, are there any other livestreaming platforms that you plan to support internally?

Sure! For instance, one of the next close targets will be [Owncast](https://owncast.online/) self-hosted livestreaming solution and its rather simple chat API. The problem with adding a new platform support, as always, is finding enough time to study the API and enough people that would find this support valuable. For example, with enough demand, Kick and Trovo support are also likely to be added.

Besides Twitch, YouTube and Owncast, however, your best bet would be to create an **external** backend for the platform you're interested in, and then incorporate it into this repo in the `/external` directory. When the backend is mature enough, we can always sit and think about how to convert it into an internal one to be distributed inside the StreamGoose binary.


## Credits

Architectural design by [Luxferre](https://luxferre.top) and members of [@foxyshadow](https://twitch.tv/foxyshadow)'s Discord server.

Initial reference implementation by Luxferre.

Noto Color Emoji font by Google, distributed under the [Open Font License](https://openfontlicense.org).

All files, except the bundled font and image assets, are released into the public domain with no warranties.
