opendtu-logger/README.md

OpenDTU logger

OpenDTU logger connects to the OpenDTU livedata websocket and captures metrics. These metrics are inserted into a PostgreSQL database. Optionally, TimescaleDB can be used.

Install instructions

Docker Compose is the preferred way to install OpenDTU Logger, but using the binary is also possible.

Docker

docker pull git.hollander.online/energy/opendtu-logger:0.0

Preferably, run the Docker image using the Docker compose examples provided in the ./docker folder.

Docker Compose

The docker folder in this repository contains example Docker compose files. The compose.with-database-grafana.yml file contains a setup suitable for a complete deployment.

To use it, take the following steps

Enter the docker folder

cd docker

Copy example.with-database.env to .env. E.g.:

cp example.with-database.env .env

• Edit the REMOTE_URL to point to your OpenDTU.
• In DB_URL alter the password field. Set the same value in. PG_PASSWORD

Deploy the Docker containers by running the following command from the docker folder.

docker compose -f compose.with-database-grafana.yml up

Docker should now start downloading the required containers and bring up OpenDTU Logger, TimescaleDB and Grafana. If everything seems ok, press CTRL + C to stop the containers. Next, execute

docker compose -f compose.with-database-grafana.yml up -d

The -d flag ensures that the containers are started in the background, so you can safely close your terminal window. Containers running in the background can be stopped by running compose again with the down command.

docker compose -f compose.with-database-grafana.yml down

Setting up Grafana

Grafana should be available at http://localhost:3000.

The initial login username and password are admin:admin. After the initial login, Grafana will request you to change the password to something more secure.

Configuring the OpenDTU Logger data source

After logging in, go to Connections -> Data sources -> Add new data source.

• Under SQL select PostgreSQL.
• Change the following fields
  • Host URL: timescaledb:5432
  • Database name: opendtu_logger
  • Username: postgres
  • Password: The password you set in the .env. file in the previous step.
  • TLS/SSL Mode: disable
  • TimescaleDB: set the slider to enable
• Click Save and test
  • You should get a green box with a Database Connection OK message.

Adding the OpenDTU Logger dashboards

Now that the data source has been configured, we can add our dashboards.

• Go to Dashboards -> New -> Import.
• Upload PV.json from the grafana folder in this repository
• After uploading, Grafana requests you to configure a data source. Select the data source configured in the previous step. Click Import. The dashboard should now be ready.
• If you want more dashboards, import the other dashboards from the grafana folder as well, or create your own.

Binary

Go to the releases page and download the most recent stable release.

Running the binary as local user

REMOTE_URL="opendtu:80" DB_URL="host=localhost port=5432 user=postgres password=secret dbname=dtu sslmode=disable" TIMESCALEDB_ENABLED="true" TZ="Europe/Amsterdam" ./opendtu-logger

Creating a more permanent setup on Linux using systemd

sudo adduser --system --group --disabled-login --home /opt/opendtu-logger opendtu-logger

sudo mkdir /opt/opendtu-logger
sudo mkdir /opt/opendtu-logger/data
sudo mv ./opendtu-logger /opt/opendtu-logger/

sudo chown -R opendtu-logger:opendtu-logger /opt/opendtu-logger

sudo nano /etc/systemd/system/opendtu-logger.service

Add the contents of systemd/opendtu-logger.service and alter the REMOTE_URL and DB_URL environment variables to match your setup.

PostgreSQL setup

To install PostgreSQL on Debian / Ubuntu run

sudo apt install postgresql

The OpenDTU logger logs to a PostgreSQL database and optionally supports TimescaleDB. Create a separate database and user using the following commands.

CREATE DATABASE opendtu_logger;
CREATE USER opendtu_logger WITH password 'SECRET';
GRANT ALL privileges ON DATABASE opendtu_logger TO opendtu_logger;
GRANT ALL ON SCHEMA public TO opendtu_logger;

Optional: enable the TimescaleDB extension. To make use of TimescaleDB, follow the steps outlined in their documentation. Then, enable TimescaleDB support for the opendtu_logger database by executing the following commands

\c opendtu_logger
CREATE EXTENSION OF NOT EXISTS timescaledb;

Environment variables

The following environment variables are required for OpenDTU Logger to be configured.

REMOTE_URL="opendtu:80"
DB_URL="host=localhost port=5432 user=postgres password=secret dbname=dtu sslmode=disable"
TIMESCALEDB_ENABLED="true"
TZ="Europe/Amsterdam"

• REMOTE_URLis used to specify the IP address or hostname OpenDTU is running on. E.g.: 192.168.1.6 or opendtu.internal
• DB_URL specifies the credentials required to connect to the PostgreSQL database. All connection parameters documented by golang's lib/pq package can be used.
• TIMESCALEDB_ENABLED should be set to true or false.
• TZ is used to ensure data is recorded with the right timestamp. Choose the timezone valid for your location from this Wikipedia page.

Grafana

The grafana folder contains example dashboards which can be imported into existing Grafana installs, or into the provided Docker Compose with Grafana setup.

• The opendtu_ dashboards contain visualisations for each table the OpenDTU logger records to.
• PV.json is a dashboard which combines the visualisations of each table that I found to be most useful.
• Zonnepanelen.json is the same as PV.json, but in Dutch language.

Known limitations

• The logger will currently log every event posted by OpenDTU to the websocket. Checks still need to be added to determine the uniqueness of an event. For more information, see also this OpenDTU Github issue.
• Upon restart of the OpenDTU, the OpenDTU Logger binary will need to be restarted as well. When using the provided compose.yml files with Docker, or using the systemd service file when using the binary, ensures this happens automatically.