home-assistant-addons/p1-logger/DOCS.md

# Documentation

NB: This documentation page is a work-in-progress at the moment of writing. The ESPhome section still needs to be written in a beginner-friendly way. However, experienced ESPHome users should be able to figure the configuration out themselves using [the ESPhome configuration available in the P1 logger git repo](https://git.hollander.online/energy/p1-logger/src/branch/main/esphome) and the details provided below.

## Installation instructions

These instructions contain 5 steps:

1. This add-on requires a PostgreSQL database, optionally with TimescaleDB support. In this example, we'll Expaso's TimescaleDB add-on in order to store data.
2. Secondly, we install and setup the Mosquitto MQTT broker from the official add-ons repository. MQTT is a standard used to exchange IoT data.
3. Thirdly, we setup ESPhome to commission a P1 reader, which will send P1 meter messages to the MQTT Broker.
4. Then, we setup P1 Logger (this add-on) to monitor the MQTT broker for new messages and record data into the database.
5. Finally, we setup Grafana to visualise the data recorded in the database.

### Install Expaso's TimescaleDB add-on

1. In Home Assistant, go to Settings -> Add-ons -> Add-on store
2. Click the three dots in the top-right corner, select 'Repositories'.
3. Add <https://github.com/Expaso/hassos-addons> .

After a few seconds, the TimescaleDB add-on should appear in the add-on store. Install it.
After installing TimescaleDB, go to Configuration. Click the three dots in the top-right corner and select "Edit in YAML". Then, make the following changes:

- Add `p1_logger` to `databases` and `timescaledb_enabled` like so:

```yml
databases:
  - homeassistant
  - p1_logger
timescale_enabled:
  - homeassistant
  - p1_logger
```

Leave the other settings as-is.
Save the configuration, go to the "Info" tab and start TimescaleDB.

### Mosquitto MQTT Broker setup

#### Install Mosquitto

1. In Home Assistant, go to Settings -> Add-ons -> Add-on store
2. Search for "Mosquitto" and install it from the "Official add-ons" repository.
3. Start the add-on.

#### Configure Mosquitto as a HA broker

1. Go to Settings -> Devices & Services -> Integrations.
2. For "MQTT", click "Configure". After this completes, click "Finish".

#### Add users

##### Add the P1 reader user

1. Click on your name in the bottom-left corner.
2. Under "User settings", make sure that "Advanced mode" is enabled.
3. Go to Settings -> People and switch to the "Users" tab.
4. Add a user for the P1 reader:
   - Display name: P1 reader
   - Username: p1_reader
   - Password and Confirm password: Choose a secure password. Make sure to save it for later use.
   - Local access only: Switch this to "On" if Home Assistant is installed on a system in the same home as the P1 reader.
   - Administrator: Leave this disabled.

##### Add the P1 logger user

1. Click on your name in the bottom-left corner.
2. Under "User settings", make sure that "Advanced mode" is enabled.
3. Go to Settings -> People and switch to the "Users" tab.
4. Add a user for the P1 logger:
   - Display name: P1 logger
   - Username: p1_logger
   - Password and Confirm password: Choose a secure password. Make sure to save it for later use.
   - Local access only: Switch this to "On".
   - Administrator: Leave this disabled.

### ESPHome setup

TODO: A detailed description on how to setup a P1 reader with ESPHome will follow soon.

For experienced ESPHome users, have a look at [the ESPhome configuration available in the P1 logger git repo](https://git.hollander.online/energy/p1-logger/src/branch/main/esphome).

Line 225 contains the following code snippet:

```yml
      on_value:
        - mqtt.publish_json:
            topic: p1/metrics
            payload: |-
                root["t"] = id(dsmr_timestamp).state;
                root["dt1"] = id(energy_delivered_tariff1).state * 1000;
                root["dt2"] = id(energy_delivered_tariff2).state * 1000;
                root["rt1"] = id(energy_returned_tariff1).state * 1000;
                root["rt2"] = id(energy_returned_tariff2).state * 1000;
                root["d"] = id(power_delivered).state * 1000;
                root["r"] = id(power_returned).state * 1000;
                root["f"] = id(electricity_failures).state;
                root["fl"] = id(electricity_long_failures).state;
                root["g"] = id(gas_delivered).state * 1000;
                root["v1"] = id(voltage_l1).state;
                root["v2"] = id(voltage_l2).state;
                root["v3"] = id(voltage_l3).state;
                root["c1"] = id(current_l1).state;
                root["c2"] = id(current_l2).state;
                root["c3"] = id(current_l3).state;
                root["d1"] = id(power_delivered_l1).state * 1000;
                root["d2"] = id(power_delivered_l2).state * 1000;
                root["d3"] = id(power_delivered_l3).state * 1000;
                root["r1"] = id(power_returned_l1).state * 1000;
                root["r2"] = id(power_returned_l2).state * 1000;
                root["r3"] = id(power_returned_l3).state * 1000;
```

**Add this `on_value` function to the *last* `dsmr` sensor that is defined. In this case, `power_returned_l3`.** Additionally, make sure that all `id`'s referenced in this function are defined within the sensors above this last one. Adjust the function as needed by e.g. removing L2 and L3 for single phase connections, removing gas when not applicable or renaming `id`'s when this suits your current setup better.

### Setting up P1 Logger

Having setup the PostgreSQL / TimescaleDB database and Mosquitto MQTT broker, we can now install and configure P1 Logger. Add the [add-on repository](https://git.hollander.online/energy/home-assistant-addons) using the [instructions](https://git.hollander.online/energy/home-assistant-addons) provided. Then, click "Install" to install this add-on. Go to "Configuration".

- If you're using Expaso's TimescaleDB add-on, the `db` setting will **not** require any changes.
- MQTT
  - `mqtt_broker`:
    - Default: `tcp://core-mosquitto:1883`.
    - In case of using an external broker, use a syntax like `tls://mqtt.example.org:8883` or `tcp://mqtt.example.org:1883`
  - `mqtt_topic`: `p1/metrics`
  - `mqtt_user`: `p1_logger`
  - **`mqtt_password`: Enter the password you used in the previous step while creating the `p1_logger` user.**
- log_level: `INFO`
  - Valid values are "`DEBUG`, `INFO`, `WARN` and `ERROR`.
- In `tz`, enter your timezone. The default is "Europe/Amsterdam". If you are in a different location, choose your timezone from [this list](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List).
- Optionally, if you don't want to use TimescaleDB, disable `timescaledb`.

Save the configuration, go to the "Info" tab and start P1 Logger.

Check the "Log" page. If the setup is succesfull, it should contain a message stating: `"level":"INFO","msg":"Connected to MQTT broker, subscribing to topic...","topic":"p1/metrics"`.

### Setting up Grafana

From the Home Assistant Community Add-ons repository, install Grafana. Click "Start" to start the add-on and enable "Show in sidebar". Grafana will now appear in the sidebar. Open it.

- In the top-left corner, click the three bars (open menu) -> Connections.
- Using the search box, enter `PostgreSQL`. Click the resulting "PostgreSQL".
- In the top-right corner, click "Add new data source"
- Use the following settings
  - Name: `p1_logger`
  - Host URL: `77b2833f-timescaledb`
  - Database name: `p1_logger`
  - Username: `postgres`
  - Password: `homeassistant`
  - Version: `16` or the closest version to it.
  - TimescaleDB: Toggle the switch to enable TimescaleDB support.
- Click "Save and Test". A "Database Connection OK" message should appear.
- In the top-right corner, click the `+` sign, "Import dashboard".
- Go to <https://git.hollander.online/energy/p1-logger/src/branch/main/grafana>
  - In this folder, a number of sample dashboards is available.
    - English: `P1-meter-readings.json`
    - Dutch: `P1-metergegevens.json`
    - German: `P1-Messdaten.json`
  - Click the dashboard you want, and then "Raw".
    - Copy and paste the text into the "Import via dashboard JSON model" box.
    - Click "Load".
  - In the `p1` field, select `p1_logger` as the data source.
  - Click import. You should now be able to view the data, collected by P1 Logger and stored into TimescaleDB.
  - Feel free to try out the other dashboards as well.

## Security notice

Using the aforementioned steps, you should now have a working P1 Logger setup. However, it is important to keep your data safe. [Expaso recommends](https://github.com/Expaso/hassos-addons/tree/master/timescaledb#usage) changing the default postgres `homeassistant` password. I recommend following their recommendation and also installing the pgAdmin4 add-on using the [instructions](https://github.com/Expaso/hassos-addons/tree/master/pgadmin4) they provide. In case you're doing this after setting up P1 Logger, remember to also update this password in the P1 Logger config and the Grafana data source connection settings.

## Feedback

If you'd like to provide feedback about this project, or in case you'd like to ask questions, please refer to [this page](https://pieterhollander.nl/author/pieter-hollander/) for contact information (e-mail, Github or LinkedIn), or use the [contact form](https://pieterhollander.nl/#contact) on my website.