Setting up InfluxDB and Grafana

Online monitoring with telemetry is essential for most data taking operations. This monitoring data can be nicely displayed in a browser with Grafana in combination with a time series database like InfluxDB.

Setting up Docker Compose

The easiest way to set up InfluxDB and Grafana is with Docker Compose. Installation instructions for Docker and the Docker Compose plugin can be found on the Docker website. Alternatively, since Docker is not trivial to set up and requires root, podman-docker together with podman-compose can be used instead if available in the distribution repositories.

A file named docker-compose.yaml needs to be created with the following content:

services:

  influxdb:
    image: docker.io/library/influxdb:2
    ports:
      - "8086:8086"
    volumes:
      - ./influxdb-config:/etc/influxdb2
      - ./influxdb-data:/var/lib/influxdb2

  grafana:
    image: docker.io/grafana/grafana:latest
    ports:
      - "3000:3000"
    depends_on:
      - influxdb
    user: "0"
    volumes:
      - ./grafana:/var/lib/grafana

Then the containers can be started with:

docker compose up -d

The containers can be stopped with:

docker compose down

Setting up InfluxDB

InfluxDB can be accessed via http://localhost:8086. When first visiting the website, a page for setup appears as shown in Figure 38.

Figure 38 InfluxDB setup page

Here, “constellation” as should be chosen as the initial bucket name. After clicking “Continue”, a page with an API key appears.

Important

The API key shown on this page must be copied for use in the Influx satellite.

Then the configuration can be finalized by clicking “Configure Later”.

Tip

It is recommended to setup a maximum duration for which monitoring data is stored. This can be done by going to “Load Data” → “Buckets”, and then opening the settings for the “constellation” bucket as shown in Figure 39.

Figure 39 InfluxDB bucket settings

Finally, it is recommended to create a custom API token for Grafana, that only reads from the “constellation” bucket. This can be done by going to “Load Data” → “API Token”, and clicking “Generate API Token” → “Custom API Token” as shown in Figure 40.

Figure 40 InfluxDB custom API token settings

Setting up Grafana

Grafana can be accessed via http://localhost:3000. The default username and password are both admin.

First, InfluxDB needs to be added as a data source. This can be done by clicking on “Connections” → “Data sources” and then “Add data source”. On that page InfluxDB can be selected as shown in Figure 41.

Figure 41 Grafana add data source

On the configuration page, the following settings need to be adjusted:

• Query language: Flux

• URL: http://influxdb:8086

• Basic auth: off

• Organization: InfluxDB organization name

• Token: Custom API token created for Grafana

• Default Bucket: constellation

• Min time interval: 1s

An example configuration is shown in Figure 42.

Figure 42 Grafana InfluxDB settings

Adding a Dashboard

To add a dashboard, monitoring data needs to be available. The Influx satellite can be used to add monitoring data to InfluxDB. It requires the InfluxDB organization name and the API key shown during the setup phase.

Generating Monitoring Data

For testing, the Mariner satellite can be used in combination with the Influx satellite.

The Mariner satellite can be started with:

SatelliteMariner -g edda -n Nine

The Influx satellite can be started with:

SatelliteInflux -g edda -n DB

The satellites need to be configured. A sample configuration might look like this:

[Mariner.Nine]
voltage = 1.0
current = 1.0
sample_period = 5.0

[Influx.DB]
org = "Insert organization name here"
token = "Insert read-write API token here"

Assuming the configuration is stored in config.toml, the CLI controller can be started with:

Controller -g edda -c config.toml

The satellites can be initialized with the configuration using:

constellation.initialize(cfg)

Now, the Mariner should send a BRIGHTNESS metric to the Influx satellite, which forwards it to the database.

Creating a Query

To add a visualization in the dashboard, a Flux query needs to be made to the database. The easiest way to do this is using the InfluxDB Data Explorer shown in Figure 43. There, the bucket (constellation), the measurement (sending satellite) and the filed (metric) can be chosen. On the right, the aggregate function should be set to “last” and then the query can be submitted.

Figure 43 InfluxDB Query Builder

The Flux query can be copied by switching to the “Script Editor” as shown in Figure 44.

Figure 44 InfluxDB Script Editor

Adding a Visualization

First, a new dashboard needs to be created in Grafana by clicking on “Dashboards” and then “Create dashboard”. It should be noted that dashboards need to be saved manually, so it is recommend to directly save the new dashboard by clicking “Save dashboard” in the top right corner, where a title for dashboard can be given.

In the top right corner new visualizations can be added. After selecting InfluxDB, the visualization panel editor opens where the Flux query can be pasted as shown in Figure 45.

Figure 45 Grafana panel editor

To get a first view of the monitoring data, the query needs to be refreshed by clicking “Refresh” above the panel. Left of the “Refresh” button, the time range can be adjusted. The panel can be further adjusted using the right side bar as shown in Figure 46. For example, a proper title and a unit (under “Standard options”) can be added, the last value can be shown (under “Legend” → “Values”), a warning for values above or below a threshold can be added, and values far apart in time can be disconnected (“Graph styles” → “Disconnected values”).

Figure 46 Grafana panel editor after edits

Finally, the dashboard needs to be saved with the new panel. To automatically refresh the dashboard, the right triangle next to the “Refresh” button in dashboard overview can be extended to select a refresh interval as shown in Figure 47.

Figure 47 Grafana dashboard

Advanced Visualizations

Transforming Data

Monitoring data can also be transformed in the Query for example to change the units. To multiply a value by 0.01, this can be added in front of the yield part of the query:

  |> map(fn: (r) => ({ r with _value: r._value * 0.01 }))

Note

Integer and floating point types can not be multiplied with each other in the Flux query language. If an integer needs to be changed to a float float(v: r._value) can be used:

  |> map(fn: (r) => ({ r with _value: float(r._value) * 0.01 }))

An example of full query with transformed values would be:

from(bucket: "constellation")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "Mariner.Nine")
  |> filter(fn: (r) => r["_field"] == "BRIGHTNESS")
  |> aggregateWindow(every: v.windowPeriod, fn: last, createEmpty: false)
  |> map(fn: (r) => ({ r with _value: float(v: r._value) * 0.01 }))
  |> yield(name: "last")

See also

For more details on creating queries, see InfluxDB’s Flux documentation.

Displaying Text

Text can be displayed using a Stat visualization using the following query:

from(bucket: "constellation")
  |> range(start: 0, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "Mariner.Nine")
  |> filter(fn: (r) => r["_field"] == "RUN_ID")
  |> group()
  |> last()
  |> keep(columns: ["_value"])

To display the text, Fields need to changed to _value under Value options.

Displaying Rates

Rates can be displayed using the aggregate.rate method:

import "experimental/aggregate"

from(bucket: "constellation")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "PyDevNullReceiver.Receiver" or r["_measurement"] == "PyRandomTransmitter.Sender")
  |> filter(fn: (r) => r["_field"] == "RX_BYTES" or r["_field"] == "TX_BYTES")
  |> aggregateWindow(every: v.windowPeriod, fn: last, createEmpty: false)
  |> aggregate.rate(
    every: 30s,
    unit: 1s,
    groupColumns: ["_measurement", "_field"],
  )
  |> yield(name: "last")