Semantic Services Monitoring

Overview

The Ontotext Semantic Services use Grafana to monitor the performance and health of the deployed services.

The data is gathered via Telegraf and stored in InfluxDB. This way, the Semantic Services performance and health are monitored in real time.

To find out more about how to use Grafana with InfluxDB, please visit Using InfluxDB in Grafana.

You can also monitor the Semantic Services performance from their web-based administration interface, the Workbench.

Dashboards

The following dashboards are available for download:

Docker

../_images/health_docker.png
  • Metrics - Semantic Objects
    Monitors essential data about the Semantic Objects, such as CPU usage, memory usage, JVM thread count, etc.
../_images/soaas.png ../_images/gdb.png
  • Query execution details
    Monitors data about each query execution details, such as time spent for parsing, executing, validation, etc.
../_images/queries.png

To monitor the overall Docker environment, you can download a dashboard that fits your needs from the official Grafana website.

Kubernetes

../_images/alerts.jpg ../_images/gdb.png
  • Query execution details
    Monitors data about each query execution details, such as time spent for parsing, executing, validation, etc.
../_images/queries.png

To monitor the overall Kubernetes environment, you can download a dashboard that fits your needs from the official Grafana website.

Each of them provides valuable real-time information about different components of the Ontotext Semantic Services.

Quick Start

If you are deploying the Ontotext Semantic Services via our sample Helm charts, it has Telegraf, InfluxDB, and Grafana pre-configured to work together. The dashboards shown above are preloaded as well.

Docker Configuration

If you are manually configuring the monitoring services, you will need to import them in Grafana. See how to import them here.

Follow the below instructions on how to configure the monitoring. Here is an example Docker Compose monitoring.yaml for deploying the monitoring services.

Telegraf has its own telegraf.conf configuration file. It needs to be mounted when starting the Docker container, so make sure it exists under config/ folder next to the Docker Compose file or update the volume mounting to point to the correct telegraf.conf location.

To deploy the example Docker Compose file, use:

docker-compose -f monitoring.yaml up -d

After deploying the Docker Compose, you can access Grafana on http://localhost:9991. It will have automatically provisioned the bundled dashboards and data sources. The default administrative user is admin with password admin. You will be prompted to update the default password upon your first login.

Note

For installing and deploying the full Ontotext Semantic Services, including security and monitoring, see the Installation section for available deployment scenarios.

To stop and remove the monitoring services, use:

docker-compose -f monitoring.yaml down

To remove all persisted data as well, use:

docker-compose -f monitoring.yaml down --volumes

Custom Provisioning

If you wish to override the default provisioning, you can uncomment the following Grafana volume mount and supply your own dashboards and data sources:

- ./grafana/provisioning/:/etc/grafana/provisioning/

Example directory structure and configurations for Grafana provisioning:

grafana/provisioning/ consists of three folders:

  • dashboards - to provision already available dashboards.

    For Grafana to load your dashboards, you need to place the following dashboard.yaml:

apiVersion: 1

providers:
  - name: 'influxdb'
    orgId: 1
    folder: ''
    type: file
    disableDeletion: false
    editable: true
    options:
      path: /etc/grafana/provisioning/dashboards

  • datasources - to provision your data source (InfluxDB)

    The datasource.yaml configuration should look like this:

apiVersion: 1

datasources:
- name: metrics
  type: influxdb
  access: proxy
  orgId: 1
  url: http://influxdb:8086
  database: metrics
  isDefault: false
  version: 1
  editable: true
  jsonData:
    httpMode: POST

  • notifiers - to provision your notification settings.

    The notifiers.yaml should look like this:

notifiers:
  - name: Slack
    type: slack
    uid: slackNotifier
    org_id: 1
    org_name: Ontotext.
    is_default: true
    send_reminder: true
    frequency: 1h
    disable_resolve_message: false
    # See `Supported Settings` section for settings supporter for each
    # alert notification type.
    settings:
      recipient: ""
      uploadImage: true
      url: webHook
  - name: Gmail
    type: email
    uid: emailNotifier
    org_id: 1
    org_name: Ontotext.
    is_default: false
    send_reminder: true
    frequency: 1h
    disable_resolve_message: false
    # See `Supported Settings` section for settings supporter for each
    # alert notification type.
    settings:
      addresses: "test@example.com"
      uploadImage: true

delete_notifiers:
  - name: Slack
    uid: slackNotifier
    org_id: 1
    org_name: Ontotext.
  - name: Gmail
    uid: emailNotifier.
    org_id: 1
    org_name: Ontotext.

Configuring Notifications & Alerts

Grafana can send notifications based on set alerts. It offers multiple integrations with e-mail and push notification services.

See more on how to configure them here.

To set an alert, follow the steps here.