Skip to content

Set and get Deployment metadata using the API

Introduction

ShimmerCat has designed and implemented a graphical monitoring system, see an example in the image below. The tool allows the client and the internal Cloud Managed Services department to access detailed information on the client platform. We can know, among other, how many requests are handled in a period of time, the average response time, and meanwhile, how many resources (CPU, memory, I/O) are being used. We can also configure alerts in response to certain events, so we can make decisions from them, such as expanding the hardware of a particular client in the face of increased traffic to their servers.

Image

This process requires three components, collector, database and visualizer. There are different of each type, and the compatibilities between them are very diverse, in addition to performance, ease of use and installation between them.

The collector, is responsible for collecting all the data of our machine, continuously, in real time, in order to send it to the database, which will be the one who stores it, and finally the viewer is the one who shows us a graph with the data.

In this "sea of ‚Äč‚Äčoptions" we have opted to works with Prometheus and Grafana, but before we have said that we need three components, and what happens is that Prometheus acts both as a database, and as a collector (Prometheus Node Exporter and Prometheus Haproxy Exporter) and Grafana will be our visualizer.

What is Prometheus?

Prometheus, a Cloud Native Computing Foundation project, is a systems and service monitoring system. It collects metrics from configured targets at given intervals, evaluates rule expressions, displays the results, and can trigger alerts if some condition is observed to be true.

Prometheus's main features are:

  • a multi-dimensional data model with time series data identified by metric name and key/value pairs
  • PromQL, a flexible query language to leverage this dimensionality
  • no reliance on distributed storage; single server nodes are autonomous
  • time series collection happens via a pull model over HTTP
  • pushing time series is supported via an intermediary gateway
  • targets are discovered via service discovery or static configuration
  • multiple modes of graphing and dashboarding support

What is Grafana?

Grafana, is an open-source, general purpose dashboard and graph composer, which runs as a web application. It supports graphite, InfluxDB, Prometheus or opentsdb as backends. Grafana allows you to query, visualize, alert on and understand your metrics no matter where they are stored. In the example image below you can see a Grafana dashboard with monitoring for edge server metrics like memory usage and system load.

Image

Install and configure exporters.

We have a project that contains the ansible recipes that are responsible for the installation of exporters (Prometheus Node Exporter and Prometheus Haproxy Exporter)

You should clone the repository and execute the roles install_prometheus_node_exporter andinstall_haproxy_exporter, simply by running the playbook 3.0-monitoring.yml. Before running the mentioned playbook, be sure to update your inventory (production file) as well as the variables haproxy_auth_pass, prometheus_node_exporter_port and haproxy_exporter_port in the group_vars/edges.yml file.

ansible-playbook -i production 3.0-monitoring.yml

Update the deployment metadata

To add and/or update the deployment metadata you can use our ansible-sc_pack-all-in-one repository, specifically the deployment_metadata role.

You should clone the repository and execute deployment_metadata role, simply by running the playbook 4.0-deployment_metadata.yml.

Before running the mentioned playbook, be sure to update your inventory (inventories/production/hosts file, specifically, you should add the IP vps-ip of the VPS that you want to monitor in the group variable [edges_deployment_metadata]) as well as the variables in the inventories/production/group_vars/edges_deployment_metadata.yml file.

./ansible_helper.sh -i inventories/production 4.0-deployment_metadata.yml

Below is an explanation of the variables used:

  • domains:

    • domains to monitor
  • deployment:

    • tags: the deployment site tag
    • at_directory: the folder where sc_pack has been deployed.
  • datacenter_place

    • place_name: city where the VPS is located.
    • provider_company: data center provider: eg. UpCloud, IBM, AWS.
    • country_code: Country code in which the VPS is located.
  • edge_server:

    • server_name : VPS identifier.
  • edge_telemetry_endpoints:

    • telemetry_class: haproxy_exporter or node_exporter.
    • http_port: haproxy_exporter (9101) or node_exporter (9112) port.
    • metrics_path: haproxy_exporter (/metrics) or node_exporter (/metrics/node_exporter) metrics path.
  • deployment_telemetry_endpoint:

    • telemetry_class: shimmercat_exporter
    • http_port: shimmercat http port, deployed in deployment at_directory variable.
    • metrics_path: /.well-known/shimmercat/metric-scrapping/prometheus.io/
    • at_deployment_id: the deployment site id, deployed in deployment at_directory variable.
  • edge_server_network_access:

    • use_for_monitoring: True or False
    • used_by_traffic: True or False
    • description: some information about the VPS subject to monitoring.
    • ip_address: VPS IP

An example of this data can be seen below:

request_data = {
        'domains': ['www.demo.com'],
        'deployment': {
            'tags': 'demo',
            'at_directory': '/srv/deployment_demo',
        },
        'datacenter_place': {
            'place_name': 'Stockholms',
            'provider_company': 'Upcloud',
            'country_code': 'SE',
        },
        'edge_server': {
            'server_name': 'vps_demo_stockholms_1'
        },
        'edge_telemetry_endpoints': [
            {
                'telemetry_class': 'haproxy_exporter',
                'http_port': 9101,
                'metrics_path': '/metrics',
            },
            {
                'telemetry_class': 'node_exporter',
                'http_port': 9112,
                'metrics_path': '/metrics/node_exporter',
            },
        ],
        'deployment_telemetry_endpoint': {
            'telemetry_class': 'shimmercat_exporter',
            'http_port': 8010,
            'metrics_path': '/.well-known/shimmercat/metric-scrapping/prometheus.io/',
            'at_deployment_id': 1,
        },
        'edge_server_network_access': {
            'use_for_monitoring': True,
            'used_by_traffic': True,
            'description': 'A demo',
            'ip_address': '127.0.0.1'
        }
    }

Get the prometheus config data

To get the prometheus config data using the API you can use something like that:

curl -X GET https://accelerator.shimmercat.com/presentation-api/v1/prometheus-conf/by-deployment-tag/<deployment-tag>/ -H 'Authorization: Token <your_authentication_token>' -H 'Content-type: application/json'

Response example:

{
  'prometheus_config': [
    {
      'metrics_path': '/.well-known/shimmercat/metric-scrapping/prometheus.io/',
      'job_name': 'shimmercat__at__deployment_demo_server_vps_demo_stockholms_1_port_8010',
      'static_configs': [
        {
          'targets': [
            '127.0.0.1:8010'
          ],
          'labels': {
            'customer': 'demo',
            'at_edge__server_name': 'vps_demo_stockholms_1',
            'telemetry_level': 'deployment',
            'at_edge__datacenter__datacenter_provider_company': 'UpCloud',
            'at_directory': '/srv/deployment_demo',
            'at_edge__datacenter__country_code': 'SE',
            'at_edge__edge_ip_address': '127.0.0.1',
            'at_edge__datacenter__datacenter_place': 'Stockholms'
          }
        }
      ],
      'scheme': 'http'
    },
    {
      'metrics_path': '/metrics',
      'job_name': 'haproxy_exporter__at__vps_demo_stockholms_1',
      'static_configs': [
        {
          'targets': [
            '127.0.0.1:9101'
          ],
          'labels': {
            'edge_ip_address': '127.0.0.1',
            'server_name': 'vps_demo_stockholms_1',
            'telemetry_level': 'edge_server',
            'datacenter__datacenter_provider_company': 'UpCloud',
            'datacenter__datacenter_place': 'Stockholms',
            'datacenter__country_code': 'SE'
          }
        }
      ],
      'scheme': 'http'
    },
    {
      'metrics_path': '/metrics/node_exporter',
      'job_name': 'node_exporter__at__vps_demo_stockholms_1',
      'static_configs': [
        {
          'targets': [
            '127.0.0.1:9112'
          ],
          'labels': {
            'edge_ip_address': '127.0.0.1',
            'server_name': 'vps_demo_stockholms_1',
            'telemetry_level': 'edge_server',
            'datacenter__datacenter_provider_company': 'UpCloud',
            'datacenter__datacenter_place': 'Stockholms',
            'datacenter__country_code': 'SE'
          }
        }
      ],
      'scheme': 'http'
    }
  ],
  'customer': 'demo'
}