Python

Host metrics refer to the metrics collected from the operating system of the host where your applications are running. These metrics include CPU, memory, disk, and network usage. Understanding host metrics is crucial as it helps you identify potential problems or bottlenecks that could affect the overall performance of your applications.

In this tutorial, we will show you how to collect host metrics, send them to GreptimeDB and visualize them.

Create Service

To experience the full power of GreptimeCloud, you need to create a service which contains a database with authentication. Open the GreptimeCloud console, signup and login. Then click the New Service button and config the following:

• Service Name: The name you want to describe your service.
• Description: More information about your service.
• Region: Select the region where the database is located.
• Plan: Select the pricing plan you want to use.

Now create the service and we are ready to write some metrics to it.

Write data

Prerequisites

• Python 3.10+

Example Application

In this section, we will create a quick start demo and showcase the core code to collect host metrics and send them to GreptimeDB. The demo is based on OTLP/HTTP. For reference, you can obtain the entire demo on GitHub.

To begin, create a new directory named quick-start-python to house our project and create a new file named requirements.txt in the directory and add the following:

opentelemetry-api==1.19.0
opentelemetry-exporter-otlp-proto-common==1.19.0
opentelemetry-exporter-otlp-proto-http==1.19.0
opentelemetry-instrumentation==0.40b0
opentelemetry-instrumentation-system-metrics==0.40b0
opentelemetry-proto==1.19.0
opentelemetry-sdk==1.19.0

Install the dependencies:

pip install -r requirements.txt

Once the required packages are installed,create a new file named main.py and write the code to create a metric export object that sends metrics to GreptimeDB. For the configuration about the exporter, please refer to OTLP integration documentation in GreptimeDB or GreptimeCloud.

from opentelemetry import metrics
from opentelemetry.instrumentation.system_metrics import SystemMetricsInstrumentor
from opentelemetry.sdk.resources import SERVICE_NAME, Resource
from opentelemetry.exporter.otlp.proto.http.metric_exporter import OTLPMetricExporter

auth = f"{username}:{password}"
b64_auth = base64.b64encode(auth.encode()).decode("ascii")
endpoint = f"https://{host}/v1/otlp/v1/metrics"
exporter = OTLPMetricExporter(
    endpoint=endpoint,
    headers={"Authorization": f"Basic {b64_auth}", "x-greptime-db-name": db},
    timeout=5)

Then attach the exporter to the MetricReader and start the host metrics collection:

metric_reader = PeriodicExportingMetricReader(exporter, 5000)
provider = MeterProvider(resource=resource, metric_readers=[metric_reader])

# Sets the global default meter provider
metrics.set_meter_provider(provider)
configuration = {
    "system.memory.usage": ["used", "free", "cached"],
    "system.cpu.time": ["idle", "user", "system", "irq"],
    "process.runtime.memory": ["rss", "vms"],
    "process.runtime.cpu.time": ["user", "system"],
}
SystemMetricsInstrumentor(config=configuration).instrument()

For more details about the code, you can refer to the OpenTelemetry Documentation.

Congratulations on successfully completing the core section of the demo! You can now run the complete demo by following the instructions in the README.md file on the GitHub repository.

The connection information can be found on the service page of GreptimeCloud console.

Visualize Data

Visualizing data in panels and monitoring metrics is important in a developer's daily work. From the GreptimeCloud console, click on Open Prometheus Workbench, then click on + New Ruleset and Add Group. Name the group host-monitor and add panels.

To add panels for all the tables you're concerned with, select a table and click on Add Panel one by one. Once you've added all the necessary panels, click on the Save button to save them. You can then view the panels in your daily work to monitor the metrics. Additionally, you can set up alert rules for the panels to be notified when the metrics exceed the threshold.