Prometheus ​

GreptimeDB can be used as long-term storage for Prometheus. Using GreptimeDB as a Prometheus backend is a seamless experience. Since Prometheus has built-in support for setting up basic authentication information during the configuration of remote write and read, all you need to do is add your configured username and password to the config YAML file and you're good to go!

To configure Prometheus, use the following settings in the Prometheus configuration file (prometheus.yml):

remote_write:
- url: http://localhost:4000/v1/prometheus/write?db=public&physical_table=greptime_physical_table
#  basic_auth:
#    username: greptime_user
#    password: greptime_pwd

remote_read:
- url: http://localhost:4000/v1/prometheus/read?db=public
#  basic_auth:
#    username: greptime_user
#    password: greptime_pwd

remote_write:
- url: http://localhost:4000/v1/prometheus/write?db=public&physical_table=greptime_physical_table
#  basic_auth:
#    username: greptime_user
#    password: greptime_pwd

remote_read:
- url: http://localhost:4000/v1/prometheus/read?db=public
#  basic_auth:
#    username: greptime_user
#    password: greptime_pwd

The db parameter in the URL represents the database that we want to write data. It's optional. By default, the database is public. If you want to write to another database, you can create a new database and replace public with the new database name.

GreptimeDB automatically groups multiple Prometheus metrics (../clients/prometheus#data-model) into the corresponding logical tables, so you do not need to specify the logical table in the URL of remote_write.

The optional physical_table parameter in the URL represents the physical table, which reduces storage overhead for small tables and improves columnar compression efficiency. If unspecified, the greptime_physical_table is used by default; if the specified physical table does not exist, it will be automatically created.

The physical_table parameter only works when the with_metric_engine is enabled in the configuration file. It is enabled by default.

Here is a table of URL parameters:

Show tables when writing successfully:

show tables;

show tables;

+---------------------------------------------------------------+
| Tables                                                        |
+---------------------------------------------------------------+
| go_memstats_heap_inuse_bytes                                  |
| go_memstats_last_gc_time_seconds                              |
| net_conntrack_listener_conn_closed_total                      |
| prometheus_remote_storage_enqueue_retries_total               |
| prometheus_remote_storage_exemplars_pending                   |
| prometheus_remote_storage_read_request_duration_seconds_count |
| prometheus_rule_group_duration_seconds                        |
| prometheus_rule_group_duration_seconds_count                  |
| ......                                                        |
+---------------------------------------------------------------+

+---------------------------------------------------------------+
| Tables                                                        |
+---------------------------------------------------------------+
| go_memstats_heap_inuse_bytes                                  |
| go_memstats_last_gc_time_seconds                              |
| net_conntrack_listener_conn_closed_total                      |
| prometheus_remote_storage_enqueue_retries_total               |
| prometheus_remote_storage_exemplars_pending                   |
| prometheus_remote_storage_read_request_duration_seconds_count |
| prometheus_rule_group_duration_seconds                        |
| prometheus_rule_group_duration_seconds_count                  |
| ......                                                        |
+---------------------------------------------------------------+

Prometheus Metrics in GreptimeDB ​

When the metrics are written into GreptimeDB by remote write endpoint, they will be transformed as follows:

A primary key with all label columns will be created automatically. When a new label is added, it will be added into primary key automatically too.

Example: Prometheus metrics in GreptimeDB Table ​

prometheus_remote_storage_samples_total{instance="localhost:9090", job="prometheus",
remote_name="648f0c", url="http://localhost:4000/v1/prometheus/write"} 500

prometheus_remote_storage_samples_total{instance="localhost:9090", job="prometheus",
remote_name="648f0c", url="http://localhost:4000/v1/prometheus/write"} 500

This example will be transformed as a row in the table prometheus_remote_storage_samples_total：

A note for VictoriaMetrics remote write ​

VictoriaMetrics slightly modified Prometheus remote write protocol for better compression. The protocol is automatically enabled when you are using vmagent to send data to a compatible backend.

GreptimeDB has this variant supported, too. Just configure GreptimeDB's remote write url for vmagent. For example, if you have GreptimeDB installed locally:

vmagent -remoteWrite.url=http://localhost:4000/v1/prometheus/write

vmagent -remoteWrite.url=http://localhost:4000/v1/prometheus/write