Trace Data Modeling
This section currently in the experimental stage and may be adjusted in future versions.
In this section, we will cover how trace data is modeled in GreptimeDB as tables.
We reuse the concept of Pipeline for trace data modeling. However, note that at the moment, only built-in pipelines are supported.
Data Model Versioning
First, the data types and features in GreptimeDB are evolving. For forward-compatibility, we use the pipeline name for data model versioning. Currently we have the following built-in pipeline for trace:
greptime_trace_v1: Flattens attributes into individual table columns.greptime_trace_v2: Stores attributes in three fixedJSON2columns.
Specify the data model in OpenTelemetry OTLP/HTTP requests with the
x-greptime-pipeline-name header. Choose v1 for flattened attribute columns,
or v2 for JSON2 attributes without adding a table column for every attribute key.
We may introduce new data models by adding new available pipeline names. Note that a new pipeline may not be compatible with previous ones, so you are recommended to use it in a new table.
greptime_trace_v2
The v2 model stores one row per span with a fixed set of 19 columns. It uses JSON2 for attributes, so new attribute keys do not add table columns.
| Columns | SQL type | Meaning |
|---|---|---|
timestamp | TIMESTAMP(9) | Span start time; the time index. |
timestamp_end | TIMESTAMP(9) | Span end time. |
duration_nano | BIGINT | Span duration in nanoseconds. |
trace_id, span_id, parent_span_id | STRING | Trace and span identifiers. |
span_kind, span_name | STRING | Span kind and operation name. |
span_status_code, span_status_message | STRING | Span status. |
trace_state | STRING | W3C trace state. |
scope_name, scope_version | STRING | Instrumentation scope. |
service_name | STRING | Nullable tag and primary key, extracted from the service.name resource attribute. |
span_attributes, scope_attributes, resource_attributes | JSON2 | Attribute objects, including nested values. |
span_events, span_links | JSON | Event and link arrays; [] when empty. |
Unlike v1, v2 keeps service.name in resource_attributes as well as extracting
it into service_name. If the resource does not provide a usable service name,
the tag is NULL.
The table is created automatically on the first write. The default table name
is still opentelemetry_traces. V2 uses the same trace-ID partitioning, skipping
indexes on service_name, trace_id, and parent_span_id, append-only mode,
and auxiliary service/operation tables described below.
Use a new table when switching from v1 to v2. Changing the pipeline header does not migrate existing data: writes using v2 are rejected for a table marked as v1, and vice versa. Existing v1 tables can continue using the v1 pipeline. See Ingestion and Query for exporter configuration and SQL examples.
V2 also supports the semantic graph, including built-in entity declarations, explicit declarations, and relationship derivation. Span pairing can cross v1 and v2 tables. Entity declarations retain v1 attribute reference names, which resolve to keys in JSON2; see Declaring entities and relationships.
greptime_trace_v1
The greptime_trace_v1 data model is pretty straight-forward. By default,
trace data is stored in a table named opentelemetry_traces. You can customize
the table name by specifying the x-greptime-trace-table-name header in your
OTLP/HTTP requests.
- Each row represents a single span.
- It maps most common data fields from OpenTelemetry Trace data model to GreptimeDB's table columns.
service_nameis extracted fromresource_attributes["service.name"]and used as a Tag (part of the Primary Key).timestampis the start time of the span and is used as the Time Index.- A new
duration_nanocolumn is generated usingend_time - start_time. - All attributes fields are flattened into columns using the name pattern:
[span|resource|scope]_attributes.[attribute_key].- Note:
resource_attributes.service.nameis excluded from flattening as it is already stored in theservice_namecolumn. - If the attribute value is a compound type like
ArrayorKvlist, it is serialized toJSONtype of GreptimeDB.
- Note:
- Compound fields,
span_linksandspan_eventsare stored asJSONtype.
The table will be automatically generated when your first data item arrived. It also follows our schema-less principle to update the table schema automatically for new columns, for example, the new attribute fields.
For greptime_trace_v1, GreptimeDB also reconciles attribute column types while
the schema evolves. When a trace table already exists, the existing table schema
is authoritative for compatible incoming values. Scalar values can be coerced to
an existing compatible column type, and existing Int64 attribute columns may be
widened to Float64 when incoming values contain both integers and floats. If a
span still cannot be written, GreptimeDB may reject that span while accepting the
other spans in the request.
The following example shows a span generated by OpenTelemetry Django instrumentation:
timestamp | 2025-05-07 10:03:29.657544
timestamp_end | 2025-05-07 10:03:29.661714
duration_nano | 4169970
trace_id | fb60d19aa36fdcb7d14a71ca0b9b42ae
span_id | 49806a2671f2ddcb
span_kind | SPAN_KIND_SERVER
span_name | POST todos/
span_status_code | STATUS_CODE_UNSET
span_status_message |
trace_state |
scope_name | opentelemetry.instrumentation.django
scope_version | 0.51b0
service_name | myproject
span_attributes.http.request.method | POST
span_attributes.url.full |
span_attributes.server.address | django:8000
span_attributes.network.peer.address |
span_attributes.server.port | 8000
span_attributes.network.peer.port |
span_attributes.http.response.status_code | 201
span_attributes.network.protocol.version | 1.1
resource_attributes.telemetry.sdk.language | python
resource_attributes.telemetry.sdk.name | opentelemetry
resource_attributes.telemetry.sdk.version | 1.30.0
span_events | []
span_links | []
parent_span_id | eccc18b6fc210f31
span_attributes.db.system |
span_attributes.db.name |
span_attributes.db.statement |
span_attributes.url.scheme | http
span_attributes.url.path | /todos/
span_attributes.client.address | 10.89.0.5
span_attributes.client.port | 44302
span_attributes.user_agent.original | python-requests/2.32.3
span_attributes.http.route | todos/
To check the table definition, you can use the show create table opentelemetry_traces
statement. For a newly created table, an output like this is expected:
Table | opentelemetry_traces
Create Table | CREATE TABLE IF NOT EXISTS "opentelemetry_traces" ( +
| "timestamp" TIMESTAMP(9) NOT NULL, +
| "timestamp_end" TIMESTAMP(9) NULL, +
| "duration_nano" BIGINT NULL, +
| "trace_id" STRING NULL SKIPPING INDEX WITH(granularity = '10240', type = 'BLOOM'), +
| "span_id" STRING NULL, +
| "span_kind" STRING NULL, +
| "span_name" STRING NULL, +
| "span_status_code" STRING NULL, +
| "span_status_message" STRING NULL, +
| "trace_state" STRING NULL, +
| "scope_name" STRING NULL, +
| "scope_version" STRING NULL, +
| "service_name" STRING NULL SKIPPING INDEX WITH(granularity = '10240', type = 'BLOOM'),+
| "span_attributes.http.request.method" STRING NULL, +
| "span_attributes.url.full" STRING NULL, +
| "span_attributes.server.address" STRING NULL, +
| "span_attributes.network.peer.address" STRING NULL, +
| "span_attributes.server.port" BIGINT NULL, +
| "span_attributes.network.peer.port" BIGINT NULL, +
| "span_attributes.http.response.status_code" BIGINT NULL, +
| "span_attributes.network.protocol.version" STRING NULL, +
| "resource_attributes.telemetry.sdk.language" STRING NULL, +
| "resource_attributes.telemetry.sdk.name" STRING NULL, +
| "resource_attributes.telemetry.sdk.version" STRING NULL, +
| "span_events" JSON NULL, +
| "span_links" JSON NULL, +
| "parent_span_id" STRING NULL SKIPPING INDEX WITH(granularity = '10240', type = 'BLOOM'),+
| "span_attributes.db.system" STRING NULL, +
| "span_attributes.db.name" STRING NULL, +
| "span_attributes.db.statement" STRING NULL, +
| "span_attributes.url.scheme" STRING NULL, +
| "span_attributes.url.path" STRING NULL, +
| "span_attributes.client.address" STRING NULL, +
| "span_attributes.client.port" BIGINT NULL, +
| "span_attributes.user_agent.original" STRING NULL, +
| "span_attributes.http.route" STRING NULL, +
| TIME INDEX ("timestamp"), +
| PRIMARY KEY ("service_name") +
| ) +
| PARTITION ON COLUMNS ("trace_id") ( +
| trace_id < '1', +
| trace_id >= 'f', +
| trace_id >= '1' AND trace_id < '2', +
| trace_id >= '2' AND trace_id < '3', +
| trace_id >= '3' AND trace_id < '4', +
| trace_id >= '4' AND trace_id < '5', +
| trace_id >= '5' AND trace_id < '6', +
| trace_id >= '6' AND trace_id < '7', +
| trace_id >= '7' AND trace_id < '8', +
| trace_id >= '8' AND trace_id < '9', +
| trace_id >= '9' AND trace_id < 'a', +
| trace_id >= 'a' AND trace_id < 'b', +
| trace_id >= 'b' AND trace_id < 'c', +
| trace_id >= 'c' AND trace_id < 'd', +
| trace_id >= 'd' AND trace_id < 'e', +
| trace_id >= 'e' AND trace_id < 'f' +
| ) +
| ENGINE=mito +
| WITH( +
| append_mode = 'true', +
| table_data_model = 'greptime_trace_v1' +
| )
Shared Table Behavior
The following behavior applies to both v1 and v2.
Partition Rules
Ensure that the first character of trace_id is evenly distributed to avoid skew
between partitions.
We included default partition
rules for
trace table on the trace_id column based on the first character of it. This is
optimised for retrieve trace spans by the trace id.
The partition rule introduces 16 partitions for the table, a reasonable starting point for a 3-5 datanode setup. Ingestion rate, query concurrency, and total region count across the cluster may call for a different partition count.
To customize the partition rule, you can:
- Create a new table ahead-of-time by copying the DDL output by
show create tableon original table, update thePARTITION ONsection to include your own rules. - Use
x-greptime-hintsHTTP header in your OTLP ingestion request, include a hinttrace_table_partitions=nwherenis the partition number. Setnto0or1to disable partitioning. Any other value must be a power of two between2and65536; the table creation fails otherwise.
Index
We include skipping
index on service_name,
trace_id, and parent_span_id for most typical queries.
In real-world, you may want to speed up queries on other fields like an attribute field. It's possible by apply additional index on these fields using alter table statement.
Unlike partition rules, index can be created on existing table and be affective on new data.
Partial Success
When a trace request contains multiple spans, GreptimeDB may accept the spans
that can be written and reject only the spans that fail deterministic validation,
such as incompatible schema or values. In this case the OTLP response contains a
partial_success section with rejected_spans and error_message. If all spans
are rejected, GreptimeDB returns 400 Bad Request.
Append-only Mode
By default, trace table created by OpenTelemetry API are in append only mode.
TTL
If you want to set a TTL for a trace table, send x-greptime-hints: ttl=7d
with the OTLP request to set a seven-day TTL when the table is created. See table options.
You can apply TTL on trace table.
Auxiliary Tables
When you ingest trace data, GreptimeDB automatically creates two auxiliary
tables to facilitate searching for services and operations. These tables are
named by appending _services and _operations to your main trace table name.
By default, these are named opentelemetry_traces_services and
opentelemetry_traces_operations. If you customize the main trace table name
using the x-greptime-trace-table-name HTTP header, the auxiliary tables will
be named accordingly (e.g., <custom_table_name>_services and
<custom_table_name>_operations).
Services Table (opentelemetry_traces_services)
This table stores the list of unique service names found in the trace data.
- Columns:
timestamp: A constant timestamp (2100-01-01 00:00:00) used for all entries.service_name: The name of the service (Tag).
Operations Table (opentelemetry_traces_operations)
This table stores the list of unique operations (service, span name, and span kind) found in the trace data.
- Columns:
timestamp: A constant timestamp (2100-01-01 00:00:00) used for all entries.service_name: The name of the service (Tag).span_name: The name of the span (Tag).span_kind: The kind of the span (Tag).