v0.14.0
Β· 16 min read
Release date: April 25, 2025
π Highlightsβ
- Bulk Insert: Adds support for high-performance bulk data ingestion.
- Otel Traces: Official releases otel traces.
- Fulltext Enhancements: Improves fulltext search with new functions (e.g.,
matches_term
,@@
operator), backend switching, and term optimizations. - Flow Engine: Introduces batching mode and dual engine capability for more flexible data processing.
- Partitioning: Supports column-wise and UUID-based partition rules for better scalability.
- Region Management: New tools for region sync, monitoring, leader region collection, and region failover handling.
- PromQL Engine: Enhanced query performance, range manipulation speed, and stability.
- SQL & API Improvements: Adds
REPLACE INTO
, new regex and JSON features, and various compatibility fixes. - Performance & Reliability: Optimizations for memory, storage, and parallel computation; numerous bug fixes for stability.
- Observability: Per-region metrics, expanded Grafana dashboards, and better profiling tools.
- Protocol & Integration: Implements Arrow Flight "DoPut" for fast data loading; improves PostgreSQL protocol support.
Plus: Many dependency upgrades, improved config and error handling, and new contributors joined!
π₯οΈ Dashboardβ
- Upgrade web dashboard to v0.9, supports SQL explain visualization and editor improvements.
Breaking changesβ
- refactor!: Remove
Value::DateTime
andValueRef::DateTime
. by @linyihai in #5616 - refactor!: make pipeline a required parameter when ingesting trace by @sunng87 in #5828
π Featuresβ
- feat: add
vec_dim
function by @xiaoniaoyouhuajiang in #5587 - feat: impl topk and bottomk by @killme2008 in #5602
- feat: skip printing full config content in sqlness by @waynexia in #5618
- feat(log-query): implement compound filter and alias expr by @waynexia in #5596
- feat: alias for boolean by @killme2008 in #5639
- feat: support some IP related functions by @waynexia in #5614
- feat: rewrite
json_encode_path
togeo_path
using compound type by @waynexia in #5640 - feat: get tables by ids in catalog manager by @fengjiachun in #5645
- feat: opentelemetry trace new data modeling by @sunng87 in #5622
- feat: update dashboard to v0.8.0 by @ZonaHex in #5666
- feat: support exact filter on time index column by @evenyag in #5671
- feat: introduce roaring bitmap to optimize sparse value scenarios by @zhongzc in #5603
- feat: time series distribution in scanner by @waynexia in #5675
- feat(promql): supports quantile and count_values by @killme2008 in #5652
- feat: also get index file&expose mito in metrics by @discord9 in #5680
- feat: add description for each grafana panel by @sunng87 in #5673
- feat: update promql-parser to 0.5 for duration literal by @sunng87 in #5682
- feat: add hint for logical region in RegionScanner by @evenyag in #5684
- feat: add a gauge for download tasks by @evenyag in #5681
- feat: alter region follower by @fengjiachun in #5676
- feat: support export command export data to s3 by @yihong0618 in #5585
- feat: enhancement information_schema.flows by @AntiTopQuark in #5623
- feat: add mysql kvbackend by @CookiePieWw in #5528
- feat: add Docker image tag information to step summary in dev-build workflow by @waynexia in #5692
- feat: expose virtual_host_style config for s3 storage by @evenyag in #5696
- feat: make empty parent_span_id null for v1 by @sunng87 in #5690
- feat: add simple extract processor by @shuiyisong in #5688
- feat: move default data path from /tmp to current directory by @sunng87 in #5719
- feat: add
vec_subvector
function by @SNC123 in #5683 - feat: load manifest manually in mito engine by @discord9 in #5725
- feat: disable http timeout by @sunng87 in #5721
- feat: update pipeline header name to x-greptime-pipeline-name by @sunng87 in #5710
- feat: add
vec_kth_elem
function by @Pikady in #5674 - feat: remove duplicated peer definition by @Wenbin1002 in #5728
- feat: add mysql election logic by @CookiePieWw in #5694
- feat: introduce
install_manifest_to
forRegionManifestManager
by @WenyXu in #5742 - feat: support regex in simple filter by @waynexia in #5753
- feat(flow): frontend client for handle sql by @discord9 in #5761
- feat: support explain analyze verbose by @evenyag in #5763
- feat: add/correct some kafka-related metrics by @waynexia in #5757
- feat: utilize blob metadata properties by @zhongzc in #5767
- feat(mito): allow skipping wal while creating tables by @v0y4g3r in #5740
- feat: introduce
RegionFollowerClient
trait by @WenyXu in #5771 - feat(flow): utils function for recording rule by @discord9 in #5768
- feat: add
AddRegionFollower
andRemoveRegionFollower
admin fn by @WenyXu in #5780 - feat: impl show region by @WenyXu in #5782
- feat: simple implementation of DictionaryVector by @waynexia in #5758
- feat: expose scanner metrics to df execution metrics by @evenyag in #5699
- feat(pipeline): support table name suffix templating in pipeline by @shuiyisong in #5775
- feat: implement
sync_region
for mito engine by @WenyXu in #5765 - feat(flow): time window expr by @discord9 in #5785
- feat: add limit for the number of running procedures by @WenyXu in #5793
- feat: introduce
CustomizedRegionLeaseRenewer
by @WenyXu in #5762 - feat: introduce read preference by @WenyXu in #5783
- feat: introduce
CollectLeaderRegionHandler
by @WenyXu in #5811 - feat: add backend field to fulltext options by @zhongzc in #5806
- feat: support REPLACE INTO statement by @evenyag in #5820
- feat(remote-wal): add remote wal prune procedure by @CookiePieWw in #5714
- feat: add
matches_term
function by @zhongzc in #5817 - feat: implement
sync_region
for metric engine by @WenyXu in #5826 - feat: introduce poison mechanism for procedure by @WenyXu in #5822
- feat: add
region_id
toCountdownTaskHandlerExt
by @WenyXu in #5834 - feat(remote-wal): send flush request when pruning remote wal by @CookiePieWw in #5825
- feat: add term as fulltext index request by @zhongzc in #5843
- feat: shorten possible wrong query range by @waynexia in #5849
- feat: make
admin_fn
macro usable outside common_function module by @WenyXu in #5850 - feat: pushdown select distinct in some cases by @evenyag in #5847
- feat(flow): batching mode engine by @discord9 in #5807
- feat: add
catalog_manager
toProcedureServiceHandler
by @WenyXu in #5873 - feat: apply terms with fulltext tantivy backend by @zhongzc in #5869
- feat: apply terms with fulltext bloom backend by @zhongzc in #5884
- feat: add query engine options by @waynexia in #5895
- feat: support altering fulltext backend by @zhongzc in #5896
- feat: report per-region metrics on region server by @waynexia in #5893
- feat: optimize
matches_term
with constant term pre-compilation by @zhongzc in #5886 - feat(flow): dual engine by @discord9 in #5881
- feat: Column-wise partition rule implementation by @v0y4g3r in #5804
- feat: support
@@
(AtAt) operator for term matching by @waynexia in #5902 - feat(mito): bulk insert request handling on datanode by @v0y4g3r in #5831
- feat: Add query pipeline http api by @linyihai in #5819
- feat: add json parse processor by @shuiyisong in #5910
- feat: implement Arrow Flight "DoPut" in Frontend by @MichaelScofield in #5836
- feat: sync region followers after altering regions by @WenyXu in #5901
- feat: prevent migrating a leader region to a peer that already has a region follower by @WenyXu in #5923
- feat: introduce
high_watermark
for remote wal logstore by @CookiePieWw in #5877 - feat: enable submitting wal prune procedure periodically by @CookiePieWw in #5867
- feat: update readme by @killme2008 in #5936
- feat: implement otel-arrow protocol for GreptimeDB by @v0y4g3r in #5840
- feat: support building
metasrv
with selector from plugins by @WenyXu in #5942 - feat: add
exclude_peer_ids
toSelectorOptions
by @WenyXu in #5949 - feat: add format support for promql http api (not prometheus) by @sunng87 in #5939
- feat: update dashboard to v0.9.0 by @ZonaHex in #5948
- feat: introduce flush metadata region task for metric engine by @WenyXu in #5951
- feat: node excluder by @MichaelScofield in #5964
- feat(flow): use batching mode&fix sqlness by @discord9 in #5903
- feat: track region failover attempts and adjust timeout by @WenyXu in #5952
- feat: enhance selector with node exclusion support by @WenyXu in #5966
- feat: improve observability of region migration procedure by @WenyXu in #5967
- feat: remove hyper parameter from promql functions by @waynexia in #5955
- feat: allow forced region failover for local WAL by @WenyXu in #5972
π Bug Fixesβ
- fix: check physical region before use by @waynexia in #5612
- fix: update typos rules to fix ci by @yihong0618 in #5621
- fix: speed up cargo build using sallow clone by @yihong0618 in #5620
- fix: increase timeout for opening candidate region and log elapsed time by @WenyXu in #5627
- fix: out of bound during bloom search by @zhongzc in #5625
- fix: refactor region leader state validation by @WenyXu in #5626
- fix: properly display CJK characters in table/column comments by @v0y4g3r in #5633
- fix: prevent failover of regions to the same peer by @WenyXu in #5632
- fix: interval rewrite rule that messes up show create flow function by @v0y4g3r in #5642
- fix: unable to install software-properties-common in dev builder by @daviderli614 in #5643
- fix(metric-engine): group DDL requests by @WenyXu in #5628
- fix: window sort not apply when other column alias to time index name by @discord9 in #5634
- fix: recover plan schema after dist analyzer by @waynexia in #5665
- fix: flaky test in sqlness by fix random port by @yihong0618 in #5657
- fix: skip schema check to avoid schema mismatch brought by metadata by @v0y4g3r in #5662
- fix: use
DateTime
instead ofNaiveDateTime
by @shuiyisong in #5669 - fix: update column requirements to use Column type instead of String by @waynexia in #5672
- fix: correct stalled count by @evenyag in #5678
- fix: FlowInfoValue's compatibility by @discord9 in #5695
- fix: conversion from TableMeta to TableMetaBuilder by @v0y4g3r in #5693
- fix: mysql prepare bool value by @discord9 in #5732
- fix: mysql prepare limit&offset param by @discord9 in #5734
- fix: wrap table name with `` by @CookiePieWw in #5748
- fix: handle nullable default value by @discord9 in #5747
- fix: properly give placeholder types by @discord9 in #5760
- fix: support name matcher in label values by @evenyag in #5773
- fix: typo variadic by @waynexia in #5800
- fix: close issue #3902 since upstream fixed by @yihong0618 in #5801
- fix: correct error status code by @WenyXu in #5802
- fix: interval cast expression can't work in range query, #5805 by @killme2008 in #5813
- fix: skip replacing exprs of the DistinctOn node by @evenyag in #5823
- fix: throw errors instead of ignoring by @WenyXu in #5792
- fix: get root cause of the procedure when converting to pb by @evenyag in #5841
- fix: remove metadata region options by @v0y4g3r in #5852
- fix: include follower peers in region distribution by @WenyXu in #5844
- fix: gRPC connection pool leak by @fengys1996 in #5876
- fix: always rejects write while downgrading region by @WenyXu in #5842
- fix: flaky test on windows by @yihong0618 in #5890
- fix: ensure logical regions are synced during region sync by @WenyXu in #5878
- fix: memtable panic by @v0y4g3r in #5894
- fix: preserve timestamp precision of irate by @waynexia in #5904
- fix: support duration to interval conversion in PostgreSQL protocol by @v0y4g3r in #5913
- fix: oom for sqlness test in container by @yihong0618 in #5917
- fix: label values potential panic by @waynexia in #5921
- fix: avoid double schema projection in file format readers by @WenyXu in #5918
- fix: anchor regex string to fully match in promql by @waynexia in #5920
- fix: wrong error msg in pipeline by @shuiyisong in #5937
- fix: filter doesn't consider default values after schema change by @evenyag in #5912
- fix: remove obsolete failover detectors after region leader change by @WenyXu in #5944
- fix: parse flow expire after interval by @discord9 in #5953
- fix: use max in flushed entry id and topic latest entry id by @CookiePieWw in #5946
- fix: store flow query ctx on creation by @discord9 in #5963
- fix: try prune one less by @discord9 in #5965
- fix: upgrade sqlparse and validate align in range query by @killme2008 in #5958
- fix: security update by @sunng87 in #5982
- fix: conn timeout&refactor: better err msg by @discord9 in #5974
π Refactorβ
- refactor: add pipeline concept to OTLP traces and remove OTLP over gRPC by @sunng87 in #5605
- refactor: simplify udf by @MichaelScofield in #5617
- refactor: rename table function to admin function by @waynexia in #5636
- refactor: remove or deprecated existing UDAF implementation by @waynexia in #5637
- refactor: remove cluster id field by @v0y4g3r in #5610
- refactor(mito): move wal sync task to background by @v0y4g3r in #5677
- refactor: update jaeger api implementation for new trace modeling by @sunng87 in #5655
- refactor: remove trace id from primary key in
opentelemetry_traces
table by @zyy17 in #5733 - refactor: make frontend instance clear by @fengys1996 in #5754
- refactor: move
list_flow_stats
toClusterInfo
trait. by @WenyXu in #5774 - refactor: remove useless partition legacy code by @v0y4g3r in #5786
- refactor: remove useless region follower legacy code by @WenyXu in #5787
- refactor: remove useless region follower legacy code by @WenyXu in #5795
- refactor: skip re-taking arrays in memtable if possible by @MichaelScofield in #5779
- refactor: remove deprecated find_unique method by @waynexia in #5790
- refactor: remove mode option in configuration files by @fengys1996 in #5809
- refactor: allow bloom filter search to apply
and
conjunction by @zhongzc in #5770 - refactor: remove prom store write dispatch by @shuiyisong in #5812
- refactor: remove backoff config by @WenyXu in #5808
- refactor: add time range for jager get operations API by @zyy17 in #5791
- refactor: improve performance for Jaeger APIs by @zyy17 in #5838
- refactor: check and fix super import by @waynexia in #5846
- refactor(flow): make start flownode clearer by @discord9 in #5848
- refactor: abstract index source from fulltext index applier by @zhongzc in #5845
- refactor: improve error code handling in status code conversion by @WenyXu in #5851
- refactor: simplify tls key read code and add sec1 key support by @sunng87 in #5856
- refactor: avoid empty display in errors by @MichaelScofield in #5858
- refactor: add
partition_rules_for_uuid()
by @zyy17 in #5743 - refactor: unify all dashboards and use
dac
tool to generate intermediate dashboards by @zyy17 in #5933 - refactor:
run_pipeline
parameters by @shuiyisong in #5954
π Documentationβ
- docs: tsbs update by @discord9 in #5608
- docs: update cluster dashboard to make opendal panel works by @evenyag in #5661
- docs: adds news to readme by @killme2008 in #5735
- docs: rfc for wal purge by @CookiePieWw in #5475
- docs: update readme by @killme2008 in #5891
- docs: memory profile scripts by @discord9 in #5922
- docs: fix some units and adds the opendal errors panel by @evenyag in #5962
β‘ Performanceβ
- perf: rm coalesce batch when target_batch_size > fetch limit by @discord9 in #5658
- perf(prom): optimize label values query by @WenyXu in #5653
- perf: support in list in simple filter by @waynexia in #5709
- perf: introduce
simd_json
for parsing ndjson by @shuiyisong in #5794 - perf: evolve promql execution engine by @waynexia in #5691
- perf: faster range manipulate for promql by @waynexia in #5859
- perf: avoid duplicate computation in promql by @waynexia in #5863
- perf: keep compiled regex in SimpleFilterEvaluator to avoid re-compiling by @evenyag in #5919
- perf: optimize fulltext zh tokenizer for ascii-only text by @zhongzc in #5975
π§ͺ Testingβ
- test: looser condition by @discord9 in #5816
- test: add tests to ensure nested data structure for identity pipeline by @sunng87 in #5888
- test: update configs to enable auto wal prune by @CookiePieWw in #5938
- test: use random seed for window sort fuzz test by @discord9 in #5950
βοΈ Miscellaneous Tasksβ
- chore: bump version to 0.13.0 by @WenyXu in #5611
- chore: support specifying
skipping
index in pipeline by @shuiyisong in #5635 - ci: remove ubuntu 20.04 runners by @sunng87 in #5545
- ci: bump dev-builder image version to 2024-12-25-a71b93dd-20250305072908 by @daviderli614 in #5651
- chore: improve
/v1/jaeger/api/trace/{trace_id}
's resp by @shuiyisong in #5663 - chore: impl ref and ref_mut for json like by @paomian in #5679
- chore: make memorykv write happily by @fengjiachun in #5686
- chore: check region wal provider on startup to avoid inconsistency by @v0y4g3r in #5687
- chore: support
application/x-ndjson
for log ingest by @shuiyisong in #5697 - chore: support
tag
in transform by @shuiyisong in #5701 - chore: support
inverted
index in pipeline by @shuiyisong in #5700 - chore: update flate2 version by @yihong0618 in #5706
- chore: use
Bytes
instead of string in bulk ingestion by @shuiyisong in #5717 - chore: bump version to 0.14.0 by @waynexia in #5711
- chore: revert commit update flate2 version (#5706)" by @yihong0618 in #5715
- chore: add some method for log query handler by @paomian in #5685
- chore: merge error files under pipeline crate by @shuiyisong in #5738
- chore: add datanode write rows to grafana dashboard by @sunng87 in #5745
- chore: support custom time index selector for identity pipeline by @shuiyisong in #5750
- chore: ut and some fix by @CookiePieWw in #5752
- chore: remove
Transformer
trait by @shuiyisong in #5772 - chore: upgrade some dependencies by @fengys1996 in #5777
- chore: accept table options in auto create table from hints by @shuiyisong in #5776
- chore: expose some methods by @fengys1996 in #5784
- chore: expose modules by @WenyXu in #5810
- chore: remove patch.crates-io for rustls by @fengys1996 in #5832
- chore: update datafusion family by @MichaelScofield in #5814
- chore: remove obsolete way of passing http configurations through env by @MichaelScofield in #5864
- chore: un-allow clippy's "readonly_write_lock" by @MichaelScofield in #5862
- chore: remove repl by @MichaelScofield in #5860
- ci: not push latest image when schedule release by @daviderli614 in #5883
- chore: upgrade opendal to 0.52 by @WenyXu in #5857
- chore: add heartbeat metrics by @fengjiachun in #5929
- chore: rm dev opt level 3 by @discord9 in #5932
- chore: use
once_cell
to avoid parse everytime in pipeline exec by @shuiyisong in #5943 - chore: better buckets for heartbeat stat size histogram by @fengjiachun in #5945
- chore: assert plugin uniqueness by @MichaelScofield in #5947
- chore: remove unused attribute by @fengys1996 in #5960
- chore: update dac tools image and docs by @zyy17 in #5961
- chore: remove coderabbit by @killme2008 in #5969
Buildβ
- build: use ubuntu-22.04 base image release
dev-build
image by @daviderli614 in #5554
New Contributorsβ
- @Wenbin1002 made their first contribution in #5728
- @Pikady made their first contribution in #5674
- @SNC123 made their first contribution in #5683
- @xiaoniaoyouhuajiang made their first contribution in #5587
All Contributorsβ
We would like to thank the following contributors from the GreptimeDB community:
@AntiTopQuark, @CookiePieWw, @MichaelScofield, @Pikady, @SNC123, @Wenbin1002, @WenyXu, @ZonaHex, @daviderli614, @discord9, @evenyag, @fengjiachun, @fengys1996, @killme2008, @linyihai, @paomian, @shuiyisong, @sunng87, @v0y4g3r, @waynexia, @xiaoniaoyouhuajiang, @yihong0618, @zhongzc, @zyy17