Blog | GreptimeDB

v0.7.1

2024年3月14日 · 阅读需 3 分钟

Release date: March 14, 2024

This is a patch release, containing an important bug fix while decoding the Prometheus remote write proto https (#3505).

Future critical bug fixes were delivered at v0.7.2. It's highly recommended to upgrade to v0.7.2 if you're using v0.7.

Breaking changes

fix!: remove error message from http header to avoid panic by @sunng87 in #3506

Bug fixes

fix: add support for influxdb basic auth by @shuiyisong in #3437
fix: fix incorrect COM_STMT_PREPARE reply by @WenyXu in #3463
fix: impl RecordBatchStream method explicitly by @shuiyisong in #3482
fix: make max-txn-ops limit valid by @fengjiachun in #3481
fix: fix f64 has no sufficient precision during parsing by @WenyXu in #3483
fix: freeze data buffer in shard by @evenyag in #3468
fix: allow passing extra table options by @evenyag in #3484
fix(common-time): allow building nanos timestamp from parts split from i64::MIN by @zhongzc in #3493
fix: adjust fill behavior of range query by @Taylor-lagrange in #3489
fix: decoding prometheus remote write proto doesn't reset the value by @waynexia in #3505
fix: correctly generate sequences when the value is pre-existed by @MichaelScofield in #3502

Improvements

feat: impl some "set"s to adapt to some client apps by @MichaelScofield in #3443
feat: update dashboard to v0.4.8 by @ZonaHex in #3450
feat: max-txn-ops option by @fengjiachun in #3458
feat(influxdb): add db query param support for v2 write api by @etolbakov in #3445
feat: support first_value/last_value in range query by @Taylor-lagrange in #3448
feat: clamp function by @waynexia in #3465
feat(fuzz): validate columns by @WenyXu in #3485
feat: to_timezone function by @tisonkun in #3470
feat(flow): accumulator for aggr func by @discord9 in #3396
feat(flow): plan def by @discord9 in #3490
feat: improve prom write requests decode performance by @v0y4g3r in #3478
feat(fuzz): add insert target by @zhongzc in #3499
feat(fuzz): add alter table target by @WenyXu in #3503
feat: support decode gzip if influxdb write specify it by @tisonkun in #3494
feat: Partition memtables by time if compaction window is provided by @evenyag in #3501

Others

build(deps): upgrade opendal to 0.45.1 by @tisonkun in #3432
chore: add bin opt to build cmd by @WenyXu in #3440
chore: specify binary name by @WenyXu in #3449
test: add fuzz test for create table by @WenyXu in #3441
refactor: separate the quote char and value by @WenyXu in #3455
perf: Reduce decode overhead during pruning keys in the memtable by @evenyag in #3415
chore: remove repetitive words by @gcmutator in #3469
refactor: introduce new Output with OutputMeta by @shuiyisong in #3466
refactor: make http api returns non-200 status code by @crwen in #3473
ci: use another mirror for etcd image by @waynexia in #3486
perf: more benchmarks for memtables by @evenyag in #3491
refactor: validate constraints eagerly by @tisonkun in #3472
ci: attempt to setup docker cache for etcd by @sunng87 in #3488
chore: bump version to v0.7.1 by @evenyag in #3510
docs: add v0.7 TSBS benchmark result by @evenyag in #3512
ci: add bin options to reduce build burden by @WenyXu in #3518

Contributors

We would like to thank the following contributors from the GreptimeDB community:

Eugene Tolbakov, JeremyHi, LFC, Lei HUANG, Ning Sun, Ruihang Xia, WU Jingdi, Weny Xu, Yingwen, Zhenchi, ZonaHe, crwen, discord9, gcmutator, shuiyisong, tison

v0.7.0

2024年3月12日 · 阅读需 9 分钟

Release date: March 12, 2024

This is a feature release, read the release blog to learn more detail about the feature and performance updates.

v0.7 represents a crucial leap toward achieving production readiness; it implements production-ready features for cloud-native monitoring scenarios.

Highlights

Metric Engine: handles a large number of small tables, making it particularly suitable for cloud-native monitoring.
Region Migration: enhances the user experience and simplifies region migrations with straightforward SQL commands.
Inverted Index: dramatically improves the efficiency of locating data segments relevant to user queries, significantly reducing the IO operations needed for scanning data files and thus accelerating the query process.

Breaking changes

Switch prom remote write to metric engine by @waynexia in #3198
Rename initialize_region_in_background to init_regions_in_background by @WenyXu in #3216
New partition grammar in SQL parser by @waynexia in #3347

Bug fixes

fix: change back GREPTIME_DB_HEADER_NAME header key name by @shuiyisong in #3184
fix(index): S3 EntityTooSmall error by @zhongzc in #3192
fix: remove name matcher from processed matcher list by @waynexia in #3213
fix: fix default value cannot accept negative number by @WenyXu in #3217
fix: only register region keeper while creating physical table by @WenyXu in #3223
fix: fix MockInstance rebuild issue by @WenyXu in #3218
fix: security update for shlex and h2 by @sunng87 in #3227
fix: fix create table ddl return incorrect table id by @WenyXu in #3232
fix: init parquet reader metrics twice by @evenyag in #3242
fix: IntermediateWriter closes underlying writer twice by @waynexia in #3248
fix: decouple columns in projection and prune by @waynexia in #3253
fix(Copy From): fix incorrect type casts by @WenyXu in #3264
fix: SQL insertion and column default constraint aware of timezone by @killme2008 in #3266
fix: cli export database default value by @tisonkun in #3259
fix: use fe_opts after setup_frontend_plugins in standalone by @shuiyisong in #3275
fix: fix incorrect StatusCode parsing by @WenyXu in #3281
fix(util): join_path function should not trim leading / by @dalprahcd in #3280
fix: correct the case sensitivity behavior for PromQL by @waynexia in #3296
fix: $TARGET_BIN not found when docker run the image by @daviderli614 in #3297
fix(index): sanitize S3 upload buffer size by @zhongzc in #3300
fix: commit_short sqlness test case by @tisonkun in #3313
fix: bump libgit2-sys from 0.16.1+1.7.1 to 0.16.2+1.7.2 by @dependabot in #3316
fix: split write metadata request by @fengjiachun in #3311
fix(index): encode string type to original data to enable fst regex to work by @zhongzc in #3324
fix: disable ansi control char when stdout is redirected by @waynexia in #3332
fix: typo in lint config by @waynexia in #3358
fix: treat "0" and "1" as valid boolean values. by @MichaelScofield in #3370
fix: remove unused imports in memtable_util.rs by @v0y4g3r in #3376
fix: resets dict builder keys counter and avoid unnecessary pruning by @evenyag in #3386
fix: throw errors instead of panic by @WenyXu in #3391
fix: some read metrics by @v0y4g3r in #3404
fix: partition region id by @fengjiachun in #3414
fix: show table names not complete from information_schema by @killme2008 in #3417
fix: mitigate memory spike during startup by @niebayes in #3418
fix: complete interceptors for all frontend entry by @shuiyisong in #3428

Improvements

feat: add modulo function by @etolbakov in #3147
feat: support HTTP&gRPC&pg set timezone by @Taylor-lagrange in #3125
feat(mito): enable inverted index by @zhongzc in #3158
feat: upgrade pgwire to 0.19 by @sunng87 in #3157
feat: let tables API return a stream by @fengjiachun in #3170
feat: add tests-fuzz crate by @WenyXu in #3173
feat: tables stream with CatalogManager by @fengjiachun in #3180
feat: adds date_format function by @killme2008 in #3167
feat: auto config cache size according to memory size by @QuenKar in #3165
feat: precise filter for mito parquet reader by @waynexia in #3178
feat: adds parse options for SQL parser by @killme2008 in #3193
feat(tests-fuzz): add CreateTableExprGenerator & AlterTableExprGenerator by @WenyXu in #3182
feat: make procedure able to return output by @WenyXu in #3201
feat: copy database from by @v0y4g3r in #3164
feat: introduce information schema provider cache by @WenyXu in #3208
feat: lazy initialize vector builder on write by @waynexia in #3210
feat: make query be aware of timezone setting by @killme2008 in #3175
feat: add create alter table expr translator by @WenyXu in #3203
feat: read column and region info from state cache by @waynexia in #3222
feat: enable concurrent write by @WenyXu in #3214
feat: add Arrow IPC output format for http rest api by @sunng87 in #3177
feat: change Range Query’s default align behavior aware of timezone by @killme2008 in #3219
feat: update dashboard to v0.4.7 by @ZonaHex in #3229
feat: http sql api return schema on empty resultset by @sunng87 in #3237
feat: add pg create alter table expr translator by @WenyXu in #3206
feat: read metadata from write cache by @QuenKar in #3224
feat: batch create ddl by @fengjiachun in #3194
feat: add insert/select generator & translator by @WenyXu in #3240
feat: return request outdated error on handling alter by @waynexia in #3239
feat: don't map semantic type in metric engine by @waynexia in #3243
feat: create tables in batch on prom write by @fengjiachun in #3246
feat: Only allow inserts and deletes operations to be executed in parallel by @fengjiachun in #3257
feat: basic types for Dataflow Framework by @discord9 in #3186
feat: initial configuration for grafana dashboard by @waynexia in #3263
feat: support fraction part in timestamp by @discord9 in #3272
feat: use simple filter to prune memtable by @waynexia in #3269
feat: Basic Definitions for Expression&Functions for Dataflow by @discord9 in #3267
feat: support cache for batch_get in CachedMetaKvBackend by @fengys1996 in #3277
feat: put all filter exprs in a filter plan separately by @waynexia in #3288
feat: administration functions by @killme2008 in #3236
feat(mito): adjust seg size of inverted index to finer granularity instead of row group level by @zhongzc in #3289
feat(mito): add options to ignore building index for specific column ids by @zhongzc in #3295
feat: organize tracing on query path by @waynexia in #3310
feat: impl partitions and region_peers information schema by @killme2008 in #3278
feat: batch get physical table routes by @fengjiachun in #3319
feat: Defines structs in the merge tree memtable by @evenyag in #3326
feat(metric-engine): set index options for data region by @zhongzc in #3330
feat: data buffer and related structs by @v0y4g3r in #3329
feat: Implement KeyDictBuilder for the merge tree memtable by @evenyag in #3334
feat: replace pk index with pk_weight during freeze by @v0y4g3r in #3343
feat: merge tree data parts by @v0y4g3r in #3346
feat: Defines more structs and methods for a partitioned merge tree by @evenyag in #3348
feat(flow): impl ScalarExpr&Scalar Function by @discord9 in #3283
feat: impl migrate_region and procedure_state SQL function by @killme2008 in #3325
feat: add isnull function by @KKould in #3360
feat: skip filling NULL for put and delete requests by @waynexia in #3364
feat: impl merge reader for DataParts by @v0y4g3r in #3361
feat: Implement write and fork for the new memtable by @evenyag in #3357
feat: distinguish between different read paths by @v0y4g3r in #3369
feat: Add freeze and fork method to the memtable by @evenyag in #3374
feat: merge tree dedup reader by @v0y4g3r in #3375
feat: Implement iter for the new memtable by @evenyag in #3373
feat: Implement dedup for the new memtable and expose the config by @evenyag in #3377
feat: change how region id maps to region worker by @waynexia in #3384
feat: make tls certificates/keys reloadable (part 1) by @sunng87 in #3335
feat(grafana): enable shared tooltip, add raft engine throughput by @waynexia in #3387
feat: Implement partition eviction and only add value size to write buffer size by @evenyag in #3393
feat: enable zstd compression and encodings in merge tree data part by @v0y4g3r in #3380
feat(flow): impl for MapFilterProject by @discord9 in #3359
feat: flush or compact table and region functions by @killme2008 in #3363
feat: Use a partition level map to look up pk index by @evenyag in #3400
feat(index): measure memory usage in global instead of single-column and add metrics by @zhongzc in #3383
feat: enable ArrowFlight compression by @tisonkun in #3403
feat: zero copy on split rows by @fengjiachun in #3407
feat: Support automatic DNS lookup for kafka bootstrap servers by @J0HN50N133 in #3379
feat: add configuration for tls watch option by @sunng87 in #3395
feat: employ sparse key encoding for shard lookup by @v0y4g3r in #3410
feat: support Create Table ... Like by @KKould in #3372
feat: tableref cache by @fengjiachun in #3420
feat: add verbose support for tql explain/analyze by @etolbakov in #3390
feat: reduce a clone of string by @fengjiachun in #3422
feat: Correct server metrics and add more metrics for scan by @evenyag in #3426
feat: support tracing rule sampler by @Taylor-lagrange in #3405
feat: decode prom requests to grpc by @v0y4g3r in #3425
feat: implement multi-dim partition rule by @waynexia in #3409

Contributors

We would like to thank the following contributors from the GreptimeDB community:

Cancai Cai, Eugene Tolbakov, Hudson C. Dalprá, JeremyHi, JohnsonLee, Kould, LFC, Lei HUANG, Ning Sun, Ruihang Xia, SteveLauC, WU Jingdi, Wei, Weny Xu, Yingwen, Yiran, Zhenchi, ZonaHe, dennis zhuang, dimbtp, discord9, fys, liyang, niebayes, shuiyisong, tison

Breaking changes​

Bug fixes​

Improvements​

Others​

Contributors​

Breaking changes​

Bug fixes​

Improvements​

Contributors​

Breaking changes

Bug fixes

Improvements

Others

Contributors

Breaking changes

Bug fixes

Improvements

Contributors