# Sqlness Test

> Instructions for running SQL tests in GreptimeDB using the `sqlness` test suite, including file types, case organization, and running tests.

# Sqlness Test

## Introduction

SQL is an important user interface for `GreptimeDB`. We have a separate test suite for it (named `sqlness`).

## Sqlness manual

### Case file

Sqlness has three types of file

- `.sql`: test input, SQL only
- `.result`: expected test output, SQL and its results
- `.output`: different output, SQL and its results

Both `.result` and `.output` are output (execution result) files. The difference is that `.result` is the
the standard (expected) output, and `.output` is the error output. Therefore, if you see `.output` files generated,
it means this test gets a different result and indicates it fails. You should
check change logs to solve the problem.  

You only need to write test SQL in `.sql` file, and run the test. On the first run it produces
an `.output` file because there is no `.result` to compare with. If you can make sure the content in
`.output` is correct, you can rename it to `.result`, which means it is the expected output.

And at any time there should only be two file types, `.sql` and `.result` -- otherwise, an existing `.output`
file means your test fails. That's why we should not ignore `.output` file type in `.gitignore`, instead, track
it and make sure it doesn't exist.

### Case organization

The root dir of input cases is `tests/cases`. It contains several sub-directories stand for different test
modes. E.g., `standalone/` contains all the tests to run under `greptimedb standalone start` mode.

Under the first level of sub-directory (e.g. the `cases/standalone`), you can organize your cases as you like.
Sqlness walks through every file recursively and runs them.

## Run the test

Unlike other tests, this harness is in a binary target form. You can run it with

```shell
cargo run --bin sqlness-runner bare
```

It automatically finishes the following procedures: compile `GreptimeDB`, start it, grab tests and feed it to
the server, then collect and compare the results. You only need to check whether there are new `.output` files.
If not, congratulations, the test is passed 🥳!

### Run a specific test

```shell
cargo sqlness bare -t your_test
```

If you specify a second argument, only test cases containing the specified string in their names will be executed. Sqlness also supports filtering based on environment. The filter is accepted as a regex string and the case name will be examined in the format of `env:case`.