Skip to content

Commit

Permalink
format test files
Browse files Browse the repository at this point in the history
  • Loading branch information
xudong963 committed Dec 16, 2022
1 parent 6215dd3 commit 02c2f77
Show file tree
Hide file tree
Showing 311 changed files with 8,399 additions and 12,678 deletions.
2 changes: 1 addition & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

16 changes: 8 additions & 8 deletions docs/doc/14-sql-commands/00-ddl/20-table/60-optimize-table.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,9 @@ Snapshot, segment, and block are the concepts Databend uses for data storage. Da

![](/img/sql/storage-structure.PNG)

Databend automatically creates snapshots of a table when data updates occur, so a snapshot represents a version of the table's data. When working with Databend, you're most likely to access a snapshot with the snapshot ID when you retrieve and query a previous version of the table' data with the [AT](../../20-query-syntax/dml-at.md) clause.
Databend automatically creates snapshots of a table when data updates occur, so a snapshot represents a version of the table's data. When working with Databend, you're most likely to access a snapshot with the snapshot ID when you retrieve and query a previous version of the table' data with the [AT](../../20-query-syntax/dml-at.md) clause.

A snapshot is a JSON file that does not save the table's data but indicate the segments the snapshot links to. If you run [FUSE_SNAPSHOT](../../../15-sql-functions/111-system-functions/fuse_snapshot.md) against a table, you can find the saved snapshots for the table.
A snapshot is a JSON file that does not save the table's data but indicate the segments the snapshot links to. If you run [FUSE_SNAPSHOT](../../../15-sql-functions/111-system-functions/fuse_snapshot.md) against a table, you can find the saved snapshots for the table.

A segment is a JSON file that organizes the storage blocks (at least 1, at most 1,000) where the data is stored. If you run [FUSE_SEGMENT](../../../15-sql-functions/111-system-functions/fuse_segment.md) against a snapshot with the snapshot ID, you can find which segments are referenced by the snapshot.

Expand Down Expand Up @@ -49,7 +49,7 @@ select if(count(*)>100,'The table needs compact now','The table does not need co

### Compacting Segments Only

The optimization merges both segments and blocks of a table by default. However, you can choose to compact the segments only when a table has too many segments.
The optimization merges both segments and blocks of a table by default. However, you can choose to compact the segments only when a table has too many segments.

Databend recommends optimizing the segments only for a table when the table has more than 1,000 segments and the average number of blocks per segment is less than 500.

Expand All @@ -71,11 +71,11 @@ OPTIMIZE TABLE [database.]table_name [ PURGE | COMPACT | ALL | [SEGMENT] [LIMIT

Purges the historical data of table. Only the latest snapshot (including the segments, blocks and table statistic file referenced by this snapshot) will be kept.
(For more explanations of table statistic file, see [ANALYZE TABLE](./80-analyze-table.md).)

- `OPTIMIZE TABLE <table_name> COMPACT [LIMIT <segment_count>]`

Compacts the table data by merging small blocks and segments into larger ones.

- This command creates a new snapshot (along with compacted segments and blocks) of the most recent table data without affecting the existing storage files, so the storage space won't be released until you run `OPTIMIZE TABLE <table_name> PURGE`.
- Depending on the size of the given table, it may take quite a while to complete the execution.
- The option LIMIT sets the maximum number of segments to be compacted. In this case, Databend will select and compact the latest segments.
Expand Down Expand Up @@ -127,7 +127,7 @@ mysql> select snapshot_id, segment_count, block_count, row_count from fuse_snaps
| f1c1f24d9d3742b9a05c827a3f03e0e0 | 17 | 17 | 10000001 |
| f150212c302244fa9780cc6337138c6e | 16 | 16 | 10000000 |
+----------------------------------+---------------+-------------+-----------+
mysql> -- newly generated blocks are too small, let's compact them
mysql> -- newly generated blocks are too small, let's compact them
mysql> optimize table t compact;
mysql> select snapshot_id, segment_count, block_count, row_count from fuse_snapshot('default', 't');
+----------------------------------+---------------+-------------+-----------+
Expand All @@ -142,7 +142,7 @@ mysql> select snapshot_id, segment_count, block_count, row_count from fuse_snaps
| f150212c302244fa9780cc6337138c6e | 16 | 16 | 10000000 |
+----------------------------------+---------------+-------------+-----------+

mysql> -- If we do not care about the historical data
mysql> -- If we do not care about the historical data
mysql> optimize table t purge;
Query OK, 0 rows affected (0.08 sec)
Read 0 rows, 0.00 B in 0.041 sec., 0 rows/sec., 0.00 B/sec.
Expand Down
1 change: 0 additions & 1 deletion docs/doc/14-sql-commands/90-explain-cmds/explain.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,6 @@ Explanation plans may include a variety of operators, depending on the SQL state

```sql
EXPLAIN select t.number from numbers(1) as t, numbers(1) as t1 where t.number = t1.number;

----
Project
├── columns: [number (#0)]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ statement error table db1.tbl1 does not exist
create table db1.tbl1 (id int);
```

`statement query <desired_query_schema_type> <options> <labels>`: the sql statement output is expected success with desired result.
`query <desired_query_schema_type> <options> <labels>`: the sql statement output is expected success with desired result.

`desired_query_schema_type` represent the schema type of the query result. which is documented in https://github.com/gregrahn/sqllogictest/blob/master/about.wiki#test-script-format

Expand All @@ -67,9 +67,8 @@ create table db1.tbl1 (id int);
for example: the following sql would be ok with output table

```text
statement query III
query III
select number, number + 1, number + 999 from numbers(10);
----
0 1 999
1 2 1000
Expand All @@ -86,9 +85,8 @@ select number, number + 1, number + 999 from numbers(10);
The follow sql configured match mysql label first then default label

```text
statement query III label(mysql)
query III label(mysql)
select number, number + 1, number + 999 from numbers(10);
----
0 1 999
1 2 1000
Expand Down
39 changes: 13 additions & 26 deletions tests/fuse-compat/compat-logictest/fuse_compat_read
Original file line number Diff line number Diff line change
@@ -1,80 +1,67 @@
statement query I
query I
select c_bool FROM fuse_compat_table;

----
0

statement query I
query I
select c_tinyint FROM fuse_compat_table;

----
127

statement query I
query I
select c_smallint FROM fuse_compat_table;

----
3267

statement query I
query I
select c_int FROM fuse_compat_table;

----
2147483647


statement query I
query I
select c_bigint FROM fuse_compat_table;

----
9223372036854775807


statement query F
query F
select c_float FROM fuse_compat_table;

----
3.4

statement query F
query F
select c_double FROM fuse_compat_table;

----
1.7

statement query T
query T
select c_date FROM fuse_compat_table;

----
9999-12-31

statement query T
query T
select c_timestamp FROM fuse_compat_table;

----
1991-01-01 00:00:00.000000

statement query T
query T
select c_varchar FROM fuse_compat_table;

----
varchar


statement query T
query T
select c_array FROM fuse_compat_table;

----
[1,2,3,["a","b","c"]]

statement query T
query T
select c_object FROM fuse_compat_table;

----
{"a":1,"b":{"c":2}}

statement query T
query T
select c_variant FROM fuse_compat_table;

----
[1,{"a":1,"b":{"c":2}}]
17 changes: 6 additions & 11 deletions tests/logictest/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,9 +96,8 @@ Runner supported: mysql handler, http handler, clickhouse handler.
This is a query demo(query_label is optional):

```
statement query III label(mysql)
query III label(mysql)
select number, number + 1, number + 999 from numbers(10);
----
0 1 999
1 2 1000
Expand Down Expand Up @@ -138,9 +137,8 @@ regex_type_map = {
2. A demo of regex likes:

```
statement query TTTRRTIIII
query TTTRRTIIII
SELECT * FROM system.tables WHERE database='db1';
----
default db1 t1 FUSE $ANYTHING $DATE NULL 0 0 0 0
```
Expand All @@ -155,9 +153,8 @@ The space is used to split results into columns, space in regex expression shoul

```
skipif clickhouse
statement query I
query I
select 1;
----
1
```
Expand All @@ -166,19 +163,17 @@ select 1;

```
onlyif mysql
statement query I
query I
select 1;
----
1
```

3. if some test has a flaky failure, and you want to ignore it, simply add skipped before statement query. (Remove it after the problem is solved)

```
statement query skipped I
query skipped I
select 1;
----
1
```
Expand All @@ -194,7 +189,7 @@ select 1;
complete.py can auto-complete test file for you, It does as follow steps:

1. Get SQLs from source-file, whether an SQL file or logictest file.
2. Execute SQL one by one, if SQL fetches results, add statement query; if SQL fetches nothing, add statement ok; if SQL gets an error, add statement error.
2. Execute SQL one by one, if SQL fetches results, add query; if SQL fetches nothing, add statement ok; if SQL gets an error, add statement error.

### Usage

Expand Down
2 changes: 1 addition & 1 deletion tests/logictest/gen_suites.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
"""

STATEMENT_QUERY = """statement query {query_options_with_labels}
STATEMENT_QUERY = """query {query_options_with_labels}
{statement}
{results}
Expand Down
12 changes: 6 additions & 6 deletions tests/logictest/logictest.py
Original file line number Diff line number Diff line change
Expand Up @@ -384,7 +384,7 @@ def assert_query_equal(self, f, resultset, statement, with_regex=False):
resultset[1],
resultset[0].group("label"),
),
errorType="statement query get result not equal to expected(with regex expression)",
errorType="query get result not equal to expected(with regex expression)",
runner=self.kind,
)

Expand All @@ -398,7 +398,7 @@ def assert_query_equal(self, f, resultset, statement, with_regex=False):
resultset[1],
resultset[0].group("label"),
),
errorType="statement query get result not equal to expected",
errorType="query get result not equal to expected",
runner=self.kind,
)

Expand All @@ -412,21 +412,21 @@ def assert_execute_query(self, statement):
raise LogicError(
runner=self.kind,
message=str(err),
errorType="statement query execute with exception",
errorType="query execute with exception",
)
try:
f = format_value(actual, len(statement.s_type.query_type))
except Exception:
raise LogicError(
message=f"{statement} statement type is query but get no result",
errorType="statement query get no result",
errorType="query get no result",
runner=self.kind,
)

if statement.results is None or len(statement.results) == 0:
raise LogicError(
message=f"{statement} no result found by query",
errorType="statement query get empty result",
errorType="query get empty result",
runner=self.kind,
)
with_regex = False
Expand All @@ -451,7 +451,7 @@ def assert_execute_query(self, statement):
if not hasResult:
raise LogicError(
message=f"{statement} no result found in test file",
errorType="statement query has no result in test file",
errorType="query has no result in test file",
runner=self.kind,
)

Expand Down
Loading

0 comments on commit 02c2f77

Please sign in to comment.