v1.2.697

What's Changed

Exciting New Features ✨

• feat: add server to admin udfs api by @everpcpc in #17141
• feat(sqlsmith): Support generate fuzz query sql by @b41sh in #17153
• feat(ci): add ttc (tcp testing container) by @sundy-li in #17160
• feat(query): support direct the spill data to a different bucket by @forsaken628 in #17159
• feat(cluster): support system-managed cluster by @zhang2014 in #17051
• feat(query): support send spill file stats to client by @sundy-li in #17186
• feat(query): load ndjson support ts with diff units. by @youngsofun in #17203
• feat(sqlsmith): fuzz query support sqllogic test cases by @b41sh in #17216
• feat: hilbert clustering by @zhyass in #17045
• feat: support vacuum aggregating index by @SkyFan2002 in #17231
• feat(cluster): support warehouse level show processlist and kill query by @zhang2014 in #17249
• feat: orc support fill missing tuple field by @youngsofun in #17247
• feat(sqlsmith): fuzz test support read Sqlite and Duckdb test by @b41sh in #17301
• feat(ddl): support alter table cast tuple type column by @b41sh in #17310
• feat: support vacuum inverted index by @SkyFan2002 in #17291
• feat: Add disable_list_batch for webhdfs by @Xuanwo in #17345
• feat: Add username support for webhdfs by @Xuanwo in #17350
• feat(ddl): Support add stored computed column by @b41sh in #17347
• feat: introduce block meta cache by @dantengsky in #17360
• feat(query): support desc procedure by @TCeason in #17364
• feat(query): support Support from to simplify select * from by @TCeason in #17372
• feat(query): Virtual column support alias name by @b41sh in #17365
• feat(query): support timestamp diff return interval type by @TCeason in #17381
• feat(query): support extract EPOCH from timestamp by @TCeason in #17385
• feat(query): interval support agg func sum by @TCeason in #17379
• feat(query): support DATE_TRUNC('WEEK', EXPR)->DATE by @TCeason in #17407
• feat(auth): auto retry on jwks key not found by @flaneur2020 in #17410
• feat(query): Allow extract parts from interval type by @TCeason in #17417
• feat(query): array function support variant type as argument by @b41sh in #17428
• feat(query): add bucket name to storage http request metrics by @everpcpc in #17431
• feat: deprecate prefix_filter and change default level of logging. by @youngsofun in #17444

Thoughtful Bug Fix 🔧

• fix(ci): flaky test by @zhyass in #17149
• fix(query): return semantic error for SRFs in WHERE and HAVING clauses by @takaebato in #17167
• fix(cluster): disable forward warehouse request in self-managed cluster by @zhang2014 in #17176
• fix(query): check state is allocated or not when oom by @sundy-li in #17179
• fix(cluster): fix unassign warehouse nodes failure by @zhang2014 in #17195
• fix(cluster): skip the recovery for warehouses that are not running state. by @zhang2014 in #17193
• fix(query): fix aggregating index conflict with min/max rewrite by @b41sh in #17182
• fix: name resolution bug in quoted ident case sensitive mode by @zhyass in #17198
• fix(query): record state_rows in aggregate payload by @sundy-li in #17194
• fix: system.tables table visibility by @TCeason in #17202
• fix(query): interval total microseconds add overflow check by @TCeason in #17215
• fix(cluster): fix warehouse audit log not working with add cluster by @zhang2014 in #17219
• fix(query): fix register function working with nullable scalar by @sundy-li in #17217
• fix(query): stick the created_by infos in parquet writer by @sundy-li in #17220
• fix: Upload to stage in streaming way instead by @Xuanwo in #17230
• fix(cluster): fix heartbeat self managed after network fail by @zhang2014 in #17241
• fix(query): fix array_agg failed if the argument is NULL by @b41sh in #17244
• fix(query): increase state_rows in copy agg state rows by @sundy-li in #17252
• fix(query): hash table scatter will always send agg meta by @sundy-li in #17245
• fix(query): fix lazy columns missed in constant table scan by @Damon07 in #17258
• fix: vacuum index not work if index is dropped by create or replace by @SkyFan2002 in #17263
• fix(query): window function support params by @b41sh in #17282
• fix: copy into table collect files twice some times. by @youngsofun in #17300
• fix: set query_kind earlier to ensure it takes effect. by @youngsofun in #17302
• fix: use full path as cache key for parquet meta. by @youngsofun in #17313
• fix: column not found in analyze by @zhyass in #17321
• fix(query): unsupport datetime format item should not return panic error by @TCeason in #17323
• fix: support returning rows larger than 10MB. by @youngsofun in #17344
• fix(query): fix histogram binder and checker by @sundy-li in #17342
• fix(query): add check window frame by @sundy-li in #17316
• fix(query): RawValueSource::parse_fallback fully parse input by @forsaken628 in #17340
• fix(query): aviod building TransformSortMergeLimit with a big number limit by @forsaken628 in #17339
• fix(query): disallow push down filters that use columns in window order by columns or function argument columns by @forsaken628 in #17353
• fix(query): procedure support string type by @TCeason in #17325
• fix(query): interval order by is incorrect by @forsaken628 in #17357
• fix(query): cast string to timestamp overflow by @TCeason in #17376
• fix(query): fix time parser with enable_strict_datetime_parser = 0 by @sundy-li in #17399
• fix(query): fix lambda function bind column failed by @b41sh in #17402
• fix(query): fix months_between overflow error by @TCeason in #17403
• fix(query): interval type support cast to string by @TCeason in #17395
• fix(query): Align TRIM functions semantically with Snowflake by @forsaken628 in #17375
• fix(query): fix resolve decimal by @sundy-li in #17409
• fix(ci): fix Link Checker by @sundy-li in #17416
• fix(query): to_yyyymmddhhmmss only support timestamp type by @TCeason in #17422
• fix(query): group checker subquery by @sundy-li in #17420
• fix(query): support vacuum temp files for create and refresh by @zhang2014 in #17421
• fix(query): if arg is Scalar::null or a null column ignore null will invalid by @TCeason in #17423
• fix(query): cte recursive check by @sundy-li in #17427
• fix(query): fix variant value can not cast to not null type by @b41sh in #17434
• fix(doc): fix load tpcds tpch document by @sundy-li in #17438
• fix(query): fix join subquery scalar rewrite by @sundy-li in #17441
• fix: fix inconsistency comment about bitmap by @forsaken628 in #17452

Code Refactor 🎉

• refactor: make fuse_amend implementation more convenient for on-premise deployments by @dantengsky in #17147
• refactor: Bump OpenDAL to 0.51 by @Xuanwo in #17161
• refactor: Optimize When Querying the tables_history Table with the Condition table_id = n by @TCeason in #17166
• refactor: Use opendal to implement recover object instead by @Xuanwo in #17165
• refactor: include key information in errors when listing meta-service key-values by @drmingdrmer in #17192
• refactor: create/drop_warehouse return WarehouseInfo by @TCeason in #17254
• refactor: only set global settings need to check super privilege by @TCeason in #17255
• refactor(query): only call once finish callback by @sundy-li in #17293
• refactor(query): record state_offsets to reduce the memory leak of agg by @sundy-li in #17285
• refactor(query): introduce udf runtime pool by @forsaken628 in #17304
• refactor(query): Multiple states Aggregate function by @forsaken628 in #17148
• refactor: rename components of the replace into impl by @dantengsky in #17355
• refactor: refine replace into by caching individual BlockMeta by @dantengsky in #17368
• refactor(query): refactor variant cast to types by @sundy-li in #17435

Build/Testing/CI Infra Changes 🔌

• ci: fix release type for nightly cron by @everpcpc in #17429
• ci: fix missing return for stable release by @everpcpc in #17454
• ci: fix publish binary to R2 by @everpcpc in #17456

Documentation 📔

• docs:Update release-readme.txt by @wubx in #17221

Others 📒

• chore: Split functions crate into multi crates by @TCeason in #17133
• chore(query): bump ast 0.1.3 by @sundy-li in #17145
• chore: fix cargo check by @TCeason in #17152
• chore: adjust fuse_time_travel_size() by @SkyFan2002 in #17164
• chore: more logging for vacuum drop table execution by @SkyFan2002 in #17168
• chore: Remove unused settings for ssl by @Xuanwo in #17173
• chore: check schema for small parquet file too. by @youngsofun in #17175
• chore(ci): support backport release by @everpcpc in #17185
• chore(ci): auto add pr-backport label by @everpcpc in #17190
• chore: simplify setting API by @drmingdrmer in #17138
• chore: fix flaky datetime test by @TCeason in #17188
• chore: try split number cast function impl by @TCeason in #17155
• chore: add udf compat ci by @TCeason in #17184
• chore(ci): upload sccache log for build by @everpcpc in #17201
• chore(planner): improve histogram cardinality estimation by @Dousir9 in #17200
• chore: Fix warning for backtrace while compiling by @Xuanwo in #17204
• chore(ci): fix sccache log path by @everpcpc in #17206
• chore(ci): update sccache log artifact name by @everpcpc in #17211
• chore(ci): adjust build & test workflows by @everpcpc in #17213
• chore: bring back function set_cache_capacity by @dantengsky in #17196
• chore: try fix flaky ci by @TCeason in #17222
• chore(ci): fix release find last tag by @everpcpc in #17226
• chore(ci): fix release bump log by @everpcpc in #17227
• chore(ci): stable release ignore patch versions by @everpcpc in #17228
• chore(ci): better generate release by @everpcpc in #17232
• chore(ci): fix release sha for backport by @everpcpc in #17238
• chore: fix flaky dictionary test by @b41sh in #17199
• chore(ci): presign artifact for download by @everpcpc in #17253
• chore(sqllogictests): Check container and stop first to avoid conflict running by @b41sh in #17261
• chore: Bump opendal to pinned tag version by @Xuanwo in #17259
• chore(ci): fix release bump version by @everpcpc in #17277
• chore(ci): fix release patch version by @everpcpc in #17278
• chore(ci): fix gen next patch release by @everpcpc in #17281
• chore(ci): unit test with 16c64g by @everpcpc in #17283
• chore: try fix flaky ci by @TCeason in #17265
• chore(ci): add bendsql binary to release by @everpcpc in #17289
• chore: databend-test print more stderr by @TCeason in #17288
• chore: Bump opendal to include a fix for gcs by @Xuanwo in #17290
• chore: make error message of conflicting transactions more user-friendly by @SkyFan2002 in #17307
• chore: add enable_prune_cache setting by @dqhl76 in #17308
• chore(query): improve fuse row fetch, use accumulating by @sundy-li in #17309
• chore(ci): fix pack binaries by @everpcpc in #17332
• chore(ci): fix pack binaries in release by @everpcpc in #17335
• chore: fix flaky test by @SkyFan2002 in #17333
• chore: Clean up not used crates by @Xuanwo in #17260
• chore: enable sort spill in recluster by @zhyass in #17236
• chore(query): improve udf server headers by @sundy-li in #17349
• chore(ci): intergration ttc-go by @forsaken628 in #17280
• chore(ci): fix intergration ttc-go by @forsaken628 in #17363
• chore: remove unused _row_number by @zhyass in #17336
• chore: remove unused code by @SkyFan2002 in #17371
• chore(query): bump jiff crate by @sundy-li in #17366
• chore(query): fix hash join hang by @Dousir9 in #17389
• chore(query): row fetch will be fetched by distinct block batch by @sundy-li in #17391
• chore(udf): tune the log message about udf decode error by @flaneur2020 in #17414
• chore: refine copy into tests by @dantengsky in #17413
• chore(metrics): add metric about opendal retries count by @flaneur2020 in #16724
• chore: Use tagged version of opendal by @Xuanwo in #17419
• chore: use Box::leak() to ensure logging guard lives for the program's lifetime by @drmingdrmer in #17425
• chore(query): cast variant to other type allow cast JSON null as SQL NULL by @b41sh in #17412
• chore(query): add new session setting: query_tag by @TCeason in #17430
• chore: update enterprise edition license url by @TCeason in #17445
• chore(query): testing tpch/tpcds using 1G scale factor by @sundy-li in #17449

New Contributors

• @takaebato made their first contribution in #17167
• @Damon07 made their first contribution in #17258

Full Changelog: v1.2.680...v1.2.697