2023/08/22

* New Feature
  * Add aggregation transform module
  * Add shuffle transform module
  * Add matchingEngine sink module
  * Add localNeo4j sink module
  * Support dataflow settings option
  * Support requestTag, dataBoost for spanner source module
  * Support commitTimestampFields for spanner sink module in batch
  * Support renameTables for spanner source module
  * Support default value for source schema
  * Support select functions for filter transform module
* Improvement
  * Update dependencies
  * Support valueCaptureType NEW_ROW, NEW_VALUES for spanner cdc
  * Add reshuffle for pdfextract transform module
  * Support failure output for spanner sink module
  * Support put upsert for insert and update
  * Support local sql file for beamsql transform module
* Bugfix
  * Bugfix batch file_loads for bigquery sink module
  * Bugfix spanner delete mode
  * Bugfix avro merge
  * Bugfix datastore entity array,nested field

2023/02/24

* New Feature
  * Add firestore source/sink module
  * Add sequence transform module
  * Support spanner change stream for spanner source module
  * Support expression for filter condition
  * Support json,datetime type
* Improvement
  * Update Dependencies
  * Support allow_commit_timestamp fields for spanner sink module
  * Support ordering key for pubsub sink module
  * Support generating nested and flatten rows for dummy source module
  * Support local execution for spanner source/sink module
* Bugfix
  * Bugfix not working checkIntervalSeconds
  * Bugfix parsing timestamp for json to record
  * Bugfix empty table for Jdbc table read
  * Bugfix BigQuery sink module in streaming mode

2022/06/07

* New Features
  * Add drivefile source module
  * copyfile sink module
  * Add tokenize transform module
* Improvement
  * Update Beam SDK version to 2.39
  * Update Apache Solr version to 9.0
  * Support external config files for solrindex sink module
  * Support numeric type, array type for jdbc source module
  * Support seeking table read fomr jdbc source module
  * Support specifing username and password using secret manager resource name for jdbc source module
  * Support using default IAM user as DB user for jdbc source module
  * Add enableRampupThrottling option for datastore sink module
  * Add distinct agg UDFs for beamsql transform module

2022/02/21

* New Features
  * Add union transform module
  * Support delete op and cell timestamp for Bigtable sink module
* Improvement
  * Beam SDK version up to 2.36
  * Add build files for Cloud Build
* Bugfix
  * Fix schemaUpdateOptions was not working for streaming mode

2022/01/25

* New Features
  * Support trigger settings for window transform module
  * Add UDFs for beamsql transorm module
  * Add websocket source module experimental
* Improvement
  * Beam SDK version up to 2.35
* Bugfix
  * Fixed the description of local run

Update log4j2 from 2.15.0 to 2.17.0

Bugfix

* update log4j2 to 2.15.0

* New Feature

  * Support heatmap for SolrIndexSink
  * Support query priority for SpannerSource
  * Support excludeFromIndexFields for DatastoreSink
  * Support date,timestamp type as filter condition for FilterTransform
* Bugfix
  * Bugfix prefix for StorageSink
  * Bugfix withKey error for DatastoreSource
  * Bugfix example config for spanner-to-bigquery.json
  * Bugfix for SolrIndexSink
* Improvement
  * Update Apache Beam SDK to 2.32
  * Provide a more secure configuration pom file (pom_secure.xml)

* New Features

  * Add Cloud BigTable Sink
  * Add AutoML(Vertex AI endpoints) Transform
  * Support template key for Cloud Datastore Sink
  * Support Numeric type for Jdbc Source
  * Support message input format for PubSub Source
  * Support json additional fields validation for PubSub Source
  * Support skip parameter to ignore the module for all config modules
* Dependency Update
  * Beam SDK version update from 2.29 to 3.31
  * Remove unuse libraries
  * Update dependency log4j, solr.
* Bugfix
  * Array type conversion bug fix for Beam Row
  * Template avro array data feed bugfix
  * Nested record insertion to BigQuery sink bugfix
  * Add missing dependency hadoop client for Parquet use

Merge pull request #35 from mercari/update-bigquery-sink-streaming-in…

…sert

Update bigquery sink streaming insert