CHANGES.txt

=================
Changes for Crate
=================

Unreleased
==========

 - only log tracebacks with debug-level when udc failed

2014/01/15 0.21.1
=================

 - fix Double URL encoding of udc parameters

 - fix NPE on close of UDCService

2014/01/15 0.21.0
=================

 - changed default port ranges for http to 4200-4300 and for transport 4300-4400

 - added ssl truststore including StartCom CA, fixes UDC

 - fixed default cluster name (especially on linux platforms)

 - implemented UDC module

 - implemented generation of an unique ClusterId on cluster start-up

 - added parameter expressions to presto based sql-parser

 - added support for subscript expressions to the new presto based sql-parser

2014/01/10 0.20.4
=================

 - admin-ui: fixed number format, fixed table missing + underreplicated shards count,
             show first table if selected table was deleted/closed

 - ignore unknown types in legacy mappings

2014/01/09 0.20.3
=================

 - admin-ui: fixed table css + data reload flickering + sidebar button
             re-enabled blender font at docu

 - added presto based sql-parser to the source tree

2014/01/08 0.20.2
=================

 - admin-ui: implemented tables overview page

2014/01/08 0.20.1
=================

 - admin-ui: fixed display/number of under-replicated docs while cluster state is red

 - admin-ui: display long numbers as rounded units

 - implemented ``object`` type within ``CREATE TABLE`` statements
   allowing to create nested tables/documents with SQL

 - renamed all occurrences of ``craty`` to ``object``

 - fix irregular deadlock while collect with ScriptValuesExpression

2013/12/20 0.20.0
=================

 - admin-ui: implemented general overview page

 - optimized serialization performance for `count(distinct colName)`

 - fix: fix NPE on ``stats.shards`` with unknown column in where clause

 - now an error is thrown if the DISTINCT keyword is used globally. Currently
   only `count(DISTINCT colName)` is supported. Other queries have to be
   re-written to use the `group by` clause.

 - removed inline junit-benchmarks sources and added dependency

 - crate.yml configuration file is now also read without specifying
   `-Des.config` in the start script.

 - new global expression ``sys.cluster.name`` created.
   Can be queried in group by queries and global aggregate queries.

 - removed inline junit-benchmarks sources and added dependency

 - crate.yml configuration file is now also read without specifying
   `-Des.config` in the start script.

 - admin-ui: added simple loading indicator on submitting a SQL query via console,
             also display query result status including query duration

 - optimized group by performance

2013/12/17 0.19.14
==================

 - fix: ignore closed tables at `information_schema`

 - fix: forbid references to other tables

 - fix: more stability for InformationSchemaServiceTest

2013/12/17 0.19.13
==================

 - admin-ui: resolve cluster health state using SQL now
   also display empty values at status-bar if cluster is not reachable/disconnected

 - allow nested columns in aggregate functions

2013/12/16 0.19.12
==================

 - the `order by` clause now works on both alias and non-alias as long as the
   columns/alias are unique within the statement.

 - fix COUNT(*) requests on "virtual" tables like ``information_schema`` and ``stats.shards``

2013/12/13 0.19.11
==================

 - fixed an error that occured if a blob is uploaded using a client that chunks
   the request but doesn't support 100-Continue.

2013/12/11 0.19.10
==================

 - implemented basic version of new admin-ui

 - fixed a thread deadlock that could occur if many group by queries run in
   parallel

 - changed threadpool settings to avoid handler blocking each other on group by queries

2013/12/05 0.19.9
=================

 - benchmark test setup improvements

 - removed randomness from test setup

 - renamed crate admin plugin to `admin-ui`

 - updated elasticsearch to 0.90.7

 - Queries on the shard stats table using aggregates without group by are supported now.

 - Also list unassigned shards when querying shard stats table.

 - fix: fixed stalled cluster on using more than one COUNT(DISTINCT col) in a statement

 - performance-tuning: faster ANY processing in ``group by`` requests

 - Shard stats table is using a pool for it's MemoryIndex now.

 - Implemented ``stats`` schema with virtual table ``shards``, usable for gathering real-time
   statistics of shards.

 - performance-tuning: concrete serialization of aggstates

 - ``COUNT`` aggregate function can now be used on column names, counting all rows with values that
   are not ``NULL``.

 - ``ANY`` aggregate function

 - the `COUNT` aggregation function now supports the `DISTINCT` keyword.

 - aggregations are now allowed on normal ``SELECT`` statements, not only ``GROUP BY`` statements

 - ``AVG`` aggregate function

 - ``SUM`` aggregate function

 - ``MAX`` aggregate function

 - minor group by optimization (replaced idxMap with value extractors)

 - grouping performance tuning

 - added more trace logs

 - ``MIN`` aggregate function, basic infrastructure for column/value based aggregation

2013/11/26 0.19.8
=================

 - added more trace logs

 - fixed classpath overloading issue.

2013/11/25 0.19.7
=================

 - fixed a bug that occured if the offset in a group by query was larger then
   the number of rows in the result-set

 - fix: group by did not work correctly on nested columns due to missing mapping

 - group by performance improvements.

 - the `group by` clause now also works if information_schema tables are
   queried.

2013/11/22 0.19.6
=================

 - fix: added sql query duration to SQL xcontent response

2013/11/22 0.19.5
=================

 - fix: offset didn't work correctly for group by queries.

 - fixed primary key handling in inout plugin

 - fixed a bug where special fields might not get applied in inout
   plugin import when a sub-object occurs in a row

 - error that group by doesn't work on columns that contain arrays is now
   already raised in the query parsing step if possible.

 - improved test-run speed by not starting nodes unnecessarily

 - further optimization of the internal data structure serialization used while
   executing a group by query.

 - Implemented Crate Version & Build class
   - use Crate version number at main rest endpoint
   - use ES version as `es_version`
   - use Version.CURRENT for distribution & create_tag.sh
   - removed static versions.json, read it from main rest endpoint instead

 - Throw a proper error if a multi-value field is encountered during group by
   instead of returning an incorrect result.

 - refactoring of Doctest TestCases

 - internal: support for mapping ip-types (stored as string)

 - Added ``duration`` property to SQLResponse, reflecting the duration of a SQL query.

 - optimized internal data structures that are used when executing a group by
   query. This should reduce memory consumption and speed up serialization.

 - map builtin column ``_score`` explicitly to double

 - type guessing on added columns (as root column or in dynamic object/craty)

 - interpret nested types in indexmapping as craty

 - Increased default ``SELECT`` limit to 10000.
   Also use this default limit on ``GROUP BY`` queries if no limit is defined explicit.

 - Fix: NPE was thrown when using a star select on an index with no defined mappings

 - new es upstream commit: new setting ``index.mapper.map_source`` to control whether source
   values get mapped when returning

 - fix: correct type validation and mapping on all handler nodes, even on nodes without table-shard
   according to table definition

 - nested columns of object types now listed in ``information_schema.columns``

2013/11/18 0.19.4
=================

 - Fix: proper error handling while uploading files to an index with disabled blob support

 - test hickup fixes and speed improvements

 - proper Error handling when issuing statements against a table-alias

 - rename column ``expressions`` of ``information_schema.indices`` to ``columns``
   and change type to array of strings

2013/11/14 0.19.3
=================

 - Schema check for table aliases can be disabled by config now.

2013/11/13 0.19.2
=================

 - Added support for table alias usage on ``SELECT`` queries. Only possible if all tables are
   using the same schema.

 - Windows build: favor crate jars over all other jars at classpath

 - Fix: make test-setup create data dirs as temporary directories
   that get deleted after test runs

 - Fix: use nodeName instead of nodeId for path var ${node} on ``COPY`` SQL command

 - Added support for ${node}, ${cluster} and ${table} variables inside the `path` value of a
   ``COPY`` SQL command

 - Fix: using regex pattern on ``COPY`` did not work properly

2013/11/13 0.19.1
=================

 - added column ``routing_column`` to table ``information_schema.tables``,
   listing the column whose value determines on which shard a document gets stored

2013/11/12 0.19.0
=================

 - Added support for importing data using the SQL `copy <table> from <path>` command.

2013/11/11 0.18.6
=================

 - fix NPE if a null value was passed inside a complex type.

2013/11/11 0.18.5
=================

 - bugfix: complex types like lists and dictionaries now also have the mapping
   applied to the values inside.  For instance if a list of datetimes (in a iso
   datetime string) is passed to the `/_sql` REST endpoint these values are
   converted to long.

2013/11/08 0.18.4
=================

 - the sql `count()` function now also be called with parameter substitution
   (`count(?)`) and with primary key primary key columns (`count(pk_col)`).
   This way SQLAlchemy's ORM functionality can be used with Crate.


2013/11/07 0.18.3
=================

 - values of a row inserted using the sql parameter substitution feature are
   now also mapped to the correct format if the table has a defined type for
   the column.

 - fix: default SQL settings were missing at distribution config file

 - fix: default cluster name(and so the log file) was not `crate`

 - bugfix: SQL statements with `group by` and `limit` clauses returned wrong
   result sets.

 - make information_schema behave like "normal" crate-tables

 - bugfix: NullPointerException when querying for non existent information_schema tables

 - Added the ``routines`` table to information_schema to contain analyzers, tokenizers,
   token-filters and char-filters

2013/11/06 0.18.2
=================

 - Fix: use update-by-path instead of merge on primary-keys queries as well

 - Added virtual table ``indices`` to the ``information_schema``, holds information about all
   available indices.

 - Added virtual table ``columns`` to the ``information_schema``, holds information about all
   available columns.

 - Added the ``table_constraints`` table to information_schema

 - Table index definition documentation overhaul

 - bugfix: the `LIKE` operand now also works on nested columns

2013/11/01 0.18.1
=================

 - Added index method ``plain`` which is the default behaviour

2013/10/31 0.18.0
=================

 - Added the `information_schema` schema that contains a virtual `tables` table
   which can be queries to get information about the currently available
   tables.

 - Added fulltext search support

 - Added support for indices on table creation

 - Implemented ``CREATE ANALYZER`` to create custom analyzers for fulltext-search

2013/10/29 0.17.3
=================

 - fixed a NullPointerException that occured if a `select` statement  got
   optimized to a primary key lookup and returned an object column.

 - fixed a NullPointerException that occured when `null` was passed as `args`
   argument to the `_sql` REST endpoint and the statement contained a `group
   by` clause.

2013/10/28 0.17.2
=================

 - bugfix: the `LIKE` operand only worked with literals, not with parameter
   substitution.

2013/10/25 0.17.1
=================

 - On update, values of kind `object` or `nested` will be overwritten instead of merged.

 - fixed sigar library loading which broke os stats of nodes

2013/10/25 0.17.0
=================

 - Implemented support for optimistic concurrent updates and deletes.

 - Use new crate-cli (crash) version with row count output at doctests.

 - Prevent table creation with routing definition on non-primary key columns.

 - removed `*` and `?` as wildcard characters for the `LIKE` clause to be more
   consistent with the SQL standard.

 - Optimize simple Select Queries with where clauses like <pk_col>=<constant_value> OR <pk_col>=<constant_value>
   and without order-by- or group-by-clauses to use MultiGetRequests

2013/10/21 0.16.0
=================

 - Added support for the SQL `LIKE` operand.

 - Implemented ``drop table``.

 - Implemented the `group by` sql clause.

 - Implemented ``CREATE TABLE`` with core data types, replicas, routing and shard settings.

 - Support for: WHERE <column_name> IN ( <value_1>, <value_2>, ..., <value_n>)

 - Query Planner Optimizes Select / Update / Delete Queries where:

   - Where clause is: <pk_col> = <constant_value>
   - Where clause is like: <pk_col> = <constant_value> OR <pk_col> = <constant_value> OR <pk_col> = ...
   - Where clause is like: <pk_col> = <constant_value> AND ...
   - Where clause is like: <pk_col> IN (<constant_value>) [ OR <pk_col> = <constant_value> ... ]

2013/10/11 0.15.1
=================

 - Rowcount in Integrationtests

2013/10/11 0.15.0
=================

 - created Benchmark to check performance and verify improvements by query planner

 - added rowcount to all SELECT responses

 - fixed filtering by a boolean constant

 - implemented SQL query planner:
   - optimize primary key queries to hit only one single shard

 - set explicit default limit on select to 1000 rows.

2013/10/08 0.14.0
=================

 - set ``primary keys`` definition under the type mappings ``_meta`` instead of index
   ``settings.crate``

 - added support for `select count(*) from ...`

 - updated elasticsearch to 0.90.5

 - included the python clients sqlalchemy documentation

2013/10/02 0.13.2
=================

 - updated crate-python client documentation

 - If a shard or version conflict occurs during select query or update the
   errors are now correctly thrown. Extended the documentation to mention the
   VerionConflictException

 - don't load any JS files from externals domains (CDN) at the admin web interface

2013/10/02 0.13.1
=================

 - include rowcount of affected rows into update responses

 - [test] create temporary directory for test data, will be deleted after test run

2013/10/02 0.13.0
=================

 - add support for sql update statements.

 - set the ``keyword analyzer`` as the default analyzer

 - bugfix: use ES operation type ``create`` for SQL ``INSERT``.

2013/09/27 0.12.0
=================

 - implemented support for selecting and querying inner/nested object properties.

 - calling _sql with `null` in the `args` no longer causes an error.

 - fixed a regression that caused chunk requests to only work for blobs

 - included foundationDB/akiban ``sql-parser`` code into the ``sql`` module

2013/09/20 0.11.3
=================

 - sql parameter substitution now also works for the limit and offset clauses

2013/09/18 0.11.2
=================

 - fixed support for ``object`` type mappings, objects are now supported
   by the transport layer (so also by the java client library).
   In addition the `/_sql`  endpoint now properly accepts nested objects inside
   the `args` key of the request body.

2013/09/18 0.11.1
=================

 - fixed the namespace error in the startscript that resulted in crate not
   starting correctly in the background.

2013/09/17 0.11.0
=================

 - the `_sql` endpoint now supports SQL parameter substitution.

 - DELETE and INSERT SQL statements are now supported.

2013/09/05 0.10.2
=================

 - added java client library in client project

2013/09/04 0.10.1
=================

 - fixed a serialization error in the sql module

2013/09/03 0.10.0
=================

 - blob support is now included

 - sql response format changed

 - use crate fork for elasticsearch dependency

 - use gradle instead of maven

 - integrate plugins from submodules to internal folders

 - removed timefacts plugin since it is not 0.90.3 compatible

 NOTE:
    this update makes crate incompatible with other elasticsearch-plugins

2013/08/08 0.9.2
================

 - updated crate-sql-plugin to 0.1.0

2013/07/24 0.9.1
================

 - fixed scrolling issue in the sql console of the admin interface

2013/07/24 0.9.0
================

 - changed the admin interface to open the linked plugins and documentation in
   a frame.

 - added a custom sphinx theme for the documentation.

 - included sql and inout plugin documentation

 - sphinx generated text documentation added to distribution

 - docs are now tested in the maven verify phase

 - added a link to the documentation in the crate-admin interface available
   under `/admin`

 - added a SQL console to the crate admin interface.

 - added the crate-sql-plugin. Therefore it is now possible to query crate
   using SQL.

 - docs are now available under /_plugins/docs

2013/07/10 0.8.5
================

 - added default config path to start script

 - reverted loading behaviour for settings

 - added DEVELOP.rst file

 - tarball now include CHANGES file

2013/07/09 0.8.4
================

 - added whitelist to allow some hard coded system properties

 - altered start scripts to use valid system properties

2013/07/09 0.8.3
================

 - loading system settings given by the command line as well

2013/07/08 0.8.2
================

 - fixed wrong inclusion path of sigar libs

2013/07/08 0.8.1
================

 - updated elasticsearch-inout-plugin to 0.5.0

2013/07/05 0.8.0
================

 - updated elasticsearch-timefacets-plugin to 0.8.0

 - updated elasticsearch to 0.90.2

 - organized project structure

 - included elasticsearch-inout-plugin and
   elasticsearch-timefacets-plugin as git submodule instead of maven
   dependencs

 - added /admin rest endpoint redirecting to crate-admin

 - added crate-admin site plugin

 - upgraded bigdesk to 2.2.1, which fixes cache memory display issues

2013/06/26 0.7.0
================

 - added elasticsearch-cratedefaults-plugin as sub-module

 - mvn package now also creates a zip file in addition to the tar.gz

 - updated elasticsearch-timefacets-plugin to 0.7.1

2013/06/10 0.6.0
================

 - updated elasticsearch-timefacets-plugin to 0.7.0

 - updated elasticsearch-inout-plugin to 0.4.0

 - updated elasticsearch to 0.90.1

2013/05/28 0.5.0
================

 - included elasticsearch-timefacets-plugin
   (https://github.com/crate/elasticsearch-timefacets-plugin)

2013/05/15 0.4.0
================

 - included segmentspy as site-plugin
   (https://github.com/polyfractal/elasticsearch-segmentspy)

 - included bigdesk as site-plugin
   (https://github.com/lukas-vlcek/bigdesk)

 - included elasticsearch-head as site-plugin
   (https://github.com/mobz/elasticsearch-head)

2013/05/15 0.3.0
================

 - updated elasticsearch-inout-plugin to 0.3.0

 - updated elasticsearch to 0.90.0

2013/05/02 0.2.1
================

 - bugfix: used wrong groupId in common-bin.xml

2013/05/02 0.2.0
================

 - updated elasticsearch-inout-plugin to 0.2.0-SNAPSHOT

2013/04/25 0.1.1
================

 - added missing sigar libraries in rpm as well

2013/04/25 0.1.0
================

 - added missing sigar libraries

 - depending on elasticsearch-inout-plugin

2013/04/17 0.0.8
================

 - updated export plugin to 0.0.3

2013/04/15 0.0.7
================

 - increased version of elasticsearch

 - increased version of elasticsearch-export-plugin

2013/04/15 0.0.6
================

 - added NOTICE file