Fixes yiisoft#7640: Implemented custom data types support. Added JSON…

… support for MySQL and PostgreSQL, array support for PostgreSQL

.travis.yml

@@ -46,9 +46,9 @@ cache:
     - $HOME/.composer/cache
     - $HOME/.npm
 
-# try running against postgres 9.3
+# try running against postgres 9.6
 addons:
-  postgresql: "9.3"
+  postgresql: "9.6"
   code_climate:
     repo_token: 2935307212620b0e2228ab67eadd92c9f5501ddb60549d0d86007a354d56915b
 
@@ -89,7 +89,7 @@ matrix:
       addons:
         code_climate:
           repo_token: 2935307212620b0e2228ab67eadd92c9f5501ddb60549d0d86007a354d56915b
-        postgresql: "9.3"
+        postgresql: "9.6"
         apt:
           packages:
             - mysql-server-5.6

docs/guide/db-active-record.md

@@ -472,7 +472,7 @@ $customer->loadDefaultValues();
 
 ### Attributes Typecasting <span id="attributes-typecasting"></span>
 
-Being populated by query results [[yii\db\ActiveRecord]] performs automatic typecast for its attribute values, using
+Being populated by query results, [[yii\db\ActiveRecord]] performs automatic typecast for its attribute values, using
 information from [database table schema](db-dao.md#database-schema). This allows data retrieved from table column
 declared as integer to be populated in ActiveRecord instance with PHP integer, boolean with boolean and so on.
 However, typecasting mechanism has several limitations:
@@ -490,7 +490,33 @@ converted during saving process.
 
 > Tip: you may use [[yii\behaviors\AttributeTypecastBehavior]] to facilitate attribute values typecasting
   on ActiveRecord validation or saving.
+
+Since 2.0.14, Yii ActiveRecord supports complex data types, such as JSON or multidimensional arrays.
+
+#### JSON in MySQL and PostgreSQL
+
+After data population, the value from JSON column will be automatically decoded from JSON according to standard JSON
+decoding rules.
+
+To save attribute value to a JSON column, ActiveRecord will automatically create a [[yii\db\JsonExpression|JsonExpression]] object
+that will be encoded to a JSON string on [QueryBuilder](db-query-builder.md) level.
+
+#### Arrays in PostgreSQL
+
+After data population, the value from Array column will be automatically decoded from PgSQL notation to an [[yii\db\ArrayExpression|ArrayExpression]]
+object. It implements PHP `ArrayAccess` interface, so you can use it as an array, or call `->getValue()` to get the array itself.
+
+To save attribute value to an array column, ActiveRecord will automatically create an [[yii\db\ArrayExpression|ArrayExpression]] object
+that will be encoded by [QueryBuilder](db-query-builder.md) to an PgSQL string representation of array.
+
+You can also use conditions for JSON columns:
+
+```php
+$query->andWhere(['=', 'json', new ArrayExpression(['foo' => 'bar'])
+```
 
+To learn more about expressions building system read the [Query Builder – Adding custom Conditions and Expressions](db-query-builder.md#adding-custom-conditions-and-expressions)
+article.
 
 ### Updating Multiple Rows <span id="updating-multiple-rows"></span>
 

docs/guide/db-query-builder.md

@@ -160,12 +160,12 @@ are in the ["Quoting Tables" section of the "Database Access Objects" guide](gui
 ### [[yii\db\Query::where()|where()]] <span id="where"></span>
 
 The [[yii\db\Query::where()|where()]] method specifies the `WHERE` fragment of a SQL query. You can use one of
-the three formats to specify a `WHERE` condition:
+the four formats to specify a `WHERE` condition:
 
 - string format, e.g., `'status=1'`
 - hash format, e.g. `['status' => 1, 'type' => 2]`
 - operator format, e.g. `['like', 'name', 'test']`
-
+- object format, e.g. `new LikeCondition('name', 'LIKE', 'test')`
 
 #### String Format <span id="string-format"></span>
 
@@ -306,6 +306,41 @@ the operator can be one of the following:
 Using the Operator Format, Yii internally uses parameter binding so in contrast to the [string format](#string-format), here
 you do not have to add parameters manually.
 
+#### Object Format <span id="object-format"></span>
+
+Object Form is available since 2.0.14 and is both most powerful and most complex way to define conditions.
+You need to follow it either if you want to build your own abstraction over query builder or if you want to implement
+your own complex conditions.
+
+Instances of condition classes are immutable. Their only purpose is to store condition data and provide getters
+for condition builders. Condition builder is a class that holds the logic that transforms data 
+stored in condition into the SQL expression.
+
+Internally the formats described above are implicitly converted to object format prior to building raw SQL,
+so it is possible to combine formats in a single condition:
+
+```php
+$query->andWhere(new OrCondition([
+    new InCondition('type', 'in', $types),
+    ['like', 'name', '%good%'],
+    'disabled=false'
+]))
+```
+
+Conversion from operator format into object format is performed according to
+[[yii\db\QueryBuilder::conditionClasses|QueryBuilder::conditionClasses]] property, that maps operators names
+to representative class names:
+
+- `AND`, `OR` -> `yii\db\conditions\ConjunctionCondition`
+- `NOT` -> `yii\db\conditions\NotCondition`
+- `IN`, `NOT IN` -> `yii\db\conditions\InCondition`
+- `BETWEEN`, `NOT BETWEEN` -> `yii\db\conditions\BetweenCondition`
+
+And so on.
+
+Using the object format makes it possible to create your own conditions or to change the way default ones are built.
+See [Creating Custom Conditions and Expressions](#creating-custom-conditions-and-expressions) chapter to learn more.
+
 
 #### Appending Conditions <span id="appending-conditions"></span>
 
@@ -758,3 +793,170 @@ $unbufferedDb->close();
 ```
 
 > Note: unbuffered query uses less memory on the PHP-side, but can increase the load on the MySQL server. It is recommended to design your own code with your production practice for extra massive data, [for example, divide the range for integer keys, loop them with Unbuffered Queries](https://github.com/yiisoft/yii2/issues/8420#issuecomment-296109257).
+
+### Adding custom Conditions and Expressions <span id="adding-custom-conditions-and-expressions"></span>
+
+As it was mentioned in [Conditions – Object Fromat](#object-format) chapter, is is possible to create custom condition
+classes. For example, let's create a condition that will check that specific columns are less than some value.
+Using the operator format, it would look like the following:
+
+```php
+[
+    'and',
+    '>', 'posts', $minLimit,
+    '>', 'comments', $minLimit,
+    '>', 'reactions', $minLimit,
+    '>', 'subscriptions', $minLimit
+]
+```
+
+When such condition applied once, it is fine. In case it is used multiple times in a single query it can
+be optimized a lot. Let's create a custom condition object to demonstrate it.
+
+Yii has a [[yii\db\conditions\ConditionInterface|ConditionInterface]], that must be used to mark classes, that represent
+a condition. It requires `fromArrayDefinition()` method implementation, in order to make possible to create condition
+from array format. In case you don't need it, you can implement this method with exception throwing.
+
+Since we create our custom condition class, we can build API that suits our task the most. 
+
+```php
+namespace app\db\conditions;
+
+class AllGreaterCondition implements \yii\db\conditions\ConditionInterface
+{
+    private $columns;
+    private $value;
+
+    /**
+     * @param string[] $columns Array of columns that must be greater, than $value
+     * @param mixed $value the value to compare each $column against.
+     */
+    public function __construct(array $columns, $value)
+    {
+        $this->columns = $columns;
+        $this->value = $value;
+    }
+
+    public static function fromArrayDefinition($operator, $operands)
+    {
+        throw new InvalidParamException('Not implemented yet, but we will do it later');
+    }
+
+    public function getColumns() { return $this->columns; }
+    public function getValue() { return $this->vaule; }
+}
+```
+
+So we can create a condition object:
+
+```php
+$conditon = new AllGreaterCondition(['col1', 'col2'], 42);
+```
+
+But `QueryBuilder` still does not know, to to make an SQL condition out of this object.
+Now we need to create a builder for this condition. It must implement [[yii\db\ExpressionBuilderInterface]]
+that requires us to implement a `build()` method. 
+
+```php
+namespace app\db\conditions;
+
+class AllGreaterConditionBuilder implements \yii\db\ExpressionBuilderInterface
+{
+    use \yii\db\Condition\ExpressionBuilderTrait; // Contains constructor and `queryBuilder` property.
+
+    /**
+     * @param AllGreaterCondition $condition the condition to be built
+     * @param array $params the binding parameters.
+     */ 
+    public function build(ConditionInterface $condition, &$params)
+    {
+        $value = $condition->getValue();
+
+        $conditions = [];
+        foreach ($condition->getColumns() as $column) {
+            $conditions[] = new SimpleCondition($column, '>', $value);
+        }
+
+        return $this->queryBuider->buildCondition(new AndCondition($conditions), $params);
+    }
+}
+```
+
+Then simple let [[yii\db\QueryBuilder|QueryBuilder]] know about our new condition – add a mapping for it to 
+the `expressionBuilders` array. It could be done right from the application configuration:
+
+```php
+'db' => [
+    'class' => 'yii\db\mysql\Connection',
+    // ...
+    'queryBuilder' => [
+        'expressionBuilders' => [
+            'app\db\conditions\AllGreaterCondition' => 'app\db\conditions\AllGreaterConditionBuilder',
+        ],
+    ],
+],
+```
+
+Now we can use our condition in `where()`:
+
+```php
+$query->andWhere(new AllGreaterCondition(['posts', 'comments', 'reactions', 'subscriptions'], $minValue));
+```
+
+If we want to make it possible to create our custom condition using operator format, we should declare it in
+[[yii\db\QueryBuilder::conditionClasses|QueryBuilder::conditionClasses]]:
+
+```php
+'db' => [
+    'class' => 'yii\db\mysql\Connection',
+    // ...
+    'queryBuilder' => [
+        'expressionBuilders' => [
+            'app\db\conditions\AllGreaterCondition' => 'app\db\conditions\AllGreaterConditionBuilder',
+        ],
+        'conditionClasses' => [
+            'ALL>' => 'app\db\conditions\AllGreaterCondition',
+        ],
+    ],
+],
+```
+
+And create a real implementation of `AllGreaterCondition::fromArrayDefinition()` method 
+in `app\db\conditions\AllGreaterCondition`:
+
+```php
+namespace app\db\conditions;
+
+class AllGreaterCondition implements \yii\db\conditions\ConditionInterface
+{
+    // ... see the implementation above
+
+    public static function fromArrayDefinition($operator, $operands)
+    {
+        return new static($operands[0], $operands[1]);
+    }
+}
+```
+
+After that, we can create our custom condition using shorter operator format:
+
+```php
+$query->andWhere(['ALL>', ['posts', 'comments', 'reactions', 'subscriptions'], $minValue]);
+```
+
+You might notice, that there was two concepts used: Expressions and Conditions. There is a [[yii\db\ExpressionInterface]]
+that should be used to mark objects, that require an Expression Builder class, that implements
+[[yii\db\ExpressionBuilderInterface]] to be built. Also there is a [[yii\db\condition\ConditionInterface]], that extends
+[[yii\db\ExpressionInterface|ExpressionInterface]] and should be used to objects, that can be created from array definition
+as it was shown above, but require builder as well.
+
+To summarise:
+
+- Expression – is a Data Transfer Object (DTO) for a dataset, that can be somehow compiled to some SQL 
+statement (an operator, string, array, JSON, etc).
+- Condition – is an Expression superset, that aggregates multiple Expressions (or scalar values) that can be compiled
+to a single SQL condition.
+
+You can create your own classes that implement [[yii\db\ExpressionInterface|ExpressionInterface]] to hide the complexity
+of transforming data to SQL statements. You will learn more about other examples of Expressions in the
+[next article](db-active-record.md);

framework/CHANGELOG.md

@@ -157,6 +157,7 @@ Yii Framework 2 Change Log
 - Enh #4495:  Added closure support in `yii\i18n\Formatter` (developeruz)
 - Enh #5786: Allowed to use custom constructors in ActiveRecord-based classes (ElisDN, klimov-paul)
 - Enh #6644: Added `yii\helpers\ArrayHelper::setValue()` (LAV45)
+- Enh #7640: Implemented custom data types support. Added JSON support for MySQL and PostgreSQL, array support for PostgreSQL (silverfire, cebe)
 - Enh #7823: Added `yii\filters\AjaxFilter` filter (dmirogin)
 - Enh #9438: `yii\web\DbSession` now relies on error handler to display errors (samdark)
 - Enh #9703, #9709: Added `yii\i18n\Formatter::asWeight()` and `::asLength()` formatters (nineinchnick, silverfire)

framework/UPGRADE.md

@@ -58,6 +58,20 @@ Upgrade from Yii 2.0.13
 
 * `yii\base\Security::compareString()` is now throwing `yii\base\InvalidParamException` in case non-strings are compared.
 
+* `yii\db\ExpressionInterface` has been introduced to represent a wider range of SQL expressions. In case you check for
+  `instanceof yii\db\Expression` in your code, you might consider changing that to checking for the interface and use the newly
+  introduced methods to retrieve the expression content.
+
+* `yii\db\PdoValue` class has been introduced to replace a special syntax that was used to declare PDO parameter type 
+when binding parameters to an SQL command, for example: `['value', \PDO::PARAM_STR]`.
+You should use `new PdoValue('value', \PDO::PARAM_STR)` instead. Old syntax will be removed in Yii 2.1.
+
+* `yii\db\QueryBuilder::conditionBuilders` property and method-based condition builders are no longer used. 
+Class-based conditions and builders are introduces instead to provide more flexibility, extensibility and
+space to customization. In case you rely on that property or override any of default condition builders, follow the 
+special [guide article](http://www.yiiframework.com/doc-2.0/guide-db-query-builder.html#adding-custom-conditions-and-expressions)
+to update your code.
+
 * Log targets (like `yii\log\EmailTarget`) are now throwing `yii\log\LogRuntimeException` in case log can not be properly exported.
 
 Upgrade from Yii 2.0.12

framework/caching/DbCache.php

@@ -10,6 +10,7 @@
 use Yii;
 use yii\base\InvalidConfigException;
 use yii\db\Connection;
+use yii\db\PdoValue;
 use yii\db\Query;
 use yii\di\Instance;
 
@@ -195,7 +196,7 @@ protected function setValue($key, $value, $duration)
             $command = $db->createCommand()
                 ->update($this->cacheTable, [
                     'expire' => $duration > 0 ? $duration + time() : 0,
-                    'data' => [$value, \PDO::PARAM_LOB],
+                    'data' => new PdoValue($value, \PDO::PARAM_LOB),
                 ], ['id' => $key]);
             return $command->execute();
         });
@@ -228,7 +229,7 @@ protected function addValue($key, $value, $duration)
                     ->insert($this->cacheTable, [
                         'id' => $key,
                         'expire' => $duration > 0 ? $duration + time() : 0,
-                        'data' => [$value, \PDO::PARAM_LOB],
+                        'data' => new PdoValue($value, \PDO::PARAM_LOB),
                     ])->execute();
             });