Merge branch 'release/0.5.4' into next

CHANGELOG.md

@@ -15,6 +15,15 @@
 
 - In the API, the `next` link is not being displayed anymore if there are no more entities to return. [#635](https://github.com/Mashape/kong/pull/635)
 
+## [0.5.4] - 2015/12/03
+
+### Fixed
+
+- Mashape Analytics plugin (aka Galileo):
+  - Improve stability under heavy load. [#757](https://github.com/Mashape/kong/issues/757)
+  - base64 encode ALF request/response bodies, enabling proper support for Galileo bodies inspection capabilities. [#747](https://github.com/Mashape/kong/pull/747)
+  - Do not include JSON bodies in ALF `postData.params` field. [#766](https://github.com/Mashape/kong/pull/766)
+
 ## [0.5.3] - 2015/11/16
 
 ### Fixed
@@ -373,8 +382,9 @@ First version running with Cassandra.
 - CLI `bin/kong` script.
 - Database migrations (using `db.lua`).
 
-[unreleased]: https://github.com/mashape/kong/compare/0.5.3...next
-[0.5.3]: https://github.com/mashape/kong/compare/0.5.3...0.5.3
+[unreleased]: https://github.com/mashape/kong/compare/0.5.4...next
+[0.5.4]: https://github.com/mashape/kong/compare/0.5.3...0.5.4
+[0.5.3]: https://github.com/mashape/kong/compare/0.5.2...0.5.3
 [0.5.2]: https://github.com/mashape/kong/compare/0.5.1...0.5.2
 [0.5.1]: https://github.com/mashape/kong/compare/0.5.0...0.5.1
 [0.5.0]: https://github.com/mashape/kong/compare/0.4.2...0.5.0

UPDATE.md → UPGRADE.md

@@ -1,6 +1,28 @@
-This document describes eventual additional steps that might be required to update between two versions of Kong. If nothing is described here for a particular version and platform, then assume the update will go smoothly.
+This document guides you through the process of upgrading Kong. First, check if a section named "Upgrade to Kong `x.x.x`" exists (`x.x.x`) being the version you are planning to upgrade to. If such a section does not exist, the upgrade you want to perform does not have any particular instructions, and you can simply consult the [Suggested upgrade path](#suggested-upgrade-path).
 
-## Update to Kong `0.5.x`
+## Suggested upgrade path
+
+Unless indicated otherwise in one of the upgrade paths of this document, it is possible to upgrade Kong **without downtime**:
+
+Considering that Kong is already running on your system, acquire the latest version from any of the available [installation methods](https://getkong.org/install/) and proceed to installing it, overriding your previous installation. Once done, consider that this is a good time to also modify your configuration.
+
+Then, run any new migration to upgrade your database schema:
+
+```shell
+$ kong migrations up [-c configuration_file]
+...
+[OK] Schema up to date
+```
+
+If you see the "Schema up to date" message, you only have to [reload](https://getkong.org/docs/latest/cli/#reload) Kong:
+
+```shell
+$ kong reload [-c configuration_file]
+```
+
+**Reminder**: `kong reload` leverages the Nginx `reload` signal and seamlessly starts new workers taking over the old ones until they all have been terminated. This will guarantee you no drop in your current incoming traffic.
+
+## Upgrade to Kong `0.5.x`
 
 Migrating to 0.5.x can be done **without downtime** by following those instructions. It is important that you be running Kong `0.4.2` and have the latest release of Python 2.7 on your system when executing those steps.
 
@@ -49,7 +71,7 @@ properties:
 
 ##### 2. Migration script
 
-[This Python script](/scripts/migration.py) will take care of migrating your database schema should you execute the following instructions:
+[This Python script](https://github.com/Mashape/kong/blob/0.5.0/scripts/migration.py) will take care of migrating your database schema should you execute the following instructions:
 
 ```shell
 # First, make sure you are already running Kong 0.4.2
@@ -74,11 +96,7 @@ If everything went well the script should print a success message. **At this poi
 
 ##### 3. Upgrade without downtime
 
-You can now update Kong to 0.5.x. Proceed as a regular update and install the package of your choice from the website. After updating, reload Kong to avoid downtime:
-
-```shell
-$ kong reload
-```
+You can now upgrade Kong to `0.5.x.` Proceed as a regular upgrade and follow the suggested upgrade path, in particular the `kong reload` command.
 
 ##### 4. Purge your Cassandra cluster
 
@@ -116,7 +134,7 @@ The old routes are still maintained but will be removed in upcoming versions. Co
   - The route to retrieve enabled plugins is now under `/plugins/enabled`.
   - The route to retrieve a plugin's configuration schema is now under `/plugins/schema/{plugin name}`.
 
-## Update to Kong `0.4.2`
+## Upgrade to Kong `0.4.2`
 
 The configuration format for specifying the port of your Cassandra instance changed. Replace:
 
@@ -136,7 +154,7 @@ cassandra:
       - "localhost:9042"
 ```
 
-## Update to Kong `0.3.x`
+## Upgrade to Kong `0.3.x`
 
 Kong now requires a patch on OpenResty for SSL support. On Homebrew you will need to reinstall OpenResty.
 

kong-0.5.3-1.rockspec → kong-0.5.4-1.rockspec

@@ -1,5 +1,5 @@
 package = "kong"
-version = "0.5.3-1"
+version = "0.5.4-1"
 supported_platforms = {"linux", "macosx"}
 source = {
   url = "git://github.com/Mashape/kong",
@@ -97,22 +97,22 @@ build = {
 
     ["kong.plugins.base_plugin"] = "kong/plugins/base_plugin.lua",
 
-    ["kong.plugins.basic-auth.migrations.cassandra"] = "kong/plugins/basic-auth/migrations/cassandra.lua",
+    ["kong.plugins.basic-auth.schema.migrations"] = "kong/plugins/basic-auth/schema/migrations.lua",
     ["kong.plugins.basic-auth.crypto"] = "kong/plugins/basic-auth/crypto.lua",
     ["kong.plugins.basic-auth.handler"] = "kong/plugins/basic-auth/handler.lua",
     ["kong.plugins.basic-auth.access"] = "kong/plugins/basic-auth/access.lua",
     ["kong.plugins.basic-auth.schema"] = "kong/plugins/basic-auth/schema.lua",
     ["kong.plugins.basic-auth.api"] = "kong/plugins/basic-auth/api.lua",
     ["kong.plugins.basic-auth.daos"] = "kong/plugins/basic-auth/daos.lua",
 
-    ["kong.plugins.key-auth.migrations.cassandra"] = "kong/plugins/key-auth/migrations/cassandra.lua",
+    ["kong.plugins.key-auth.schema.migrations"] = "kong/plugins/key-auth/schema/migrations.lua",
     ["kong.plugins.key-auth.handler"] = "kong/plugins/key-auth/handler.lua",
     ["kong.plugins.key-auth.access"] = "kong/plugins/key-auth/access.lua",
     ["kong.plugins.key-auth.schema"] = "kong/plugins/key-auth/schema.lua",
     ["kong.plugins.key-auth.api"] = "kong/plugins/key-auth/api.lua",
     ["kong.plugins.key-auth.daos"] = "kong/plugins/key-auth/daos.lua",
 
-    ["kong.plugins.oauth2.migrations.cassandra"] = "kong/plugins/oauth2/migrations/cassandra.lua",
+    ["kong.plugins.oauth2.schema.migrations"] = "kong/plugins/oauth2/schema/migrations.lua",
     ["kong.plugins.oauth2.handler"] = "kong/plugins/oauth2/handler.lua",
     ["kong.plugins.oauth2.access"] = "kong/plugins/oauth2/access.lua",
     ["kong.plugins.oauth2.schema"] = "kong/plugins/oauth2/schema.lua",
@@ -143,13 +143,13 @@ build = {
     ["kong.plugins.mashape-analytics.schema"] = "kong/plugins/mashape-analytics/schema.lua",
     ["kong.plugins.mashape-analytics.buffer"] = "kong/plugins/mashape-analytics/buffer.lua",
 
-    ["kong.plugins.rate-limiting.migrations.cassandra"] = "kong/plugins/rate-limiting/migrations/cassandra.lua",
+    ["kong.plugins.rate-limiting.schema.migrations"] = "kong/plugins/rate-limiting/schema/migrations.lua",
     ["kong.plugins.rate-limiting.handler"] = "kong/plugins/rate-limiting/handler.lua",
     ["kong.plugins.rate-limiting.access"] = "kong/plugins/rate-limiting/access.lua",
     ["kong.plugins.rate-limiting.schema"] = "kong/plugins/rate-limiting/schema.lua",
     ["kong.plugins.rate-limiting.daos"] = "kong/plugins/rate-limiting/daos.lua",
 
-    ["kong.plugins.response-ratelimiting.migrations.cassandra"] = "kong/plugins/response-ratelimiting/migrations/cassandra.lua",
+    ["kong.plugins.response-ratelimiting.schema.migrations"] = "kong/plugins/response-ratelimiting/schema/migrations.lua",
     ["kong.plugins.response-ratelimiting.handler"] = "kong/plugins/response-ratelimiting/handler.lua",
     ["kong.plugins.response-ratelimiting.access"] = "kong/plugins/response-ratelimiting/access.lua",
     ["kong.plugins.response-ratelimiting.header_filter"] = "kong/plugins/response-ratelimiting/header_filter.lua",
@@ -185,14 +185,14 @@ build = {
     ["kong.plugins.ip-restriction.access"] = "kong/plugins/ip-restriction/access.lua",
     ["kong.plugins.ip-restriction.schema"] = "kong/plugins/ip-restriction/schema.lua",
 
-    ["kong.plugins.acl.migrations.cassandra"] = "kong/plugins/acl/migrations/cassandra.lua",
+    ["kong.plugins.acl.schema.migrations"] = "kong/plugins/acl/schema/migrations.lua",
     ["kong.plugins.acl.handler"] = "kong/plugins/acl/handler.lua",
     ["kong.plugins.acl.access"] = "kong/plugins/acl/access.lua",
     ["kong.plugins.acl.schema"] = "kong/plugins/acl/schema.lua",
     ["kong.plugins.acl.api"] = "kong/plugins/acl/api.lua",
     ["kong.plugins.acl.daos"] = "kong/plugins/acl/daos.lua",
 
-    ["kong.plugins.acl.migrations.cassandra"] = "kong/plugins/acl/migrations/cassandra.lua",
+    ["kong.plugins.acl.schema.migrations"] = "kong/plugins/acl/schema/migrations.lua",
     ["kong.plugins.acl.handler"] = "kong/plugins/acl/handler.lua",
     ["kong.plugins.acl.access"] = "kong/plugins/acl/access.lua",
     ["kong.plugins.acl.schema"] = "kong/plugins/acl/schema.lua",
@@ -208,15 +208,15 @@ build = {
     ["kong.api.routes.plugins"] = "kong/api/routes/plugins.lua",
     ["kong.api.routes.plugins"] = "kong/api/routes/plugins.lua",
 
-    ["kong.plugins.jwt.migrations.cassandra"] = "kong/plugins/jwt/migrations/cassandra.lua",
+    ["kong.plugins.jwt.schema.migrations"] = "kong/plugins/jwt/schema/migrations.lua",
     ["kong.plugins.jwt.handler"] = "kong/plugins/jwt/handler.lua",
     ["kong.plugins.jwt.access"] = "kong/plugins/jwt/access.lua",
     ["kong.plugins.jwt.schema"] = "kong/plugins/jwt/schema.lua",
     ["kong.plugins.jwt.api"] = "kong/plugins/jwt/api.lua",
     ["kong.plugins.jwt.daos"] = "kong/plugins/jwt/daos.lua",
     ["kong.plugins.jwt.jwt_parser"] = "kong/plugins/jwt/jwt_parser.lua",
 
-    ["kong.plugins.hmac-auth.migrations.cassandra"] = "kong/plugins/hmac-auth/migrations/cassandra.lua",
+    ["kong.plugins.hmac-auth.schema.migrations"] = "kong/plugins/hmac-auth/schema/migrations.lua",
     ["kong.plugins.hmac-auth.handler"] = "kong/plugins/hmac-auth/handler.lua",
     ["kong.plugins.hmac-auth.access"] = "kong/plugins/hmac-auth/access.lua",
     ["kong.plugins.hmac-auth.schema"] = "kong/plugins/hmac-auth/schema.lua",

kong/cli/migrations.lua

@@ -31,7 +31,7 @@ end
 local config_path = cutils.get_kong_config_path(args.config)
 local configuration = config.load(config_path)
 local dao_factory = dao.load(configuration)
-local migrations = Migrations(dao_factory)
+local migrations = Migrations(dao_factory, configuration)
 
 local kind = args.type
 if kind ~= "all" and kind ~= "core" then
@@ -69,38 +69,33 @@ if args.command == "list" then
 
 elseif args.command == "up" then
 
-  local function migrate(identifier)
+  local function before(identifier)
     cutils.logger:info(string.format(
       "Migrating %s on keyspace \"%s\" (%s)",
       cutils.colors.yellow(identifier),
       cutils.colors.yellow(dao_factory._properties.keyspace),
       dao_factory.type
     ))
+  end
 
-    local err = migrations:migrate(identifier, function(identifier, migration)
-      if migration then
-        cutils.logger:info(string.format(
-          "%s migrated up to: %s",
-          identifier,
-          cutils.colors.yellow(migration.name)
-        ))
-      end
-    end)
-    if err then
-      cutils.logger:error_exit(err)
-    end
+  local function on_each_success(identifier, migration)
+    cutils.logger:info(string.format(
+      "%s migrated up to: %s",
+      identifier,
+      cutils.colors.yellow(migration.name)
+    ))
   end
 
   if kind == "all" then
-    migrate("core")
-    for _, plugin_name in ipairs(configuration.plugins_available) do
-      local has_migrations = utils.load_module_if_exists("kong.plugins."..plugin_name..".migrations."..dao_factory.type)
-      if has_migrations then
-        migrate(plugin_name)
-      end
+    local err = migrations:run_all_migrations(before, on_each_success)
+    if err then
+      cutils.logger:error_exit(err)
     end
   else
-    migrate(kind)
+    local err = migrations:run_migrations(kind, before, on_each_success)
+    if err then
+      cutils.logger:error_exit(err)
+    end
   end
 
   cutils.logger:success("Schema up to date")
@@ -111,20 +106,26 @@ elseif args.command == "down" then
     cutils.logger:error_exit("You must specify 'core' or a plugin name for this command.")
   end
 
-  cutils.logger:info(string.format(
-    "Rollbacking %s in keyspace \"%s\" (%s)",
-    cutils.colors.yellow(kind),
-    cutils.colors.yellow(dao_factory._properties.keyspace),
-    dao_factory.type
-  ))
+  local function before(identifier)
+    cutils.logger:info(string.format(
+      "Rollbacking %s in keyspace \"%s\" (%s)",
+      cutils.colors.yellow(identifier),
+      cutils.colors.yellow(dao_factory._properties.keyspace),
+      dao_factory.type
+    ))
+  end
+
+  local function on_success(identifier, migration)
+    if migration then
+      cutils.logger:success("\""..identifier.."\" rollbacked: "..cutils.colors.yellow(migration.name))
+    else
+      cutils.logger:success("No migration to rollback")
+    end
+  end
 
-  local rollbacked, err = migrations:rollback(kind)
+  local err = migrations:run_rollback(kind, before, on_success)
   if err then
     cutils.logger:error_exit(err)
-  elseif rollbacked then
-    cutils.logger:success("\""..kind.."\" rollbacked: "..cutils.colors.yellow(rollbacked.name))
-  else
-    cutils.logger:success("No migration to rollback")
   end
 
 elseif args.command == "reset" then

kong/cli/utils/signal.lua

@@ -150,7 +150,7 @@ end
 local function prepare_database(args_config)
   local kong_config = get_kong_config(args_config)
   local dao_factory = dao.load(kong_config)
-  local migrations = require("kong.tools.migrations")(dao_factory)
+  local migrations = require("kong.tools.migrations")(dao_factory, kong_config)
 
   local keyspace_exists, err = dao_factory.migrations:keyspace_exists()
   if err then
@@ -159,11 +159,24 @@ local function prepare_database(args_config)
     cutils.logger:info("Database not initialized. Running migrations...")
   end
 
-  local err = migrations:migrate_all(kong_config, function(identifier, migration)
-    if migration then
-      cutils.logger:success(string.format("%s migrated up to: %s", identifier, cutils.colors.yellow(migration.name)))
-    end
-  end)
+  local function before(identifier)
+    cutils.logger:info(string.format(
+      "Migrating %s on keyspace \"%s\" (%s)",
+      cutils.colors.yellow(identifier),
+      cutils.colors.yellow(dao_factory._properties.keyspace),
+      dao_factory.type
+    ))
+  end
+
+  local function on_each_success(identifier, migration)
+    cutils.logger:info(string.format(
+      "%s migrated up to: %s",
+      identifier,
+      cutils.colors.yellow(migration.name)
+    ))
+  end
+
+  local err = migrations:run_all_migrations(before, on_each_success)
   if err then
     cutils.logger:error_exit(err)
   end

kong/constants.lua

@@ -1,4 +1,4 @@
-local VERSION = "0.5.3"
+local VERSION = "0.5.4"
 
 return {
   NAME = "kong",

kong/dao/cassandra/factory.lua

@@ -115,7 +115,7 @@ end
 -- Useful for huge DDL operations such as migrations
 -- @param {string} queries Semicolon separated string of queries
 -- @param {boolean} no_keyspace Won't set the keyspace if true
--- @return {string} error if any
+-- @return {table} error if any
 function CassandraFactory:execute_queries(queries, no_keyspace)
   local ok, err
   local session = cassandra:new()

kong/dao/cassandra/plugins.lua

@@ -37,11 +37,11 @@ function Plugins:_unmarshall(t)
 end
 
 -- @override
-function Plugins:update(t)
+function Plugins:update(t, full)
   if not t.consumer_id then
     t.consumer_id = constants.DATABASE_NULL_ID
   end
-  return Plugins.super.update(self, t)
+  return Plugins.super.update(self, t, full)
 end
 
 function Plugins:find_distinct()