Working on 1.4 compatibility

Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    skull_island (1.2.12)
+    skull_island (1.4.0)
       deepsort (~> 0.4)
       erubi (~> 1.8)
       json (~> 2.1)

README.md

@@ -1,6 +1,6 @@
 # Skull Island
 
-A full-featured SDK for [Kong](https://konghq.com/kong/) 1.1.x/1.2.x (with support for migrating from 0.14.x). Note that this is unofficial (meaning this project is in no way officially endorsed, recommended, or related to Kong [as a company](https://konghq.com/) or an [open-source project](https://github.com/Kong/kong)). It is also in no way related to the [pet toy company](https://www.kongcompany.com/) by the same name (but hopefully that was obvious).
+A full-featured SDK for [Kong](https://konghq.com/kong/) 1.4.x (with support for migrating from 0.14.x, 1.1.x, and 1.2.x). Note that this is unofficial (meaning this project is in no way officially endorsed, recommended, or related to Kong [as a company](https://konghq.com/) or an [open-source project](https://github.com/Kong/kong)). It is also in no way related to the [pet toy company](https://www.kongcompany.com/) by the same name (but hopefully that was obvious).
 
 ![Gem](https://img.shields.io/gem/v/skull_island)
 ![Travis (.org)](https://img.shields.io/travis/jgnagy/skull_island)
@@ -30,15 +30,15 @@ gem install skull_island
 Or add this to your Gemfile:
 
 ```ruby
-gem 'skull_island',  '~> 1.2'
+gem 'skull_island',  '~> 1.4'
 ```
 
 Or add this to your .gemspec:
 
 ```ruby
 Gem::Specification.new do |spec|
  # ...
- spec.add_runtime_dependency 'skull_island', '~> 1.2'
+ spec.add_runtime_dependency 'skull_island', '~> 1.4'
  # ...
 end
 ```
@@ -141,25 +141,25 @@ Note that `--test` has a high likelihood of generating errors with a complicated
 
 #### Importing with Projects
 
-Skull Island 1.2.1 introduces the ability to use a special top-level key in the configuration called `project` that uses meta-data to track which resources belong to a project. This meta-data can safely be added at another time as this tool will "adopt" otherwise matching resources into a project.
+Skull Island 1.2.1 introduced the ability to use a special top-level key in the configuration called `project` that uses meta-data to track which resources belong to a project. This meta-data can safely be added at another time as this tool will "adopt" otherwise matching resources into a project.
 
 To use this functionality, either add the `project` key to your configuration file (usually directly below the `version` key) with some value that will be unique on your gateway, or use `--project foo` (where `foo` is the name of your project) as a CLI flag.
 
 When using the `project` feature of Skull Island, the CLI tool will automatically clean up old resources no longer found in your config file. This is, in fact, the _only_ circumstance under which this tool actually removes resources. Use this feature with care, as it can delete large swaths of your configuration if used incorrectly. It is **critical** that this value is unique since this project functionality is used to delete resources.
 
 ### Migrating
 
-With Skull Island, it is possible to migrate a configuration from a 0.14.x gateway to one compatible with a 1.2.x gateway. If you have a previous export, you can just run `skull_island migrate /path/to/export.yml` and you'll receive a 1.2 compatible config on standard out. If you'd prefer, you can have that config written to a file as well (just like the export command) like so:
+With Skull Island, it is possible to migrate a configuration from a 0.14.x, 1.1.x, or 1.2.x gateway to the most recent compatible gateway. If you have a previous export, you can just run `skull_island migrate /path/to/export.yml` and you'll receive a 1.4 compatible config on standard out. If you'd prefer, you can have that config written to a file as well (just like the export command) like so:
 
 ```
 skull_island migrate /path/to/export.yml /output/location/migrated.yml
 ```
 
-While this hasn't been heavily tested for all possible use-cases, any configuration generated or usable by the `'~> 0.14'` version of this gem should safely convert using the migration command. This tool also makes no guarantees about plugin functionality, configuration compatibility across versions, or that the same plugins are installed and available in your newer gateway. It should go without saying that you should **test and confirm** that all of your functionality was successfully migrated.
+While this hasn't been heavily tested for all possible use-cases, any configuration generated or usable by the `'~> 0.14'` or `'~> 1.2'` version of this gem should safely convert using the migration command. This tool also makes no guarantees about plugin functionality, configuration compatibility across versions, or that the same plugins are installed and available in your newer gateway. It should go without saying that you should **test and confirm** that all of your functionality was successfully migrated.
 
-If you don't have a previous export, you'll need to install an older version of this gem using `gem install --version '~> 0.14' skull_island`, then perform an `export`, then you can switch back to the latest version of the gem for migrating and importing.
+If you don't have a previous export, you'll need to install an older version of this gem using something like `gem install --version '~> 0.14' skull_island`, then perform an `export`, then you can switch back to the latest version of the gem for migrating and importing.
 
-While it would be possible to make migration _automatic_ for the `import` command, `skull_island` intentionally doesn't do this to avoid the appearance that the config is losslessly compatible across versions. In reality, the newer config version has additional features (like tagging) that will likely be used heavily. It makes sense to this author to maintain the migration component and the normal functionality as distinct features to encourage the use of the newer capabilities in 1.1+.
+While it would be possible to make migration _automatic_ for the `import` command, `skull_island` intentionally doesn't do this to avoid the appearance that the config is losslessly compatible across versions. In reality, the newer config version has additional features (like tagging) that are used heavily by skull_island. It makes sense to this author to maintain the migration component and the normal functionality as distinct features to encourage the use of the newer capabilities in 1.1 and beyond.
 
 ### Reset A Gateway
 
@@ -188,7 +188,7 @@ If you're wondering what version of `skull_island` is installed, use:
 ```
 $ skull_island version
 
-SkullIsland Version: 1.2.5
+SkullIsland Version: 1.4.1
 ```
 
 ### File Format
@@ -197,7 +197,7 @@ The import/export/migrate CLI functions produce YAML with support for embedded R
 
 ```yaml
 ---
-version: '1.2'
+version: '1.4'
 project: FooV2
 certificates: []
 consumers:
@@ -267,7 +267,7 @@ plugins:
 
 All top-level keys (other than `version` and `project`) require an Array as a parameter, either by providing a list of entries or an empty Array (`[]`). The above shows how to use the `lookup()` function to refer to another resource. This "looks up" the resource type (`service` in this case) by `name` (`search_api` in this case) and resolves its `id`. This function can also be used to lookup a `route` or `upstream` by its `name`, or a `consumer` by its `username`. Note that Kong itself doesn't _require_ `route` resources to have unique names, so you'll need to enforce that practice yourself for `lookup` to be useful for Routes.
 
-Note that while this configuration looks a lot like the [DB-less](https://docs.konghq.com/1.1.x/db-less-and-declarative-config/) configuration (and even may, at times, be interchangeable), this is merely a coincidence. **Skull Island doesn't support the DB-less mode for Kong.** This may potentially change in the future, but for now it is not a goal of this project.
+Note that while this configuration looks a lot like the [DB-less](https://docs.konghq.com/1.4.x/db-less-and-declarative-config/) configuration (and even may, at times, be interchangeable), this is merely a coincidence. **Skull Island doesn't support the DB-less mode for Kong.** This may potentially change in the future, but for now it is not a goal of this project.
 
 #### Embedded Ruby
 
@@ -277,6 +277,8 @@ While technically _any_ Ruby is valid, the following are pretty helpful for temp
 
 * `ENV.fetch('VARIABLE_NAME', 'default value')` - This allows looking up the environment variable `VARIABLE_NAME` and using its value, or, if it isn't defined, it uses `default value` as the value. With this we could change `host: api.example.com` to `host: <%= ENV.fetch('API_HOST', 'api.example.com') %>`. With this, if `API_HOST` is provided, it'll use that, otherwise it will default to `api.example.com`. This is especially helpful for sensitive information; you can version control the configuration but pass in things like credentials via environment variables at runtime.
 
+Note also that 1.4.x and beyond of Skull Island support two phases of embedded ruby: first, a simple phase that treats the **entire file** as just text, allowing you to use the full power of ruby for things like loops, conditional logic, and more; the second phase is applied for individual attributes within the rendered YAML document. This is where the `lookup()` function above is used. 
+
 ## SDK Usage
 
 The API Client requires configuration before it can be used. For now, this is a matter of calling `APIClient.configure()`, passing a Hash, with Symbols for keys:
@@ -554,6 +556,7 @@ resource = Resources::Service.new
 
 # These attributes can be set and read
 resource.protocol        = 'http'
+resource.client_certificate = { 'id' => '77e32ff2-...' }
 resource.connect_timeout = 60000
 resource.host            = 'example.com'
 resource.port            = 80
@@ -591,6 +594,7 @@ resource = Resources::Upstream.new
 
 # These attributes can be set and read
 resource.name          = 'service.v1.xyz'
+resource.algorithm     = 'round-robin'
 resource.hash_on       = 'none'
 resource.hash_fallback = 'none'
 resource.slots         = 1000

lib/skull_island.rb

@@ -39,6 +39,7 @@
 require 'skull_island/api_client'
 require 'skull_island/simple_api_client'
 require 'skull_island/resource_collection'
+require 'skull_island/helpers/cli_erb'
 require 'skull_island/helpers/meta'
 require 'skull_island/helpers/resource'
 require 'skull_island/helpers/resource_class'

lib/skull_island/cli.rb

@@ -11,6 +11,7 @@ module SkullIsland
   # Base CLI for SkullIsland
   class CLI < Thor
     include Helpers::Migration
+    include Helpers::CliErb
 
     class_option :verbose, type: :boolean
 
@@ -29,7 +30,7 @@ def export(output_file = '-')
 
       validate_server_version
 
-      output = { 'version' => '1.2' }
+      output = { 'version' => '1.4' }
       output['project'] = options['project'] if options['project']
 
       [
@@ -54,7 +55,7 @@ def import(input_file = '-')
       raw ||= acquire_input(input_file, options['verbose'])
 
       # rubocop:disable Security/YAMLLoad
-      input = YAML.load(raw)
+      input = YAML.load(erb_preprocess(raw))
       # rubocop:enable Security/YAMLLoad
 
       validate_config_version input['version']
@@ -184,9 +185,9 @@ def acquire_input(input_file, verbose = false)
     end
 
     def validate_config_version(version)
-      if version && ['1.1', '1.2'].include?(version)
+      if version && ['1.4'].include?(version)
         validate_server_version
-      elsif version && ['0.14', '1.0'].include?(version)
+      elsif version && ['0.14', '1.0', '1.1', '1.2'].include?(version)
         warn '[CRITICAL] Config version is too old. Try `migrate` instead of `import`.'
         exit 2
       else
@@ -206,7 +207,7 @@ def validate_migrate_version(version)
 
     def validate_server_version
       server_version = SkullIsland::APIClient.about_service['version']
-      if server_version.match?(/^1.[12]/)
+      if server_version.match?(/^1.[4]/)
         true
       else
         warn '[CRITICAL] Server version mismatch!'

lib/skull_island/helpers/cli_erb.rb

@@ -0,0 +1,20 @@
+# frozen_string_literal: true
+
+module SkullIsland
+  module Helpers
+    # Performs a simple, first pass ERb preprocess on the entire input file for the CLI
+    module CliErb
+      def erb_preprocess(input)
+        warn '[INFO] Preprocessing template' if options['verbose']
+        # rubocop:disable Security/Eval
+        eval(Erubi::Engine.new(input).src)
+        # rubocop:enable Security/Eval
+      end
+
+      # At this phase, we want to leave this alone...
+      def lookup(type, value)
+        "<%= lookup :#{type}, '#{value}' %>"
+      end
+    end
+  end
+end

lib/skull_island/helpers/migration.rb

@@ -6,7 +6,9 @@ module Helpers
     module Migration
       def migrate_config(config)
         if config['version'] == '0.14'
-          migrate_0_14_to_1_1(config)
+          migrate_config migrate_0_14_to_1_1(config)
+        elsif ['1.1', '1.2', '1.3'].include?(config['version'])
+          migrate_1_1_to_1_4(config)
         else
           false # Just return false if it can't be migrated
         end
@@ -29,6 +31,12 @@ def migrate_0_14_to_1_1(config)
         new_config['version'] = '1.1'
         new_config
       end
+
+      def migrate_1_1_to_1_4(config)
+        new_config = config.dup
+        new_config['version'] = '1.4'
+        new_config
+      end
     end
   end
 end

lib/skull_island/helpers/resource.rb

@@ -16,7 +16,7 @@ def datetime_from_params(params, actual_key)
 
       # rubocop:disable Style/GuardClause
       # rubocop:disable Security/Eval
-      def delayed_set(property, data, key)
+      def delayed_set(property, data, key = property.to_s)
         if data[key]
           value = recursive_erubi(data[key])
           send(

lib/skull_island/resources/access_control_list.rb

@@ -21,8 +21,8 @@ def self.batch_import(data, verbose: false, test: false)
 
         data.each_with_index do |resource_data, index|
           resource = new
-          resource.delayed_set(:group, resource_data, 'group')
-          resource.delayed_set(:consumer, resource_data, 'consumer')
+          resource.delayed_set(:group, resource_data)
+          resource.delayed_set(:consumer, resource_data)
           resource.import_update_or_skip(index: index, verbose: verbose, test: test)
           known_ids << resource.id
         end

lib/skull_island/resources/basicauth_credential.rb

@@ -24,9 +24,9 @@ def self.batch_import(data, verbose: false, test: false)
 
         data.each_with_index do |resource_data, index|
           resource = new
-          resource.delayed_set(:username, resource_data, 'username')
-          resource.delayed_set(:password, resource_data, 'password') if resource_data['password']
-          resource.delayed_set(:consumer, resource_data, 'consumer')
+          resource.delayed_set(:username, resource_data)
+          resource.delayed_set(:password, resource_data) if resource_data['password']
+          resource.delayed_set(:consumer, resource_data)
           resource.import_update_or_skip(index: index, verbose: verbose, test: test)
           known_ids << resource.id
         end

lib/skull_island/resources/certificate.rb

@@ -24,8 +24,8 @@ def self.batch_import(data, verbose: false, test: false, project: nil, time: nil
 
         data.each_with_index do |resource_data, index|
           resource = new
-          resource.delayed_set(:cert, resource_data, 'cert')
-          resource.delayed_set(:key, resource_data, 'key')
+          resource.delayed_set(:cert, resource_data)
+          resource.delayed_set(:key, resource_data)
           resource.snis = resource_data['snis'] if resource_data['snis']
           resource.tags = resource_data['tags'] if resource_data['tags']
           resource.project = project if project

lib/skull_island/resources/jwt_credential.rb

@@ -25,11 +25,9 @@ def self.batch_import(data, verbose: false, test: false)
         data.each_with_index do |resource_data, index|
           resource = new
           resource.algorithm = resource_data['algorithm']
-          resource.delayed_set(:key, resource_data, 'key') if resource_data['key']
-          resource.delayed_set(:secret, resource_data, 'secret') if resource_data['secret']
-          if resource_data['rsa_public_key']
-            resource.delayed_set(:rsa_public_key, resource_data, 'rsa_public_key')
-          end
+          resource.delayed_set(:key, resource_data) if resource_data['key']
+          resource.delayed_set(:secret, resource_data) if resource_data['secret']
+          resource.delayed_set(:rsa_public_key, resource_data) if resource_data['rsa_public_key']
           resource.delayed_set(:consumer, resource_data, 'consumer')
           resource.import_update_or_skip(index: index, verbose: verbose, test: test)
           known_ids << resource.id

lib/skull_island/resources/keyauth_credential.rb

@@ -21,8 +21,8 @@ def self.batch_import(data, verbose: false, test: false)
 
         data.each_with_index do |resource_data, index|
           resource = new
-          resource.delayed_set(:key, resource_data, 'key')
-          resource.delayed_set(:consumer, resource_data, 'consumer')
+          resource.delayed_set(:key, resource_data)
+          resource.delayed_set(:consumer, resource_data)
           resource.import_update_or_skip(index: index, verbose: verbose, test: test)
           known_ids << resource.id
         end

lib/skull_island/resources/plugin.rb

@@ -31,13 +31,13 @@ def self.batch_import(data, verbose: false, test: false, project: nil, time: nil
           resource.name = resource_data['name']
           resource.enabled = resource_data['enabled']
           resource.run_on = resource_data['run_on'] if resource_data['run_on']
-          resource.delayed_set(:config, resource_data, 'config') if resource_data['config']
+          resource.delayed_set(:config, resource_data) if resource_data['config']
           resource.tags = resource_data['tags'] if resource_data['tags']
           resource.project = project if project
           resource.import_time = (time || Time.now.utc.to_i) if project
-          resource.delayed_set(:consumer, resource_data, 'consumer')
-          resource.delayed_set(:route, resource_data, 'route')
-          resource.delayed_set(:service, resource_data, 'service')
+          resource.delayed_set(:consumer, resource_data)
+          resource.delayed_set(:route, resource_data)
+          resource.delayed_set(:service, resource_data)
           resource.import_update_or_skip(index: index, verbose: verbose, test: test)
           known_ids << resource.id
         end