refactor: change code structure; update code conventions (maddevsio#290)

bkzhn · May 19, 2022 · 3c304ed · 3c304ed
1 parent 57308b2
commit 3c304ed
Show file tree

Hide file tree

Showing 53 changed files with 365 additions and 412 deletions.
diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md
@@ -22,6 +22,7 @@ Please note we have a code of conduct, please follow it in all your interactions
       - [Resource names](#resource-names)
       - [Variable names](#variable-names)
       - [Output names](#output-names)
+      - [Resources order in .tf files](#resources-order-in-tf-files)
     - [Names of terraform files, directories, and modules](#names-of-terraform-files-directories-and-modules)
       - [General configuration files](#general-configuration-files)
       - [Specific configuration files](#specific-configuration-files)
@@ -292,13 +293,21 @@ resource "aws_route_table_association" "intra" {
 - If the return value is a list, it must have a plural name
 - Use description for outputs
 
+#### Resources order in .tf files
+
+Resources in `.tf` files should be described in the following order:
+1. locals
+2. data
+3. modules
+4. resources
+
 ### Names of terraform files, directories, and modules
 
 #### General configuration files
 
 Each terraform module and configuration contains a set of general files ending in `.tf`:
 
-- `main.tf` - contains terraform settings if it is the top layer; or the main working code if it is a module
+- `main.tf` - contains terraform settings and resources that can't be somehow grouped if it is the top layer; or the main working code if it is a module
 - `variables.tf` - module input values
 - `outputs.tf` - module output values
 
@@ -308,27 +317,27 @@ Besides these, there may be:
 - `providers.tf` - contains settings from terraform providers, e.g. `aws`, `kubernetes`, etc
 - `iam.tf` - IAM configurations of policies, roles, etc
 
-This is not a full list; each configuration, module, or layer may need additional files and manifests. The objective is to name them as succinctly and closer in meaning to the content as possible. Do not use prefixes.
+This is not a full list; each configuration, module, or layer may need additional files and manifests. The objective is to name them as succinctly and closer in meaning to the content as possible. Do not use prefixes (for files inside modules).
 
 > Terraform itself doesn't care how many files you create. It collects all layer and module manifests into one object, builds dependencies, and executes.
 
 #### Specific configuration files
 
-These configuration files and manifests include the following: `data "template_file"` or `templatefile ()` template resources, a logical resource group placed in a separate `.tf` file, one or more deployments to k8s using `resource "helm_release"`, module initialization, aws resources that do not require a separate module, etc.
+These configuration files and manifests include a logical resource group placed in a separate `.tf` file.
 
 > It should be noted that since some kind of a logical group of resources is being, why not move it all into a separate module. But it turned out that it is easier to manage helm releases, templates for them, and additional resources in separate `.tf` files at the root of a layer. And for many such configurations, when moving to modules, the amount of code can double + what we move to modules is usually what we are going to reuse.
 
-Each specific `.tf` file must begin with a prefix indicating the service or provider to which the main resource or group being created belongs, e.g. `aws`. Optionally, the type of service is indicated next, e.g. `iam`. Next comes the name of the main service or resource or resource group declared inside, and after that, an explanatory suffix can optionally be added if there are several such files. All the parts of the name are separated by hyphens`
+Each specific `.tf` file must begin with a prefix indicating the service or provider to which the main resource or group being created belongs, e.g. `aws`. Next comes the name of the main service or resource or resource group declared inside, and after that, an explanatory suffix can optionally be added if there are several such files. All the parts of the name are separated by hyphens`
 
-So the formula looks like this: `provider|servicename`-[`optional resource/service type`]-`main resourcename|group-name`-[`optional suffix`].tf
+So the formula looks like this: `provider|servicename`-`main resourcename|group-name`-[`optional suffix`].tf
 
 Examples:
 
-- `aws-vpc.tf` - terraform manifest describing the creation of a single vpc
+- `aws-vpc.tf` - terraform manifest describing the creation of a group resources for vpc (vpc + vpc endpoints)
 - `aws-vpc-stage.tf` - terraform manifest describing the creation of one of several vpc, for staging
-- `eks-namespaces.tf` - group of namespaces created in the EKS cluster
 - `eks-external-dns.tf` - contains the description of external-dns service deployment to the EKS cluster
-- `aws-ec2-pritunl.tf` - contains the initialization of the module that creates an EC2 instance in AWS with pritunl configured
+
+If a resource isn't related to any others (for example: `resource "aws_iam_account_password_policy" "default"`), it can be stored in the `main.tf` file. 
 
 #### Modules
 
@@ -338,24 +347,24 @@ Examples:
 
 - `eks-rbac-ci` - module for creating rbac for CI inside the EKS cluster
 - `aws-iam-autoscaler` - module for creating IAM policies for autoscaler
-- `aws-ec2-pritunl` - module for creating pritunl ec2 instance
+- `aws-pritunl` - module for creating pritunl ec2 instance
 
 ### Project structure
 ---
 
-| FILE / DIRECTORY| DESCRIPTION   |
-| --------------- |:-------------:|
-| docker/      | custom dockerfiles for examples |
-| examples/    | example k8s deployments |
-| helm-charts/ | directory contains custom helm charts |
-| helm-charts/certificate | helm chart which creates ssl certificate for nginx ingress |
+| FILE / DIRECTORY           |                           DESCRIPTION                           |
+| -------------------------- | :-------------------------------------------------------------: |
+| docker/                    |                 custom dockerfiles for examples                 |
+| examples/                  |                     example k8s deployments                     |
+| helm-charts/               |              directory contains custom helm charts              |
+| helm-charts/certificate    |   helm chart which creates ssl certificate for nginx ingress    |
 | helm-charts/cluster-issuer | helm chart which creates cluster-issuer using cert manager cdrs |
-| helm-charts/elk | umbrella chart to deploy elk stack |
-| helm-charts/teamcity | helm chart which deploys teamcity agent and/or server |
-|terraform/| directory contains terraform configuration files |
-|terraform/layer1-aws| directory contains aws resources |
-|terraform/layer2-k8s| directory contains resources deployed to k8s-EKS |
-|terraform/modules| directory contains terraform modules |
-|.editorconfig| |
-|.gitlab-ci.yml||
-|.pre-commit-config.yaml||
+| helm-charts/elk            |               umbrella chart to deploy elk stack                |
+| helm-charts/teamcity       |      helm chart which deploys teamcity agent and/or server      |
+| terraform/                 |        directory contains terraform configuration files         |
+| terraform/layer1-aws       |                directory contains aws resources                 |
+| terraform/layer2-k8s       |        directory contains resources deployed to k8s-EKS         |
+| terraform/modules          |              directory contains terraform modules               |
+| .editorconfig              |                                                                 |
+| .gitlab-ci.yml             |                                                                 |
+| .pre-commit-config.yaml    |                                                                 |
diff --git a/.github/workflows/terraform-ci.yml b/.github/workflows/terraform-ci.yml
@@ -1,4 +1,4 @@
-name: 'Terraform-ci'
+name: "Terraform-ci"
 
 on: [push, pull_request]
 
@@ -9,118 +9,87 @@ defaults:
 jobs:
   # Terraform validate configuration
   terraform-validate:
-    name: 'Terraform-validate'
+    name: "Terraform-validate"
     runs-on: ubuntu-latest
     container:
       image: maddevsio/terraform-utils:latest
       env:
         PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
     steps:
-    - name: Checkout
-      uses: actions/checkout@v2
-    - name: Terraform Init l1
-      working-directory: ./terraform/layer1-aws
-      run: terraform init -backend=false
-    - name: Terraform Init l2
-      working-directory: ./terraform/layer2-k8s
-      run: terraform init -backend=false
-    - name: Terraform Validate l1
-      working-directory: ./terraform/layer1-aws
-      run: terraform validate -no-color .
-    - name: Terraform Validate l2
-      working-directory: ./terraform/layer2-k8s
-      run: terraform validate -no-color .
-    - name: Upload files for l1
-      uses: actions/upload-artifact@v2
-      with:
-        name: l1
-        path: ./terraform/layer1-aws/.terraform
-        retention-days: 1
-    - name: Upload files for l2
-      uses: actions/upload-artifact@v2
-      with:
-        name: l2
-        path: ./terraform/layer2-k8s/.terraform
-        retention-days: 1
+      - name: Checkout
+        uses: actions/checkout@v2
+      - name: Terraform Init l1
+        working-directory: ./terraform/layer1-aws
+        run: terraform init -backend=false
+      - name: Terraform Init l2
+        working-directory: ./terraform/layer2-k8s
+        run: terraform init -backend=false
+      - name: Terraform Validate l1
+        working-directory: ./terraform/layer1-aws
+        run: terraform validate -no-color .
+      - name: Terraform Validate l2
+        working-directory: ./terraform/layer2-k8s
+        run: terraform validate -no-color .
+      - name: Upload files for l1
+        uses: actions/upload-artifact@v2
+        with:
+          name: l1
+          path: ./terraform/layer1-aws/.terraform
+          retention-days: 1
 
   # Checks that all Terraform configuration files format
   terraform-format:
-    name: 'Terraform-format'
+    name: "Terraform-format"
     runs-on: ubuntu-latest
     container:
       image: maddevsio/terraform-utils:latest
       env:
         PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
     steps:
-    - name: Checkout
-      uses: actions/checkout@v2
-    - name: Terraform Format
-      run: terraform fmt -recursive -write=false -check .
-      working-directory: ./terraform
+      - name: Checkout
+        uses: actions/checkout@v2
+      - name: Terraform Format
+        run: terraform fmt -recursive -write=false -check .
+        working-directory: ./terraform
 
   # Checks that all Terraform configuration files tflint
   terraform-tflint:
-    name: 'Terraform-tflint'
+    name: "Terraform-tflint"
     runs-on: ubuntu-latest
     container:
       image: maddevsio/terraform-utils:latest
       env:
         PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
     steps:
-    - name: Checkout
-      uses: actions/checkout@v2
-    - name: Terraform tflint l1
-      working-directory: ./terraform/layer1-aws
-      run: tflint --no-color
-    - name: Terraform tflint l2
-      working-directory: ./terraform/layer2-k8s
-      run: tflint --no-color
+      - name: Checkout
+        uses: actions/checkout@v2
+      - name: Terraform tflint l1
+        working-directory: ./terraform/layer1-aws
+        run: tflint --no-color
+      - name: Terraform tflint l2
+        working-directory: ./terraform/layer2-k8s
+        run: tflint --no-color
 
   terraform-tfsec-l1:
-    name: 'Terraform-tfsec-l1'
+    name: "Terraform-tfsec-l1"
     needs: terraform-validate
     runs-on: ubuntu-latest
     container:
       image: tfsec/tfsec
       options: --user root
     steps:
-    - name: Checkout
-      uses: actions/checkout@v2
-    - name: Download init for l1
-      uses: actions/download-artifact@v2
-      with:
-        name: l1
-        path: ./terraform/layer1-aws/.terraform
-    - name: tfsec l1
-      working-directory: ./terraform
-      run: tfsec layer1-aws
-    - uses: geekyeggo/delete-artifact@v1
-      with:
-        name: l1
-        failOnError: false
-      if: ${{ always() }}
-
-  terraform-tfsec-l2:
-    name: 'Terraform-tfsec-l2'
-    needs: terraform-validate
-    runs-on: ubuntu-latest
-    container:
-      image: tfsec/tfsec
-      options: --user root
-    steps:
-    - name: Checkout
-      uses: actions/checkout@v2
-    - name: Download init for l2
-      uses: actions/download-artifact@v2
-      with:
-        name: l2
-        path: ./terraform/layer2-k8s/.terraform
-    - name: Terraform tfsec l2
-      working-directory: ./terraform
-      run: tfsec layer2-k8s
-    - uses: geekyeggo/delete-artifact@v1
-      with:
-        name: l2
-        failOnError: false
-      if: ${{ always() }}
-
+      - name: Checkout
+        uses: actions/checkout@v2
+      - name: Download init for l1
+        uses: actions/download-artifact@v2
+        with:
+          name: l1
+          path: ./terraform/layer1-aws/.terraform
+      - name: tfsec l1
+        working-directory: ./terraform
+        run: tfsec layer1-aws
+      - uses: geekyeggo/delete-artifact@v1
+        with:
+          name: l1
+          failOnError: false
+        if: ${{ always() }}
diff --git a/terraform/layer1-aws/README.md b/terraform/layer1-aws/README.md
@@ -22,7 +22,7 @@
 | <a name="module_acm"></a> [acm](#module\_acm) | terraform-aws-modules/acm/aws | 3.3.0 |
 | <a name="module_eks"></a> [eks](#module\_eks) | terraform-aws-modules/eks/aws | 18.9.0 |
 | <a name="module_eventbridge"></a> [eventbridge](#module\_eventbridge) | terraform-aws-modules/eventbridge/aws | 1.14.0 |
-| <a name="module_pritunl"></a> [pritunl](#module\_pritunl) | ../modules/aws-ec2-pritunl | n/a |
+| <a name="module_pritunl"></a> [pritunl](#module\_pritunl) | ../modules/aws-pritunl | n/a |
 | <a name="module_r53_zone"></a> [r53\_zone](#module\_r53\_zone) | terraform-aws-modules/route53/aws//modules/zones | 2.5.0 |
 | <a name="module_vpc"></a> [vpc](#module\_vpc) | terraform-aws-modules/vpc/aws | 3.12.0 |
 | <a name="module_vpc_cni_irsa"></a> [vpc\_cni\_irsa](#module\_vpc\_cni\_irsa) | terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks | 4.14.0 |
@@ -33,7 +33,7 @@
 | Name | Type |
 |------|------|
 | [aws_cloudtrail.main](https://registry.terraform.io/providers/aws/4.10.0/docs/resources/cloudtrail) | resource |
-| [aws_ebs_encryption_by_default.this](https://registry.terraform.io/providers/aws/4.10.0/docs/resources/ebs_encryption_by_default) | resource |
+| [aws_ebs_encryption_by_default.default](https://registry.terraform.io/providers/aws/4.10.0/docs/resources/ebs_encryption_by_default) | resource |
 | [aws_iam_account_password_policy.default](https://registry.terraform.io/providers/aws/4.10.0/docs/resources/iam_account_password_policy) | resource |
 | [aws_kms_key.eks](https://registry.terraform.io/providers/aws/4.10.0/docs/resources/kms_key) | resource |
 | [aws_s3_bucket.cloudtrail](https://registry.terraform.io/providers/aws/4.10.0/docs/resources/s3_bucket) | resource |
@@ -60,7 +60,7 @@
 |------|-------------|------|---------|:--------:|
 | <a name="input_allowed_account_ids"></a> [allowed\_account\_ids](#input\_allowed\_account\_ids) | List of allowed AWS account IDs | `list` | `[]` | no |
 | <a name="input_allowed_ips"></a> [allowed\_ips](#input\_allowed\_ips) | IP addresses allowed to connect to private resources | `list(any)` | `[]` | no |
-| <a name="input_aws_account_password_policy"></a> [aws\_account\_password\_policy](#input\_aws\_account\_password\_policy) | n/a | `any` | <pre>{<br>  "allow_users_to_change_password": true,<br>  "create": true,<br>  "hard_expiry": true,<br>  "max_password_age": "90",<br>  "minimum_password_length": "14",<br>  "password_reuse_prevention": "10",<br>  "require_lowercase_characters": true,<br>  "require_numbers": true,<br>  "require_symbols": true,<br>  "require_uppercase_characters": true<br>}</pre> | no |
+| <a name="input_aws_account_password_policy"></a> [aws\_account\_password\_policy](#input\_aws\_account\_password\_policy) | n/a | `any` | <pre>{<br>  "allow_users_to_change_password": true,<br>  "create": true,<br>  "hard_expiry": false,<br>  "max_password_age": "90",<br>  "minimum_password_length": "14",<br>  "password_reuse_prevention": "10",<br>  "require_lowercase_characters": true,<br>  "require_numbers": true,<br>  "require_symbols": true,<br>  "require_uppercase_characters": true<br>}</pre> | no |
 | <a name="input_aws_cis_benchmark_alerts"></a> [aws\_cis\_benchmark\_alerts](#input\_aws\_cis\_benchmark\_alerts) | AWS CIS Benchmark alerts configuration | `any` | <pre>{<br>  "email": "[email protected]",<br>  "enabled": "false",<br>  "rules": {<br>    "aws_config_changes_enabled": true,<br>    "cloudtrail_configuration_changes_enabled": true,<br>    "console_login_failed_enabled": true,<br>    "consolelogin_without_mfa_enabled": true,<br>    "iam_policy_changes_enabled": true,<br>    "kms_cmk_delete_or_disable_enabled": true,<br>    "nacl_changes_enabled": true,<br>    "network_gateway_changes_enabled": true,<br>    "organization_changes_enabled": true,<br>    "parameter_store_actions_enabled": true,<br>    "route_table_changes_enabled": true,<br>    "s3_bucket_policy_changes_enabled": true,<br>    "secrets_manager_actions_enabled": true,<br>    "security_group_changes_enabled": true,<br>    "unauthorized_api_calls_enabled": true,<br>    "usage_of_root_account_enabled": true,<br>    "vpc_changes_enabled": true<br>  }<br>}</pre> | no |
 | <a name="input_az_count"></a> [az\_count](#input\_az\_count) | Count of avaiablity zones, min 2 | `number` | `3` | no |
 | <a name="input_cidr"></a> [cidr](#input\_cidr) | Default CIDR block for VPC | `string` | `"10.0.0.0/16"` | no |

diff --git a/terraform/layer1-aws/aws-acm.tf b/terraform/layer1-aws/aws-acm.tf
@@ -1,3 +1,13 @@
+data "aws_acm_certificate" "main" {
+  count = var.create_acm_certificate ? 0 : 1
+
+  domain = var.domain_name
+  statuses = [
+    "ISSUED",
+  "PENDING_VALIDATION"]
+  most_recent = true
+}
+
 module "acm" {
   source  = "terraform-aws-modules/acm/aws"
   version = "3.3.0"
@@ -11,13 +21,3 @@ module "acm" {
 
   tags = local.tags
 }
-
-data "aws_acm_certificate" "main" {
-  count = var.create_acm_certificate ? 0 : 1
-
-  domain = var.domain_name
-  statuses = [
-    "ISSUED",
-  "PENDING_VALIDATION"]
-  most_recent = true
-}