Plugin for secrets management in Helm

Developed and used on all environments in BaseCRM.

First internal version of the plugin used pure PGP and the whole secret file was encrypted as one.

A current version of the plugin using Golang sops as backend which could be integrated in future into Helm itself, but currently, it is only shell wrapper.

What kind of problems this plugin solves:

• Simple replaceable layer integrated with helm command for encrypting, decrypting, view secrets files stored in any place. Currently using SOPS as backend.
• Support for YAML/JSON structures encryption - Helm YAML secrets files
• Encryption per value where visual Diff should work even on encrypted files
• On the fly decryption for git diff
• On the fly decryption and cleanup for helm install/upgrade with this plugin helm bash command wrapper
• Multiple key management solutions like PGP and AWS KMS at same time
• Simple adding/removing keys
• With AWS KMS permissions managment for keys
• Secrets files directory tree seperation with recursive .sops.yaml files search
• Extracting sub elements from encrypted file structure
• Encrypt only part of a file if needed. Example encrypted file

Install

SOPS install

Just install plugin using helm plugin install https://github.com/futuresimple/helm-secrets and sops will be installed using hook when helm > 2.3.x

You can always install manually for MacOS:

brew install sops

For Linux RPM or DEB, sops is available here: Dist Packages

SOPS git diff

Git config part is installed with a plugin but to be fully functional need .gitattributes file inside the root directory of charts repo with content

*.yaml diff=sopsdiffer

More info on sops page

Using Helm plugin manager (> 2.3.x)

helm plugin install https://github.com/futuresimple/helm-secrets

Pre Helm 2.3.0 Installation

Get a release tarball from the releases page.

Unpack the tarball in your helm plugins directory ($(helm home)/plugins).

For example:

curl -L $TARBALL_URL | tar -C $(helm home)/plugins -xzv

Helm-wrapper configuration

By default helm-wrapper is configured to use KMS profiles and do not encrypt/decrypt secrets.yaml in charts templates. Set you own options as ENV variables if you like overwrite default kms enabled and decrypt charts disabled.

DECRYPT_CHARTS=false helm-wrapper ....

or/and

KMS_USE=true helm-wrapper ....

If you like to use it in different way just change this lines.

Usage and examples

$ helm secrets help

Available commands:

  enc           Encrypt chart secrets file
  dec           Decrypt chart secrets file
  dec-deps      Decrypt chart's dependencies' secrets files
  view          Print chart secrets decrypted
  edit          Edit chart secrets and encrypt at the end

Any of this command have its own help

SOPS as alternative usage in shell

As alternative, you can use sops for example for edit just type

sops <SECRET_FILE_PATH>

Mozilla sops official usage page

Use case

We use vars for Helm Charts from separate directory tree with structure like this:

helm_vars/
├── .sops.yaml
├── projectX
|   ├── .sops.yaml
│   ├── production
│   │   └── us-east-1
│   │       └── java-app
│   │           └── hello-world
│   │               ├── secrets.yaml
│   │               └── values.yaml
│   ├── sandbox
│   │   └── us-east-1
│   │       └── java-app
│   │           └── hello-world
│   │               ├── secrets.yaml
│   │               └── values.yaml
|   ├── secrets.yaml
│   └── values.yaml
├── projectY
|   ├── .sops.yaml
│   ├── production
│   │   └── us-east-1
│   │       └── java-app
│   │           └── hello-world
│   │               ├── secrets.yaml
│   │               └── values.yaml
│   ├── sandbox
│   │   └── us-east-1
│   │       └── java-app
│   │           └── hello-world
│   │               ├── secrets.yaml
│   │               └── values.yaml
|   ├── secrets.yaml
│   └── values.yaml
├── secrets.yaml
└── values.yaml

As you can see we can run different PGP or KMS keys per project, globally or per any tree level. Thanks to this we can isolate tree on different CI/CD instances using same GIT repository. As we use simple -f option when running helm-wrapper we can just use encrypted secrets.yaml and all this secrets will be decrypted and cleaned on the fly before and after helm run.

.sops.yaml file example

---
creation_rules:
        # Encrypt with AWS KMS
        - kms: 'arn:aws:kms:us-east-1:222222222222:key/111b1c11-1c11-1fd1-aa11-a1c1a1sa1dsl1+arn:aws:iam::222222222222:role/helm_secrets'

        # As failover encrypt with PGP
          pgp: '000111122223333444AAAADDDDFFFFGGGG000999'

        # For more help look at https://github.com/mozilla/sops

Multiple KMS and PGP are allowed.

Everything is described in SOPS docs - links in this project description.

Tips

Prevent committing decrypted files to git

If you like to secure situation when decrypted file is committed by mistake to git you can add your secrets.yaml.dec files to you charts project .gitignore

As the second level of securing this situation is to add for example .sopscommithook file inside your charts repository local commit hook. This will prevent committing decrypted files without sops metadata.

.sopscommithook content example:

#!/bin/sh

for FILE in $(git diff-index HEAD | grep <your vars dir> | grep "secrets.y" | cut -f2 -d$'\t'); do
    if file "$FILE" | grep -q -C10000 "sops:" | grep -q "version:"
    then
        echo "!!!!! $FILE" 'File is not encrypted !!!!!'
        echo "Run: helm secrets enc <file path>"
        exit 1
    fi
done
exit