Kapitan is a tool to manage complex deployments using jsonnet and jinja2.
Use Kapitan to manage your Kubernetes manifests, your documentation, your Terraform configuration or even simplify your scripts.
Join our community on #kapitan or visit https://kapitan.dev
How is it different from Helm? Please look at our FAQ!
- Main Features
- Quickstart
- Example
- Main concepts
- Typical folder structure
- Usage
- Modes of operation
- Credits
- FAQ
- Related projects
- Use the Inventory as the single source of truth to tie together deployments, resources and documentation. based on reclass
- Use Jsonnet to create json/yaml based configurations (e.g. Kubernetes, Terraform);
- Use Jinja2 to create text based templates for scripts and documentation;
- Manage secrets with GPG or gCloud KMS and define who can access them, without compromising collaboration with other users.
- Create dynamically generated documentation about a single deployment (i.e. ad-hoc instructions) or all deployments at once (i.e. global state of deployments)
docker run -t --rm -v $(pwd):/src:delegated deepmind/kapitan -h
On Linux you can add -u $(id -u) on docker run in order for kapitan to not change file permissions.
For CI/CD usage, check out ci/
Kapitan needs Python 3.6.
Install Python 3.6:
Linux: sudo apt-get update && sudo apt-get install -y python3.6-dev python3-pip python3-yaml
Mac: brew install python3 libyaml
Install Kapitan:
User ($HOME/.local/lib/python3.6/bin on Linux or $HOME/Library/Python/3.6/bin on macOS):
pip3 install --user --upgrade kapitan
System-wide (not recommended):
sudo pip3 install --upgrade kapitan
The example below compiles 3 targets inside the examples/kubernetes folder.
Each target represents a different namespace in a minikube cluster
These targets generate the following resources:
- Kubernetes
Namespacefor the targets - Kubernetes
StatefulSetfor ElasticSearch Master node - Kubernetes
StatefulSetfor ElasticSearch Client node - Kubernetes
StatefulSetfor ElasticSearch Data node - Kubernetes
Serviceto expose ElasticSearch discovery port - Kubernetes
Serviceto expose ElasticSearch service port - Kubernetes
StatefulSetfor MySQL - Kubernetes
Serviceto expose MySQL service port - Kubernetes
Secretfor MySQL - Scripts to configure kubectl context to control the targets and helpers to apply/delete objects.
- Documentation
$ cd examples/kubernetes
$ kapitan compile
Compiled minikube-mysql
Compiled minikube-es
A component is an application that will be deployed to a kubernetes cluster. This includes all necessary kubernetes objects (StatefulSet, Services, ConfigMaps) defined in jsonnet. It may also include scripts, config files and dynamically generated documentation defined using Jinja templates.
This is a hierarchical database of variables that are passed to the targets during compilation.
By default, Kapitan will look for an inventory/ directory to render the inventory from.
There are 2 types of objects inside the inventory:
Classes define variables that are shared across many targets. You can have for example a component.elasticsearch class with all the default values for targets using elasticsearch. Or a production or dev class to enable / disable certain features based on the type of target.
You can always override values further up the tree (i.e. in the inventory target file or in a class that inherits another class)
Classifying almost anything will help you avoid repetition (DRY) and will force you to organise parameters hierarchically.
For example, the snippet below, taken from the example elasticsearch class, declares
what parameters are needed for the elasticsearch component:
$ cat inventory/classes/component/elasticsearch.yml
parameters:
elasticsearch:
image: "quay.io/pires/docker-elasticsearch-kubernetes:5.5.0"
java_opts: "-Xms512m -Xmx512m"
replicas: 1
masters: 1
roles:
master:
image: ${elasticsearch:image}
java_opts: ${elasticsearch:java_opts}
replicas: ${elasticsearch:replicas}
masters: ${elasticsearch:masters}
data:
image: ${elasticsearch:image}
java_opts: ${elasticsearch:java_opts}
replicas: ${elasticsearch:replicas}
masters: ${elasticsearch:masters}
client:
image: ${elasticsearch:image}
java_opts: ${elasticsearch:java_opts}
replicas: ${elasticsearch:replicas}
masters: ${elasticsearch:masters}
ingest:
image: ${elasticsearch:image}
java_opts: ${elasticsearch:java_opts}
replicas: ${elasticsearch:replicas}
masters: ${elasticsearch:masters}
...
Or in the mysql class example, we declare the generic variables that will be shared by all targets that import the component and what to compile.
We include a secret that is referencing a GPG encrypted value in secrets/mysql/root/password, or if the file doesn't exist, it will dynamically generate a random b64-encoded password, encrypt it and save it into the file.
$ cat inventory/classes/component/mysql.yml
parameters:
mysql:
storage: 10G
storage_class: standard
image: mysql:latest
users:
root:
# If 'secrets/mysql/root/password' doesn't exist, it will gen a random b64-encoded password
password: ?{gpg:mysql/root/password|randomstr|base64}
# password: ?{gkms:mysql/root/password|randomstr|base64}
kapitan:
compile:
- output_path: manifests
input_type: jsonnet
input_paths:
- components/mysql/main.jsonnet
output_type: yaml
- output_path: scripts
input_type: jinja2
input_paths:
- scripts
- output_path: .
output_type: yaml
input_type: jinja2
input_paths:
- docs/mysql/README.md
A target usually represents a single namespace in a kubernetes cluster and defines all components, scripts and documentation that will be generated for that target.
Inside the inventory target files you can include classes and define new values or override any values inherited from the included classes. For example:
$ cat inventory/targets/minikube-es.yml
classes:
- common
- cluster.minikube
- component.elasticsearch
parameters:
target_name: minikube-es
elasticsearch:
replicas: 2
Targets can also be defined inside the inventory.
.
├── components
│ ├── elasticsearch
│ │ ├── configmap.jsonnet
│ │ ├── deployment.jsonnet
│ │ ├── main.jsonnet
│ │ └── service.jsonnet
│ └── nginx
│ ├── configmap.jsonnet
│ ├── deployment.jsonnet
│ ├── main.jsonnet
│ ├── nginx.conf.j2
│ └── service.jsonnet
├── inventory
│ ├── classes
│ │ ├── cluster
│ │ │ ├── cluster1.yml
│ │ │ └── cluster2.yml
│ │ ├── component
│ │ │ ├── elasticsearch.yml
│ │ │ ├── nginx.yml
│ │ │ └── zookeeper.yml
│ │ └── environment
│ │ ├── dev.yml
│ │ └── prod.yml
│ └── targets
│ ├── dev-cluster1-elasticsearch.yml
│ ├── prod-cluster1-elasticsearch.yml
│ └── prod-cluster2-frontend.yml
├── secrets
│ ├── targets
│ │ ├── prod-cluster1-elasticsearch
│ │ │ └── password
│ ├── common
│ │ └── example-com-tls.key
├── lib
│ ├── kapitan.libjsonnet
│ └── kube.libjsonnet
└── .kapitan
$ kapitan -h
usage: kapitan [-h] [--version] {eval,compile,inventory,searchvar,secrets} ...
Kapitan is a tool to manage kubernetes configuration using jsonnet templates
positional arguments:
{eval,compile,inventory,searchvar,secrets}
commands
eval evaluate jsonnet file
compile compile targets
inventory show inventory
searchvar show all inventory files where var is declared
secrets manage secrets
optional arguments:
-h, --help show this help message and exit
--version show program's version number and exit
Additional parameters are available for each positional argument. For example:
$ kapitan compile -h
usage: kapitan compile [-h] [--search-paths JPATH [JPATH ...]] [--verbose]
[--prune] [--quiet] [--output-path PATH]
[--targets TARGET [TARGET ...]] [--parallelism INT]
[--indent INT] [--secrets-path SECRETS_PATH] [--reveal]
[--inventory-path INVENTORY_PATH] [--cache]
[--cache-paths PATH [PATH ...]]
[--ignore-version-check]
optional arguments:
-h, --help show this help message and exit
--search-paths JPATH [JPATH ...], -J JPATH [JPATH ...]
set search paths, default is ["."]
--verbose, -v set verbose mode
--prune prune jsonnet output
--quiet set quiet mode, only critical output
--output-path PATH set output path, default is "."
--targets TARGET [TARGET ...], -t TARGET [TARGET ...]
targets to compile, default is all
--parallelism INT, -p INT
Number of concurrent compile processes, default is 4
--indent INT, -i INT Indentation spaces for YAML/JSON, default is 2
--secrets-path SECRETS_PATH
set secrets path, default is "./secrets"
--reveal reveal secrets (warning: this will write sensitive
data)
--inventory-path INVENTORY_PATH
set inventory path, default is "./inventory"
--cache, -c enable compilation caching to .kapitan_cache, default
is False
--cache-paths PATH [PATH ...]
cache additional paths to .kapitan_cache, default is
[]
--ignore-version-check
ignore the version from .kapitan
These parameters can also be defined in a local .kapitan file, for example:
$ cat .kapitan
compile:
indent: 4
parallelism: 8
This is equivalent to running:
kapitan compile --indent 4 --parallelism 8
To enforce the kapitan version used for compilation (for consistency and safety), you can add version to .kapitan:
$ cat .kapitan
...
version: 0.19.0
This will compile all targets to compiled folder.
Accessing the inventory from jsonnet compile types requires you to import jsonnet/kapitan.libjsonnet, which includes the native_callback functions glueing reclass to jsonnet (amongst others).
The jsonnet snippet below imports the inventory for the target you're compiling and returns the java_opts for the elasticsearch data role:
local kap = import "lib/kapitan.libjsonnet";
local inventory = kap.inventory();
{
"data_java_opts": inventory.parameters.elasticsearch.roles.data.java_opts,
}
Jinja2 types will pass the "inventory" and whatever target vars as context keys in your template.
This snippet renders the same java_opts for the elasticsearch data role:
java_opts for elasticsearch data role are: {{ inventory.parameters.elasticsearch.roles.data.java_opts }}
Such as reading the inventory within jsonnet, Kapitan also provides a function to render a Jinja2 template file. Again, importing "kapitan.jsonnet" is needed.
The jsonnet snippet renders the jinja2 template in templates/got.j2:
local kap = import "lib/kapitan.libjsonnet";
{
"jon_snow": kap.jinja2_template("templates/got.j2", { is_dead: false }),
}
It's up to you to decide what the output is.
We support the following filters for use in Jinja2 templates:
sha256 - SHA256 hashing of text e.g. {{ text | sha256 }}
yaml - Dump text as YAML e.g. {{ text | yaml }}
b64encode - base64 encode text e.g. {{ text | b64encode }}
b64decode - base64 decode text e.g. {{ text | b64decode }}
Manages your secrets with GPG or Google Cloud KMS (beta), with plans to also support AWS KMS and Vault.
The usual flow of creating and using an encrypted secret with kapitan is:
-
Define your GPG recipients or KMS key, see common.yml class,
parameters.kapitan.secrets. You can also define these per target. -
Create your secret:
-
Manually:
GPG: kapitan secrets --write gpg:mysql/root/password -t minikube-mysql -f <password file> gKMS: kapitan secrets --write gkms:mysql/root/password -t minikube-mysql -f <password file> OR use stdin: echo -n '<password>' | kapitan secrets --write [gpg/gkms]:mysql/root/password -t minikube-mysql -f -This will inherit the secrets configuration from minikube-mysql target, encrypt and save your password into
secrets/mysql/root/password, seeexamples/kubernetes. -
Automatically:
See mysql.yml class. When referencing your secret, you can use the following functions to automatically generate, encrypt and save your secret:randomstr - Generates a random string. You can optionally pass the length you want i.e. randomstr:32 rsa - Generates an RSA 4096 private key (PKCS#8). You can optionally pass the key size i.e. rsa:2048 rsapublic - Derives an RSA public key from a private key. Required argument is the private key file i.e. rsapublic:path/to/encrypted_private_key base64 - base64 encodes your secret; to be used as a secondary function i.e. randomstr|base64 sha256 - sha256 hashes your secret; to be used as a secondary function i.e. randomstr|sha256. You can optionally pass a salt i.e randomstr|sha256:salt -> becomes sha256("salt:<generated random string>")
-
-
Use your secret in your classes/targets, like in the mysql.yml class:
users:
root:
# If 'secrets/mysql/root/password' doesn't exist, it will gen a random b64-encoded password
password: ?{gpg:mysql/root/password|randomstr|base64}
- After
kapitan compile, this will compile to the mysql_secret.yml k8s secret. If you are part of the GPG recipients, you can see the secret by running:
kapitan secrets --reveal -f examples/kubernetes/compiled/minikube-mysql/manifests/mysql_secret.yml
To setup GPG for the kubernetes examples you can run:
gpg --import examples/kubernetes/secrets/example\@kapitan.dev.pub
gpg --import examples/kubernetes/secrets/example\@kapitan.dev.key
And to trust the GPG example key:
gpg --edit-key [email protected]
gpg> trust
Please decide how far you trust this user to correctly verify other users' keys
(by looking at passports, checking fingerprints from different sources, etc.)
1 = I don't know or won't say
2 = I do NOT trust
3 = I trust marginally
4 = I trust fully
5 = I trust ultimately
m = back to the main menu
Your decision? 5
Do you really want to set this key to ultimate trust? (y/N) y
gpg> quit
Rendering the inventory for the minikube-es target:
$ kapitan inventory -t minikube-es
...
classes:
- component.namespace
- cluster.common
- common
- cluster.minikube
- component.elasticsearch
environment: base
exports: {}
parameters:
_reclass_:
environment: base
name:
full: minikube-es
short: minikube-es
cluster:
id: minikube
name: minikube
type: minikube
user: minikube
elasticsearch:
image: quay.io/pires/docker-elasticsearch-kubernetes:5.5.0
java_opts: -Xms512m -Xmx512m
masters: 1
replicas: 2
roles:
client:
image: quay.io/pires/docker-elasticsearch-kubernetes:5.5.0
java_opts: -Xms512m -Xmx512m
masters: 1
replicas: 2
data:
image: quay.io/pires/docker-elasticsearch-kubernetes:5.5.0
java_opts: -Xms512m -Xmx512m
masters: 1
replicas: 2
ingest:
image: quay.io/pires/docker-elasticsearch-kubernetes:5.5.0
java_opts: -Xms512m -Xmx512m
masters: 1
replicas: 2
master:
image: quay.io/pires/docker-elasticsearch-kubernetes:5.5.0
java_opts: -Xms512m -Xmx512m
masters: 1
replicas: 2
kapitan:
compile:
- input_paths:
- components/namespace/main.jsonnet
input_type: jsonnet
output_path: pre-deploy
output_type: yaml
- input_paths:
- components/elasticsearch/main.jsonnet
input_type: jsonnet
output_path: manifests
output_type: yaml
- input_paths:
- scripts
input_type: jinja2
output_path: scripts
- input_paths:
- docs/elasticsearch/README.md
input_type: jinja2
output_path: .
secrets:
recipients:
- fingerprint: D9234C61F58BEB3ED8552A57E28DC07A3CBFAE7C
name: [email protected]
vars:
namespace: minikube-es
target: minikube-es
kubectl:
insecure_skip_tls_verify: false
minikube:
cpus: 4
memory: 4096
version: v0.25.0
mysql:
hostname: localhost
namespace: minikube-es
target_name: minikube-es
vault:
address: https://localhost:8200
Show all inventory files where variable is declared:
$ kapitan searchvar parameters.elasticsearch.replicas
./inventory/targets/minikube-es.yml 2
./inventory/classes/component/elasticsearch.yml 1
Before developing Kapitan, we turned to Helm in an attempt to improve from our old Jinja based templating system.
We quickly discovered that Helm did not fit well with our workflow, for the following reasons (which were true at the time of the evaluation):
Helmuses Go templates to define Kubernetes (yaml) manifests. We were already unsatisfied by using Jinja and we did not see a huge improvement from our previous system, the main reason being: YAML files are not suitable to be managed by text templating frameworks.Helmdoes not have a solution for sharing values across charts, if not through subcharts. We wanted to be able to have one single place to define all values for all our templates. Sharing data between charts felt awkward and complicated.Helmis component/chart based. We wanted to have something that would treat the whole of our deployments as a whole.- We did not fancy the dependency on the tiller.
In short, we feel Helm is trying to be apt-get for Kubernetes charts, while we are trying to take you further than that.
With Kapitan, we worked to de-compose several problems that most of the other solutions are treating as one.
-
Kubernetes manifests: We like the jsonnet approach of using json as the working language. Jsonnet allows us to use inheritance and composition, and hide complexity at higher levels.
-
Configuration files: Most solutions will assume this problem is solved somewhere else. We feel Jinja (or your template engine of choice) have the upper hand here.
-
Hierarchical inventory: This is the feature that sets us apart from other solutions. We use the inventory (based on reclass) to define variables and properties that can be reused across different projects/deployments. This allows us to limit repetition, but also to define a nicer interface with developers (or CI tools) which will only need to understand YAML to operate changes.
-
Secrets: We manage most of our secrets with kapitan using the GPG and Google Cloud KMS integrations. Keys can be setup per class, per target or shared so you can easily and flexibly manage access per environment. They can also be dynamically generated on compilation, if you don't feel like generating random passwords or RSA private keys, and they can be referenced in the inventory like any other variables. We have plans to support other providers such as AWS KMS and Vault, in addition to GPG and Google Cloud KMS.
-
Canned scripts: We treat scripts as text templates, so that we can craft pre-canned scripts for the specific target we are working on. This can be used for instance to define scripts that setup clusters, contexts or allow to run kubectl with all the correct settings. Most other solutions require you to define contexts and call kubectl with the correct settings. We take care of that for you. Less ambiguity, less mistakes.
-
Documentation: We also use templates to create documentation for the targets we deploy. Documentation lived alongside everything else and it is treated as a first class citizen. We feel most other solutions are pushing the limits of their capacity in order to provide for the above problems. Helm treats everything as a text template, while jsonnet tries to do everything as json. We believe that these approaches can be blended in a powerful new way, glued together by the inventory.
- sublime-jsonnet-syntax - Jsonnet syntax highlighting for Sublime Text
- language-jsonnet - Jsonnet syntax highlighting for Atom