Skip to content

Commit

Permalink
feat: generate libraries for CRDs (jsonnet-libs#31)
Browse files Browse the repository at this point in the history
This is heavily based on https://github.com/xvzf/crd-libs.
Big shout out to @xvzf for putting this together.
  • Loading branch information
Duologic authored Jun 3, 2021
1 parent 7249057 commit ae32e1c
Show file tree
Hide file tree
Showing 45 changed files with 784 additions and 90 deletions.
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@ spec.json
vendor
k8s
.vscode/
gen/
gen/
44 changes: 44 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
FROM bitnami/kubectl:1.21.1 as kubectl
FROM rancher/k3s:v1.21.1-k3s1 as k3s
FROM mikefarah/yq:4.9.3 as yq2

FROM golang:1.16.4 as base

ENV GO111MODULE=on
WORKDIR /app

COPY go.mod .
COPY go.sum .

RUN go mod download

COPY pkg pkg
COPY main.go main.go

FROM base AS builder
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o k8s-gen .

FROM golang:1.15.2-alpine3.12 as docsonnet

RUN apk add --no-cache git
RUN go get github.com/jsonnet-libs/docsonnet
RUN go install github.com/sh0rez/docsonnet

FROM alpine:3.12

WORKDIR /app

ENV KUBECONFIG=/app/kubeconfig/kube-config.yaml
RUN chmod a+w /app

COPY --from=kubectl /opt/bitnami/kubectl/bin/kubectl /usr/local/bin
COPY --from=k3s /bin/k3s /usr/local/bin
COPY --from=yq2 /usr/bin/yq /usr/local/bin/yq2
COPY --from=builder /app/k8s-gen /usr/local/bin
COPY --from=docsonnet /go/bin/docsonnet /usr/local/bin
#
RUN apk add --no-cache bash curl

COPY scripts .

ENTRYPOINT ["./gen.sh"]
43 changes: 43 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
.PHONY: configure build debug run test push push-image

IMAGE_NAME ?= k8s-crds
IMAGE_PREFIX ?= jsonnet-libs
IMAGE_TAG ?= 0.0.1

INPUT_DIR ?= ${PWD}/libs/k8s-alpha
OUTPUT_DIR ?= ${PWD}/gen

ABS_INPUT_DIR := $(shell realpath $(INPUT_DIR))
JSONNET_FILE := $(ABS_INPUT_DIR)/config.jsonnet
ABS_OUTPUT_DIR := $(shell realpath $(OUTPUT_DIR))

# Requires Go implementation of Jsonnet
# Implementation of `-c` argument pending: https://github.com/google/jsonnet/issues/195
configure:
jsonnet -c -m $(ABS_INPUT_DIR) -S $(JSONNET_FILE)

build:
docker build -t $(IMAGE_PREFIX)/$(IMAGE_NAME):$(IMAGE_TAG) .

debug: configure build
docker run --rm -ti \
--user $(shell id -u):$(shell id -g) \
-v $(ABS_INPUT_DIR):/config \
-v $(ABS_OUTPUT_DIR):/output \
--entrypoint /bin/bash \
$(IMAGE_PREFIX)/$(IMAGE_NAME):$(IMAGE_TAG)

run: configure build
docker run --rm -ti \
--user $(shell id -u):$(shell id -g) \
-v $(ABS_INPUT_DIR):/config \
-v $(ABS_OUTPUT_DIR):/output \
$(IMAGE_PREFIX)/$(IMAGE_NAME):$(IMAGE_TAG) /config /output

test: build

push: build test push-image

push-image:
docker push $(IMAGE_PREFIX)/$(IMAGE_NAME):$(IMAGE_TAG)
docker push $(IMAGE_PREFIX)/$(IMAGE_NAME):latest
50 changes: 31 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,23 @@ respective Kubernetes versions.

```bash
# Generate for all versions
$ k8s-gen
$ k8s-gen -c libs/k8s-alpha/config.yml
Generating ...

# Only a subset (e.g. for development)
$ k8s-gen 1.18 1.17
$ k8s-gen -c libs/k8s-alpha/config.yml 1.18 1.17
```

### Generating from CRDs

With [k3s](https://k3s.io/), it is possible to generate jsonnet libraries from
CRDs.This can be executed with a makefile target to run it in a container:

```bash
$ make run INPUT_DIR=libs/istio/
```


## Customizing

Because the generator only creates the most minimal yet functional code, more
Expand All @@ -32,22 +42,29 @@ For that, `k8s-gen` implements two methods for extending:

### `custom` patches

The [`custom/`](https://github.com/jsonnet-libs/k8s/tree/master/custom)
The [`custom/`](https://github.com/jsonnet-libs/k8s/tree/master/libs/k8s-alpha/custom)
directory contains a set of `.libsonnet` files, that are _automatically merged_
with the generated result in `main.libsonnet`, so they become part of the
exported API.

Current Patches:

```
custom/
├── core
│   ├── apps.libsonnet # Constructors for `core/v1`, ported from `ksonnet-gen` and `kausal.libsonnet`
│   ├── batch.libsonnet # Constructors for `batch/v1beta1`, `batch/v2alpha1`, ported from `kausal.libsonnet`
│   ├── core.libsonnet # Constructors for `apps/v1`, `apps/v1beta1`, ported from `ksonnet-gen` and `kausal.libsonnet`
│   ├── mapContainers.libsonnet # Adds `mapContainers` functions for fields that support them
│   └── rbac.libsonnet # Adds helper functions to rbac objects
└── istio # Placeholder for istio CRD support
libs/k8s-alpha/
├── config.jsonnet # Config to generate the k8s-alpha jsonnet libraries
├── config.yml # Generated from config.jsonnet
├── README.md.tmpl # Template for the index of the generated docs
└── custom
└── core
   ├── apps.libsonnet # Constructors for `core/v1`, ported from `ksonnet-gen` and `kausal.libsonnet`
├── autoscaling.libsonnet # Extends `autoscaling/v2beta2`
   ├── batch.libsonnet # Constructors for `batch/v1beta1`, `batch/v2alpha1`, ported from `kausal.libsonnet`
   ├── core.libsonnet # Constructors for `apps/v1`, `apps/v1beta1`, ported from `ksonnet-gen` and `kausal.libsonnet`
├── list.libsonnet # Adds `core.v1.List`
   ├── mapContainers.libsonnet # Adds `mapContainers` functions for fields that support them
   ├── rbac.libsonnet # Adds helper functions to rbac objects
└── volumeMounts.libsonnet # Adds helper functions to mount volumes
```

### Extensions
Expand All @@ -59,13 +76,8 @@ need to added by the user themselves.
Extensions can be applied as so:

```jsonnet
(import "github.com/jsonnet-libs/k8s/1.18/main.libsonnet")
+ (import "github.com/jsonnet-libs/k8s/extensions/<name>.libsonnet")
(import "github.com/jsonnet-libs/k8s-alpha/1.21/main.libsonnet")
+ (import "github.com/jsonnet-libs/k8s-alpha/extensions/<name>.libsonnet")
```

Current Patches:
```
extensions/
├── core # Placeholder for core extensions
└── istio # Placeholder for istio CRD support
```
There are currently no extension patches.
49 changes: 0 additions & 49 deletions config.yml

This file was deleted.

Empty file removed custom/istio/.gitkeep
Empty file.
Empty file removed extensions/core/.gitkeep
Empty file.
Empty file removed extensions/istio/.gitkeep
Empty file.
10 changes: 10 additions & 0 deletions gen_all.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#!/usr/bin/env bash
set -euo pipefail
set -x

rm -rf gen./
mkdir -p gen/

for dir in libs/*; do
[ -d "$dir" ] && make run INPUT_DIR="$dir" OUTPUT_DIR=gen/
done
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/jsonnet-libs/k8s

go 1.13
go 1.16

require (
github.com/fatih/camelcase v1.0.0
Expand Down
9 changes: 6 additions & 3 deletions go.sum
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/fatih/camelcase v1.0.0 h1:hxNvNX/xYBp0ovncs8WyWZrOrpBNub/JfaMvbURyft8=
github.com/fatih/camelcase v1.0.0/go.mod h1:yN2Sb0lFhZJUdVvtELVWefmrXpuZESvPmqwoZc+/fpc=
github.com/fatih/color v1.9.0 h1:8xPHl4/q1VyqGIPif1F+1V3Y3lSmrq01EabUW3CoW5s=
github.com/fatih/color v1.9.0/go.mod h1:eQcE1qtQxscV5RaZvpXrrb8Drkc3/DdQ+uUYCNjL+zU=
github.com/go-clix/cli v0.1.2-0.20200502172020-b8f4629e879a h1:nh+UOawbjKgiUAJAgi8JHctNebEu6mjwDXsv8Xdln8w=
github.com/go-clix/cli v0.1.2-0.20200502172020-b8f4629e879a/go.mod h1:dYJevXraB9mXZFhz5clyQestG0qGcmT5rRC/P9etoRQ=
Expand All @@ -15,16 +15,20 @@ github.com/hashicorp/go-multierror v1.0.0 h1:iVjPR7a6H0tWELX5NxNe7bYopibicUzc7uP
github.com/hashicorp/go-multierror v1.0.0/go.mod h1:dHtQlpGsu+cZNNAkkCN/P3hoUDHhCYQXV3UM06sGGrk=
github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pty v1.1.1 h1:VkoXIwSboBpnk99O/KFauAEILuNHv5DVFKZMBN/gUgw=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/mattn/go-colorable v0.1.4 h1:snbPLB8fVfU9iwbbo30TPtbLRzwWu6aJS6Xh4eaaviA=
github.com/mattn/go-colorable v0.1.4/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-isatty v0.0.11 h1:FxPOTFNqGkuDUGi3H/qkUbQO4ZiBa2brKq5r0l8TGeM=
github.com/mattn/go-isatty v0.0.11/go.mod h1:PhnuNfih5lzO57/f3n+odYbM4JtupLOxQOAqxQCu2WE=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/posener/complete v1.2.3 h1:NP0eAhjcjImqslEwo/1hq7gpajME0fTLTezBKDqfXqo=
github.com/posener/complete v1.2.3/go.mod h1:WZIdtGGp+qx0sLrYKtIRAruyNpv6hFCicSgv7Sy7s/s=
github.com/sergi/go-diff v1.1.0 h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
github.com/sergi/go-diff v1.1.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM=
github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
Expand All @@ -33,12 +37,11 @@ github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+
github.com/stretchr/testify v1.4.0 h1:2E4SXV/wtOkTonXsotYi4li6zVWxYlZuYNCXe9XRJyk=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037 h1:YyJpGZS1sBuBCzLAR1VEpK193GlqGZbnPFnPV/5Rsb4=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.4 h1:/eiJrUcujPVeJ3xlSWaiNi3uSVmDGBK1pDHUHAnao1I=
gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
10 changes: 10 additions & 0 deletions libs/cert-manager/README.md.tmpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
permalink: /
---

# Jsonnet library for %(name)s

The Jsonnet Kubernetes library is a generated with
[`k8s`](https://github.com/jsonnet-libs/k8s).

%(pages)s
27 changes: 27 additions & 0 deletions libs/cert-manager/config.jsonnet
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
local name = 'cert-manager';
local repository = 'github.com/jsonnet-libs/' + name + '-lib';
local specs = [
{
output: '1.3',
openapi: 'http://localhost:8001/openapi/v2',
prefix: '^io\\.cert-manager\\..*',
crd: 'https://github.com/jetstack/cert-manager/releases/download/v1.3.1/cert-manager.crds.yaml',
localName: 'cert_manager',
repository: 'github.com/jsonnet-libs/cert-manager-lib',
},
];

{
'config.yml': std.manifestYamlDoc({
repository: repository,
specs: specs,
}, true),

'docs/README.md': (importstr 'README.md.tmpl') % {
name: name,
pages: std.join('\n', [
'- [%(output)s](%(output)s/README.md)' % spec
for spec in specs
]),
},
}
8 changes: 8 additions & 0 deletions libs/cert-manager/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
"repository": "github.com/jsonnet-libs/cert-manager-lib"
"specs":
- "crd": "https://github.com/jetstack/cert-manager/releases/download/v1.3.1/cert-manager.crds.yaml"
"localName": "cert_manager"
"openapi": "http://localhost:8001/openapi/v2"
"output": "1.3"
"prefix": "^io\\.cert-manager\\..*"
"repository": "github.com/jsonnet-libs/cert-manager-lib"
10 changes: 10 additions & 0 deletions libs/cert-manager/docs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
permalink: /
---

# Jsonnet library for cert-manager

The Jsonnet Kubernetes library is a generated with
[`k8s`](https://github.com/jsonnet-libs/k8s).

- [1.3](1.3/README.md)
10 changes: 10 additions & 0 deletions libs/cnrm/README.md.tmpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
permalink: /
---

# Jsonnet library for %(name)s

The Jsonnet Kubernetes library is a generated with
[`k8s`](https://github.com/jsonnet-libs/k8s).

%(pages)s
27 changes: 27 additions & 0 deletions libs/cnrm/config.jsonnet
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
local name = 'cnrm';
local repository = 'github.com/jsonnet-libs/' + name + '-lib';
local specs = [
{
output: '1.33',
openapi: 'http://localhost:8001/openapi/v2',
prefix: '^com\\.google\\.cloud\\.cnrm\\..*',
crd: 'https://raw.githubusercontent.com/GoogleCloudPlatform/k8s-config-connector/1.33.0/install-bundles/install-bundle-workload-identity/crds.yaml',
localName: 'cnrm',
repository: 'github.com/jsonnet-libs/cnrm-lib',
},
];

{
'config.yml': std.manifestYamlDoc({
repository: repository,
specs: specs,
}, true),

'docs/README.md': (importstr 'README.md.tmpl') % {
name: name,
pages: std.join('\n', [
'- [%(output)s](%(output)s/README.md)' % spec
for spec in specs
]),
},
}
Loading

0 comments on commit ae32e1c

Please sign in to comment.