Skip to content

Commit

Permalink
Merge pull request kubernetes-sigs#2332 from hswong3i/registry_fixup
Browse files Browse the repository at this point in the history
Registry Addon Fixup
  • Loading branch information
rsmitty authored Mar 22, 2018
2 parents 6ac7840 + bb1eb9f commit 4175431
Show file tree
Hide file tree
Showing 23 changed files with 137 additions and 696 deletions.
3 changes: 3 additions & 0 deletions inventory/sample/group_vars/k8s-cluster.yml
Original file line number Diff line number Diff line change
Expand Up @@ -171,6 +171,9 @@ istio_enabled: false

# Registry deployment
registry_enabled: false
# registry_namespace: "{{ system_namespace }}"
# registry_storage_class: ""
# registry_disk_size: "10Gi"

# Local volume provisioner deployment
local_volume_provisioner_enabled: false
Expand Down
89 changes: 48 additions & 41 deletions roles/kubernetes-apps/registry/README.md
Original file line number Diff line number Diff line change
@@ -1,36 +1,39 @@
# Private Docker Registry in Kubernetes
Private Docker Registry in Kubernetes
=====================================

Kubernetes offers an optional private Docker registry addon, which you can turn
on when you bring up a cluster or install later. This gives you a place to
on when you bring up a cluster or install later. This gives you a place to
store truly private Docker images for your cluster.

## How it works
How it works
------------

The private registry runs as a `Pod` in your cluster. It does not currently
The private registry runs as a `Pod` in your cluster. It does not currently
support SSL or authentication, which triggers Docker's "insecure registry"
logic. To work around this, we run a proxy on each node in the cluster,
logic. To work around this, we run a proxy on each node in the cluster,
exposing a port onto the node (via a hostPort), which Docker accepts as
"secure", since it is accessed by `localhost`.

## Turning it on
Turning it on
-------------

Some cluster installs (e.g. GCE) support this as a cluster-birth flag. The
Some cluster installs (e.g. GCE) support this as a cluster-birth flag. The
`ENABLE_CLUSTER_REGISTRY` variable in `cluster/gce/config-default.sh` governs
whether the registry is run or not. To set this flag, you can specify
`KUBE_ENABLE_CLUSTER_REGISTRY=true` when running `kube-up.sh`. If your cluster
does not include this flag, the following steps should work. Note that some of
whether the registry is run or not. To set this flag, you can specify
`KUBE_ENABLE_CLUSTER_REGISTRY=true` when running `kube-up.sh`. If your cluster
does not include this flag, the following steps should work. Note that some of
this is cloud-provider specific, so you may have to customize it a bit.

### Make some storage

The primary job of the registry is to store data. To do that we have to decide
where to store it. For cloud environments that have networked storage, we can
use Kubernetes's `PersistentVolume` abstraction. The following template is
The primary job of the registry is to store data. To do that we have to decide
where to store it. For cloud environments that have networked storage, we can
use Kubernetes's `PersistentVolume` abstraction. The following template is
expanded by `salt` in the GCE cluster turnup, but can easily be adapted to
other situations:

<!-- BEGIN MUNGE: EXAMPLE registry-pv.yaml.in -->
```yaml
``` yaml
kind: PersistentVolume
apiVersion: v1
metadata:
Expand Down Expand Up @@ -64,14 +67,15 @@ just want to kick the tires on this without committing to it, you can easily
adapt the `ReplicationController` specification below to use a simple
`emptyDir` volume instead of a `persistentVolumeClaim`.

## Claim the storage
Claim the storage
-----------------

Now that the Kubernetes cluster knows that some storage exists, you can put a
claim on that storage. As with the `PersistentVolume` above, you can start
claim on that storage. As with the `PersistentVolume` above, you can start
with the `salt` template:

<!-- BEGIN MUNGE: EXAMPLE registry-pvc.yaml.in -->
```yaml
``` yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
Expand All @@ -90,15 +94,16 @@ spec:
This tells Kubernetes that you want to use storage, and the `PersistentVolume`
you created before will be bound to this claim (unless you have other
`PersistentVolumes` in which case those might get bound instead). This claim
`PersistentVolumes` in which case those might get bound instead). This claim
gives you the right to use this storage until you release the claim.

## Run the registry
Run the registry
----------------

Now we can run a Docker registry:

<!-- BEGIN MUNGE: EXAMPLE registry-rc.yaml -->
```yaml
``` yaml
apiVersion: v1
kind: ReplicationController
metadata:
Expand Down Expand Up @@ -146,12 +151,13 @@ spec:
```
<!-- END MUNGE: EXAMPLE registry-rc.yaml -->

## Expose the registry in the cluster
Expose the registry in the cluster
----------------------------------

Now that we have a registry `Pod` running, we can expose it as a Service:

<!-- BEGIN MUNGE: EXAMPLE registry-svc.yaml -->
```yaml
``` yaml
apiVersion: v1
kind: Service
metadata:
Expand All @@ -171,14 +177,15 @@ spec:
```
<!-- END MUNGE: EXAMPLE registry-svc.yaml -->

## Expose the registry on each node
Expose the registry on each node
--------------------------------

Now that we have a running `Service`, we need to expose it onto each Kubernetes
`Node` so that Docker will see it as `localhost`. We can load a `Pod` on every
`Node` so that Docker will see it as `localhost`. We can load a `Pod` on every
node by creating following daemonset.

<!-- BEGIN MUNGE: EXAMPLE ../../saltbase/salt/kube-registry-proxy/kube-registry-proxy.yaml -->
```yaml
``` yaml
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
Expand Down Expand Up @@ -217,7 +224,7 @@ spec:
<!-- END MUNGE: EXAMPLE ../../saltbase/salt/kube-registry-proxy/kube-registry-proxy.yaml -->

When modifying replication-controller, service and daemon-set defintions, take
care to ensure _unique_ identifiers for the rc-svc couple and the daemon-set.
care to ensure *unique* identifiers for the rc-svc couple and the daemon-set.
Failing to do so will have register the localhost proxy daemon-sets to the
upstream service. As a result they will then try to proxy themselves, which
will, for obvious reasons, not work.
Expand All @@ -226,29 +233,30 @@ This ensures that port 5000 on each node is directed to the registry `Service`.
You should be able to verify that it is running by hitting port 5000 with a web
browser and getting a 404 error:

```console
``` console
$ curl localhost:5000
404 page not found
```

## Using the registry
Using the registry
------------------

To use an image hosted by this registry, simply say this in your `Pod`'s
`spec.containers[].image` field:

```yaml
``` yaml
image: localhost:5000/user/container
```

Before you can use the registry, you have to be able to get images into it,
though. If you are building an image on your Kubernetes `Node`, you can spell
out `localhost:5000` when you build and push. More likely, though, you are
though. If you are building an image on your Kubernetes `Node`, you can spell
out `localhost:5000` when you build and push. More likely, though, you are
building locally and want to push to your cluster.

You can use `kubectl` to set up a port-forward from your local node to a
running Pod:

```console
``` console
$ POD=$(kubectl get pods --namespace kube-system -l k8s-app=kube-registry-upstream \
-o template --template '{{range .items}}{{.metadata.name}} {{.status.phase}}{{"\n"}}{{end}}' \
| grep Running | head -1 | cut -f1 -d' ')
Expand All @@ -260,15 +268,14 @@ Now you can build and push images on your local computer as
`localhost:5000/yourname/container` and those images will be available inside
your kubernetes cluster with the same name.

# More Extensions
More Extensions
===============

- [Use GCS as storage backend](gcs/README.md)
- [Enable TLS/SSL](tls/README.md)
- [Enable Authentication](auth/README.md)
- [Use GCS as storage backend](gcs/README.md)
- [Enable TLS/SSL](tls/README.md)
- [Enable Authentication](auth/README.md)

## Future improvements
Future improvements
-------------------

* Allow port-forwarding to a Service rather than a pod (#15180)


[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/cluster/addons/registry/README.md?pixel)]()
- Allow port-forwarding to a Service rather than a pod (\#15180)
4 changes: 4 additions & 0 deletions roles/kubernetes-apps/registry/defaults/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,7 @@ registry_image_repo: registry
registry_image_tag: 2.6
registry_proxy_image_repo: gcr.io/google_containers/kube-registry-proxy
registry_proxy_image_tag: 0.4

registry_namespace: "{{ system_namespace }}"
registry_storage_class: ""
registry_disk_size: "10Gi"
26 changes: 0 additions & 26 deletions roles/kubernetes-apps/registry/files/images/Dockerfile

This file was deleted.

24 changes: 0 additions & 24 deletions roles/kubernetes-apps/registry/files/images/Makefile

This file was deleted.

23 changes: 0 additions & 23 deletions roles/kubernetes-apps/registry/files/images/rootfs/bin/boot

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

Loading

0 comments on commit 4175431

Please sign in to comment.