Skip to content

Commit

Permalink
Add documentation to install prometheus and grafana
Browse files Browse the repository at this point in the history
  • Loading branch information
aledbf committed Aug 8, 2018
1 parent c141e2b commit 322dae6
Show file tree
Hide file tree
Showing 6 changed files with 277 additions and 0 deletions.
41 changes: 41 additions & 0 deletions deploy/monitoring/configuration.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: prometheus-configuration
labels:
name: prometheus-configuration
namespace: ingress-nginx
data:
prometheus.yml: |-
global:
scrape_interval: 10s
scrape_configs:
- job_name: 'ingress-nginx-endpoints'
kubernetes_sd_configs:
- role: pod
namespaces:
names:
- ingress-nginx
relabel_configs:
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
action: keep
regex: true
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scheme]
action: replace
target_label: __scheme__
regex: (https?)
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
action: replace
target_label: __metrics_path__
regex: (.+)
- source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
action: replace
target_label: __address__
regex: ([^:]+)(?::\d+)?;(\d+)
replacement: $1:$2
- source_labels: [__meta_kubernetes_service_name]
regex: prometheus-service
action: drop
57 changes: 57 additions & 0 deletions deploy/monitoring/grafana.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
labels:
name: grafana
name: grafana
namespace: ingress-nginx
spec:
selector:
matchLabels:
app: grafana
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
labels:
app: grafana
spec:
containers:
- image: grafana/grafana
name: grafana
ports:
- containerPort: 3000
protocol: TCP
resources:
limits:
cpu: 500m
memory: 2500Mi
requests:
cpu: 100m
memory: 100Mi
volumeMounts:
- mountPath: /var/lib/grafana
name: data
restartPolicy: Always
volumes:
- emptyDir: {}
name: data

---

apiVersion: v1
kind: Service
metadata:
name: grafana
namespace: ingress-nginx
spec:
ports:
- port: 3000
protocol: TCP
targetPort: 3000
selector:
app: grafana
type: NodePort
89 changes: 89 additions & 0 deletions deploy/monitoring/prometheus.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: prometheus-server
namespace: ingress-nginx
rules:
- apiGroups: [""]
resources:
- services
- endpoints
- pods
verbs: ["get", "list", "watch"]

---

apiVersion: v1
kind: ServiceAccount
metadata:
name: prometheus-server
namespace: ingress-nginx

---

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: prometheus-server
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: prometheus-server
subjects:
- kind: ServiceAccount
name: prometheus-server
namespace: ingress-nginx

---

apiVersion: apps/v1
kind: Deployment
metadata:
name: prometheus-server
namespace: ingress-nginx
spec:
replicas: 1
selector:
matchLabels:
app: prometheus-server
template:
metadata:
labels:
app: prometheus-server
spec:
serviceAccountName: prometheus-server
containers:
- name: prometheus
image: prom/prometheus:v2.3.2
args:
- "--config.file=/etc/prometheus/prometheus.yml"
- "--storage.tsdb.path=/prometheus/"
ports:
- containerPort: 9090
volumeMounts:
- name: prometheus-config-volume
mountPath: /etc/prometheus/
- name: prometheus-storage-volume
mountPath: /prometheus/
volumes:
- name: prometheus-config-volume
configMap:
name: prometheus-configuration
- name: prometheus-storage-volume
emptyDir: {}

---

apiVersion: v1
kind: Service
metadata:
name: prometheus-service
namespace: ingress-nginx
spec:
selector:
app: prometheus-server
type: NodePort
ports:
- port: 9090
targetPort: 9090
Binary file added docs/images/grafana.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/prometheus-dashboard.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
90 changes: 90 additions & 0 deletions docs/user-guide/monitoring.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Prometheus and Grafana installation

This tutorial will show you how to install [Prometheus](https://prometheus.io/) and [Grafana](https://grafana.com/) for scraping the metrics of the NGINX Ingress controller.

!!! Important: this example uses `emptyDir` volumes for Prometheus and Grafana. This means once the pod gets terminated you will lose all the data.

## Before You Begin

The NGINX Ingress controller should already be deployed according to the deployment instructions [here](../deploy/index.md).

Note that the yaml files used in this tutorial are stored in the [deploy/monitoring](https://github.com/kubernetes/ingress-nginx/tree/master/deploy/monitoring) folder of the GitHub repository [kubernetes/ingress-nginx](https://github.com/kubernetes/ingress-nginx).

## Deploy and configure Prometheus Server

The Prometheus server must be configured so that it can discover endpoints of services. If a Prometheus server is already running in the cluster and if it is configured in a way that it can find the ingress controller pods, no extra configuration is needed.

If there is no existing Prometheus server running, the rest of this tutorial will guide you through the steps needed to deploy a properly configured Prometheus server.

Running the following command deploys the prometheus configuration in Kubernetes:

```console
kubectl create -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/monitoring/configuration.yaml
configmap "prometheus-configuration" created
```

Running the following command deploys prometheus in Kubernetes:

```console
kubectl create -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/monitoring/prometheus.yaml
clusterrole "prometheus-server" created
serviceaccount "prometheus-server" created
clusterrolebinding "prometheus-server" created
deployment "prometheus-server" created
service "prometheus-service" created
```

### Prometheus Dashboard

Open Prometheus dashboard in a web browser:

```console
kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default-http-backend ClusterIP 10.103.59.201 <none> 80/TCP 3d
ingress-nginx NodePort 10.97.44.72 <none> 80:30100/TCP,443:30154/TCP,10254:32049/TCP 5h
prometheus NodePort 10.98.233.86 <none> 9090:32630/TCP 1m
```

Obtain the IP address of the nodes in the running cluster:

```console
kubectl get nodes -o wide
```

In some cases where the node only have internal IP adresses we need to execute:

```console
kubectl get nodes --selector=kubernetes.io/role!=master -o jsonpath={.items[*].status.addresses[?\(@.type==\"InternalIP\"\)].address}
10.192.0.2 10.192.0.3 10.192.0.4
```

Open your browser and visit the following URL: _http://{node IP address}:{prometheus-svc-nodeport}_ to load the Prometheus Dashboard.

According to the above example, this URL will be http://10.192.0.3:32630

![Dashboard](../images/prometheus-dashboard.png)

### Grafana

```console
kubectl create -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/monitoring/grafana.yaml
```

```console
kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default-http-backend ClusterIP 10.103.59.201 <none> 80/TCP 3d
ingress-nginx NodePort 10.97.44.72 <none> 80:30100/TCP,443:30154/TCP,10254:32049/TCP 5h
prometheus NodePort 10.98.233.86 <none> 9090:32630/TCP 10m
grafana NodePort 10.98.233.86 <none> 9090:31086/TCP 10m
```

Open your browser and visit the following URL: _http://{node IP address}:{grafana-svc-nodeport}_ to load the Grafana Dashboard.
According to the above example, this URL will be http://10.192.0.3:31086

The username and password is `admin`

After the login you can import the Grafana dashboard from _https://github.com/kubernetes/ingress-nginx/tree/master/deploy/grafana/dashboards_

![Dashboard](../images/grafana.png)

0 comments on commit 322dae6

Please sign in to comment.