更新efk配置和文档

fjibj · Mar 13, 2018 · c25e736 · c25e736
1 parent 92afc3b
commit c25e736
Show file tree

Hide file tree

Showing 9 changed files with 732 additions and 2 deletions.
diff --git a/README.md b/README.md
@@ -52,7 +52,7 @@
 ## 参考阅读
 
 - 建议阅读 [rootsongjc-Kubernetes指南](https://github.com/rootsongjc/kubernetes-handbook) 原理和实践指南。
-- 建议阅读 [feisky-Kubernetes指南](https://github.com/feiskyer/kubernetes-handbook/blob/master/SUMMARY.md) 原理和部署章节。
+- 建议阅读 [feisky-Kubernetes指南](https://github.com/feiskyer/kubernetes-handbook/blob/master/zh/SUMMARY.md) 原理和部署章节。
 - 建议阅读 [opsnull-安装教程](https://github.com/opsnull/follow-me-install-kubernetes-cluster) 二进制手工部署。
 
 ## 版权

diff --git a/docs/guide/efk.md b/docs/guide/efk.md
@@ -1 +1,54 @@
 ## EFK
+
+`EFK` 插件是`k8s`项目的一个日志解决方案，它包括三个组件：[Elasticsearch](),[Fluentd](),[Kibana]()；Elasticsearch 是日志存储和日志搜索引擎，Fluentd 负载把`k8s`集群的日志发送给 Elasticsearch, Kibana 是可视化界面查看和检索存储在 Elasticsearch 的数据。
+
+### 部署
+
+下载官方最新[release](https://github.com/kubernetes/kubernetes/release)，进入目录: `kubernetes/cluster/addons/fluentd-elasticsearch`，参考官方配置的基础上使用本项目`manifests/efk/`部署，以下为几点主要的修改：
+
++ 官方提供的`kibana-deployment.yaml`中的参数`SERVER_BASEPATH`在k8s v1.8 版本以后部署需要按照本项目调整
++ 修改官方docker镜像，方便国内下载加速
+
+请使用`kubectl create -f /etc/ansible/manifests/efk/`进行安装
+
+注意：Fluentd 是以 DaemonSet 形式运行且只会调度到有`beta.kubernetes.io/fluentd-ds-ready=true`标签的节点，所以对需要收集日志的节点逐个打上标签：
+
+``` bash
+$ kubectl label nodes 192.168.1.2 beta.kubernetes.io/fluentd-ds-ready=true
+node "192.168.1.2" labeled
+```
+
+### 验证
+
+``` bash
+kubectl get pods -n kube-system|grep -E 'elasticsearch|fluentd|kibana'
+elasticsearch-logging-0                    1/1       Running   0          19h
+elasticsearch-logging-1                    1/1       Running   0          19h
+fluentd-es-v2.0.2-6c95c                    1/1       Running   0          17h
+fluentd-es-v2.0.2-f2xh8                    1/1       Running   0          8h
+fluentd-es-v2.0.2-pv5q5                    1/1       Running   0          8h
+kibana-logging-d5cffd7c6-9lz2p             1/1       Running   0          1m
+```
+kibana Pod 第一次启动时会用较长时间(10-20分钟)来优化和 Cache 状态页面，可以查看 Pod 的日志观察进度，等待 `Ready` 状态
+
+``` bash
+$ kubectl logs -n kube-system kibana-logging-d5cffd7c6-9lz2p -f
+...
+{"type":"log","@timestamp":"2018-03-13T07:33:00Z","tags":["listening","info"],"pid":1,"message":"Server running at http://0:5601"}
+{"type":"log","@timestamp":"2018-03-13T07:33:00Z","tags":["status","ui settings","info"],"pid":1,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
+```
+
+### 访问 Kibana
+
+这里介绍 `kube-apiserver`方式访问，获取访问 URL
+
+``` bash
+$ kubectl cluster-info | grep Kibana
+Kibana is running at https://192.168.1.10:8443/api/v1/namespaces/kube-system/services/kibana-logging/proxy
+```
+浏览器访问 URL：`https://192.168.1.10:8443/api/v1/namespaces/kube-system/services/kibana-logging/proxy`，然后使用`basic auth`或者`证书` 的方式认证后即可，关于认证可以参考[dashboard文档](dashboard.md)
+
+首次登陆需要在`Management` - `Index Patterns` 创建 `index pattern`，可以使用默认的 logstash-* pattern，点击 Create; 创建Index后，稍等几分钟就可以在 Discover 菜单看到 ElasticSearch logging 中汇聚的日志；
+
+
+[前一篇](ingress.md) -- [目录](index.md) -- [后一篇](harbor.md)
diff --git a/docs/guide/index.md b/docs/guide/index.md
@@ -6,7 +6,7 @@
 - 安装 [dashboard](dashboard.md)
 - 安装 [heapster](heapster.md)
 - 安装 [ingress](ingress.md)
-- 安装 efk
+- 安装 [efk](efk.md)
 - 安装 [harbor](harbor.md)
 
 ### 基础特性演示

diff --git a/manifests/efk/es-service.yaml b/manifests/efk/es-service.yaml
@@ -0,0 +1,17 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: elasticsearch-logging
+  namespace: kube-system
+  labels:
+    k8s-app: elasticsearch-logging
+    kubernetes.io/cluster-service: "true"
+    addonmanager.kubernetes.io/mode: Reconcile
+    kubernetes.io/name: "Elasticsearch"
+spec:
+  ports:
+  - port: 9200
+    protocol: TCP
+    targetPort: db
+  selector:
+    k8s-app: elasticsearch-logging
diff --git a/manifests/efk/es-statefulset.yaml b/manifests/efk/es-statefulset.yaml
@@ -0,0 +1,111 @@
+# RBAC authn and authz
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: elasticsearch-logging
+  namespace: kube-system
+  labels:
+    k8s-app: elasticsearch-logging
+    kubernetes.io/cluster-service: "true"
+    addonmanager.kubernetes.io/mode: Reconcile
+---
+kind: ClusterRole
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: elasticsearch-logging
+  labels:
+    k8s-app: elasticsearch-logging
+    kubernetes.io/cluster-service: "true"
+    addonmanager.kubernetes.io/mode: Reconcile
+rules:
+- apiGroups:
+  - ""
+  resources:
+  - "services"
+  - "namespaces"
+  - "endpoints"
+  verbs:
+  - "get"
+---
+kind: ClusterRoleBinding
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  namespace: kube-system
+  name: elasticsearch-logging
+  labels:
+    k8s-app: elasticsearch-logging
+    kubernetes.io/cluster-service: "true"
+    addonmanager.kubernetes.io/mode: Reconcile
+subjects:
+- kind: ServiceAccount
+  name: elasticsearch-logging
+  namespace: kube-system
+  apiGroup: ""
+roleRef:
+  kind: ClusterRole
+  name: elasticsearch-logging
+  apiGroup: ""
+---
+# Elasticsearch deployment itself
+apiVersion: apps/v1beta2
+kind: StatefulSet
+metadata:
+  name: elasticsearch-logging
+  namespace: kube-system
+  labels:
+    k8s-app: elasticsearch-logging
+    version: v5.6.4
+    kubernetes.io/cluster-service: "true"
+    addonmanager.kubernetes.io/mode: Reconcile
+spec:
+  serviceName: elasticsearch-logging
+  replicas: 2
+  selector:
+    matchLabels:
+      k8s-app: elasticsearch-logging
+      version: v5.6.4
+  template:
+    metadata:
+      labels:
+        k8s-app: elasticsearch-logging
+        version: v5.6.4
+        kubernetes.io/cluster-service: "true"
+    spec:
+      serviceAccountName: elasticsearch-logging
+      containers:
+      #- image: gcr.io/google-containers/elasticsearch:v5.6.4
+      - image: mirrorgooglecontainers/elasticsearch:v5.6.4
+        name: elasticsearch-logging
+        resources:
+          # need more cpu upon initialization, therefore burstable class
+          limits:
+            cpu: 1000m
+          requests:
+            cpu: 100m
+        ports:
+        - containerPort: 9200
+          name: db
+          protocol: TCP
+        - containerPort: 9300
+          name: transport
+          protocol: TCP
+        volumeMounts:
+        - name: elasticsearch-logging
+          mountPath: /data
+        env:
+        - name: "NAMESPACE"
+          valueFrom:
+            fieldRef:
+              fieldPath: metadata.namespace
+      volumes:
+      - name: elasticsearch-logging
+        emptyDir: {}
+      # Elasticsearch requires vm.max_map_count to be at least 262144.
+      # If your OS already sets up this number to a higher value, feel free
+      # to remove this init container.
+      initContainers:
+      - image: alpine:3.6
+        command: ["/sbin/sysctl", "-w", "vm.max_map_count=262144"]
+        name: elasticsearch-logging-init
+        securityContext:
+          privileged: true