forked from easzlab/kubeasz
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
8 changed files
with
166 additions
and
93 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| ## 06-安装网络组件.md | ||
|
|
||
| 首先回顾下K8S网络设计原则，在配置集群网络插件或者实践K8S 应用/服务部署请时刻想到这些原则： | ||
|
|
||
| - 1.每个Pod都拥有一个独立IP地址，Pod内所有容器共享一个网络命名空间 | ||
| - 2.集群内所有Pod都在一个直接连通的扁平网络中，可通过IP直接访问 | ||
| - 所有容器之间无需NAT就可以直接互相访问 | ||
| - 所有Node和所有容器之间无需NAT就可以直接互相访问 | ||
| - 容器自己看到的IP跟其他容器看到的一样 | ||
| - 3.Service cluster IP尽可在集群内部访问，外部请求需要通过NodePort、LoadBalance或者Ingress来访问 | ||
|
|
||
| `Container Network Interface (CNI)`是目前CNCF主推的网络模型，它由两部分组成： | ||
|
|
||
| - CNI Plugin负责给容器配置网络，它包括两个基本的接口 | ||
| - 配置网络: AddNetwork(net *NetworkConfig, rt *RuntimeConf) (types.Result, error) | ||
| - 清理网络: DelNetwork(net *NetworkConfig, rt *RuntimeConf) error | ||
| - IPAM Plugin负责给容器分配IP地址 | ||
|
|
||
| Kubernetes Pod的网络是这样创建的： | ||
| - 0.每个Pod除了创建时指定的容器外，都有一个kubelet启动时指定的`基础容器`，比如：`mirrorgooglecontainers/pause-amd64` `registry.access.redhat.com/rhel7/pod-infrastructure` | ||
| - 1.首先 kubelet创建`基础容器`生成network namespace | ||
| - 2.然后 kubelet调用网络CNI driver，由它根据配置调用具体的CNI 插件 | ||
| - 3.然后 CNI 插件给`基础容器`配置网络 | ||
| - 4.最后 Pod 中其他的容器共享使用`基础容器`的网络 | ||
|
|
||
| 本项目基于CNI driver 调用各种网络插件来配置kubernetes的网络，常用CNI插件有 `flannel` `calico` `weave`等等，这些插件各有优势，也在互相借鉴学习优点，比如：在所有node节点都在一个二层网络时候，flannel提供hostgw实现，避免vxlan实现的udp封装开销，估计是目前最高效的；calico也针对L3 Fabric，推出了IPinIP的选项，利用了GRE隧道封装；因此这些插件都能适合很多实际应用场景。 | ||
|
|
||
| 项目当前内置支持的网络插件有：`calico` `flannel` `kube-router` | ||
|
|
||
| ### 安装讲解请 | ||
|
|
||
| - [安装calico](06.calico.md) | ||
| - [安装flannel](06.flannel.md) | ||
| - [安装kube-router](06.kube-router.md) | ||
|
|
||
| ### 参考 | ||
| - [kubernetes.io networking docs](https://kubernetes.io/docs/concepts/cluster-administration/networking/) | ||
| - [feiskyer-kubernetes指南网络章节](https://github.com/feiskyer/kubernetes-handbook/blob/master/zh/network/network.md) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,107 @@ | ||
| # kube-router 网络组件 | ||
|
|
||
| kube-router是一个简单、高效的网络插件，它提供一揽子解决方案： | ||
| - 基于GoBGP 提供Pod 网络互联（Routing） | ||
| - 使用ipsets优化的iptables 提供网络策略支持（Firewall/NetworkPolicy） | ||
| - 基于IPVS/LVS 提供高性能服务代理（Service Proxy） | ||
|
|
||
| 更多介绍请前往`https://github.com/cloudnativelabs/kube-router` | ||
|
|
||
| ## 配置 | ||
|
|
||
| 本项目提供多种网络插件可选，如果需要安装kube-router，请在/etc/ansible/hosts文件中设置变量 `CLUSTER_NETWORK="kube-router"` | ||
|
|
||
| - kube-router需要在所有master节点和node节点安装 | ||
|
|
||
| ## 安装 | ||
|
|
||
| - 单步安装已经集成：`ansible-playbook 90.setup.yml` | ||
| - 分步安装请执行：`ansible-playbook 06.network.yml` | ||
|
|
||
| ## 验证 | ||
|
|
||
| - 1.pod间网络联通性：略 | ||
|
|
||
| - 2.host路由表 | ||
|
|
||
| ``` bash | ||
| # master上路由 | ||
| root@master1:~$ ip route | ||
| ... | ||
| 172.20.1.0/24 via 192.168.1.2 dev ens3 proto 17 | ||
| 172.20.2.0/24 via 192.168.1.3 dev ens3 proto 17 | ||
| ... | ||
|
|
||
| # node3上路由 | ||
| root@node3:~$ ip route | ||
| ... | ||
| 172.20.0.0/24 via 192.168.1.1 dev ens3 proto 17 | ||
| 172.20.1.0/24 via 192.168.1.2 dev ens3 proto 17 | ||
| 172.20.2.0/24 dev kube-bridge proto kernel scope link src 172.20.2.1 | ||
| ... | ||
| ``` | ||
|
|
||
| - 3.bgp连接状态 | ||
|
|
||
| ``` bash | ||
| # master上 | ||
| root@master1:~$ netstat -antlp|grep router|grep LISH|grep 179 | ||
| tcp 0 0 192.168.1.1:179 192.168.1.3:58366 ESTABLISHED 26062/kube-router | ||
| tcp 0 0 192.168.1.1:42537 192.168.1.2:179 ESTABLISHED 26062/kube-router | ||
|
|
||
| # node3上 | ||
| root@node3:~$ netstat -antlp|grep router|grep LISH|grep 179 | ||
| tcp 0 0 192.168.1.3:58366 192.168.1.1:179 ESTABLISHED 18897/kube-router | ||
| tcp 0 0 192.168.1.3:179 192.168.1.2:43928 ESTABLISHED 18897/kube-router | ||
|
|
||
| ``` | ||
|
|
||
| - 4.NetworkPolicy有效性，验证参照[这里](guide/networkpolicy.md) | ||
|
|
||
| - 5.ipset列表查看 | ||
|
|
||
| ``` bash | ||
| ipset list | ||
| ... | ||
| Name: kube-router-pod-subnets | ||
| Type: hash:net | ||
| Revision: 6 | ||
| Header: family inet hashsize 1024 maxelem 65536 timeout 0 | ||
| Size in memory: 672 | ||
| References: 2 | ||
| Members: | ||
| 172.20.1.0/24 timeout 0 | ||
| 172.20.2.0/24 timeout 0 | ||
| 172.20.0.0/24 timeout 0 | ||
|
|
||
| Name: kube-router-node-ips | ||
| Type: hash:ip | ||
| Revision: 4 | ||
| Header: family inet hashsize 1024 maxelem 65536 timeout 0 | ||
| Size in memory: 416 | ||
| References: 1 | ||
| Members: | ||
| 192.168.1.1 timeout 0 | ||
| 192.168.1.2 timeout 0 | ||
| 192.168.1.3 timeout 0 | ||
| ... | ||
| ``` | ||
|
|
||
| - 6.ipvs虚拟服务器查看 | ||
|
|
||
| ``` bash | ||
| # 首先创建测试应用 | ||
| $ kubectl run nginx --image=nginx --replicas=3 --port=80 --expose | ||
|
|
||
| # 查看ipvsadm输出 | ||
| $ ipvsadm | ||
| IP Virtual Server version 1.2.1 (size=4096) | ||
| Prot LocalAddress:Port Scheduler Flags | ||
| -> RemoteAddress:Port Forward Weight ActiveConn InActConn | ||
| TCP 10.68.0.1:https rr persistent 10800 # 这个kubernetes虚拟服务地址 | ||
| -> 192.168.1.1:6443 Masq 1 0 0 | ||
| TCP 10.68.199.39:http rr # 这个是测试应用nginx的虚拟服务地址 | ||
| -> 172.20.1.5:http Masq 1 0 0 | ||
| -> 172.20.2.6:http Masq 1 0 0 | ||
| -> 172.20.2.8:http Masq 1 0 0 | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters