前面一直用k8s的flannel来建立主机间docker容器的互联，但是当仅有两台机器用来做测试的时刻，安装一个flannel也是挺纠结的：麻烦、还有未知的问题，起一个服务在那里总会有那么些担忧。
其实可以直接通过建立路由来实现两台机器间容器的互联互通：Docker多台宿主机间的容器互联-centos7（直接路由）
两台主机（centos7/docker-1.12.6）：
- 192.168.191.140 kube-master
- 192.168.191.141 kube-worker1
安装/配置docker
这里不多讲了，参考 Kubeadm部署kubernetes 进行docker的安装。
建立新网卡，修改docker配置使用新网卡
1
| yum install net-tools bridge-utils -y
|
1
2
3
4
5
6
7
| setenforce 0
vi /etc/selinux/config
SELINUX=disabled
systemctl stop firewalld
systemctl disable firewalld
|
1
2
3
4
| echo 1 > /proc/sys/net/ipv4/ip_forward
vi /etc/sysctl.conf
net.ipv4.ip_forward = 1
|
先停docker！先停docker！先停docker！（好像docker会缓冲bridge的ip）
1
2
3
4
| service docker stop
brctl addbr kbr0
ip link set dev docker0 down
ip link del dev docker0
|
下面的配置，两台机不同，如下：
| 192.168.191.140 kube-master |
192.168.191.141 kube-worker1 |
| vi /etc/sysconfig/network-scripts/ifcfg-kbr0 |
|
| DEVICE=kbr0 |
DEVICE=kbr0 |
| ONBOOT=yes |
ONBOOT=yes |
| BOOTPROTO=static |
BOOTPROTO=static |
| IPADDR=172.17.3.1 |
IPADDR=172.17.4.1 |
| NETMASK=255.255.255.0 |
NETMASK=255.255.255.0 |
| GATEWAY=172.17.3.0 |
GATEWAY=172.17.4.0 |
| USERCTL=no |
USERCTL=no |
| TYPE=Bridge |
TYPE=Bridge |
| IPV6INIT=no |
IPV6INIT=no |
| ———————————————– |
——————————————————– |
| vi /etc/sysconfig/network-scripts/route-ens33 （ip对应的网卡名称） |
|
| 172.17.4.0/24 via 192.168.191.141 dev ens33 |
172.17.3.0/24 via 192.168.191.140 dev ens33 |
| ———————————————– |
——————————————————– |
参考： Configuring Static Routes
1
2
3
4
| vi /usr/lib/systemd/system/docker.service
ExecStart=/usr/bin/dockerd --bridge=kbr0
systemctl daemon-reload
|
先起网卡！先起网卡！先起网卡！
1
2
3
| service network restart
systemctl start docker
|
最终效果
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
| | 192.168.191.140 kube-master | 192.168.191.141 kube-worker1
|:------------------------------------------------------------------------------|:-------------------------------------------------------
| [root@kube-master ~]# ifconfig | [root@kube-worker1 ~]# ifconfig
| ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 | ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
| inet 192.168.191.140 netmask 255.255.255.0 broadcast 192.168.191.255| inet 192.168.191.141 netmask 255.255.255.0 broadcast 192.168.191.255
| inet6 fe80::1186:2fe5:9ee5:8790 prefixlen 64 scopeid 0x20<link> | inet6 fe80::3995:4490:e2e7:1d0f prefixlen 64 scopeid 0x20<link>
| ether 00:0c:29:40:2d:15 txqueuelen 1000 (Ethernet) | ether 00:0c:29:2e:67:4d txqueuelen 1000 (Ethernet)
| RX packets 18010 bytes 10754845 (10.2 MiB) | RX packets 19871 bytes 12247126 (11.6 MiB)
| RX errors 0 dropped 0 overruns 0 frame 0 | RX errors 0 dropped 0 overruns 0 frame 0
| TX packets 4797 bytes 475332 (464.1 KiB) | TX packets 5647 bytes 561624 (548.4 KiB)
| TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 | TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
| |
| kbr1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 | kbr0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
| inet 172.17.3.1 netmask 255.255.255.0 broadcast 172.17.3.255 | inet 172.17.4.1 netmask 255.255.255.0 broadcast 172.17.4.255
| ether 00:00:00:00:00:00 txqueuelen 1000 (Ethernet) | ether 00:00:00:00:00:00 txqueuelen 1000 (Ethernet)
| RX packets 179 bytes 13932 (13.6 KiB) | RX packets 139 bytes 10492 (10.2 KiB)
| RX errors 0 dropped 0 overruns 0 frame 0 | RX errors 0 dropped 0 overruns 0 frame 0
| TX packets 43 bytes 3894 (3.8 KiB) | TX packets 36 bytes 3004 (2.9 KiB)
| TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 | TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
| |
| lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 | lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
| inet 127.0.0.1 netmask 255.0.0.0 | inet 127.0.0.1 netmask 255.0.0.0
| inet6 ::1 prefixlen 128 scopeid 0x10<host> | inet6 ::1 prefixlen 128 scopeid 0x10<host>
| loop txqueuelen 1 (Local Loopback) | loop txqueuelen 1 (Local Loopback)
| RX packets 140 bytes 11644 (11.3 KiB) | RX packets 215 bytes 18260 (17.8 KiB)
| RX errors 0 dropped 0 overruns 0 frame 0 | RX errors 0 dropped 0 overruns 0 frame 0
| TX packets 140 bytes 11644 (11.3 KiB) | TX packets 215 bytes 18260 (17.8 KiB)
| TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 | TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
| |
| [root@kube-master ~]# route -n | [root@kube-worker1 ~]# route -n
| Kernel IP routing table | Kernel IP routing table
| Destination Gateway Genmask Flags Metric Ref Use Iface | Destination Gateway Genmask Flags Metric Ref Use Iface
| 0.0.0.0 192.168.191.2 0.0.0.0 UG 100 0 0 ens33 | 0.0.0.0 192.168.191.2 0.0.0.0 UG 100 0 0 ens33
| 172.17.3.0 0.0.0.0 255.255.255.0 U 427 0 0 kbr1 | 172.17.3.0 192.168.191.140 255.255.255.0 UG 100 0 0 ens33
| 172.17.4.0 192.168.191.141 255.255.255.0 UG 100 0 0 ens33 | 172.17.4.0 0.0.0.0 255.255.255.0 U 425 0 0 kbr0
| 192.168.191.0 0.0.0.0 255.255.255.0 U 100 0 0 ens33 | 192.168.191.0 0.0.0.0 255.255.255.0 U 100 0 0 ens33
| [root@kube-master ~]# | [root@kube-worker1 ~]#
| [root@kube-master ~]# docker run -ti --rm busybox sh | [root@kube-worker1 ~]# docker run -ti --rm busybox sh
| / # ifconfig | / # ifconfig
| eth0 Link encap:Ethernet HWaddr 02:42:AC:11:03:02 | eth0 Link encap:Ethernet HWaddr 02:42:AC:11:04:02
| inet addr:172.17.3.2 Bcast:0.0.0.0 Mask:255.255.255.0 | inet addr:172.17.4.2 Bcast:0.0.0.0 Mask:255.255.255.0
| inet6 addr: fe80::42:acff:fe11:302/64 Scope:Link | inet6 addr: fe80::42:acff:fe11:402/64 Scope:Link
| UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 | UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
| RX packets:23 errors:0 dropped:0 overruns:0 frame:0 | RX packets:16 errors:0 dropped:0 overruns:0 frame:0
| TX packets:15 errors:0 dropped:0 overruns:0 carrier:0 | TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
| collisions:0 txqueuelen:0 | collisions:0 txqueuelen:0
| RX bytes:1870 (1.8 KiB) TX bytes:1222 (1.1 KiB) | RX bytes:1296 (1.2 KiB) TX bytes:648 (648.0 B)
| |
| lo Link encap:Local Loopback | lo Link encap:Local Loopback
| inet addr:127.0.0.1 Mask:255.0.0.0 | inet addr:127.0.0.1 Mask:255.0.0.0
| inet6 addr: ::1/128 Scope:Host | inet6 addr: ::1/128 Scope:Host
| UP LOOPBACK RUNNING MTU:65536 Metric:1 | UP LOOPBACK RUNNING MTU:65536 Metric:1
| RX packets:0 errors:0 dropped:0 overruns:0 frame:0 | RX packets:0 errors:0 dropped:0 overruns:0 frame:0
| TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 | TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
| collisions:0 txqueuelen:1 | collisions:0 txqueuelen:1
| RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) | RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
| / # ping 172.17.4.2 |
| PING 172.17.4.2 (172.17.4.2): 56 data bytes | / # ping 172.17.3.2
| 64 bytes from 172.17.4.2: seq=0 ttl=62 time=2.598 ms | PING 172.17.3.2 (172.17.3.2): 56 data bytes
| 64 bytes from 172.17.4.2: seq=1 ttl=62 time=1.569 ms | 64 bytes from 172.17.3.2: seq=0 ttl=62 time=1.421 ms
| 64 bytes from 172.17.4.2: seq=2 ttl=62 time=1.194 ms | 64 bytes from 172.17.3.2: seq=1 ttl=62 time=1.446 ms
| ^C | ^C
| --- 172.17.4.2 ping statistics --- | --- 172.17.3.2 ping statistics ---
| 3 packets transmitted, 3 packets received, 0% packet loss | 2 packets transmitted, 2 packets received, 0% packet loss
| round-trip min/avg/max = 1.194/1.787/2.598 ms | round-trip min/avg/max = 1.421/1.433/1.446 ms
|
|-------------------------------------------------------------------------------|--------------------------------------------------------
|
效果还不错，什么都没有安装route两台机器的docker就互联互通了。二三台机器使用这种方式最省事的，并且理论上效率也是最高的。
其他参考
–END