虚拟机环境
| IP | 版本 | 角色 |
|---|---|---|
| 10.211.55.46 | CentOS 7.6 | k8s-master |
| 10.211.55.47 | CentOS 7.6 | k8s-node-1 |
| 10.211.55.48 | CentOS 7.6 | k8s-node-2 |
关闭不必要的麻烦
# 三台机器同样操作
[root@k8s-master ~]# systemctl disable firewalld && systemctl stop firewalld && setenforce 0
安装 Docker
这里安装的 Docker 是
1.12.6，因为 k8s 1.6.1 版本最高支持 Docker 为 1.12.x。三台机器上都是一样的操作。
- 安装所需组件
[root@k8s-master ~]# yum install -y libtool-ltdl policycoreutils-python yum-utils
- 下载 Docker rpm
[root@k8s-master ~]# wget https://yum.dockerproject.org/repo/main/centos/7/Packages/docker-engine-1.12.6-1.el7.centos.x86_64.rpm
[root@k8s-master ~]# wget https://yum.dockerproject.org/repo/main/centos/7/Packages/docker-engine-selinux-1.12.6-1.el7.centos.noarch.rpm
- 安装
[root@k8s-master ~]# rpm -ivh docker-engine-selinux-1.12.6-1.el7.centos.noarch.rpm
[root@k8s-master ~]# rpm -ivh docker-engine-1.12.6-1.el7.centos.x86_64.rpm
[root@k8s-master ~]# systemctl enable docker && systemctl start docker
- 拉取镜像，并修改成 k8s 集群启动时需要的镜像名称
docker pull warrior/pause-amd64:3.0
docker tag warrior/pause-amd64:3.0 gcr.io/google_containers/pause-amd64:3.0
docker pull warrior/etcd-amd64:3.0.17
docker tag warrior/etcd-amd64:3.0.17 gcr.io/google_containers/etcd-amd64:3.0.17
docker pull mirrorgooglecontainers/kube-apiserver-amd64:v1.6.1
docker tag mirrorgooglecontainers/kube-apiserver-amd64:v1.6.1 gcr.io/google_containers/kube-apiserver-amd64:v1.6.1
docker pull mirrorgooglecontainers/kube-scheduler-amd64:v1.6.1
docker tag mirrorgooglecontainers/kube-scheduler-amd64:v1.6.1 gcr.io/google_containers/kube-scheduler-amd64:v1.6.1
docker pull mirrorgooglecontainers/kube-controller-manager-amd64:v1.6.1
docker tag mirrorgooglecontainers/kube-controller-manager-amd64:v1.6.1 gcr.io/google_containers/kube-controller-manager-amd64:v1.6.1
docker pull mirrorgooglecontainers/kube-proxy-amd64:v1.6.1
docker tag mirrorgooglecontainers/kube-proxy-amd64:v1.6.1 gcr.io/google_containers/kube-proxy-amd64:v1.6.1
docker pull gysan/dnsmasq-metrics-amd64:1.0
docker tag gysan/dnsmasq-metrics-amd64:1.0 gcr.io/google_containers/dnsmasq-metrics-amd64:1.0
docker pull warrior/k8s-dns-kube-dns-amd64:1.14.1
docker tag warrior/k8s-dns-kube-dns-amd64:1.14.1 gcr.io/google_containers/k8s-dns-kube-dns-amd64:1.14.1
docker pull warrior/k8s-dns-dnsmasq-nanny-amd64:1.14.1
docker tag warrior/k8s-dns-dnsmasq-nanny-amd64:1.14.1 gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64:1.14.1
docker pull warrior/k8s-dns-sidecar-amd64:1.14.1
docker tag warrior/k8s-dns-sidecar-amd64:1.14.1 gcr.io/google_containers/k8s-dns-sidecar-amd64:1.14.1
docker pull awa305/kube-discovery-amd64:1.0
docker tag awa305/kube-discovery-amd64:1.0 gcr.io/google_containers/kube-discovery-amd64:1.0
docker pull gysan/exechealthz-amd64:1.2
docker tag gysan/exechealthz-amd64:1.2 gcr.io/google_containers/exechealthz-amd64:1.2
Master
修改系统配置
所有机器
[root@k8s-master ~]# cat <<EOF > /etc/sysctl.d/kubernetes.conf
> net.bridge.bridge-nf-call-ip6tables = 1
> net.bridge.bridge-nf-call-iptables = 1
> EOF
[root@k8s-master ~]# sysctl -p /etc/sysctl.d/kubernetes.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
每个节点都要操作。
添加 yum 源
[root@k8s-master ~]# cat <<EOF > /etc/yum.repos.d/kubernetes.repo
> [kubernetes]
> name=Kubernetes
> baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
> enabled=1
> gpgcheck=0
> EOF
[root@k8s-master ~]# yum makecache fast
安装
- 安装
kubelet、kubeadm、kubectl、kubernetes-cni。
[root@k8s-master ~]# yum -y install socat kubelet-1.6.1 kubeadm-1.6.1 kubectl-1.6.1 kubernetes-cni-0.5.1
- 启动
kubelet
[root@k8s-master ~]# systemctl enable kubelet && systemctl start kubelet
初始化 Master
[root@k8s-master ~]# kubeadm init --kubernetes-version=v1.6.1
[kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters.
[init] Using Kubernetes version: v1.6.1
[init] Using Authorization mode: RBAC
[preflight] Running pre-flight checks
[certificates] Generated CA certificate and key.
[certificates] Generated API server certificate and key.
[certificates] API Server serving cert is signed for DNS names [k8s-master kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 10.211.55.46]
[certificates] Generated API server kubelet client certificate and key.
[certificates] Generated service account token signing key and public key.
[certificates] Generated front-proxy CA certificate and key.
[certificates] Generated front-proxy client certificate and key.
[certificates] Valid certificates and keys now exist in "/etc/kubernetes/pki"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/controller-manager.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/scheduler.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/admin.conf"
[apiclient] Created API client, waiting for the control plane to become ready
[apiclient] All control plane components are healthy after 11.279598 seconds
[apiclient] Waiting for at least one node to register
[apiclient] First node has registered after 4.503921 seconds
[token] Using token: 786d52.a19313e9d0c4eed0
[apiconfig] Created RBAC rules
[addons] Created essential addon: kube-proxy
[addons] Created essential addon: kube-dns
Your Kubernetes master has initialized successfully!
To start using your cluster, you need to run (as a regular user):
sudo cp /etc/kubernetes/admin.conf $HOME/
sudo chown $(id -u):$(id -g) $HOME/admin.conf
export KUBECONFIG=$HOME/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
http://kubernetes.io/docs/admin/addons/
You can now join any number of machines by running the following on each node
as root:
kubeadm join --token 786d52.a19313e9d0c4eed0 10.211.55.46:6443
- 复制配置文件到用户目录下
[root@k8s-master ~]# sudo cp /etc/kubernetes/admin.conf $HOME/
[root@k8s-master ~]# sudo chown $(id -u):$(id -g) $HOME/admin.conf
[root@k8s-master ~]# export KUBECONFIG=$HOME/admin.conf
安装 Node
记得安装 Docker
- yum 源
[root@k8s-node-1 ~]# cat <<EOF > /etc/yum.repos.d/kubernetes.repo
> [kubernetes]
> name=Kubernetes
> baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
> enabled=1
> gpgcheck=0
> EOF
[root@k8s-node-1 ~]# yum makecache fast
- 安装
[root@k8s-node-1 ~]# yum -y install socat kubelet-1.6.1 kubeadm-1.6.1 kubectl-1.6.1 kubernetes-cni-0.5.1
[root@k8s-node-1 ~]# systemctl enable kubelet && systemctl start kubelet
- 加入集群
# 使用 Master 中的提示 token 加入
[root@k8s-node-1 ~]# kubeadm join --token 786d52.a19313e9d0c4eed0 10.211.55.46:6443
[kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters.
[preflight] Running pre-flight checks
[discovery] Trying to connect to API Server "10.211.55.46:6443"
[discovery] Created cluster-info discovery client, requesting info from "https://10.211.55.46:6443"
[discovery] Cluster info signature and contents are valid, will use API Server "https://10.211.55.46:6443"
[discovery] Successfully established connection with API Server "10.211.55.46:6443"
[bootstrap] Detected server version: v1.6.1
[bootstrap] The server supports the Certificates API (certificates.k8s.io/v1beta1)
[csr] Created API client to obtain unique certificate for this node, generating keys and certificate signing request
[csr] Received signed certificate from the API server, generating KubeConfig...
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf"
Node join complete:
* Certificate signing request sent to master and response
received.
* Kubelet informed of new secure connection details.
Run 'kubectl get nodes' on the master to see this machine join.
加入成功，在主节点上查看节点信息：
[root@k8s-master ~]# kubectl get nodes
NAME STATUS AGE VERSION
k8s-master NotReady 11m v1.6.1
k8s-node-1 NotReady 34s v1.6.1
另一台节点，如法炮制即可。
- 最终结果
[root@k8s-master ~]# kubectl get nodes
NAME STATUS AGE VERSION
k8s-master NotReady 15m v1.6.1
k8s-node-1 NotReady 4m v1.6.1
k8s-node-2 NotReady 1s v1.6.1
问题
节点的状态都是 NotReady
通过命令 kubectl get nodes 可以看到节点的状态都是 NotReady，这是因为还没有安装 CNI 网络插件。网络插件的选择有很多种，可以参考网络查件。其实在初始化好 Master 之后，有如下提示：
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
http://kubernetes.io/docs/admin/addons/
这里选择网络插件 weave：
[root@k8s-master ~]# kubectl apply -f https://git.io/weave-kube-1.6
serviceaccount "weave-net" created
clusterrole "weave-net" created
clusterrolebinding "weave-net" created
role "weave-net" created
rolebinding "weave-net" created
daemonset "weave-net" created
- 这时候查看节点，发现还是
NotReady。查看节点详情：
[root@k8s-master ~]# kubectl describe node k8s-node-1
Name: k8s-node-1
Role:
Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/os=linux
kubernetes.io/hostname=k8s-node-1
Annotations: node.alpha.kubernetes.io/ttl=0
volumes.kubernetes.io/controller-managed-attach-detach=true
Taints: <none>
CreationTimestamp: Thu, 18 Apr 2019 00:19:36 -0400
Phase:
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
OutOfDisk False Thu, 18 Apr 2019 00:31:37 -0400 Thu, 18 Apr 2019 00:19:36 -0400 KubeletHasSufficientDisk kubelet has sufficient disk space available
MemoryPressure False Thu, 18 Apr 2019 00:31:37 -0400 Thu, 18 Apr 2019 00:19:36 -0400 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Thu, 18 Apr 2019 00:31:37 -0400 Thu, 18 Apr 2019 00:19:36 -0400 KubeletHasNoDiskPressure kubelet has no disk pressure
Ready False Thu, 18 Apr 2019 00:31:37 -0400 Thu, 18 Apr 2019 00:19:36 -0400 KubeletNotReady runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Addresses: 10.211.55.47,10.211.55.47,k8s-node-1
Capacity:
cpu: 2
memory: 1877572Ki
pods: 110
Allocatable:
cpu: 2
memory: 1775172Ki
pods: 110
System Info:
Machine ID: ef56949a51de1447b8661f55966dba12
System UUID: EF56949A-51DE-1447-B866-1F55966DBA12
Boot ID: e1d7798b-c9db-42c6-a3f4-c417b4b09ff3
Kernel Version: 3.10.0-957.el7.x86_64
OS Image: CentOS Linux 7 (Core)
Operating System: linux
Architecture: amd64
Container Runtime Version: docker://1.12.6
Kubelet Version: v1.6.1
Kube-Proxy Version: v1.6.1
ExternalID: k8s-node-1
Non-terminated Pods: (2 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits
--------- ---- ------------ ---------- --------------- -------------
kube-system kube-proxy-nfl7l 0 (0%) 0 (0%) 0 (0%) 0 (0%)
kube-system weave-net-96m1k 20m (1%) 0 (0%) 0 (0%) 0 (0%)
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
CPU Requests CPU Limits Memory Requests Memory Limits
------------ ---------- --------------- -------------
20m (1%) 0 (0%) 0 (0%) 0 (0%)
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
12m 12m 1 kubelet, k8s-node-1 Normal Starting Starting kubelet.
12m 12m 1 kubelet, k8s-node-1 Warning ImageGCFailed unable to find data for container /
12m 12m 2 kubelet, k8s-node-1 Normal NodeHasSufficientDisk Node k8s-node-1 status is now: NodeHasSufficientDisk
12m 12m 2 kubelet, k8s-node-1 Normal NodeHasSufficientMemory Node k8s-node-1 status is now: NodeHasSufficientMemory
12m 12m 2 kubelet, k8s-node-1 Normal NodeHasNoDiskPressure Node k8s-node-1 status is now: NodeHasNoDiskPressure
12m 12m 1 kube-proxy, k8s-node-1 Normal Starting Starting kube-proxy.
在 Conditions.Ready 的 Message 里面显示 runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized。
- 解决办法
# 删除掉里面最后一行中的 "$KUBELET_NETWORK_ARGS"
[root@k8s-node-1 ~]# vim /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
# 重启 kubelet
[root@k8s-node-1 ~]# systemctl enable kubelet && systemctl start kubelet
# 重新加入集群
[root@k8s-node-1 ~]# kubeadm reset && kubeadm join --token 786d52.a19313e9d0c4eed0 10.211.55.46:6443
查看节点：
[root@k8s-master ~]# kubectl get nodes
NAME STATUS AGE VERSION
k8s-master NotReady 48m v1.6.1
k8s-node-1 Ready 37m v1.6.1
k8s-node-2 NotReady 33m v1.6.1
其他节点如法炮制。
文章评论