高可用k8s集群搭建：初始化其他master节点

先在另外两台机器上安装 kubectl、kubelet

  参考离线安装kubeadm、kubectl、kubelet），并修改 kubelet 配置(参考初始化第一个master节点中方法)以使用自定义镜像

配置 master 节点

  将第一个 master 节点 /etc/kubernetes 下内容拷贝其他两个机器，在 ca 证书所在目录（/etc/kubernetes/pki/）执行以下脚本创建证书

openssl genrsa -out apiserver-kubelet-client.key 2048
openssl req -new -key apiserver-kubelet-client.key -out apiserver-kubelet-client.csr -subj "/O=system:masters,/CN=kube-apiserver-kubelet-client"
openssl x509 -req -set_serial $(date +%s%N) -in apiserver-kubelet-client.csr -CA ca.crt -CAkey ca.key -out apiserver-kubelet-client.crt -days 365 -extensions v3_req -extfile req.conf

openssl genrsa -out controller-manager.key 2048
openssl req -new -key controller-manager.key -out controller-manager.csr -subj "/CN=system:kube-controller-manager"
openssl x509 -req -set_serial $(date +%s%N) -in controller-manager.csr -CA ca.crt -CAkey ca.key -out controller-manager.crt -days 365 -extensions v3_req -extfile req.conf

openssl genrsa -out scheduler.key 2048
openssl req -new -key scheduler.key -out scheduler.csr -subj "/CN=system:kube-scheduler"
openssl x509 -req -set_serial $(date +%s%N) -in scheduler.csr -CA ca.crt -CAkey ca.key -out scheduler.crt -days 365 -extensions v3_req -extfile req.conf

openssl genrsa -out admin.key 2048
openssl req -new -key admin.key -out admin.csr -subj "/O=system:masters/CN=kubernetes-admin"
openssl x509 -req -set_serial $(date +%s%N) -in admin.csr -CA ca.crt -CAkey ca.key -out admin.crt -days 365 -extensions v3_req -extfile req.conf

openssl genrsa -out $(hostname).key 2048
openssl req -new -key $(hostname).key -out $(hostname).csr -subj "/O=system:nodes/CN=system:node:$(hostname)"
openssl x509 -req -set_serial $(date +%s%N) -in $(hostname).csr -CA ca.crt -CAkey ca.key -out $(hostname).crt -days 365 -extensions v3_req -extfile req.conf

其中 req.conf 文件内容如下

[niuhp@localhost pki]$ cat req.conf 
[ v3_req ]
# Extensions to add to a certificate request
keyUsage = critical, digitalSignature, keyEncipherment
extendedKeyUsage = clientAuth

将 apiserver-kubelet-client.key和apiserver-kubelet-client.crt 两个文件覆盖原文件（/etc/kubernetes/pki/目录下），读取其他证书的内容并修改到指定的配置文件，证书内容不能直接读取使用，需要用base64加密，如cat ca.crt | base64 -w 0,需要修改的配置文件有：

• admin.conf 使用 ca,crt、admin.crt、admin.key
• controller-manager.conf 使用 ca,crt、controller-manager.crt、controller-manager.key
• kubelet.conf 使用 ca,crt、${hostname}.crt、${hostname}.key
• scheduler.conf 使用 ca,crt、scheduler.crt、scheduler.key
  以 kubelet 为例,当前机器 hostname 为 master2

  [gadmin@spa-38-174-112 k8s]$ cat kubelet.conf 
  apiVersion: v1
  clusters:
  - cluster:
      certificate-authority-data: ## cat ca.crt | base64 -w 0 显示的内容
      server: https://192.168.17.100:6443
    name: kubernetes
  contexts:
  - context:
      cluster: kubernetes
      user: system:node:master2  ##需要改为当前hostname
    name: system:node:master2@kubernetes  ##需要改为当前hostname
  current-context: system:master2@kubernetes  ##需要改为当前hostname
  kind: Config
  preferences: {}
  users:
  - name: system:node:master2  ##需要改为当前hostname
    user:
      client-certificate-data: ## cat master2.crt | base64 -w 0 显示的内容
      client-key-data: ## cat master2.key | base64 -w 0 显示的内容

  修改完成后需要重启 kubelet，sudo systemctl daemon-reload && sudo systemctl restart kubelet,验证下：

  [niuhp@localhost k8s]$ kubectl get csr
  NAME                                                   AGE       REQUESTOR                    CONDITION
  csr-bbc5w                                              2h        system:node:vm-192-168-17-101   Approved,Issued
  csr-psz9b                                              9m        system:node:vm-192-168-17-102   Approved,Issued
  csr-zkvr9                                              1m        system:node:vm-192-168-17-103   Approved,Issued

  修改机器标签

    执行以下命令将新节点标记为 master ,并查看节点状态

  [niuhp@localhost ]$ kubectl label node vm-192-168-17-102 node-role.kubernetes.io/master=
  [niuhp@localhost ]$ kubectl label node vm-192-168-17-103 node-role.kubernetes.io/master=
  [niuhp@localhost ]$ kubectl get no
  NAME                STATUS     ROLES     AGE       VERSION
  vm-192-168-17-101   Ready      master    2h        v1.8.4
  vm-192-168-17-102   Ready      master    9m        v1.8.4
  vm-192-168-17-103   Ready      master    1m        v1.8.4

  扩容dns及heapster

  [niuhp@localhost ]$ kubectl scale --replicas=3 deployment kube-dns -n kube-system
  [niuhp@localhost ]$ kubectl scale --replicas=3 deployment heapster -n kube-system