高可用k8s集群搭建：搭建etcd集群

  众所周知 kubernetes 使用 etcd 来保存集群数据，官方的 kubeadm 安装方式会创建一个单节点的 etcd 容器，为了保证集群高可用，我们需要搭建一个 etcd 集群，这里使用镜像gcr.io/google_containers/etcd-amd64:3.0.17搭建主要是为了避免使用其他版本带来一些兼容性问题，也是因为 kubernetes 一直使用这个镜像。

系统要求

• 64位centos7系统
• 已安装 docker ，安装步骤请戳

单节点etcd服务

  在控制台执行以下命令

REGISTRY=gcr.io/google_containers/etcd-amd64
ETCD_VERSION=3.0.17
NODE1=127.0.0.1
sudo docker run -d \
  -p 2379:2379 \
  -p 2380:2380 \
  --volume=/var/lib/etcd-data:/etcd-data \
  --name etcd ${REGISTRY}:${ETCD_VERSION} \
  /usr/local/bin/etcd \
  --data-dir=/etcd-data --name node1 \
  --initial-advertise-peer-urls http://${NODE1}:2380 --listen-peer-urls http://0.0.0.0:2380 \
  --advertise-client-urls http://${NODE1}:2379 --listen-client-urls http://0.0.0.0:2379 \
  --initial-cluster node1=http://${NODE1}:2380

验证安装

[niuhp@localhost etcd]$ sudo docker exec etcd /bin/sh -c "/usr/local/bin/etcd --version"
etcd Version: 3.0.17
Git SHA: cc198e2
Go Version: go1.6.4
Go OS/Arch: linux/amd64
[niuhp@localhost etcd]$ sudo docker exec etcd /bin/sh -c "ETCDCTL_API=3 /usr/local/bin/etcdctl version"
etcdctl version: 3.0.17
API version: 3.0
[niuhp@localhost etcd]$ sudo docker exec etcd /bin/sh -c "ETCDCTL_API=3 /usr/local/bin/etcdctl endpoint health"
127.0.0.1:2379 is healthy: successfully committed proposal: took = 999.973µs
[niuhp@localhost etcd]$ sudo docker exec etcd /bin/sh -c "ETCDCTL_API=3 /usr/local/bin/etcdctl put foo bar"
OK
[niuhp@localhost etcd]$ sudo docker exec etcd /bin/sh -c "ETCDCTL_API=3 /usr/local/bin/etcdctl get foo"
foo
bar

三节点etcd集群

机器准备

• node1 192.168.17.101
• node2 192.168.17.102
• node3 192.168.17.103

  分别在每个node上执行以下命令

• 在 node1 上执行以下命令

  REGISTRY=gcr.io/google_containers/etcd-amd64
  ETCD_VERSION=3.0.17
  TOKEN=token-etcd-cluster
  CLUSTER_STATE=new
  NAME_1=etcd1
  NAME_2=etcd2
  NAME_3=etcd3
  HOST_1=192.168.17.101
  HOST_2=192.168.17.102
  HOST_3=192.168.17.103
  CLUSTER=${NAME_1}=http://${HOST_1}:2380,${NAME_2}=http://${HOST_2}:2380,${NAME_3}=http://${HOST_3}:2380
  DATA_DIR=/home/niuhp/etcd/data
  
  THIS_NAME=${NAME_1}
  THIS_IP=${HOST_1}
  
  sudo docker stop etcd || true
  sudo docker rm etcd || true
  sudo docker run -d \
    -p 2379:2379 \
    -p 2380:2380 \
    --volume=${DATA_DIR}:/etcd-data \
    --name etcd ${REGISTRY}:${ETCD_VERSION} \
    /usr/local/bin/etcd \
    --data-dir=/etcd-data --name ${THIS_NAME} \
    --initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://0.0.0.0:2380 \
    --advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://0.0.0.0:2379 \
    --initial-cluster ${CLUSTER} \
    --initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}

• 分别将上述命令中 THIS_NAME=${NAME_1}、THIS_IP=${HOST_1} 改为 THIS_NAME=${NAME_2}、THIS_IP=${HOST_2}和THIS_NAME=${NAME_3}、THIS_IP=${HOST_3}并依次在 node2、node3 上执行即可
• 验证安装，和单节点服务验证类似，分别在三个节点上执行

  [niuhp@localhost etcd]$ sudo docker exec etcd /bin/sh -c "/usr/local/bin/etcd --version"
  etcd Version: 3.0.17
  Git SHA: cc198e2
  Go Version: go1.6.4
  Go OS/Arch: linux/amd64
  [niuhp@localhost etcd]$ sudo docker exec etcd /bin/sh -c "ETCDCTL_API=3 /usr/local/bin/etcdctl version"
  etcdctl version: 3.0.17
  API version: 3.0
  [niuhp@localhost etcd]$ sudo docker exec etcd /bin/sh -c "ETCDCTL_API=3 /usr/local/bin/etcdctl endpoint health"
  127.0.0.1:2379 is healthy: successfully committed proposal: took = 1.54972ms

  另外可以通过在其中一个节点 put 值，在所有节点 get 检查结果是否一致，如在 node1 上执行 put 操作

  [niuhp@localhost etcd]$ sudo docker exec etcd /bin/sh -c "ETCDCTL_API=3 /usr/local/bin/etcdctl put foo value_from_node1"
  OK

  然后在三个节点上执行 get 操作均可以得到相同的结果value_from_node1,确定集群安装正确

  [niuhp@localhost etcd]$ sudo docker exec etcd /bin/sh -c "ETCDCTL_API=3 /usr/local/bin/etcdctl get foo"
  foo
  value_from_node1

  参考

• https://coreos.com/etcd/docs/3.0.17/op-guide/clustering.html
• https://github.com/coreos/etcd/releases/
• https://github.com/coreos/etcd/blob/master/Documentation/op-guide/container.md#docker