浙江龙网

  1. 首页 > 生活百科 > >

京东云开发者｜IoT运维 - 如何部署一套高可用K8S集群

生活百科

环境

京东云开发者｜IoT运维 - 如何部署一套高可用K8S集群

文章插图
准备工作配置ansible(deploy 主机执行)
# ssh-keygen# for i in 192.168.3.{21..28}; dossh-copy-id-i ~/.ssh/id_rsa.pub $i; done[root@deploy ~]# cat /etc/ansible/hosts[etcd]192.168.3.21192.168.3.22192.168.3.23[k8s-master]192.168.3.24192.168.3.25192.168.3.26[k8s-worker]192.168.3.27192.168.3.28[k8s:children]k8s-masterk8s-worker优化主机配置关闭防火墙和selinux
# ansible all -m shell -a "systemctl stop firewalld && systemctl disable firewalld"# ansible all -m shell -a "sed -i 's/^SELINUX=.*/SELINUX=disabled/g' /etc/selinux/config"修改limit
关闭交换分区
# swapoff -a# ansiblek8s-m shell -a "yes | cp /etc/fstab /etc/fstab_bak"# ansiblek8s-m shell -a "cat /etc/fstab_bak | grep -v swap > /etc/fstab"# ansiblek8s-m shell -a "echo vm.swappiness = 0 >> /etc/sysctl.d/k8s.conf"# ansiblek8s-m shell -a "sysctl -p /etc/sysctl.d/k8s.conf"配置ipvs# cat /root/ipvs.sh#!/bin/bashyum -y install ipvsadm ipset####创建ipvs脚本cat > /etc/sysconfig/modules/ipvs.modules << EOF#!/bin/bashmodprobe -- ip_vsmodprobe -- ip_vs_rrmodprobe -- ip_vs_wrrmodprobe -- ip_vs_shmodprobe -- nf_conntrack_ipv4EOF####执行脚本，验证配置chmod 755 /etc/sysconfig/modules/ipvs.modulesbash /etc/sysconfig/modules/ipvs.moduleslsmod | grep -e ip_vs -e nf_conntrack_ipv4########################## ansible k8s-m script -a "/root/ipvs.sh"配置网桥转发规则
# cat sysctl.sh#!/bin/bashcat > /etc/sysctl.d/k8s.conf << EOFnet.bridge.bridge-nf-call-ip6tables = 1net.bridge.bridge-nf-call-iptables = 1net.ipv4.ip_forward = 1EOFcat <<EOF | tee /etc/modules-load.d/crio.confoverlaybr_netfilterEOFmodprobe overlaymodprobe br_netfiltersysctl --system# ansible k8s-m script -a "/root/sysctl.sh"配置etcd集群生成证书(ansible 主机操作)
# curl -o /usr/bin/cfssl https://pkg.cfssl.org/R1.2/cfssl_linux-amd64# curl -o /usr/bin/cfssljson https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64# curl -o /usr/bin/cfssl-certinfo https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64# chmod +x /usr/bin/cfssl*创建 CA 配置文件
# mkdir p ssl# cd /root/ssl# cat >ca-config.json <<EOF{"signing": {"default": {"expiry": "876000h"},"profiles": {"etcd": {"usages": ["signing","key encipherment","server auth","client auth"],"expiry": "876000h"}}}}EOF创建 CA 证书签名请求
# cat >ca-csr.json <<EOF{"CN": "etcd","key": {"algo": "rsa","size": 2048},"names": [{"C": "CN","ST": "beijing","L": "beijing","O": "jdt","OU": "iot"}]}EOF生成 CA 证书和私钥
# cfssl gencert -initca ca-csr.json | cfssljson -bare ca创建etcd的TLS认证证书
# cat > etcd-csr.json <<EOF{"CN": "etcd","hosts": ["192.168.3.21","192.168.3.22","192.168.3.23","192.168.3.24","192.168.3.23","192.168.3.26","etcd1","etcd2","etcd3","master1","master2","master3"],"key": {"algo": "rsa","size": 2048},"names": [{"C": "CN","ST": "beijing","L": "beijing","O": "jdt","OU": "iot"}]EOF生成 etcd证书和私钥并分发
# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=etcd etcd-csr.json | cfssljson -bare etcd# ansibleetcd -m copy -a "src=https://www.huyubaike.com/root/ssl/ dest=/export/Data/certs/"ETCD安装以及配置
创建数据目录
# ansible etcd -m shell -a "mkdir -p /export/Data/etcd_data"下载etcd并分发
# wget https://github.com/etcd-io/etcd/releases/download/v3.5.1/etcd-v3.5.1-linux-amd64.tar.gz# tar xf etcd-v3.5.1-linux-amd64.tar.gz&& cd etcd-v3.5.1-linux-amd64# ansible etcd -m copy -a "src=https://www.huyubaike.com/biancheng/etcddest=/usr/bin/"# ansible etcd -m copy -a "src=https://www.huyubaike.com/biancheng/etcdutldest=/usr/bin/"# ansible etcd -m copy -a "src=https://www.huyubaike.com/biancheng/etcdctldest=/usr/bin/"# ansible etcd -m shell -a "chmod +x /usr/bin/etcd*"配置etcd
# cat etcd_config.sh#!/bin/bash#PEER_NAME指定本节点的主机名称/域名，#PRIVATE_IP指定本节点的IP(用于后面配置文件的生成)#ETCD_CLUSTER群集列表，是所有节点信息（内容格式: 各节点名称=https://ip:端口名称任意但要有标识性）#ETCD_INITIAL_CLUSTER_TOKEN为该etcd集群Token,同一集群token一致interface_name=`cat /proc/net/dev | sed -n '3,$p' | awk -F ':' {'print $1'} | grep-E "^ " | grep -v lo | head -n1`ipaddr=`ip a | grep $interface_name| awk '{print $2}' | awk -F"/"'{print $1}' | awk -F':' '{print $NF}'`export PEER_NAME=`hostname`export PRIVATE_IP=`echo $ipaddr | tr -d '\r'`export ETCD_CLUSTER="etcd1=https://192.168.3.21:2380,etcd2=https://192.168.3.22:2380,etcd3=https://192.168.3.23:2380"export ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster-1"cat > /etc/systemd/system/etcd.service<<EOF[Unit]Description=etcdDocumentation=https://github.com/coreos/etcdConflicts=etcd.service[Service]Type=notifyRestart=alwaysRestartSec=5sLimitNOFILE=65536TimeoutStartSec=0ExecStart=/usr/bin/etcd --name ${PEER_NAME} \--data-dir /export/Data/etcd_data\--listen-client-urls https://${PRIVATE_IP}:2379 \--advertise-client-urls https://${PRIVATE_IP}:2379 \--listen-peer-urls https://${PRIVATE_IP}:2380 \--initial-advertise-peer-urls https://${PRIVATE_IP}:2380 \--cert-file=/export/Data/certs/etcd.pem \--key-file=/export/Data/certs/etcd-key.pem \--client-cert-auth \--trusted-ca-file=/export/Data/certs/ca.pem \--peer-cert-file=/export/Data/certs/etcd.pem \--peer-key-file=/export/Data/certs/etcd-key.pem \--peer-client-cert-auth \--peer-trusted-ca-file=/export/Data/certs/ca.pem \--initial-cluster ${ETCD_CLUSTER} \--initial-cluster-token etcd-cluster-1 \--initial-cluster-state new[Install]WantedBy=multi-user.targetEOF

推荐阅读


上一篇：Spring Boot 项目自定义 banner

下一篇：怎么练成轻功（轻功如何快速练成)