kubernetes 集群安装
kubernetes 集群安装

主机规划:
| 主机名 | ip地址 | 配置最低要求 | 角色 |
|---|---|---|---|
| master | 192.168.86.11 | CPU>=2,MEM>=2GB,DISK=100G | Master |
| node1 | 192.168.86.12 | CPU>=1,MEM>=1GB,DISK=100G | Worker |
| node2 | 192.168.86.13 | CPU>=1,MEM>=1GB,DISK=100G | Worker |
一、安装 ikuai
1.1 下载 ikuai
下载 iso 32位:ikuai
实验环境,选择 32 位的 ISO 下载,硬件要求比较低,在真实环境中需要安装 64 位的

1.2 创建虚拟机
- 自定义安装
- 硬件兼容性默认 Workstation 17.5 or later
- 稍后安装操作系统
- 客户机操作系统:其他,版本:其他
- 虚拟机名称,位置自取
- 处理器配置:处理器数量1,内核数量1
- 内存配置1GB
- 网络类型:仅主机
ikuai 路由器有两块网卡,第一块采用仅主机模式网络,用于与kubernetes集群通讯,第二块为NAT模式,用于访问外部网络。 - I/O控制类型:BusLogic(默认)
- 磁盘类型:IDE(默认)
此处只能选择默认的 IDE,不能修改,因为 32 位的 ikuai 无法识别其它几种磁盘类型 - 创建新虚拟磁盘
- 将虚拟磁盘存储为单个文件,8GB
- 下一步,完成
1.3 配置虚拟机
- 添加 NAT 网卡
- 导入镜像文件(iKuai8_x32_3.7.20_Build202506041743.iso)

1.4 安装 ikuai
- 点击开启此虚拟机,进入到安装界面

- 输入 1 后回车

- 输入 y 回车,系统就开始安装,安装成功后会自动重启

- 重启好后会听到蜂鸣声,并且会显示相关菜单

- 配置一定的 IP 信息,选择 2 、设置 LAN/WAN 地址,即在请输入菜单编号后输入 2 后回车

- 输入 0 来设置 LAN1 的地址,输入好后回车

- 根据示例来输入一个未被占用的 IP 地址,如 192.168.86.200/255.255.255.0 后回车

- 提示设置成功,按 esc 键返回到上级菜单

- 输入 q 后回车,退出到控制台菜单

- 最后再输入 q 后回车,锁定当前配置
如果关机,输入菜单编号 8,等待 10 秒后关机
或者暂时挂起
1.5 管理 ikuai
系统安装好后,打开浏览器,输入 http://192.168.86.200 就可访问 ikuai 的控制页面

默认用户名和密码都是 admin,输入后即可登录成功
第一次登录成功后,为了设备安全,要求修改登录密码,密码要求大小写字母和数组的组合,长度至少 8 位以上。例如输入 Ikuai888 后点击确定按钮,即可进入管理页面

ikuai 是一个纯软件实现的路由器。
- 点击左边菜单中“网络设置”,然后再点击“内外网设置“

- 提示当前网卡是虚拟机提供的性能比较弱。关闭提示框后将页面向下滚动到最后

- 内网网口变绿了,代表内网网口正在工作中。将鼠标移动到图标上就会弹出设备信息

另一个外网网口是灰色的,鼠标移动到图标上会提示空闲接口,未绑定网卡

- 点击外网网口图标,进入到配置窗口

- 点击“选择网卡”后面的下拉列表,选择eth1

- 点击“绑定”按钮进入到如下配置页面

- 将“接入方式”选择 DHCP(动态获取),让它动态分配外网访问所需要的 IP 地址等信息。然后将页面向下滚动到最后,就可以看到外网网口
也变成绿色

ikuai 路由器就安装好,并且可以为 kubernetes 集群各主机提供网络的访问能力
二、系统环境准备
2.1 安装系统
安装 redhat 系统的虚拟机教程:Linux系统安装
安装一台 4 核心CPU,4G内存,两块网卡(第一块为仅主机模式,第二块为 NAT 模式),磁盘空间为 100GB 的 Redhat 9.5 系统
将安装好系统、初始化环境后的 k8s 虚拟机克隆三台主机,搭建 kubelets 集群的 master、node1、node2
2.2 初始化环境
- 如果分区是自动分区,有/swap,则进入/etc/fstab文件中去把有/swap的行注释掉
- 修改 /etc/fstab 文件,将包含 swap 内容的行注释
- 使用 free -m 来查看 swap 是否还在使用
- swapoff -a
- 仓库
/etc/yum.repos.d/yum.repo
name(BaseOS、AppStream)
baseurl
gpgcheck(0) - 挂载
/etc/fstab
/dev/sr0 /mnt iso9660 defaults 0 0 - 防火墙
systemctl disable --now firewalld - selinux
sed -i 's/SELINUX=enforcing/SELINUX=permissive/g' /etc/selinux/config - 安装软件
net-tools nfs-utils vim wget curl bash-completion device-mapper-persistent-data psmisc tree - 修改 linux 最大连接数
/etc/security/limits.conf - 安装 ipvs
dnf install ipvsadm -y - 开启路由转发
echo 'net.ipv4.ip_forward=1' >> /etc/sysctl.conf
sysctl -p
2.2.1 配置本地仓库,设置自动挂载
- 验证挂载:
systemctl daemon-reload
mount -a
df -h /mnt
# 1. 配置本地仓库
[root@localhost ~]# vi /etc/yum.repos.d/yum.repo
[root@localhost ~]# cat /etc/yum.repos.d/yum.repo
[BaseOS]
name=BaseOS
baseurl=/mnt/BaseOS
gpgcheck=0
[AppStream]
name=AppStream
baseurl=/mnt/AppStream
gpgcheck=0
# 2. 配置自动挂载
[root@localhost yum.repos.d]# vi /etc/fstab
[root@localhost yum.repos.d]# cat /etc/fstab
....
/dev/sr0 /mnt iso9660 defaults 0 0
# 3. 验证挂载
[root@localhost yum.repos.d]# systemctl daemon-reload
[root@localhost yum.repos.d]# mount -a
[root@localhost yum.repos.d]# df -h /mnt
Filesystem Size Used Avail Use% Mounted on
/dev/sr0 11G 11G 0 100% /mnt
2.2.2 关闭防火墙和 SELinux
- 验证是否设置成功 selinux
grep permissive /etc/selinux/config
[root@localhost yum.repos.d]# sed -i 's/SELINUX=enforcing/SELINUX=permissive/g' /etc/selinux/config
[root@localhost yum.repos.d]# grep permissive /etc/selinux/config
# permissive - SELinux prints warnings instead of enforcing.
SELINUX=permissive
[root@localhost yum.repos.d]# systemctl disable --now firewalld
Removed "/etc/systemd/system/multi-user.target.wants/firewalld.service".
Removed "/etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service".
2.2.3 关闭交换分区(swap)
[root@localhost yum.repos.d]# swapoff -a
[root@localhost yum.repos.d]# sed -i 's/.*swap.*/#&/' /etc/fstab
[root@localhost yum.repos.d]# free -m
total used free shared buff/cache available
Mem: 3881 431 3513 10 118 3450
Swap: 0 0 0
# 或者进入设置挂载的文件中,注释掉有 swap 的行
- 修改 /etc/fstab 文件,将包含 swap 内容的行注释
- 使用 free -m 来查看 swap 是否还在使用
- swapoff -a
2.2.4 安装基本软件
dnf install
net-tools nfs-utils(ifconfig) vim wget curl bash-completion device-mapper-persistent-data psmisc(ptree,进程数) tree -y
[root@localhost yum.repos.d]# dnf install net-tools nfs-utils vim wget curl bash-completion device-mapper-persistent-data psmisc tree -y
2.2.5 修改 Linux 最大连接数(/etc/security/limits.conf)
[root@localhost yum.repos.d]# vim /etc/security/limits.conf
* soft nofile 655350
* hard nofile 655350
* soft nproc 655350
* hard nproc 655350
* soft memlock unlimited
* hard memlock unlimited
2.2.6 安装 ipvs(ipvsadm)
[root@localhost yum.repos.d]# dnf install ipvsadm -y
2.2.7 开启路由转发
net.ipv4.ip_forward = 1
[root@localhost yum.repos.d]# vim /etc/sysctl.conf
[root@localhost yum.repos.d]# cat /etc/sysctl.conf
# sysctl settings are defined through files in
# /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.
#
# Vendors settings live in /usr/lib/sysctl.d/.
# To override a whole file, create a new file with the same in
# /etc/sysctl.d/ and put new settings there. To override
# only specific settings, add a file with a lexically later
# name in /etc/sysctl.d/ and put new settings there.
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
net.ipv4.ip_forward = 1
三、集群准备
克隆三台 系统环境准备过程中的 k8s 虚拟机
3.1 配置 master 节点
- 修改主机名
ls /etc/NetworkManager/system-connections/命令,可查看当前网卡
3.1.1 修改仅主机网卡(IP192.168.86.11,网关指向ikuai)
[root@master ~]# nmcli c m ens160 ipv4.method manual ipv4.addresses 192.168.86.11/24 ipv4.dns "223.5.5.5 8.8.8.8" ipv4.gateway 192.168.86.200 connection.autoconnect yes
[root@master ~]# nmcli c m ens160 up
3.1.2 暂时关闭 NAT 模式的网卡
暂时关闭NAT模式网卡,以防止安装集群时出现绑定网卡错误的问题
nmcli c m ens192 connection.autoconnect no
[root@master ~]# nmcli c m ens192 connection.autoconnect no
[root@master ~]# nmcli c up ens192
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/5)
3.1.3 配置主机映射
[root@master ~]# cat /etc/hosts
192.168.86.11 master m1
192.168.86.12 node1 n1
192.168.86.13 node2 n2
192.168.86.14 minio
192.168.86.20 harbor.registry.com harbor
3.2 配置 node1 和 node2 节点
和 master 一样操作
3.3 免密设置
- 配置主机映射
- 执行命令 ssh-keygen
- ssh-copy-id 主机
- 配置主机映射:
[root@master1 kubernetes]# cat /etc/hosts
192.168.86.11 master1 m1
192.168.86.12 master2 m2
192.168.86.13 master3 m3
192.168.86.14 node1 n1
192.168.86.15 node2 n2
192.168.86.20 harbor.registry.com harbor
- 执行命令 ssh-keygen:
[root@master1 kubernetes]# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
/root/.ssh/id_rsa already exists.
Overwrite (y/n)? y
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa
Your public key has been saved in /root/.ssh/id_rsa.pub
The key fingerprint is:
SHA256:0MJxfsnasRNn7L+W+OixHo/tfYOGF+upZkpJweKxrVo root@master1
The key's randomart image is:
+---[RSA 3072]----+
| . . |
| . =.. o |
| +ooo* + |
| .o=+.B |
| oSo+ . |
| o .. o |
| E o +.=. |
| o . +.&++.|
| . .++@** +|
+----[SHA256]-----+
- 执行命令 ssh-copy-id
[root@master1 kubernetes]# ssh-copy-id n1
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@n1's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh 'n1'"
and check to make sure that only the key(s) you wanted were added.
四、搭建 Kubernetes 集群
4.1 安装 docker(三台主机)
- 添加 docker 源
dnf install yum-utils -yyum-config-manager --add-repo https://mirrors.aliyun.com/dockerce/linux/rhel/docker-ce.repo
- 安装 docker
dnf install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -y
- 配置 docker
- /etc/docker/daemon.json
- 注意自己的 harbor 地址(https://harbor.registry.com)
- 启动 docker
systemctl enable --now docker
- 验证 docker
docker version
# 添加 docker 源
[root@master ~]# dnf install yum-utils -y
Updating Subscription Management repositories.
.........
Installed:
yum-utils-4.3.0-16.el9.noarch
Complete!
# 安装 docker
[root@master ~]# yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/rhel/docker-ce.repo
Updating Subscription Management repositories.
Unable to read consumer identity
This system is not registered with an entitlement server. You can use "rhc" or "subscription-manager" to register.
Adding repo from: https://mirrors.aliyun.com/docker-ce/linux/rhel/docker-ce.repo
[root@master ~]# ls /etc/yum.repos.d/
docker-ce.repo redhat.repo yum.repo
[root@master ~]# dnf install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -y
# 配置docker
[root@master ~]# cat > /etc/docker/daemon.json <<EOF
{
"default-ipc-mode": "shareable",
"data-root": "/data/docker",
"exec-opts": ["native.cgroupdriver=systemd"],
"log-driver": "json-file",
"log-opts": {
"max-size": "100m",
"max-file": "50"
},
"insecure-registries": ["https://harbor.registry.com"],
"registry-mirrors":[
"https://docker.m.daocloud.io",
"https://docker.1ms.run",
"https://func.ink",
"https://docker.hlmirror.com",
"https://docker.imgdb.de",
"https://docker-0.unsee.tech",
"https://docker.rainbond.cc",
"https://lispy.org",
"https://docker.xiaogenban1993.com",
"https://mirror.ccs.tencentyun.com"
]
}
EOF
` 解释:`
{
"default-ipc-mode": "shareable", #把ipc共享模式打开
"data-root": "/data/docker",
"exec-opts": ["native.cgroupdriver=systemd"], #把容器的驱动改为systemd
"log-driver": "json-file", #日志文件叫json-file
"log-opts": {
"max-size": "100m", #日志存放文件数量
"max-file": "50"
},
"insecure-registries": ["https://harbor.registry.com"], #自己的harbor地址
"registry-mirrors":[ #以下都是公共的仓库地址
"https://docker.m.daocloud.io",
"https://docker.1ms.run",
"https://func.ink",
"https://docker.hlmirror.com",
"https://docker.imgdb.de",
"https://docker-0.unsee.tech",
"https://docker.rainbond.cc",
"https://lispy.org",
"https://docker.xiaogenban1993.com",
"https://mirror.ccs.tencentyun.com"
]
}
#启动docker
[root@master ~]# systemctl enable --now docker
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service.
#验证docker
[root@master ~]# docker version
Client: Docker Engine - Community
Version: 28.3.1
API version: 1.51
Go version: go1.24.4
Git commit: 38b7060
Built: Wed Jul 2 20:58:10 2025
OS/Arch: linux/amd64
Context: default
Server: Docker Engine - Community
Engine:
Version: 28.3.1
API version: 1.51 (minimum version 1.24)
Go version: go1.24.4
Git commit: 5beb93d
Built: Wed Jul 2 20:56:24 2025
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.7.27
GitCommit: 05044ec0a9a75232cad458027ca83437aae3f4da
runc:
Version: 1.2.5
GitCommit: v1.2.5-0-g59923ef
docker-init:
Version: 0.19.0
GitCommit: de40ad0
4.2 安装 CRI(三台主机)
CRI 为容器运行时,常见的有 cri-docker 和 containerd,两者任选其一安装
- 获取 cri-dockerd-0.3.18.amd64.tgz 包
- 解压文件
tar -xf cri-dockerd-0.3.17.amd64.tgz - 将文件复制到 /usr/bin目录下
- 设置文件可执行权限
chmod +x /usr/bin/cri-dockerd - 配置 cri-docker 服务
/usr/lib/systemd/system/cri-docker.service - 配置 cri-docker 套接字
/usr/lib/systemd/system/cri-docker.socket - 启动 cri-docker 服务
systemctl daemon-reload
systemctl enable --now cri-docker
systemctl is-active cri-docker
# xftp 传输 cri-dockerd-0.3.18.amd64.tgz 包到三台主机
[root@master ~]# tar -xf cri-dockerd-0.3.18.amd64.tgz
[root@master ~]# ls
anaconda-ks.cfg cri-dockerd cri-dockerd-0.3.18.amd64.tgz
[root@master ~]# cd cri-dockerd/
[root@master cri-dockerd]# ls
cri-dockerd
# 将文件复制到/usr/bin目录下
[root@master cri-dockerd]# cp cri-dockerd /usr/bin/
[root@master cri-dockerd]# ls
cri-dockerd
[root@master cri-dockerd]# ls /usr/bin/
[root@master cri-dockerd]# scp cri-dockerd n1:/usr/bin/
The authenticity of host 'n1 (192.168.86.12)' can't be established.
ED25519 key fingerprint is SHA256:ahc/chG5rVHeDdBFb5i/MzxZZ5UUF0j3TSa+rTiM+tE.
This host key is known by the following other names/addresses:
~/.ssh/known_hosts:1: node1
~/.ssh/known_hosts:4: node2
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'n1' (ED25519) to the list of known hosts.
cri-dockerd 100% 49MB 160.2MB/s 00:00
[root@master cri-dockerd]# scp cri-dockerd n2:/usr/bin/
The authenticity of host 'n2 (192.168.86.13)' can't be established.
ED25519 key fingerprint is SHA256:ahc/chG5rVHeDdBFb5i/MzxZZ5UUF0j3TSa+rTiM+tE.
This host key is known by the following other names/addresses:
~/.ssh/known_hosts:1: node1
~/.ssh/known_hosts:4: node2
~/.ssh/known_hosts:5: n1
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'n2' (ED25519) to the list of known hosts.
cri-dockerd
# cri-docker 配置服务文件
.service文件:
[root@master ~]# cat /usr/lib/systemd/system/cri-docker.service
[Unit]
Description=CRI Interface for Docker Application Container Engine
Documentation=https://docs.mirantis.com
After=network-online.target firewalld.service docker.service
Wants=network-online.target
Requires=cri-docker.socket
[Service]
Type=notify
#ExecStart=/usr/bin/cri-dockerd --network-plugin=cni --pod-infra-container-image=registry.aliyuncs.com/google_containers/pause:3.10
ExecStart=/usr/bin/cri-dockerd --container-runtime-endpoint fd:// --network-plugin=cni --pod-infra-container-image=registry.k8s.io/pause:3.10
ExecReload=/bin/kill -s HUP
TimeoutSec=0
RestartSec=2
Restart=always
StartLimitBurst=3
StartLimitInterval=60s
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
Delegate=yes
KillMode=process
[Install]
WantedBy=multi-user.target
#########
cat > /usr/lib/systemd/system/cri-docker.service <<EOF
[Unit]
Description=CRI Interface for Docker Application Container Engine
Documentation=https://docs.mirantis.com
After=network-online.target firewalld.service docker.service
Wants=network-online.target
Requires=cri-docker.socket
[Service]
Type=notify
#ExecStart=/usr/bin/cri-dockerd --network-plugin=cni --pod-infra-container-image=registry.aliyuncs.com/google_containers/pause:3.10
ExecStart=/usr/bin/cri-dockerd --container-runtime-endpoint fd:// --network-plugin=cni --pod-infra-container-image=registry.k8s.io/pause:3.10
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always
StartLimitBurst=3
StartLimitInterval=60s
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
Delegate=yes
KillMode=process
[Install]
WantedBy=multi-user.target
EOF
# 配置 cri-docker 套接字
[root@master ~]# cat /usr/lib/systemd/system/cri-docker.socket
[Unit]
Description=CRI Docker socket for the API
PartOf=cri-docker.service
[Socket]
ListenStream=%t/cri-dockerd.sock
SocketMode=0660 #模式=权限
SocketUser=root #用户
SocketGroup=docker #组
[Install]
WantedBy=sockets.targetcat
###########
[Unit]
Description=CRI Docker socket for the API
PartOf=cri-docker.service
[Socket]
ListenStream=%t/cri-dockerd.sock
SocketMode=0660
SocketUser=root
SocketGroup=docker
[Install]
WantedBy=sockets.target
# 启动 cri-docker 服务
systemctl daemon-reload
systemctl enable --now cri-docker
systemctl is-active cri-docker
先启动docker,再启动cri-docker
- 三台主机
[root@node1 ~]# systemctl daemon-reload
systemctl enable --now docker
[root@node1 ~]# [root@node1 ~]# systemctl enable --now cri-docker
Created symlink /etc/systemd/system/multi-user.target.wants/cri-docker.service → /usr/lib/systemd/system/cri-docker.service.
4.3 安装 Kubernetes
4.3.1 配置仓库(三台主机)
- 添加 kubernets 源
/etc/yum.repos.d/kubernetes.repo - 安装 kubelets
yum install -y kubelet kubeadm kubectl - 启动 kubelet 服务
systemctl enable --now cri-docker
systemctl enable --now kubelet - 验证安装
kubeadm version - 导入所需镜像,查看所需镜像
kubeadm config images list
主节点需要导入全部镜像,从节点只需导入 kube-proxy、coredns、pause 这三个镜像
添加 kubelets 源
[root@master ~]# cat <<EOF | tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.33/rpm/
enabled=1
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.33/rpm/repodata/repomd.xml.key
EOF
###########
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.33/rpm/
enabled=1
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.33/rpm/repodata/repomd.xml.key
[root@master ~]# ls /etc/yum.repos.d/
docker-ce.repo kubernetes.repo redhat.repo yum.repo
安装 kubelets、启动 kubelet 服务、验证安装
# 安装kubernetes
yum install -y kubelet kubeadm kubectl
# 启动kubernetes服务 (cri启动==三个节点都要启动、ikuai主机开启)
[root@node1 ~]# systemctl daemon-reload
[root@node1 ~]# systemctl enable --now cri-docker
Created symlink /etc/systemd/system/multi-user.target.wants/cri-docker.service → /usr/lib/systemd/system/cri-docker.service.
systemctl enable --now kubelet
[root@master yum.repos.d]# systemctl enable --now kubelet
Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service → /usr/lib/systemd/system/kubelet.service.
# 验证安装
kubeadm version
[root@master ~]# kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"33", EmulationMajor:"", EmulationMinor:"", MinCompatibilityMajor:"", MinCompatibilityMinor:"", GitVersion:"v1.33.2", GitCommit:"a57b6f7709f6c2722b92f07b8b4c48210a51fc40", GitTreeState:"clean", BuildDate:"2025-06-17T18:39:42Z", GoVersion:"go1.24.4", Compiler:"gc", Platform:"linux/amd64"}
- 导入所需镜像
主节点需要导入全部镜像,从节点只需导入 kube-proxy、coredns、pause 这三个镜像
calico-images-v3.30.2.tar
k8s-node-images-v1.33.2.tar
kubectl-proxy-1.0.tar
# xftp 传输 calico-images-v3.30.2.tar 包到三台主机
[root@master ~]# ls
anaconda-ks.cfg cri-dockerd cri-dockerd-0.3.18.amd64.tgz pause_3.10.tar.gz
[root@master ~]# docker load -i pause_3.10.tar.gz
d8bdedd33a4e: Loading layer 737.8kB/737.8kB
Loaded image: registry.k8s.io/pause:3.10
[root@master ~]# rm -rf pause_3.10.tar.gz cri-dockerd-0.3.18.amd64.tgz
[root@master ~]# ls
anaconda-ks.cfg cri-dockerd
[root@master ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
registry.k8s.io/pause 3.10 873ed7510279 13 months ago 736kB
# 查看所需镜像
kubeadm config images list
[root@master ~]# kubeadm config images list
registry.k8s.io/kube-apiserver:v1.33.2
registry.k8s.io/kube-controller-manager:v1.33.2
registry.k8s.io/kube-scheduler:v1.33.2
registry.k8s.io/kube-proxy:v1.33.2
registry.k8s.io/coredns/coredns:v1.12.0
registry.k8s.io/pause:3.10
registry.k8s.io/etcd:3.5.21-0
# xftp拖.tar包
主节点:
[root@master ~]# ls
anaconda-ks.cfg cri-dockerd k8s-images-v1.33.2.tar
[root@master ~]# docker load -i k8s-images-v1.33.2.tar
Loaded image: registry.k8s.io/pause:3.10
.........
- node:
从节点:
[root@node1 ~]# ls
calico-images-v3.30.2.tar cri-dockerd-0.3.18.amd64.tgz k8s-node-images-v1.33.2.tar
[root@node1 ~]# docker load -i k8s-images-v1.33.2.tar
Loaded image: registry.k8s.io/pause:3.10
........
4.3.2 初始化’主’节点(master)
官方链接:kubdernetes初始化
- master 主机上执行
# 完整命令:
kubeadm init --apiserver-advertise-address=192.168.86.11 \
--kubernetes-version=1.33.2 \
--service-cidr=10.10.0.0/12 \
--pod-network-cidr=10.244.0.0/16 \
--ignore-preflight-errors=all \
--cri-socket unix:///var/run/cri-dockerd.sock
[root@master ~]# kubeadm init --apiserver-advertise-address=192.168.86.11 \
--kubernetes-version=1.33.2 \
--service-cidr=10.10.0.0/12 \
--pod-network-cidr=10.244.0.0/16 \
--ignore-preflight-errors=all \
--cri-socket unix:///var/run/cri-dockerd.sock
[init] Using Kubernetes version: v1.33.2
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local master] and IPs [10.0.0.1 192.168.86.11]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost master] and IPs [192.168.86.11 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost master] and IPs [192.168.86.11 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "super-admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests"
[kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s
[kubelet-check] The kubelet is healthy after 1.001611324s
[control-plane-check] Waiting for healthy control plane components. This can take up to 4m0s
[control-plane-check] Checking kube-apiserver at https://192.168.86.11:6443/livez
[control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz
[control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez
[control-plane-check] kube-controller-manager is healthy after 2.144035s
[control-plane-check] kube-scheduler is healthy after 2.74327218s
[control-plane-check] kube-apiserver is healthy after 4.502604728s
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node master as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
[mark-control-plane] Marking the node master as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]
[bootstrap-token] Using token: kou526.7c9yr60c5zy0dwpd
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
# 复制以下两行,后续增加工作节点时用
kubeadm join 192.168.86.11:6443 --token kou526.7c9yr60c5zy0dwpd \
--discovery-token-ca-cert-hash sha256:1f7521e2cd7e02ab7a96b11ece60acc383367295f89de653fabb82aca68e35bd
- 命令解析如下:
#指定主服务器的地址是多少,也就是 master 主机的地址
--apiserver-advertise-address=192.168.86.11
#指定控制平面镜像的仓库地址(默认 registry.k8s.io),当前为阿里镜像仓库地址,也可指向自己的私有镜像仓库地址如my-registry.com/k8s
--image-repository
#指定 Kubernetes 版本(默认最新稳定版),固定集群版本,避免自动升级导致兼容性问题。
--kubernetes-version=1.33.0
#指定service资源的网络范围
--service-cidr=10.10.0.0/12:
#指定pos的网络范围
--pod-network-cidr=10.244.0.0/16
#跳过所有前置的错误检测
--ignore-preflight-errors=all
#指定 CRI(容器运行时)的 socket 路径,如果是containerd则为unix:///run/containerd/containerd.sock
--cri-socket unix:///var/run/cri-dockerd.sock
4.3.3 创建 .kube目录(master)
# 创建 .kube 目录
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config
[root@master ~]# mkdir -p $HOME/.kube
[root@master ~]# cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[root@master ~]# chown $(id -u):$(id -g) $HOME/.kube/config
kubectl get node #查看节点
4.3.4 增加工作节点(master)
# 从初始化命令的最后一行复制kubeadm join......
kubeadm join 192.168.86.11:6443 --token kou526.7c9yr60c5zy0dwpd \
--discovery-token-ca-cert-hash sha256:1f7521e2cd7e02ab7a96b11ece60acc383367295f89de653fabb82aca68e35bd \
# 初始化的最后一句 --cri-socket unix:///var/run/cri-dockerd.sock
# kubeadm token create --print-join-command 重新生成令牌(三小时时效)
- 命令解析:
#指定主节点IP和端口
kubeadm join 192.168.86.11:6443
#指定token,如果过期,需要执行 kubeadm token
--token ge53zg.14u86m07fuz4ywa7
# 命令重新申请
create --print-join-command
#指定证书
--discovery-token-ca-cert-hash
#指定 CRI(容器运行时)的 socket 路径
--cri-socket unix:///var/run/cri-dockerd.sock
4.3.5 查看集群(master)
集群三个节点都已经创建完成,但这三个节点都处于未就绪状态
- status 为 NotReady 是因为现在还没有网,后续安装 CNI
# 查看集群
[root@master ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
master NotReady control-plane 28m v1.33.2
node1 NotReady <none> 103s v1.33.2
node2 NotReady <none> 30s v1.33.2
#status为NotReady是因为现在还没有网,后续安装CNI
报错
- 打开 ikuai(相当于路由器,打开才能联网),才能 yum 下载
- swap
- 修改 /etc/fstab 文件,将包含 swap 内容的行注释
- 使用 free -m 来查看 swap 是否还在使用
- swapoff -a
- 启动 cri 必须有这个垫片才能使用 kubulet(三台节点都要启动)
- systemctl daemon-reload
- systemctl enable --now cri-docker
- 初始化的时候注意 IP 和 版本号
- kubeadm version
- 初始化过程报错:考虑 cri、kubulet 服务是否开启
- 增加工作节点时
- kubeadm join 如果报错,复制报错信息中的 --ignore-preflight-errors=all 忽略
4.4 部署网络插件
Kubernetes 集群如果要能够正常工作,所有容器都必须工作在一个扁平的网络空间中。接下就需要部署网络插件来实现这个扁平化网络空间,让 Kubernetes 集群可以正常工作
4.4.1 下载配置文件(主节点calico-v3.30.2.yaml)
可通过 xftp 传输 calico-v3.30.2.yaml
-
打开 Calico 安装说明文档:calico
-
点击 Manifest 下面的 Install Calico with Kubernetes API datastore, more than 50 nodes 链接

-
复制命令将配置文件下载到本地(或者直接从自己电脑托 .yaml文件到主机)
- 得到如下界面,根据提示:

- 下载命令:
curl https://raw.githubusercontent.com/projectcalico/calico/v3.30.2/manifests/calico-typha.yaml -o calico.yaml
`
- 得到如下界面,根据提示:
# 通过 xftp 传输calico-images-v3.30.2.tar、calico-v3.30.2.yaml
[root@master ~]# ls
anaconda-ks.cfg calico-images-v3.30.2.tar calico-v3.30.2.yaml cri-dockerd k8s-images-v1.33.2.tar
# 查看所需要的镜像文件
[root@master ~]# grep image* calico-v3.30.2.yaml
image: docker.io/calico/cni:v3.30.2
imagePullPolicy: IfNotPresent
image: docker.io/calico/cni:v3.30.2
imagePullPolicy: IfNotPresent
image: docker.io/calico/node:v3.30.2
imagePullPolicy: IfNotPresent
image: docker.io/calico/node:v3.30.2
imagePullPolicy: IfNotPresent
image: docker.io/calico/kube-controllers:v3.30.2
imagePullPolicy: IfNotPresent
- image: docker.io/calico/typha:v3.30.2
imagePullPolicy: IfNotPresent
# 三个节点都需要
[root@node1 ~]# docker load -i calico-images-v3.30.2.tar
4.4.2 修改配置文件
- 将 CALICO_IPV4POOL_IPIP 和 CALICO_IPV4POOL_VXLAN 都关闭,即使用 BGP 模式
- 将 calico.yaml 配置文件中 CALICO_IPV4POOL_CIDR 修改为初始集群时指定的 Pod 地址,即 --pod-network-cidr=10.244.0.0/16 参数所指定的地址。
[root@master ~]# vim calico-v3.30.2.yaml
3. /CALICO_IP查找,修改value值(BGP模式 Off Never Never)
4. CALICO_IPV4POOL_CIDR 修改为初始集群时指定的 Pod 地址,即 初始化主节点--pod-network-cidr=10.244.0.0/16 参数所指定的地址
# Enable IPIP
- name: CALICO_IPV4POOL_IPIP
value: "Off"
# Enable or Disable VXLAN on the default IP pool.
- name: CALICO_IPV4POOL_VXLAN
value: "Never"
# Enable or Disable VXLAN on the default IPv6 IP pool.
- name: CALICO_IPV6POOL_VXLAN
value: "Never"
# no effect. This should fall within `--cluster-cidr`.
- name: CALICO_IPV4POOL_CIDR
value: "10.224.0.0/16"
# Disable file logging so `kubectl logs` works.


4.4.3 上传镜像文件(calico-images-v3.30.2.tar)
由于 calico 镜像导入到集群节点中。三个节点都需要导入
- 通过 xftp 传输 calico-images-v3.30.2.tar/calico-v3.30.2.tar,执行命令导入
# 1. master01 节点
[root@master ~]# ls
anaconda-ks.cfg calico-images-v3.30.2.tar calico.yaml
# 2. node01 节点
[root@node1 ~]# ls
anaconda-ks.cfg calico-images-v3.30.2.tar
# 3. node02 节点
[root@node2 ~]# ls
anaconda-ks.cfg calico-images-v3.30.2.tar
4.4.4 导入镜像文件(docker load -i )
- master
[root@master ~]# docker load -i calico-images-v3.30.2.tar
73234a741a6f: Loading layer 3.694MB/3.694MB
ff353cb90f45: Loading layer 81.51MB/81.51MB
Loaded image: calico/typha:v3.30.2
0df84c117099: Loading layer 407.3MB/407.3MB
Loaded image: calico/node:v3.30.2
02d3dffb3ef1: Loading layer 117.8MB/117.8MB
Loaded image: calico/kube-controllers:v3.30.2
81a7ae79a3f2: Loading layer 158.7MB/158.7MB
5f70bf18a086: Loading layer 1.024kB/1.024kB
Loaded image: calico/cni:v3.30.2
[root@master ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
calico/typha v3.30.2 b3baa600c7ff 2 weeks ago 85.2MB
calico/node v3.30.2 cc52550d767f 2 weeks ago 405MB
calico/kube-controllers v3.30.2 761b294e2655 2 weeks ago 122MB
calico/cni v3.30.2 77a357d0d33e 2 weeks ago 162MB
registry.k8s.io/kube-apiserver v1.33.2 ee794efa53d8 3 weeks ago 102MB
registry.k8s.io/kube-proxy v1.33.2 661d404f36f0 3 weeks ago 97.9MB
registry.k8s.io/kube-scheduler v1.33.2 cfed1ff74892 3 weeks ago 73.4MB
registry.k8s.io/kube-controller-manager v1.33.2 ff4f56c76b82 3 weeks ago 94.6MB
registry.k8s.io/etcd 3.5.21-0 499038711c08 3 months ago 153MB
registry.k8s.io/coredns/coredns v1.12.0 1cf5f116067c 7 months ago 70.1MB
registry.k8s.io/pause 3.10 873ed7510279 13 months ago 736kB
[root@master ~]# ls
anaconda-ks.cfg calico-images-v3.30.2.tar calico-v3.30.2.yaml k8s-images-v1.33.2.tar
[root@master ~]# scp calico-images-v3.30.2.tar n1:~
calico-images-v3.30.2.tar 100% 733MB 155.4MB/s 00:04
[root@master ~]# scp calico-images-v3.30.2.tar n2:~
calico-images-v3.30.2.tar 100% 733MB 151.9MB/s 00:04
- node 节点
docker load -i calico-images-v3.30.2.tar
[root@node1 ~]# ls
calico-images-v3.30.2.tar cri-dockerd-0.3.18.amd64.tgz k8s-node-images-v1.33.2.tar
[root@node1 ~]# docker load -i calico-images-v3.30.2.tar
73234a741a6f: Loading layer 3.694MB/3.694MB
ff353cb90f45: Loading layer 81.51MB/81.51MB
Loaded image: calico/typha:v3.30.2
0df84c117099: Loading layer 407.3MB/407.3MB
Loaded image: calico/node:v3.30.2
02d3dffb3ef1: Loading layer 117.8MB/117.8MB
Loaded image: calico/kube-controllers:v3.30.2
81a7ae79a3f2: Loading layer 158.7MB/158.7MB
5f70bf18a086: Loading layer 1.024kB/1.024kB
Loaded image: calico/cni:v3.30.2
[root@node1 ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
calico/typha v3.30.2 b3baa600c7ff 2 weeks ago 85.2MB
calico/node v3.30.2 cc52550d767f 2 weeks ago 405MB
calico/kube-controllers v3.30.2 761b294e2655 2 weeks ago 122MB
calico/cni v3.30.2 77a357d0d33e 2 weeks ago 162MB
registry.k8s.io/pause 3.10 873ed7510279 13 months ago 736kB
[root@node2 ~]# ls
calico-images-v3.30.2.tar cri-dockerd-0.3.18.amd64.tgz k8s-node-images-v1.33.2.tar
[root@node2 ~]# docker load -i calico-images-v3.30.2.tar
73234a741a6f: Loading layer 3.694MB/3.694MB
ff353cb90f45: Loading layer 81.51MB/81.51MB
Loaded image: calico/typha:v3.30.2
0df84c117099: Loading layer 407.3MB/407.3MB
Loaded image: calico/node:v3.30.2
02d3dffb3ef1: Loading layer 117.8MB/117.8MB
Loaded image: calico/kube-controllers:v3.30.2
81a7ae79a3f2: Loading layer 158.7MB/158.7MB
5f70bf18a086: Loading layer 1.024kB/1.024kB
Loaded image: calico/cni:v3.30.2
[root@node2 ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
calico/typha v3.30.2 b3baa600c7ff 2 weeks ago 85.2MB
calico/node v3.30.2 cc52550d767f 2 weeks ago 405MB
calico/kube-controllers v3.30.2 761b294e2655 2 weeks ago 122MB
calico/cni v3.30.2 77a357d0d33e 2 weeks ago 162MB
registry.k8s.io/pause 3.10 873ed7510279 13 months ago 736kB
4.4.5 安装网络插件
如果k8s有问题,一般情况,可以先看k8s日志:
先用
kubectl get pods --all-namespaces -owide查看问题 pod,
然后用kubectl describe pod pod_name -n kube-system来查看日志
kubectl apply -f calico-v3.30.2.yaml
kubectl describe node node1
kubectl get node
kubectl get node -o wide
[root@master ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
master Ready control-plane 6h8m v1.33.2
node1 Ready <none> 5h40m v1.33.2
node2 Ready <none> 5h39m v1.33.2
# 看pod
kubectl get pods
kubectl get ns
kubectl get pod -n kube-system
kubectl get pod -o wide -n kube-system
[root@master ~]# kubectl apply -f calico-v3.30.2.yaml
poddisruptionbudget.policy/calico-kube-controllers created
poddisruptionbudget.policy/calico-typha created
serviceaccount/calico-kube-controllers created
serviceaccount/calico-node created
serviceaccount/calico-cni-plugin created
configmap/calico-config created
customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/bgpfilters.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/stagedglobalnetworkpolicies.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/stagedkubernetesnetworkpolicies.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/stagednetworkpolicies.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/tiers.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/adminnetworkpolicies.policy.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/baselineadminnetworkpolicies.policy.networking.k8s.io created
clusterrole.rbac.authorization.k8s.io/calico-kube-controllers created
clusterrole.rbac.authorization.k8s.io/calico-node created
clusterrole.rbac.authorization.k8s.io/calico-cni-plugin created
clusterrole.rbac.authorization.k8s.io/calico-tier-getter created
clusterrolebinding.rbac.authorization.k8s.io/calico-kube-controllers created
clusterrolebinding.rbac.authorization.k8s.io/calico-node created
clusterrolebinding.rbac.authorization.k8s.io/calico-cni-plugin created
clusterrolebinding.rbac.authorization.k8s.io/calico-tier-getter created
service/calico-typha created
daemonset.apps/calico-node created
deployment.apps/calico-kube-controllers created
deployment.apps/calico-typha created
[root@master ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
master Ready control-plane 6h8m v1.33.2
node1 Ready <none> 5h40m v1.33.2
node2 Ready <none> 5h39m v1.33.2
更多推荐
所有评论(0)