1. 项目概述
最近在升级Kubernetes集群时遇到了containerd版本兼容性问题,正好借这个机会记录下基于containerd v2.21和Kubernetes 1.31部署Nginx的完整过程。这个组合是目前生产环境比较推荐的稳定搭配,相比Docker方案更轻量,资源占用更少。
提示:如果你在kubeadm init时遇到"failed to create new cri runtime service"这类错误,大概率是containerd配置有问题,后文会详细说明解决方案。
2. 环境准备与依赖安装
2.1 系统要求
推荐使用Ubuntu 22.04 LTS或CentOS 8以上版本,内核版本不低于5.4。我的测试环境配置如下:
- 操作系统: Ubuntu 22.04.3 LTS
- 节点配置: 2核CPU/4GB内存/50GB磁盘
- 网络环境: 各节点间网络互通,关闭防火墙或配置好相应规则
2.2 安装containerd v2.21
首先卸载旧版本containerd(如果已安装):
sudo apt-get remove -y containerd docker.io然后安装官方最新版:
wget https://github.com/containerd/containerd/releases/download/v2.21.0/containerd-2.21.0-linux-amd64.tar.gz sudo tar Cxzvf /usr/local containerd-2.21.0-linux-amd64.tar.gz配置systemd服务:
sudo mkdir -p /usr/local/lib/systemd/system sudo curl -o /usr/local/lib/systemd/system/containerd.service https://raw.githubusercontent.com/containerd/containerd/main/containerd.service sudo systemctl daemon-reload sudo systemctl enable --now containerd2.3 安装Kubernetes 1.31组件
添加Kubernetes仓库:
sudo apt-get update && sudo apt-get install -y apt-transport-https curl curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list安装指定版本:
sudo apt-get update sudo apt-get install -y kubelet=1.31.0-00 kubeadm=1.31.0-00 kubectl=1.31.0-00 sudo apt-mark hold kubelet kubeadm kubectl3. Kubernetes集群初始化
3.1 containerd配置调整
编辑/etc/containerd/config.toml:
[plugins."io.containerd.grpc.v1.cri"] sandbox_image = "registry.k8s.io/pause:3.9" [plugins."io.containerd.grpc.v1.cri".containerd] snapshotter = "overlayfs" [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc] runtime_type = "io.containerd.runc.v2"重启服务使配置生效:
sudo systemctl restart containerd3.2 初始化控制平面
使用kubeadm初始化集群:
sudo kubeadm init \ --pod-network-cidr=10.244.0.0/16 \ --cri-socket=unix:///var/run/containerd/containerd.sock \ --kubernetes-version=v1.31.0配置kubectl:
mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config3.3 安装网络插件
这里选择Calico:
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.26.1/manifests/tigera-operator.yaml kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.26.1/manifests/custom-resources.yaml验证节点状态:
kubectl get nodes -o wide4. Nginx部署实战
4.1 创建基础部署
编写nginx-deployment.yaml:
apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.25.3 ports: - containerPort: 80应用配置:
kubectl apply -f nginx-deployment.yaml4.2 暴露服务
创建NodePort服务:
apiVersion: v1 kind: Service metadata: name: nginx-service spec: type: NodePort selector: app: nginx ports: - protocol: TCP port: 80 targetPort: 80 nodePort: 30080验证服务:
kubectl get svc nginx-service curl http://<node-ip>:300805. 常见问题排查
5.1 CRI连接问题
如果遇到"failed to create new cri runtime service"错误,检查:
- containerd.sock权限是否正确(通常应为root:root 660)
- containerd服务是否正常运行
- /etc/containerd/config.toml配置是否正确
5.2 镜像拉取失败
解决方法:
sudo crictl pull nginx:1.25.3或者配置国内镜像源:
[plugins."io.containerd.grpc.v1.cri".registry.mirrors] [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"] endpoint = ["https://registry-1.docker.io"]5.3 网络不通问题
排查步骤:
- 检查Calico Pod是否正常运行
- 验证节点间网络连通性
- 检查iptables/nftables规则
6. 性能优化建议
6.1 资源限制配置
在Deployment中添加资源限制:
resources: limits: cpu: "1" memory: "512Mi" requests: cpu: "0.5" memory: "256Mi"6.2 启用HPA
创建HorizontalPodAutoscaler:
apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: nginx-hpa spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: nginx-deployment minReplicas: 2 maxReplicas: 10 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 506.3 日志收集方案
建议配置:
kubectl apply -f https://raw.githubusercontent.com/elastic/beats/master/deploy/kubernetes/filebeat-kubernetes.yaml7. 安全加固措施
7.1 Pod安全策略
创建SecurityContext:
securityContext: runAsNonRoot: true runAsUser: 1000 capabilities: drop: - ALL7.2 网络策略
限制Nginx Pod的网络访问:
apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: nginx-policy spec: podSelector: matchLabels: app: nginx policyTypes: - Ingress - Egress ingress: - ports: - protocol: TCP port: 808. 监控与告警
8.1 Prometheus监控
安装Prometheus Operator:
kubectl apply -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/main/bundle.yaml创建ServiceMonitor:
apiVersion: monitoring.coreos.com/v1 kind: ServiceMonitor metadata: name: nginx-monitor spec: selector: matchLabels: app: nginx endpoints: - port: web interval: 30s8.2 Grafana仪表板
导入Nginx监控看板(ID: 9614):
kubectl apply -f https://raw.githubusercontent.com/grafana/grafana/main/deploy/kubernetes/helm/grafana/templates/dashboards-json-configmap.yaml9. 版本升级策略
9.1 containerd升级
推荐使用滚动升级:
sudo apt-get update sudo apt-get install containerd=2.21.0-00 sudo systemctl restart containerd9.2 Kubernetes升级
使用kubeadm升级:
sudo apt-get update sudo apt-get install kubeadm=1.31.0-00 sudo kubeadm upgrade plan sudo kubeadm upgrade apply v1.31.010. 生产环境建议
- 至少部署3个Master节点保证高可用
- 使用外部etcd集群提高稳定性
- 配置定期备份(使用velero等工具)
- 启用审计日志功能
- 考虑使用Ingress Controller替代NodePort
重要:在正式上线前,建议在测试环境充分验证所有配置,特别是网络策略和资源限制部分,避免影响生产环境稳定性。