kubernetes pod 卡在容器创建中 [英] kubernetes pods stuck at containercreating
问题描述
我有一个树莓派集群(一个主节点，三个节点)
I have a raspberry pi cluster (one master , 3 nodes)
我的基本形象是:raspbian stretch lite
My basic image is : raspbian stretch lite
我已经设置了一个基本的 kubernetes 设置，其中主节点可以看到他的所有节点(kubectl get 节点)并且它们都在运行.我使用了一个编织网络插件进行网络通信
I already set up a basic kubernetes setup where a master can see all his nodes (kubectl get nodes) and they're all running. I used a weave network plugin for the network communication
当一切都设置好后，我尝试在我的集群上运行一个 nginx pod(首先有一些副本，但现在只有 1 个 pod)，如下所示kubectl run my-nginx --image=nginx
When everything is all setup i tried to run a nginx pod (first with some replica's but now just 1 pod) on my cluster as followed kubectl run my-nginx --image=nginx
但不知何故，pod 卡在容器创建"状态，当我运行 docker 镜像时，我看不到正在拉取的 nginx 镜像.通常一个 nginx 镜像不是那么大，所以现在必须已经拉取(15 分钟).kubectl describe pods 提示pod 沙箱创建失败，kubernetes 会重新创建.
But somehow the pod get stuck in the status "Container creating" , when i run docker images i can't see the nginx image being pulled. And normally an nginx image is not that large so it had to be pulled already by now (15 minutes). The kubectl describe pods give the error that the pod sandbox failed to create and kubernetes will rec-create it.
我搜索了有关此问题的所有内容并尝试了 stackoverflow 上的解决方案(重新启动以重新启动集群，搜索了 describe pods，新的网络插件使用 flannel 进行了尝试)，但我看不出实际问题是什么.我在 Virtual box 中做了完全相同的事情(只是 ubuntu 而不是 ARM)并且一切正常.
I searched everything about this issue and tried the solutions on stackoverflow (reboot to restart cluster, searched describe pods , new network plugin tried it with flannel) but i can't see what the actual problem is. I did the exact same thing in Virtual box (just ubuntu not ARM ) and everything worked.
首先我认为这是一个权限问题，因为我以普通用户身份运行所有内容，但在 vm 中我做了同样的事情并且没有任何改变.然后我检查了 kubectl get pods --all-namespaces 以验证 weaver 网络和 kube-dns 的 pod 是否正在运行，并且那里也没有任何问题.
First i thougt it was a permission issue because i run everything as a normal user , but in vm i did the same thing and nothing changed. Then i checked kubectl get pods --all-namespaces to verify that the pods for the weaver network and kube-dns are running and also nothing wrong over there .
这是 Raspberry pi 中的防火墙问题吗?weave 网络插件是否与 arm 设备不兼容(甚至 kubernetes 网站也说是)?我猜是 api 网络问题，这就是为什么我的 pod 无法在节点上运行
Is this a firewall issue in Raspberry pi ? Is the weave network plugin not compatible (even the kubernetes website says it is) with arm devices ? I 'am guessing there is an api network problem and thats why i can't get my pod runnning on a node
日志文件
kubectl 描述 podName
kubectl describe podName
>
> Name: my-nginx-9d5677d94-g44l6 Namespace: default Node: kubenode1/10.1.88.22 Start Time: Tue, 06 Mar 2018 08:24:13
> +0000 Labels: pod-template-hash=581233850
> run=my-nginx Annotations: <none> Status: Pending IP: Controlled By: ReplicaSet/my-nginx-9d5677d94 Containers:
> my-nginx:
> Container ID:
> Image: nginx
> Image ID:
> Port: 80/TCP
> State: Waiting
> Reason: ContainerCreating
> Ready: False
> Restart Count: 0
> Environment: <none>
> Mounts:
> /var/run/secrets/kubernetes.io/serviceaccount from default-token-phdv5 (ro) Conditions: Type Status
> Initialized True Ready False PodScheduled True
> Volumes: default-token-phdv5:
> Type: Secret (a volume populated by a Secret)
> SecretName: default-token-phdv5
> Optional: false QoS Class: BestEffort Node-Selectors: <none> Tolerations: node.kubernetes.io/not-ready:NoExecute for
> 300s
> node.kubernetes.io/unreachable:NoExecute for 300s Events: Type Reason Age From
> Message ---- ------ ---- ----
> ------- Normal Scheduled 5m default-scheduler Successfully assigned my-nginx-9d5677d94-g44l6 to kubenode1 Normal
> SuccessfulMountVolume 5m kubelet, kubenode1 MountVolume.SetUp
> succeeded for volume "default-token-phdv5" Warning
> FailedCreatePodSandBox 1m kubelet, kubenode1 Failed create pod
> sandbox. Normal SandboxChanged 1m kubelet, kubenode1
> Pod sandbox changed, it will be killed and re-created.
kubectl 记录 podName
kubectl logs podName
Error from server (BadRequest): container "my-nginx" in pod "my-nginx-9d5677d94-g44l6" is waiting to start: ContainerCreating
journalctl -u kubelet 给出这个错误
journalctl -u kubelet gives this error
Mar 12 13:42:45 kubeMaster kubelet[16379]: W0312 13:42:45.824314 16379 cni.go:171] Unable to update cni config: No networks found in /etc/cni/net.d
Mar 12 13:42:45 kubeMaster kubelet[16379]: E0312 13:42:45.824816 16379 kubelet.go:2104] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
问题似乎出在我的网络插件上.在我的/etc/systemd/system/kubelet.service.d/10.kubeadm.conf 中.网络插件的标志是否存在?环境= kubelet_network_args --cni-bin-dir=/etc/cni/net.d--network-plugin=cni
The problem seems to be with my network plugin. In my /etc/systemd/system/kubelet.service.d/10.kubeadm.conf . the flags for the network plugins are present ? environment= kubelet_network_args --cni-bin-dir=/etc/cni/net.d --network-plugin=cni
推荐答案
您可以通过查找试图拉取图像的节点来查看它是否与网络相关:
You can see if it's network related by finding the node trying to pull the image:
kubectl describe pod
SSH 到节点，并在其上运行 docker pull nginx.如果手动拉取图像时遇到问题，则可能与网络有关.
SSH to the node, and run docker pull nginx on it. If it's having issues pulling the image manually, then it might be network related.
这篇关于kubernetes pod 卡在容器创建中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
