kubernetes容器丢失的响应 [英] Responses from kubernetes containers getting lost
问题描述
我有一个pod在UDP端口5060上托管SIP应用程序，并且已经创建了服务作为 NODEPORT on 5060。
规格：
spec：{
ports：[
{
port：5061，
协议：UDP，
targetPort：5060，
nodeport：5060，
name：sipu
}
]，
selector：{
app：opensips
}，
type：NodePort
}
IP
- 节点的公共IP： 192.168.144.29 。
- 节点的专用IP： 10.0.1.215。
- 容器的IP： 10.244.2.4 即可。
- docker0界面： 10.244.2.1 。
问题。
我向应用程序发送一个SIP请求，并期待一个200 OK响应，我没有得到。
为了跟踪相同的，我安装了TCPDUMP容器和节点。
在容器上，我可以看到在节点本身捕获的请求和响应数据包，我只看到请求数据包。不明白为什么数据包丢失了。
下面是容器的tcpdump：
tcpdump：listen on eth0，link-type EN10MB（Ethernet），capture size 1514 bytes
06：12：20.391171 IP（tos 0x0，ttl 64，id 13372 ，offset 0，flags [DF]，proto UDP（17），length 1010）
10.244.2.1.50867> 10.244.2.4.5060：[bad udp cksum 0x1ddc - > 0xe738！] SIP，长度：982
PUBLISH sip：service-1@opensipstest.org SIP / 2.0
Via：SIP / 2.0 / UDP 192.168.144.10:5060;branch=z9hG4bK-5672-1- 0
Max-Forwards：20
From：service< sip：service-1@opensipstest.org> ;; tag = 1
To：sut< sip：service-1 @ opensipstest。有机>
电话号码：1-5672@192.168.144.10
CSeq：1 PUBLISH
到期日：3600
事件：现在
内容长度：607
User-Agent：Sipp v1.1-TLS，version 20061124
<？xml version =1.0？>
<已删除的存在xml以减小大小>
06：12：20.401126 IP（tos 0x10，ttl 64，id 11888，offset 0，flags [DF]，proto UDP（17），length 427）
10.244.2.4.5060 > 10.244.2.1.5060：[bad udp cksum 0x1b95 - > 0xeddc！] SIP，长度：399
SIP / 2.0 200 OK
通过：SIP / 2.0 / UDP 192.168.144.10:5060;received=10.244.2.1;branch=z9hG4bK-5672-1-0
From：service< sip：service-1@opensipstest.org> ;; tag = 1
To：sut< sip：service-1@opensipstest.org> ;; tag = 332ff20b76febdd3c55f313f3efc6bb8-ca08
呼叫ID：1-5672@192.168.144.10
CSeq：1 PUBLISH
过期：3600
SIP-ETag：a.1450478491.39.2.0
服务器：OpenSIPS（1.8。 4-notls（x86_64 / linux））
内容长度：0
06：12：20.401364 IP（tos 0x0，ttl 64，id 13374，offset 0，flags [DF]，原UDP（17），长度427）
10.244.2.1.58836> 10.244.2.4.5060：[bad udp cksum 0x1b95 - > 0x1bcc！] SIP，长度：399
SIP / 2.0 200 OK
通过：SIP / 2.0 / UDP 192.168.144.10:5060;received=10.244.2.1;branch=z9hG4bK-5672-1-0
From：service< sip：service-1@opensipstest.org> ;; tag = 1
To：sut< sip：service-1@opensipstest.org> ;; tag = 332ff20b76febdd3c55f313f3efc6bb8-ca08
呼叫ID：1-5672@192.168.144.10
CSeq：1 PUBLISH
过期：3600
SIP-ETag：a.1450478491.39.2.0
服务器：OpenSIPS（1.8。 4-notls（x86_64 / linux））
内容长度：0
和tcpdump的节点：
tcpdump：listen on eth0，link-type EN10MB（以太网），捕获大小1514字节
06：12：20.390772 IP（tos 0x0，ttl 127，id 20196，offset 0，flags [none]，proto UDP（17），length 1010）
192.168.144.10.5060> 10.0.1.215.5060：[udp sum ok] SIP，length：982
PUBLISH sip：service-1@opensipstest.org SIP / 2.0
通过：SIP / 2.0 / UDP 192.168.144.10:5060; branch = z9hG4bK-5672-1-0
最大向前：20
From：service< sip：service-1@opensipstest.org> ;; tag = 1
To：sut< SIP：service-1@opensipstest.org>
电话号码：1-5672@192.168.144.10
CSeq：1 PUBLISH
到期日：3600
事件：现在
内容长度：607
User-Agent：Sipp v1.1-TLS，version 20061124
<？xml version =1.0？>
< presence xmlns =urn：ietf：params：xml：ns：pidf/>
来自Iptable的Nodeport规则
$ $ $ $ $ $ $ ud 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 destination destination destination destination
12 8622 REDIRECT udp - * * 0.0 .0.0 / 0 0.0.0.0/0 / * default / opensips：sipu * / udp dpt：5060 redir ports 40482
3 95 REDIRECT udp - * * 0.0.0.0/0 0.0.0.0/0 / *默认/ my-udp-service：* / udp dpt：6000 redir ports 47497
链KUBE-NODEPORT-HOST（1参考）
pkts bytes target prot opt out out source destination
0 0 DNAT udp - * * 0.0.0.0/0 0.0.0.0/0 / * default / opensips：sipu * / udp dpt：5060 to：10.0.1.215：40482
0 0 DNAT udp - * * 0.0.0.0/0 0.0.0.0/0 / * default / my-udp-service：* / udp dpt：6000 to：10.0.1.215：47497
如果需要，我很乐意分享更多信息。我试图挖掘我的能力，但现在我不知道该怎么看，因此请求一些帮助。
编辑
我在 TCP 上进行了同样的测试。在TCP上，它按预期工作。
谢谢
5060在默认服务节点端口范围之外：
http：/ /kubernetes.github.io/docs/user-guide/services/#type-nodeport
创建服务应该失败。
可以通过在kube-apiserver上指定--service-node-port-range来尝试其他端口，或者使用不同的范围创建集群。
您还应该检查是否可以从其他节点看到响应。
另外，当报告问题时，请指定Kubernetes版本。
I have installed kubernetes on openstack. The setup has one master and one node on coreos.
I have a pod hosting an SIP application on UDP port 5060 and I have created service as NODEPORT on 5060.
The spec:
"spec": {
"ports": [
{
"port": 5061,
"protocol": "UDP",
"targetPort": 5060,
"nodeport": 5060,
"name": "sipu"
}
],
"selector": {
"app": "opensips"
},
"type": "NodePort"
}
IPs
- Public IP of node: 192.168.144.29.
- Private IP of node: 10.0.1.215..
- IP of the container:10.244.2.4.
- docker0 interface: 10.244.2.1.
Now, the problem. I send a SIP request to the application and expect a 200 OK response, which I am not getting.
To trace the same, I installed TCPDUMP on the container and the node. On the container, I can see the request and response packet captured while on the node itself I just see the request packet. Don't understand why the packet is getting lost.
Below is tcpdump of container:
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 1514 bytes
06:12:20.391171 IP (tos 0x0, ttl 64, id 13372, offset 0, flags [DF], proto UDP (17), length 1010)
10.244.2.1.50867 > 10.244.2.4.5060: [bad udp cksum 0x1ddc -> 0xe738!] SIP, length: 982
PUBLISH sip:service-1@opensipstest.org SIP/2.0
Via: SIP/2.0/UDP 192.168.144.10:5060;branch=z9hG4bK-5672-1-0
Max-Forwards: 20
From: service <sip:service-1@opensipstest.org>;tag=1
To: sut <sip:service-1@opensipstest.org>
Call-ID: 1-5672@192.168.144.10
CSeq: 1 PUBLISH
Expires: 3600
Event: presence
Content-Length: 607
User-Agent: Sipp v1.1-TLS, version 20061124
<?xml version="1.0"?>
<deleted presence xml to reduce size>
06:12:20.401126 IP (tos 0x10, ttl 64, id 11888, offset 0, flags [DF], proto UDP (17), length 427)
10.244.2.4.5060 > 10.244.2.1.5060: [bad udp cksum 0x1b95 -> 0xeddc!] SIP, length: 399
SIP/2.0 200 OK
Via: SIP/2.0/UDP 192.168.144.10:5060;received=10.244.2.1;branch=z9hG4bK-5672-1-0
From: service <sip:service-1@opensipstest.org>;tag=1
To: sut <sip:service-1@opensipstest.org>;tag=332ff20b76febdd3c55f313f3efc6bb8-ca08
Call-ID: 1-5672@192.168.144.10
CSeq: 1 PUBLISH
Expires: 3600
SIP-ETag: a.1450478491.39.2.0
Server: OpenSIPS (1.8.4-notls (x86_64/linux))
Content-Length: 0
06:12:20.401364 IP (tos 0x0, ttl 64, id 13374, offset 0, flags [DF], proto UDP (17), length 427)
10.244.2.1.58836 > 10.244.2.4.5060: [bad udp cksum 0x1b95 -> 0x1bcc!] SIP, length: 399
SIP/2.0 200 OK
Via: SIP/2.0/UDP 192.168.144.10:5060;received=10.244.2.1;branch=z9hG4bK-5672-1-0
From: service <sip:service-1@opensipstest.org>;tag=1
To: sut <sip:service-1@opensipstest.org>;tag=332ff20b76febdd3c55f313f3efc6bb8-ca08
Call-ID: 1-5672@192.168.144.10
CSeq: 1 PUBLISH
Expires: 3600
SIP-ETag: a.1450478491.39.2.0
Server: OpenSIPS (1.8.4-notls (x86_64/linux))
Content-Length: 0
And tcpdump of node:
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 1514 bytes
06:12:20.390772 IP (tos 0x0, ttl 127, id 20196, offset 0, flags [none], proto UDP (17), length 1010)
192.168.144.10.5060 > 10.0.1.215.5060: [udp sum ok] SIP, length: 982
PUBLISH sip:service-1@opensipstest.org SIP/2.0
Via: SIP/2.0/UDP 192.168.144.10:5060;branch=z9hG4bK-5672-1-0
Max-Forwards: 20
From: service <sip:service-1@opensipstest.org>;tag=1
To: sut <sip:service-1@opensipstest.org>
Call-ID: 1-5672@192.168.144.10
CSeq: 1 PUBLISH
Expires: 3600
Event: presence
Content-Length: 607
User-Agent: Sipp v1.1-TLS, version 20061124
<?xml version="1.0"?>
<presence xmlns="urn:ietf:params:xml:ns:pidf" />
Nodeport rules from Iptable
Chain KUBE-NODEPORT-CONTAINER (1 references)
pkts bytes target prot opt in out source destination
12 8622 REDIRECT udp -- * * 0.0.0.0/0 0.0.0.0/0 /* default/opensips:sipu */ udp dpt:5060 redir ports 40482
3 95 REDIRECT udp -- * * 0.0.0.0/0 0.0.0.0/0 /* default/my-udp-service: */ udp dpt:6000 redir ports 47497
Chain KUBE-NODEPORT-HOST (1 references)
pkts bytes target prot opt in out source destination
0 0 DNAT udp -- * * 0.0.0.0/0 0.0.0.0/0 /* default/opensips:sipu */ udp dpt:5060 to:10.0.1.215:40482
0 0 DNAT udp -- * * 0.0.0.0/0 0.0.0.0/0 /* default/my-udp-service: */ udp dpt:6000 to:10.0.1.215:47497
I am happy to share more info if required. I have tried to dig in my capacity, but now I don't know what to look therefore requesting some help here.
EDIT
I did the same test on TCP. On TCP, it works as expected.
Thanks
5060 is outside the default Service Node Port Range: http://kubernetes.github.io/docs/user-guide/services/#type-nodeport
Creation of the service should have failed.
You can try a different port, or create a cluster using a different range, by specifying --service-node-port-range on kube-apiserver.
You should also check whether the response can be seen from other nodes. There are issues with "hairpin mode" when communicating within the same node.
Also, when reporting problems, please specify the Kubernetes release.
这篇关于kubernetes容器丢失的响应的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
