一个詹金斯人，两个Kubernetes集群 [英] One Jenkins, Two Kubernetes Clusters

问题描述

我正在尝试使用多个Kubernetes云，并且只有一个Jenkins.我能够将两个Kubernetes集群都注册为云，并且所有登录检查均有效.当我在Jenkins所在的集群上运行构建时，它可以正常工作，提取代码，构建映像等等.

I'm trying to use multiple Kubernetes clouds and have just a single Jenkins. I was able to get both Kubernetes clusters registered as clouds and all the login checks work. When I run a build on the cluster where Jenkins is, it works fine, pulls the code, builds an image and all that.

但是，当我将标签更改为第二个集群时，那里没有任何Jenkins，似乎永远都不想在那里建立，而总是建立在Jenkins本地的集群上.

However, when I change the label to my second cluster, that doesn't have any Jenkins there seems to never want to build there and always builds on the cluster that's local to Jenkins.

我想必我缺少一些愚蠢的东西，但看不到它是什么.

I'm missing something stupid I'm sure of it but don't see what it is.

推荐答案

您可能会从另一个群集中丢失服务帐户令牌.如果您在Jenkins的Kubernetes配置上测试集群连接->管理詹金斯，您会得到什么结果.

You might be missing the service account token from the other cluster. If you test the cluster connection on the Kubernetes config in Jenkins -> Manage Jenkins, what result do you get.

您可能需要使用:

kubectl describe sa <service account>

然后您将看到一个令牌，然后使用以下命令复制该令牌:

You will then see a token then copy that token use this command:

kubectl describe secret <token>

然后您可以复制该长令牌并将其作为kubernetes令牌或仅作为令牌放入Jenkins凭证中.

You can then copy that long token and put it in Jenkins credentials as kubernetes token or just token.

如果您使用Jenkins进行云配置，它将为您提供成功":

If you go to the cloud config in Jenkins, it will then give you "Success":

Jenkins云配置结果示例

我正面临一个新问题，您可能很快也会遇到.我为此花费了将近2个工作周.

I'm sitting with a new issue that you might also face soon. I've spent about almost 2 working weeks on this.

有2种选择，到目前为止，我还不能给您确切的答案，但是我可以提供选择和解释.

There are 2 options, I can't give you a definitive answer as of yet, but I can give options and explanations.

上下文

我有2个名为FS和TC的Kubernetes集群. 我正在使用的Jenkins在TC上运行.

I have 2 kubernetes clusters called FS and TC. The Jenkins I am using runs on TC.

从服务器确实从TC Jenkins部署在FS中，但是FS中的从服务器无法连接到TC中的Jenkins主服务器.

The slaves do deploy in FS from the TC Jenkins, however the slaves in FS would not connect to the Jenkins master in TC.

从站利用需要HOST和PORT的TCP连接.但是，TC上公开的jnlp服务是HTTP(http:/jenkins-jnlp.tc.com/).

The slaves make use of a TCP connection that requires a HOST and PORT. However, the exposed jnlp service on TC is HTTP (http:/jenkins-jnlp.tc.com/).

即使我使用

• 主机:jenkins-jnlp.tc.com
• 端口:80

它仍然会抱怨它正在获取串行数据而不是二进制数据.

It will still complain that it's getting serial data instead of binary data.

投诉

对于TC，我通过端口(50000)使用了本地jnlp服务HOST(jenkins-jnlp.svc.cluster.local).这对于我们当前的TC环境非常有效.

For TC I made use of the local jnlp service HOST (jenkins-jnlp.svc.cluster.local) with PORT (50000). This works well for our current TC environment.

解决方案

解决方案1 ​​

一个可能的解决方案将涉及在FS的从服务器和主服务器之间运行HTTP到TCP中继容器.然后将其链接到TC中的HTTP URL(http:/jenkins-jnlp.tc.com/)，封装到TCP的HTTP连接(localhost:50000)，反之亦然.

A possible solution would involve having a HTTP to TCP relay container running between the slave and master on FS. It will then be linked up to the HTTP url in TC (http:/jenkins-jnlp.tc.com/), encapsulating the HTTP connection to TCP (localhost:50000) and vice versa.

然后，FS上的从服务器可以使用从中间容器中暴露的TCP端口连接到TC主服务器.

The slaves on FS can then connect to the TC master using that TCP port being exposed from that container in the middle.

图可以更好地理解

解决方案2

人们不断抱怨，最终有人在2020年2月20日左右对Jenkins进行了新功能.他们引入了可在HTTP上运行并将其转换为从站上的TCP的Websocket.

People kept complaining and eventually someone made a new functionality to Jenkins around 20 Feb 2020. They introduced Websockets that can run over HTTP and convert it to TCP on the slave.

我确实进行了设置，但是它似乎太新了，即使FS上的从站表示已连接，但它仍无法与我一起使用，但仍无法与TC上的Jenkins主站正确通信.仍然会看到代理/从机吊舱处于脱机状态.

I did set it up, but it seems too new and is not working for me even though the slave on FS says it's connected, it's still not properly communicating with the Jenkins master on TC. It still sees the agent/slave pod as offline.

这是我使用的链接

1. 原始帖子
2. 有关Jenkins的更新说明
3. 有关Jenkins WebSocket的详细信息
4. Jenkins入站代理github
5. DockerHub jenkins-inbound-agent

1. Original post
2. Update note on Jenkins
3. Details on Jenkins WebSocket
4. Jenkins inbound-agent github
5. DockerHub jenkins-inbound-agent

结论

经过大量的摆弄，研究和摸索，我认为唯一的解决方案是解决方案#1.解决方案1的问题不存在，无法将HTTP封装到TCP并返回的简单工具或服务(据我所知，我搜索了几天).这意味着，我得自己做一个.

After a lot of fiddling, research and banging my head on the wall, I think the only solution is solution #1. Problem with solution #1, a simple tool or service to encapsulate HTTP to TCP and back does not exist (that I know of, I searched for days). This means, I'll have to make one myself.

解决方案2仍然太新，从零到零的文档可以帮助我或使设置变得容易，并且似乎存在一些错误.似乎修复这些错误的唯一方法是同时修改Jenkins和jnlp代理的代码，我什至不知道从哪里开始.

Solution #2 is still too new, zero to none docs to help me out or make setting it up easy and seems to come with some bugs. It seems the only way to fix these bugs would be to modify both Jenkins and the jnlp agent's code, which I have no idea where to even start.

这篇关于一个詹金斯人，两个Kubernetes集群的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！