我如何才能调试这个无法解释的OpenShift 3 Pod在提供Spring Boot应用程序时进入Crashloop？ [英] How can I debug this unexplained Openshift 3 pod going into crashloop when serving a spring boot app?
本文介绍了我如何才能调试这个无法解释的OpenShift 3 Pod在提供Spring Boot应用程序时进入Crashloop？的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！
问题描述
我正在尝试将一个Spring引导应用程序移到OpenShift 3集群上。我正在使用Maven Fabric 8插件来生成大部分OpenShift样板配置以及执行S2I构建。当我的Pod启动时，我可以在日志输出中看到应用程序启动了，但就在SpringBoot默认为默认配置文件(我还没有设置配置文件)之后，应用程序崩溃，我在OpenShift日志中看到的唯一输出被终止……我在谷歌上找不到任何有价值的东西，但似乎是OpenJDK试图获取比单个Pod可用的内存更多的内存。我添加了一个Fabric8片段，它限制了单个容器能够使用的内存，但在启动Pod时仍然得到相同的错误。我运行了OC Description Pod，看到了出口代码143。我的想法已经用完了，对于如何进一步调试这个问题/如何解决这类问题，我将不胜感激。我也不确定这是否相关，但即使我的Applation.yml设置为启用SSL，Fabric8创建的路由始终是一个HTTP URL，而不是HTTPS URL。我想知道这是否可能是导致Pod进入崩溃循环的原因，因为ReadinessProbe和well nessProbe无法命中执行器端点？
控制台输出、maven pom文件、Applation.yml和Fabrice8配置如下。
console output:
. ____ _ __ _ _
/\ / ___'_ __ _ _(_)_ __ __ _
( ( )\___ | '_ | '_| | '_ / _` |
\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.1.3.RELEASE)
2020-06-27 10:14:06.297 INFO 8 --- [ main] com.xib.vodacom.ans.AnsApplicationKt : Starting AnsApplicationKt v1.0.9 on vc-ans-1-bzlxc-debug with PID 8 (/deployments/vc-ans-1.0.9.jar started by ? in /deployments)
2020-06-27 10:14:06.396 DEBUG 8 --- [ main] com.xib.vodacom.ans.AnsApplicationKt : Running with Spring Boot v2.1.3.RELEASE, Spring v5.1.4.RELEASE
2020-06-27 10:14:06.397 INFO 8 --- [ main] com.xib.vodacom.ans.AnsApplicationKt : No active profile set, falling back to default profiles: default
Killed
sh-4.2$
oc describe pod output:
Secret (a volume populated by a Secret)
SecretName: certs
Optional: false
default-token-8pts4:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-8pts4
Optional: false
QoS Class: Burstable
Node-Selectors: node-role.kubernetes.io/compute=true
Tolerations: node.kubernetes.io/memory-pressure:NoSchedule
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 6m default-scheduler Successfully assigned cx-ans-prod/vc-ans-1-bzlxc to pocpap8zadsas.cc.corp
Normal Created 5m (x3 over 6m) kubelet, asdf.cc.corp Created container
Normal Started 5m (x3 over 6m) kubelet, asdfgf.cc.corp Started container
Normal Killing 5m (x2 over 6m) kubelet, asdfgf.cc.corp Killing container with id docker://spring-boot:Container failed liveness probe.. Container will be killed and recreated.
Warning Unhealthy 5m (x7 over 6m) kubelet, asdfgf.cc.corp Readiness probe failed: Get https://192.168.31.110:8080/actuator/health: dial tcp 192.168.31.110:8080: connect: connection refused
Warning Unhealthy 5m (x7 over 6m) kubelet, asdfgf.cc.corp Liveness probe failed: Get https://192.168.31.110:8080/actuator/health: dial tcp 192.168.31.110:8080: connect: connection refused
Normal Pulled 1m (x7 over 6m) kubelet, asdfgf.cc.corp Container image "docker-registry.default.svc:5000/cx-ans-prod/vc-ans@sha256:4f39366bddc1e4ce7ed3c7e320453fbe6e90c4100f860a82ec87b04e7fb7e5b1" already present on machine
C:devxib-ans-bo>
pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<modelVersion>4.0.0</modelVersion>
<groupId>com.xib</groupId>
<artifactId>vc-ans</artifactId>
<version>1.0.9</version>
<packaging>jar</packaging>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.3.RELEASE</version>
</parent>
<scm>
<connection>scm:git:git://pgitap1zatcrh.vodacom.corp:XIB/xib-ans-bo.git</connection>
<url>git://pgitap1zatcrh.vodacom.corp:XIB/xib-ans-bo.git</url>
</scm>
<properties>
<java.version>1.8</java.version>
<kotlin.version>1.3.61</kotlin.version>
<fabric8.mode>openshift</fabric8.mode>
<fabric8.namespace>cx-ans-prod</fabric8.namespace>
</properties>
<dependencies>
<dependency>
<groupId>commons-lang</groupId>
<artifactId>commons-lang</artifactId>
<version>2.3</version>
<scope>compile</scope>
<exclusions>
<exclusion>
<artifactId>tomcat-annotations-api</artifactId>
<groupId>org.apache.tomcat</groupId>
</exclusion>
<exclusion>
<artifactId>commons-logging</artifactId>
<groupId>commons-logging</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
<scope>compile</scope>
<exclusions>
<exclusion>
<artifactId>tomcat-annotations-api</artifactId>
<groupId>org.apache.tomcat</groupId>
</exclusion>
<exclusion>
<artifactId>commons-logging</artifactId>
<groupId>commons-logging</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
<scope>compile</scope>
<exclusions>
<exclusion>
<artifactId>tomcat-annotations-api</artifactId>
<groupId>org.apache.tomcat</groupId>
</exclusion>
<exclusion>
<artifactId>commons-logging</artifactId>
<groupId>commons-logging</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<scope>compile</scope>
<exclusions>
<exclusion>
<artifactId>tomcat-annotations-api</artifactId>
<groupId>org.apache.tomcat</groupId>
</exclusion>
<exclusion>
<artifactId>commons-logging</artifactId>
<groupId>commons-logging</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-webflux</artifactId>
<scope>compile</scope>
<exclusions>
<exclusion>
<artifactId>tomcat-annotations-api</artifactId>
<groupId>org.apache.tomcat</groupId>
</exclusion>
<exclusion>
<artifactId>commons-logging</artifactId>
<groupId>commons-logging</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
<scope>compile</scope>
<exclusions>
<exclusion>
<artifactId>tomcat-annotations-api</artifactId>
<groupId>org.apache.tomcat</groupId>
</exclusion>
<exclusion>
<artifactId>commons-logging</artifactId>
<groupId>commons-logging</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-data</artifactId>
<scope>compile</scope>
<exclusions>
<exclusion>
<artifactId>tomcat-annotations-api</artifactId>
<groupId>org.apache.tomcat</groupId>
</exclusion>
<exclusion>
<artifactId>commons-logging</artifactId>
<groupId>commons-logging</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-test</artifactId>
<scope>compile</scope>
<exclusions>
<exclusion>
<artifactId>tomcat-annotations-api</artifactId>
<groupId>org.apache.tomcat</groupId>
</exclusion>
<exclusion>
<artifactId>commons-logging</artifactId>
<groupId>commons-logging</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.auth0</groupId>
<artifactId>java-jwt</artifactId>
<version>3.8.1</version>
<scope>compile</scope>
<exclusions>
<exclusion>
<artifactId>tomcat-annotations-api</artifactId>
<groupId>org.apache.tomcat</groupId>
</exclusion>
<exclusion>
<artifactId>commons-logging</artifactId>
<groupId>commons-logging</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-commons</artifactId>
<scope>compile</scope>
<exclusions>
<exclusion>
<artifactId>tomcat-annotations-api</artifactId>
<groupId>org.apache.tomcat</groupId>
</exclusion>
<exclusion>
<artifactId>commons-logging</artifactId>
<groupId>commons-logging</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.module</groupId>
<artifactId>jackson-module-kotlin</artifactId>
<scope>compile</scope>
<exclusions>
<exclusion>
<artifactId>tomcat-annotations-api</artifactId>
<groupId>org.apache.tomcat</groupId>
</exclusion>
<exclusion>
<artifactId>commons-logging</artifactId>
<groupId>commons-logging</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>25.1-jre</version>
<scope>compile</scope>
<exclusions>
<exclusion>
<artifactId>tomcat-annotations-api</artifactId>
<groupId>org.apache.tomcat</groupId>
</exclusion>
<exclusion>
<artifactId>commons-logging</artifactId>
<groupId>commons-logging</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.liquibase</groupId>
<artifactId>liquibase-core</artifactId>
<scope>compile</scope>
<exclusions>
<exclusion>
<artifactId>tomcat-annotations-api</artifactId>
<groupId>org.apache.tomcat</groupId>
</exclusion>
<exclusion>
<artifactId>commons-logging</artifactId>
<groupId>commons-logging</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-stdlib-jdk8</artifactId>
<version>1.3.21</version>
<scope>compile</scope>
<exclusions>
<exclusion>
<artifactId>tomcat-annotations-api</artifactId>
<groupId>org.apache.tomcat</groupId>
</exclusion>
<exclusion>
<artifactId>commons-logging</artifactId>
<groupId>commons-logging</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-reflect</artifactId>
<version>1.3.21</version>
<scope>compile</scope>
<exclusions>
<exclusion>
<artifactId>tomcat-annotations-api</artifactId>
<groupId>org.apache.tomcat</groupId>
</exclusion>
<exclusion>
<artifactId>commons-logging</artifactId>
<groupId>commons-logging</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.6</version>
<scope>compile</scope>
<exclusions>
<exclusion>
<artifactId>tomcat-annotations-api</artifactId>
<groupId>org.apache.tomcat</groupId>
</exclusion>
<exclusion>
<artifactId>commons-logging</artifactId>
<groupId>commons-logging</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<artifactId>ojdbc8</artifactId>
<groupId>com.oracle.ojdbc</groupId>
<version>19.3.0.0</version>
<scope>compile</scope>
<exclusions>
<exclusion>
<artifactId>tomcat-annotations-api</artifactId>
<groupId>org.apache.tomcat</groupId>
</exclusion>
<exclusion>
<artifactId>commons-logging</artifactId>
<groupId>commons-logging</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.jetbrains.kotlinx</groupId>
<artifactId>kotlinx-coroutines-core</artifactId>
<version>1.0.0</version>
<scope>compile</scope>
<exclusions>
<exclusion>
<artifactId>tomcat-annotations-api</artifactId>
<groupId>org.apache.tomcat</groupId>
</exclusion>
<exclusion>
<artifactId>commons-logging</artifactId>
<groupId>commons-logging</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<artifactId>commons-logging</artifactId>
<groupId>*</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.restdocs</groupId>
<artifactId>spring-restdocs-core</artifactId>
<version>2.0.3.RELEASE</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.restdocs</groupId>
<artifactId>spring-restdocs-mockmvc</artifactId>
<version>2.0.3.RELEASE</version>
<scope>test</scope>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-stdlib-jdk8</artifactId>
<version>1.3.21</version>
</dependency>
<dependency>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-reflect</artifactId>
<version>1.3.21</version>
</dependency>
<dependency>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-stdlib-jre7</artifactId>
<version>1.3.21</version>
</dependency>
<dependency>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-stdlib-jre8</artifactId>
<version>1.3.21</version>
</dependency>
<dependency>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-stdlib-jdk7</artifactId>
<version>1.3.21</version>
</dependency>
<dependency>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-stdlib</artifactId>
<version>1.3.21</version>
</dependency>
<dependency>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-runtime</artifactId>
<version>1.3.21</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-dependencies</artifactId>
<version>2.1.2.RELEASE</version>
<scope>import</scope>
<type>pom</type>
</dependency>
<dependency>
<groupId>com.auth0</groupId>
<artifactId>java-jwt</artifactId>
<version>3.8.1</version>
<scope>compile</scope>
<type>pom</type>
</dependency>
</dependencies>
</dependencyManagement>
<build>
<sourceDirectory>${project.basedir}/src/main/kotlin</sourceDirectory>
<testSourceDirectory>${project.basedir}/src/test/kotlin</testSourceDirectory>
<plugins>
<plugin>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-maven-plugin</artifactId>
<configuration>
<args>
<arg>-Xjsr305=strict</arg>
</args>
<compilerPlugins>
<plugin>spring</plugin>
</compilerPlugins>
</configuration>
<dependencies>
<dependency>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-maven-allopen</artifactId>
<version>${kotlin.version}</version>
</dependency>
</dependencies>
</plugin>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<executable>true</executable>
</configuration>
</plugin>
<plugin>
<groupId>io.fabric8</groupId>
<artifactId>fabric8-maven-plugin</artifactId>
<version>4.4.1</version>
<configuration>
<generateRoute>true</generateRoute>
<enricher>
<config>
<fmp-service>
<name>ans-service</name>
<type>ClusterIP</type>
</fmp-service>
</config>
</enricher>
<enricher>
<config>
<fmp-maven-scm-enricher>>
<scm-tag>https</scm-tag>
</fmp-maven-scm-enricher>>
</config>
</enricher>
</configuration>
<executions>
<execution>
<goals>
<goal>resource</goal>
<goal>build</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
fabric8 deployment.yml fragment:
spec:
template:
spec:
containers:
- env:
- name: SPRING_PROFILES_ACTIVE
value: qa
volumeMounts:
- name: certs
mountPath: /opt/certs
readOnly: true
livenessProbe:
httpGet:
path: /actuator/health
port: 8080
scheme: HTTPS
initialDelaySeconds: 30
readinessProbe:
httpGet:
path: /actuator/health
port: 8080
scheme: HTTPS
initialDelaySeconds: 30
resources:
requests:
memory: "64Mi"
limits:
memory: "256Mi"
env:
- name: JVM_OPTS
value: "-Xms64M -Xmx256M"
volumes:
- name: certs
secret:
secretName: certs
items:
- key: ans.pfx
path: ans.pfx
- key: vb_pp_client_ans.jks
path: vb_pp_client_ans.jks
- key: vc_truststore.jks
path: vc_truststore.jks
spring boot application.yml:
app:
datasource:
driver-class-name: oracle.jdbc.driver.OracleDriver
url: jdbc:oracle:thin:@qgeno.cc.corp
username: asdfgf
password: asdfgf
connection-test-query: SELECT 1 from dual
pool-name: ANS_CP
minimum-idle: 3
maximum-pool-size: 20
alive:
file: /opt/apps/xib/vc-ans/running.txt
spring:
main:
allow-bean-definition-overriding: true
vb:
vb-url: "https://asdfgf.cc.corp"
hc-check-msisdn: "0821341111"
ssl-cert-password: "asdfgf"
ssl-cert-alias: "asdfgf"
keystore-path: "/opt/certs/vb_pp_client_ans.jks"
repeat-caller-url: https://asdfgf.cc.corp
server:
http:
port: 8080
ssl:
enabled: true
protocol: TLS
key-alias: lkjhj
key-store: /opt/certs/ans.pfx
key-store-password: asdfgf
trust-store: /opt/certs/vc_truststore.jks
trust-store-password: asdfgf
logging:
level:
com.xib: DEBUG
management:
server:
ssl:
key-store: /opt/certs/ans.pfx
key-store-password: asdfgf
endpoint:
health:
show-details: always
推荐答案
第143号错误码为SIGTERM，kubernetes用来关闭实例。这意味着库伯内斯想要杀死你的吊舱是有原因的。我认为有3个选项可能是这样的：
1：如您所说，如果内存分配大于指定的内存限制，则会触发。在这种情况下，Kubernetes只会杀死您的Pod并保持节点稳定，但如果是CPU限制，则Kubernetes可以限制资源分配，而不会杀死Pod。您可以通过运行以下命令检查资源限制是否低于您的应用程序要求：oc get deployment deployment_name -o yaml
2：正如@Arghya Sadhu所说，您可能以某种方式错误地配置了执行器，尝试检查是否正确设置了活动和就绪探头并给出了正确的输出。您也可以通过上面的命令获取此配置。
3：一切都运行得很好，除了你的应用程序需要花费太多时间来准备，Kubernetes认为你的应用程序行为不端。如果您将资源限制设置得太窄，并且您的Java应用程序的资源太少，无法及时做好准备，则可能会出现问题。
这篇关于我如何才能调试这个无法解释的OpenShift 3 Pod在提供Spring Boot应用程序时进入Crashloop？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
查看全文
