Spring Cloud 数据流 Pod 清理 [英] Spring Cloud Data Flow Pod Cleanup
问题描述
我们反复看到日志中出现资源配额限制问题，并且任务作业在 Kubernetes 上运行的 SCDF 上失败.问题是，running"中有这么多 Pod即使在他们完成后的状态.我了解，SCDF 不会删除 Pod，清理工作是开发人员的责任.
We are repeatedly seeing resource quota limitation issues in logs and Task jobs fail on the SCDF running on Kubernetes. Problem is, there are so many pods in "running" status even after they completed. I understand, SCDF does not delete the pods and it is developer's responsibility to cleanup.
即使我从 SCDF 仪表板 UI 运行任务执行清理，它也只会清理执行日志和任务表单 UI，但该任务创建的 Pod 仍然保留.这是预期的吗?任务执行清理不应该也删除 pods 吗?我们使用的是 Spring-Cloud-Dataflow-Server 2.4.2 Release.
Even when I run the Task Execution Cleanup from SCDF dashboard UI, it only cleans up the execution logs and task form UI but the pods created by that task still remain. Is this expected ? Shouldn't Task Execution Cleanup also delete the pods ? We are using Spring-Cloud-Dataflow-Server 2.4.2 Release.
有没有办法在执行完成后立即清理 Pod?这里有什么最佳做法吗?
Is there a way to cleanup the pods right after the execution is complete ? Any best practices here ?
推荐答案
Method - 1
您可以使用 spring-cloud-dataflow 提供的 restful api 清理任务执行.
You can clean up task executions by using restful api provided by spring-cloud-dataflow.
请求结构
DELETE /tasks/executions/{ids}?action=CLEANUP,REMOVE_DATA HTTP/1.1
Host: localhost:9393
触发 DELETE 请求.
http://<stream-url>/tasks/executions/<task-ids-seperated-by-coma>?action=CLEANUP,REMOVE_DATA
eg: http://localhost:9393/tasks/executions/1,2?action=CLEANUP,REMOVE_DATA
注意:上述api将清理用于部署任务的资源，并从底层持久性存储中删除与任务执行相关的数据.
Note: Above api will clean up resources that were used to deploy tasks and delete the data associated with task executions from the underlying persistence store.
CLEANUP:清理资源
REMOVE_DATA:从持久性存储中删除数据.
REMOVE_DATA : remove data from persistence store.
您可以传递两个操作或单个操作，具体取决于您的用例.
You can either pass both actions or single action depends on your use-case.
方法 - 2
使用 Spring Cloud 数据流外壳.
Using spring cloud dataflow shell.
进入 spring-cloud-dataflow shell 并执行以下命令.
Enter into the spring-cloud-dataflow shell and execute below command.
task execution cleanup --id <task-id>
eg: task execution cleanup --id 1
清理任务执行来自spring-cloud-dataflow shell
其他选项(仅适用于 kubernetes 平台)
如果您不想删除所有已完成的 Pod，则可以使用 kubectl 工具进行删除.
If you wan't to delete all completed pods then you can delete using kubectl tool.
kubectl delete pod --field-selector=status.phase==Succeeded -l role=spring-app -n <namespace-where-tasks-launched>
如果您不想删除所有处于 Error 状态的 Pod，请执行以下命令
If you wan't to delete all pods with Error status then execute below command
kubectl delete pod --field-selector=status.phase==Failed -l role=spring-app -n <namespace-where-tasks-launched>
这篇关于Spring Cloud 数据流 Pod 清理的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
