Flink作业.UnfulfillableSlotRequestException:无法满足插槽要求.请求资源配置文件(ResourceProfile {UNKNOWN})无法实现 [英] Flink job .UnfulfillableSlotRequestException: Could not fulfill slot req. Req resource profile (ResourceProfile{UNKNOWN}) is unfulfillable
问题描述
提交Flink作业
$ ./bin/flink run -m 10.0.2.4:6123 /streaming/mvn-flinkstreaming-scala/mvn-flinkstreaming-scala-1.0.jar
Stream processing!!!!!!!!!!!!!!!!!
org.apache.flink.streaming.api.datastream.DataStreamSink@40ef3420
------------------------------------------------------------
The program finished with the following exception:
org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException: No pooled slot available and request to ResourceManager for new slot failed
at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:607)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591)
... 31 more
Caused by: org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException: No pooled slot available and request to ResourceManager for new slot failed
... 29 more
Caused by: java.util.concurrent.CompletionException: org.apache.flink.runtime.resourcemanager.exceptions.ResourceManagerException: Could not fulfill slot request
org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: java.util.concurrent.ExecutionException: org.apache.flink.runtime.client.JobSubmissionException: Failed to submit JobGraph.
但是当我在UI中检查作业日志时,得到了另一个错误,
but when i checked the logs of the job in UI getting a different error,
Caused by: java.util.concurrent.CompletionException: org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException: No pooled slot available and request to ResourceManager for new slot failed
at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:607)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591)
... 31 more
Caused by: org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException: No pooled slot available and request to ResourceManager for new slot failed
... 29 more
Caused by: java.util.concurrent.CompletionException: org.apache.flink.runtime.resourcemanager.exceptions.ResourceManagerException: Could not fulfill slot request ea
我应该检查什么,我的配置参数如下: A)是-m ip_address:6123正确的选项或8081应该是端口... 配置.
What should I check, my configuration parameter are as below, A) is the -m ip_address:6123 right option or 8081 should be the port... config..
# Note this accounts for all memory usage within the TaskManager process, including JVM metaspace and other overhead.
taskmanager.memory.process.size: 1568m
# To exclude JVM metaspace and overhead, please, use total Flink memory size instead of 'taskmanager.memory.process.size'.
# It is not recommended to set both 'taskmanager.memory.process.size' and Flink memory.
#
# taskmanager.memory.flink.size: 1280m
# The number of task slots that each TaskManager offers. Each slot runs one parallel pipeline.
taskmanager.numberOfTaskSlots: 2
# The parallelism used for programs that did not specify and other parallelism.
parallelism.default: 2
集群开始,
$ bin/start-cluster.sh
Starting cluster.
Starting standalonesession daemon on host centos1.
Starting taskexecutor daemon on host centos2.
Starting taskexecutor daemon on host centos3.
能够在主节点中处理grep,
Able to process grep in master node,
]$ ps -ef | grep flink
root 12300 1 10 07:22 pts/0 00:00:05 java -Xms16384m -Xmx16384m -Dlog.file=/storage/flink-1.10.0/log/
找不到与任务管理器相关的查找过程, centos2〜] $ psg flink
Unable to find a process of fink related to task managers, centos2 ~]$ psg flink
这是正确的状态吗?
推荐答案
我在Flink-1.10.0
中也遇到了同样的问题.因此,请确保根据数据负载有足够的内存.
I have faced the same problem in Flink-1.10.0
. So please make sure you have enough memory according to the data load.
我得到的错误:
java.lang.OutOfMemoryError: Metaspace
- jobmanager.heap.size:1024m(默认)
- taskmanager.memory.flink.size:1280m(默认)
- taskmanager.memory.jvm-metaspace.size:256m(默认)
所以我根据数据负载增加了taskmanager.memory.jvm-metaspace.size
,它解决了我的问题.
So I have increased taskmanager.memory.jvm-metaspace.size
according to data load and it solved my issue.
有关更多详细信息,请单击此处.
For more details click here.
这篇关于Flink作业.UnfulfillableSlotRequestException:无法满足插槽要求.请求资源配置文件(ResourceProfile {UNKNOWN})无法实现的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!