缓冲池被破坏.Apache Flink flapMap运算符中发现的问题 [英] "Buffer pool is destroyed" issue found in Apache Flink flapMap Operator

问题描述

当我尝试在flatMap运算符中写入OUT集合时，出现非法状态异常(仅在高负载下):缓冲池被破坏我在这里做什么错?当flink抛出缓冲池错误吗?

When I try to write to OUT collection in flatMap operator, I get illegal state exception(only under high load):Buffer pool is destroyed What wrong am I doing here ? When flink throws Buffer pool error ?

java.lang.RuntimeException:  Buffer pool is destroyed.

    at org.apache.flink.streaming.runtime.io.RecordWriterOutput.pushToRecordWriter(RecordWriterOutput.java:110)
    at org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:89)
    at org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:45)
    at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:718)
    at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:696)
    at org.apache.flink.streaming.api.operators.TimestampedCollector.collect(TimestampedCollector.java:51)

..........
Caused by: java.lang.IllegalStateException:  Buffer pool is destroyed.
    at org.apache.flink.util.Preconditions.checkState(Preconditions.java:179)
    at org.apache.flink.runtime.io.network.buffer.BufferBuilder.append(BufferBuilder.java:79)
    at org.apache.flink.runtime.io.network.api.serialization.SpanningRecordSerializer.copyToBufferBuilder(SpanningRecordSerializer.java:95)
    at org.apache.flink.runtime.io.network.api.writer.RecordWriter.copyFromSerializerToTargetChannel(RecordWriter.java:150)
    at org.apache.flink.runtime.io.network.api.writer.RecordWriter.emit(RecordWriter.java:128)
    at org.apache.flink.runtime.io.network.api.writer.RecordWriter.emit(RecordWriter.java:101)
    at org.apache.flink.streaming.runtime.io.StreamRecordWriter.emit(StreamRecordWriter.java:81)
    at org.apache.flink.streaming.runtime.io.RecordWriterOutput.pushToRecordWriter(RecordWriterOutput.java:107)```

推荐答案

什么是缓冲池?

缓冲池是用于网络堆栈的内存段实例的固定大小的池，这意味着您在发送数据时将其放入其中，并在通过网络接收时从中获取分段.

Buffer pool is a fixed size pool of memory segments instances for the network stack, which means that you put segments into it when you send data and fetch segments from it when you receive across the network.

为什么要摧毁它?

Why is it destroyed?

由于网络环境关闭，缓冲池被破坏.

The buffer pool is destroyd because the network environment shuts down.

为什么NetworkEnvironment关闭?

Why does NetworkEnvironment shut down?

这可能是具体情况，我不太确定.我建议您在此缓冲池例外之上阅读更多日志.

It may be a specific circumtances which I'm not very sure. I suggest you read more logs above this buffer pool exception.

这篇关于缓冲池被破坏.Apache Flink flapMap运算符中发现的问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！