Flink 异步 IO 操作符与另一个同步操作符链接 [英] Flink async IO operator chaining with with another sync operator

问题描述

我有一个用例，我在 flink 中使用带有普通映射器的异步 IO 操作符.我正在使用 flink 1.8.因此，异步操作符必须位于操作符链的头部.所以我的操作员流程如下所示:

I have a usecase where I am using async IO operators with normal mappers in flink. I am using flink 1.8. So, async operator is going to have to be at the head of the operator chain. So my operator flows looks like this:

来源 ->映射器 1 ->AsyncOperator ->Mapper2 ->水槽

Source -> Mapper1 -> AsyncOperator -> Mapper2 -> Sink

因为需要异步操作符为头，所以有两个操作符链，因此有两个任务 - 1. Source + Mapper1 2. AsyncOperator+Mapper2+Sink.我对第二条链有疑问.我认为如果正确链接第二个链，则它们应该包含在单个任务中.我不确定任务线程上的异步运算符和映射器 2 之间是否存在等待时间，或者 Mapper2 在内部绑定到异步运算符的响应处理程序?理想情况下，它应该是第二个，但我找不到任何相同的文档 - 因此想知道.

Because of the requirement of async operator being head, there are two operator chains and hence two tasks- 1. Source + Mapper1 2. AsyncOperator+Mapper2+Sink. I have question regarding the second chain. I think the second chain should be comprised within a single task if they are chained correctly. I am not sure if there is a wait time between async operator and mapper 2 on the task threads or the Mapper2 gets bound to the response handler for the Async Operator internally ? Ideally, it should be second, but I can't find any documentation for the same - hence wondering.

参考:

1. https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/operators/asyncio.html

推荐答案

AsyncWaitOperator 在线程中启动一个 Emitter，因此一旦结果可用，他们就会得到发送到操作员的收集器.但请注意，如果您指定有序结果，则可能会有等待时间".由于完成顺序与传入元素的顺序不匹配.

The AsyncWaitOperator spins up an Emitter in a thread, so as soon as results are available they get sent to the operator's collector. Note though that if you specify ordered results there can be a "wait time" due to completion order not matching the order of incoming elements.

这篇关于Flink 异步 IO 操作符与另一个同步操作符链接的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！