与另一个同步操作员进行Flink异步IO操作员链接 [英] Flink async IO operator chaining with with another sync operator
问题描述
我有一个用例，其中我在flink中使用带有常规映射器的异步IO运算符.我正在使用flink 1.8.因此，异步运算符将必须处于运算符链的顶端.因此，我的运算符流程如下所示:
I have a usecase where I am using async IO operators with normal mappers in flink. I am using flink 1.8. So, async operator is going to have to be at the head of the operator chain. So my operator flows looks like this:
来源->映射器1->AsyncOperator->映射器2->下沉
Source -> Mapper1 -> AsyncOperator -> Mapper2 -> Sink
由于要求异步运算符为head，所以有两个运算符链，因此有两个任务-1. Source + Mapper1 2. AsyncOperator + Mapper2 + Sink.我对第二条链有疑问.我认为，如果正确地链接了第二个链，则应该将其包含在单个任务中.我不确定任务线程上的异步操作符和映射器2之间是否有等待时间，或者内部是否将Mapper2绑定到异步操作符的响应处理程序?理想情况下，它应该是第二个，但是我找不到相同的任何文档-因此想知道.
Because of the requirement of async operator being head, there are two operator chains and hence two tasks- 1. Source + Mapper1 2. AsyncOperator+Mapper2+Sink. I have question regarding the second chain. I think the second chain should be comprised within a single task if they are chained correctly. I am not sure if there is a wait time between async operator and mapper 2 on the task threads or the Mapper2 gets bound to the response handler for the Async Operator internally ? Ideally, it should be second, but I can't find any documentation for the same - hence wondering.
参考:
推荐答案
AsyncWaitOperator 在线程中旋转 Emitter ，因此一旦获得结果，它们就会得到发送给运营商的收藏家.请注意，如果您指定排序结果，则可能会有一个等待时间".由于完成顺序与传入元素的顺序不匹配.
The AsyncWaitOperator spins up an Emitter in a thread, so as soon as results are available they get sent to the operator's collector. Note though that if you specify ordered results there can be a "wait time" due to completion order not matching the order of incoming elements.
这篇关于与另一个同步操作员进行Flink异步IO操作员链接的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
