将 3rd 方异步 API 与 Cap'n Proto RPC 集成的好方法是什么? [英] what is a good way to integrate 3rd party asynchronous APIs with Cap'n Proto RPC?

问题描述

我有一个处理 Cap'n Proto RPC 请求的 Linux 服务器.其中一些请求需要将请求中的数据转发到另一台运行的服务器，在本例中为 Kafka 代理.librdkafka 和 Cap'n Proto KJ 库都可以使用 poll()，所以我认为操作系统将确保它们都可以异步运行，但我不确定是否需要或有益的进一步集成.有没有人有这方面的经验?

I have a Linux server which processes Cap'n Proto RPC requests. Some of those requests need to forward data in the request to another server running, in this case, a Kafka broker. The librdkafka and Cap'n Proto KJ libraries can both use poll() so i think the OS will ensure they can both run asynchronously but I'm not sure if further integration is required or beneficial. Does anyone have experience with this?

这个问题比我列出的细节要广泛一些.我将来可能会从 Cap'n Proto RPC 调用其他 API，因此我们将不胜感激.

The question is a bit broader than the specifics i've listed.. there are other APIs which i may call in future from Cap'n Proto RPC so any broad guidelines would be appreciated.

推荐答案

不幸的是，这并不是那么简单.是的，他们都使用 poll()，但问题是，一次只有一个库会调用 poll()，而且只有那个库会实际接收任何事件——另一个被卡住了.这是事件循环库的经典挑战——默认情况下它们不能一起使用.

Unfortunately, this is not so simple. Yes, they both use poll(), but the problem is, only one library will be calling poll() at a time, and only that library will actually receive any events -- the other is stuck. This is a classic challenge with event loop libraries -- by default they cannot be used together.

一种选择是尝试在单独的线程中使用库.但是，事件驱动库的设计通常假设您在单个线程中完成所有工作，否则为什么需要事件循环?

One option is to try to use the libraries in separate threads. But, often event-driven libraries design around the assumption that you're doing everything in a single thread because otherwise why would you need an event loop?

但是正确的事情"是集成事件循环.KJ 的事件循环能够与其他事件库集成.比如我把它和libuv集成到node-capnp；查看此文件的第一部分:

But "The Right Thing" is to integrate the event loops. KJ's event loop is capable of being integrated with other event libraries. For example, I integrated it with libuv for node-capnp; see the first part of this file:

https://github.com/kentonv/node-capnp/blob/master/src/node-capnp/capnp.cc

(在某个时候，我计划将此处与 libuv 相关的代码分离到 Cap'n Proto 附带的单独库中.)

(At some point I plan to separate out the libuv-related code here into a separate library shipped with Cap'n Proto.)

再举一个例子，这是 Nathan Hourt 的一个拉取请求，用于添加与 Qt 事件循环的集成——但请注意，这个不包括 I/O 集成，我认为是因为 Nathan 使用的是 AsyncIoStream 的实现 他在数据可用时手动将数据推送到其中:

For another example, here's a pull request by Nathan Hourt to add integration with Qt's event loop -- but note that this one doesn't include I/O integration, I think because Nathan is using an implementation of AsyncIoStream into which he manually pushes data when it's available:

https://github.com/sandstorm-io/capnproto/pull/253

在任何情况下，您都需要对 Kafka 使用的任何内容执行类似的操作.希望您随后将您的代码贡献给 Cap'n Proto！:)

In any case, you'll need to do something similar with whatever Kafka uses. Hopefully you'll then contribute your code back to Cap'n Proto! :)

这篇关于将 3rd 方异步 API 与 Cap'n Proto RPC 集成的好方法是什么?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！