库数据流用C [英] Library for Dataflow in C

问题描述

我该怎么办数据流（管道和过滤器，流处理，基于流的）用C？而不是与UNIX管道。

How can I do dataflow (pipes and filters, stream processing, flow based) in C? And not with UNIX pipes.

我最近碰到 stream.py 。

流是iterables用流水线机制，使数据流编程和易于并行化。

Streams are iterables with a pipelining mechanism to enable data-flow programming and easy parallelization.

的想法是把一个函数的输出，变成可迭代到另一个迭代和插头，作为另一种这样的功能的输入。虽然你已经可以做到这一点使用功能的成分，这包通过重载>>运营商提供了一个优雅的符号吧。

The idea is to take the output of a function that turns an iterable into another iterable and plug that as the input of another such function. While you can already do this using function composition, this package provides an elegant notation for it by overloading the >> operator.

我想在C复制这种功能的简单版本我特别喜欢>> 运营商，以避免函数组合乱七八糟的超载。维基百科指向<一个href=\"http://groups.google.com/group/comp.lang.c/browse_thread/thread/22211d03c3e0b5c9/f11f83cdbde4eee8\"相对=nofollow>从新闻组帖子在1990年这暗示。

I would like to duplicate a simple version of this kind of functionality in C. I particularly like the overloading of the >> operator to avoid function composition mess. Wikipedia points to this hint from a Usenet post in 1990.

为什么C 2因为我想是能够做到这一点的微控制器，并在C扩展其他高级语言（最大，钯*，Python）的。

Why C? Because I would like to be able to do this on microcontrollers and in C extensions for other high level languages (Max, Pd*, Python).

*（讽刺的是马克斯和Pd被写了，在C，专门为此目的 - 我要找的东西准系统）

* (ironic given that Max and Pd were written, in C, specifically for this purpose – I'm looking for something barebones)

推荐答案

我知道，这不是一个很好的答案，但你应该让自己的简单数据流的框架。

I know, that it's not a good answer, but you should make your own simple dataflow framework.

我写了一个原型DF服务器（和我的朋友在一起），其中有几个未实现的功能尚未：它只能传递消息的整数和触发数据，并且它不支持paralellism。我刚刚跳过此工作：组件的生产商端口具有函数指针消费者的口，这是建立在初始化列表，他们调用它（如果列表不为空）。所以，当一个事件触发时，组件执行树形步行得来速的数据流图的。因为他们与整数和触发器的工作，它的极端快。

I've written a prototype DF server (together with a friend of mine), which have several unimplemented features yet: it can only pass Integer and Trigger data in messages, and it does not supports paralellism. I've just skipped this work: the components' producer ports have a list of function pointers to consumer ports, which are set up upon the initialization, and they call it (if the list is not empty). So, when an event fires, the components perform a tree-like walk-thru of the dataflow graph. As they work with Integers and Triggers, it's extremly quick.

另外，我写了一个奇怪的成分，它有一个消费者和一个生产者端口，它只是简单地将数据传递THRU - 但在另一个线程。这是消费者的日常很快结束，因为它只是把数据，并设置一个标志生产者端线程。肮脏的，但它适合我的需要。它脱离了树走很长的过程。

Also, I've written a strange component, which have one consumer and one producer port, it just simply passes the data thru - but in another thread. It's consumer routine finishes quickly, as it just puts the data and sets a flag to the producer-side thread. Dirty, but it suits my needs: it detaches long processes of the tree-walk.

所以，你可能会认识到，它的快速任务，其中图形大小并不重要一低流量异步系统。

So, as you may recognized, it's a low-traffic asynchronous system for quick tasks, where the graph size does not matter.

可惜的是，你的问题与我的不同尽可能多的积分，就像某一张数据流系统可以从另一个不同，你需要一个同步的，paralell，流处理解决方案。

Unfortunatelly, your problem differs as many points from mine, just as many one dataflow system can differ from another, you need a synchronous, paralell, stream handling solution.

我认为，在一个DF服务器的最大的问题是调度员。并发性，碰撞，线程的优先级......正如我所说，我只是跳过这个问题，没有解决。你应该跳过这一点。而且你还要跳过其他的问题。

I think, the biggest issue in a DF server is the dispatcher. Concurrency, collision, threads, priority... as I said, I've just skipped the problem, not solved. You should skip it, too. And you also should skip other problems.

调度

在同步DF架构的情况下，所有组件都必须按周期运行一次，除非特殊情况。他们有一个简单的precondition：可用输入数据？所以，你应该只扫描直通组件，并将它们传递到一个免费的调用程序线程，如果数据是可用的。处理所有这些之后，你就会有N个剩余的组件，没有处理。你应该再次处理列表。第二处理后，您将具有M个remainings。如果N ==男，周期就结束了。

In case of a synchronous DF architecture, all the components must run once per cycle, except special cases. They have a simple precondition: is the input data available? So, you should just to scan thru the components, and pass them to a free caller thread, if data is available. After processing all of them, you will have N remaining components, which haven't processed. You should process the list again. After the second processing you will have M remainings. If N == M, the cycle is over.

我觉得某种形式的同样的东西会工作，如果组件的数量低于100只。

I think some kind of same stuff will work, if the number of components is below only 100.

绑定

是的，结合最好的办法就是可视化编程。直到整理编辑，配置类code应该使用insetad，是这样的：

Yep, the best way of binding is the visual programming. Until finishing the editor, config-like code should used insetad, something like:

 // disclaimer: not actual code
 Component* c1 = new AddComponent();
 Component* c2 = new PrintComponent();
 c2->format = "The result is %d\n";
 bind(c1->result,c2->feed);

这很容易写，好读，其他的愿望是什么？

It's easy to write, well-readable, other wish?

消息

您应该通过组件的端口之间纯粹的原始数据包。你只需要一个绑定，它包含对生产和消费国港口指针的列表，包含处理过的标志，其中的调度员使用。

You should pass pure raw packets among components' ports. You need only a list of bindings, which contain pairs of pointers of producer and consumer ports, and contains the processed flag, which the "dispatcher" uses.

调用问题

的问题是，生产的不应该调用消费者端口，但是组件;所有成分（类），变量和烧制是在组件中。因此，生产者应直接调用组件的共同切入点，通过消费者的ID给它，或者它应该调用端口，它应该调用它所属的组件的任何方法。

The problem is that producer should not call the consumer port, but the component; all component (class) variables and firings are in the component. So, the producer should call the component's common entry point directly, passing the consumer's ID to it, or it should call the port, which should call any method of the component which it belongs.

所以，如果你能有一些限制住，我说，先走了，写你的精简版框架。这是一个很好的工作，但写小部件，看看，多么聪明，他们可以连接在一起制作的应用程式是最终的乐趣。

So, if you can live with some restrictions, I say, go ahead, and write your lite framework. It's a good task, but writing small components and see, how smart can they wired together building a great app is the ultimate fun.

如果您还有其他问题，请随时问，我经常在这里扫描数据流关键字。

If you have further questions, feel free to ask, I often scan the "dataflow" keyword here.

可能的话，你可以找出一个更简单dataflowish模型程序。

Possibly, you can figure out a more simple dataflowish model for your program.

这篇关于库数据流用C的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！