如何使用Cassandra的地图缩小与或w / o猪？ [英] How to use Cassandra's Map Reduce with or w/o Pig?

问题描述

有人可以解释MapReduce如何与Cassandra .6一起工作吗？我已经阅读了字数计数的例子，但我不完全按照卡桑德拉结束与客户端结束发生了什么。

Can someone explain how MapReduce works with Cassandra .6? I've read through the word count example, but I don't quite follow what's happening on the Cassandra end vs. the "client" end.

https://svn.apache.org/repos/asf/cassandra/trunk/contrib / word_count /

例如，假设我使用Python和Pycassa，我如何加载一个新的map reduce函数，然后调用它？我的地图减少功能必须是安装在cassandra服务器上的java？如果是这样，我如何从Pycassa调用它？

For instance, let's say I'm using Python and Pycassa, how would I load in a new map reduce function, and then call it? Does my map reduce function have to be java that's installed on the cassandra server? If so, how do I call it from Pycassa?

还有提到Pig使这更容易，但我是一个完整的Hadoop noob，所以没有真正的帮助。

There's also mention of Pig making this all easier, but I'm a complete Hadoop noob, so that didn't really help.

您的答案可以使用Thrift或其他，我刚刚提到Pycassa表示客户端。我只是想了解在Cassandra集群中运行的程序与提出请求的实际服务器之间的区别。

Your answer can use Thrift or whatever, I just mentioned Pycassa to denote the client side. I'm just trying to understand the difference between what runs in the Cassandra cluster vs. the actual server making the requests.

推荐答案

根据我听到的（以及来自这里），开发人员编写MapReduce程序的方式使用Cassandra作为数据源如下。你编写一个常规的MapReduce程序（你链接到的示例是为纯Java版本），现在可用的jar提供一个CustomInputFormat，允许输入源为Cassandra（而不是默认的，即Hadoop）。

From what I've heard (and from here), the way that a developer writes a MapReduce program that uses Cassandra as the data source is as follows. You write a regular MapReduce program (the example you linked to is for the pure-Java version) and the jars that are now available provide a CustomInputFormat that allows the input source to be Cassandra (instead of the default, which is Hadoop).

如果你使用Pycassa，我会说你运气不好，直到（1）该项目的维护者添加对MapReduce的支持，或者（2）一些Python函数一起编写一个Java MapReduce程序并运行它。后者肯定是一个黑客，但会让你起来。

If you're using Pycassa I'd say you're out of luck until either (1) the maintainer of that project adds support for MapReduce or (2) you throw some Python functions together that write up a Java MapReduce program and run it. The latter is definitely a bit of a hack but would get you up and going.

这篇关于如何使用Cassandra的地图缩小与或w / o猪？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！