完全混淆了Riak + Erlang的riakc客户端的MapReduce [英] Completely confused about MapReduce in Riak + Erlang's riakc client

问题描述

我在这里感到困惑的主要原因（我认为），qfun的论据应该是什么，返回值应该是什么。 自述文件基本上没有说出任何关于这一点的例子，它给出了第二个例子和第三个参考。

The main thing I'm confused about here (I think) is what the arguments to the qfun are supposed to be and what the return value should be. The README basically doesn't say anything about this and the example it gives throws away the second and third args.

现在我只是想了解参数，而不是使用Riak做任何实践。最终，我将尝试用它重建（基于MySQL的缓慢的）财务报告系统。所以忽略了我这个目标的无意义，为什么下面给我一个 badfun 异常？

Right now I'm only trying to understand the arguments and not using Riak for anything practical. Eventually I'll be trying to rebuild our (slow, MySQL-based) financial reporting system with it. So ignoring the pointlessness of my goal here, why does the following give me a badfun exception?

数据是名称和年龄的元组（成对），以键为名。在Erlang控制台插入数据之前，我没有对JSON进行任何转换。

The data is just tuples (pairs) of Names and Ages, with the keys being the name. I'm not doing any conversion to JSON or such before inserting the data from the Erlang console.

现在有一些 {Name，Age} / code>对存储在<"people>> 中的对我想使用MapReduce（没有其他原因，不了解如何）要获得值，请在第一次使用中保持不变。

Now with some {Name, Age} pairs stored in <<"people">> I want to use MapReduce (for no other reason than to understand "how") to get the values back out, unchanged in this first use.

riakc_pb_socket:mapred(
    Pid, <<"people">>,
    [{map, {qfun, fun(Obj, _, _) -> [Obj] end}, none, true}]).

但这只是给我一个坏蛋：

This just gives me a badfun, however:

{error,<<"{\"phase\":0,\"error\":\"{badfun,#Fun<erl_eval.18.17052888>}\",\"input\":\"{ok,{r_object,<<\\\"people\\\">>,<<\\\"elaine\\\">"...>>}

如何通过我的地图函数传递数据不变？Erlang客户端的文档比README更好吗？README似乎假设你已经知道输入是什么。

How do I just pass the data through my map function unchanged? Is there any better documentation of the Erlang client than what is in the README? That README seems to assume you already know what the inputs are.

推荐答案

经过几天的玩耍，这是一个整洁的技巧，使开发变得更加轻松利用Erlang的RPC支持，事实上有运行时代码加载，以将代码分发到所有Riak节点上：

After a few days of playing around, here's a neat trick that makes development easier. Exploit Erlang's RPC support and the fact it has runtime code loading, to distribute your code across all the Riak nodes:

%% Call this somewhere during your app's initialization routine.
%% Assumes you have a list of available Riak nodes in your app's env.
load_mapreduce_in_riak() ->
  load_mapreduce_in_riak(application:get_env(app_name, riak_nodes, [])).

load_mapreduce_in_riak([]) ->
  ok;
load_mapreduce_in_riak([{Node, Cookie}|Tail]) ->
  erlang:set_cookie(Node, Cookie),
  case net_adm:ping(Node) of
    pong ->
      {Mod, Bin, Path} = code:get_object_code(app_name_mapreduce),
      rpc:call(Node, code, load_binary, [Mod, Path, Bin]);
    pang ->
      io:format("Riak node ~p down! (ping <-> pang)~n", [Node])
  end,
  load_mapreduce_in_riak(Tail).

现在您可以参考模块中的任何功能 app_name_mapreduce ，它们将对Riak集群可见。如果需要，可以使用代码：delete / 1再次删除代码。

Now you can refer to any of the functions in the module app_name_mapreduce and they'll be visible to the Riak cluster. The code can be removed again with code:delete/1, if needed.

这篇关于完全混淆了Riak + Erlang的riakc客户端的MapReduce的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！