外部客户如何通过HTTP回调来通知Oozie工作流 [英] How external clients notify Oozie workflow with HTTP callback

问题描述

假设我们有一个Oozie工作流程由3个Java动作节点启动的情况。每个Java操作都将对Oozie / Hadoop集群外部的外部Web服务（例如google.com，yahoo.com等公开的某些Web服务）进行异步HTTP调用。我认为这是可行的，因为Oozie支持自定义操作节点。

现在，我不想让Oozie不时地轮询外部Web服务，工作是在外部Web服务中完成的。我想要有外部Web服务（让我们假设我们可以自由修改它）回调Oozie为了让Oozie完成外部Web服务的工作，并进一步将一些信息传递给Oozie，并让Oozie决定采取哪些后续行动拿到。

有wiki，比如 http://www.infoq.com/articles/ExtendingOozie ，讨论异步节点的操作回调，但是我从来没有找到任何真正的示例来说明如何回调异步操作节点。有没有人有任何想法如何回调异步行动节点的工作？

非常感谢！

解决方案

查看 Oozie SSH行动实施。它只是一个类，相对简单（但有点麻烦），并展示了如何创建一个回调URL：

 字符串callbackPost = ignoreOutput？ _：
 getOozieConf（）。get（HTTP_COMMAND_OPTIONS）.replace（，%%%）; 
 
 String callBackUrl = Services.get（）。get（CallbackService.class）
 .createCallBackUrl（action.getId（），EXT_STATUS_VAR）; 
  

然后将URL作为参数传递给shell脚本。该脚本稍后仅在该URL上调用curl。外部状态ID例如是通过回调返回到Oozie的已执行进程的PID。它必须不为空/空。

提示如果您决定深入研究代码：虽然看起来代码是同步执行的，但实际上它是通过异步执行的在后台运行shell脚本的方法。

回调由Oozie的 CallbackServlet ： p>

  dagEngine.processCallback（actionId，callbackService.getExternalStatus（queryString），props）; 
  

Let us say we have a case where an Oozie workflow is started with 3 Java action nodes. Each Java action is going to make an async HTTP call to an external web services (such as some web service exposed by google.com, yahoo.com, etc.) outside the Oozie/Hadoop cluster. I assume this is doable since Oozie support custom action node.

Now, I don't want to have Oozie poll the external web services from time to time to check if the work is done in external web service. I want to have the external web service (let us assume we can modify that freely) call back Oozie to nofiy Oozie the work by external web service is done, and further pass some info back to Oozie and let Oozie decide which follow-up actions to take.

There are wikis, such as this http://www.infoq.com/articles/ExtendingOozie, talking about call back from action for async nodes, however I never have found any actual sample how call back for async action nodes works. Does anyone have any idea how does this call back for async action nodes work?

Many thanks in advance!

解决方案

Have a look at the Oozie SSH action implementation. It is just one class, relatively simple (but a bit messy), and shows how to create a callback URL:

String callbackPost = ignoreOutput ? "_" : 
getOozieConf().get(HTTP_COMMAND_OPTIONS).replace(" ", "%%%");

String callBackUrl = Services.get().get(CallbackService.class)
                .createCallBackUrl(action.getId(), EXT_STATUS_VAR);

The URL is then passed to a shell script as an argument. The script later just invokes curl on that URL. The external status id is for example PID of the executed process that is returned back to Oozie via the callback. It must not be empty/null.

Hints if you decide to delve into the code: although it looks like the code is executed synchronously, it is in fact executed asynchronously by means of running the shell script in the background.

The callback is processed by Oozie's CallbackServlet:

dagEngine.processCallback(actionId, callbackService.getExternalStatus(queryString), props);

这篇关于外部客户如何通过HTTP回调来通知Oozie工作流的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！