EMR主节点是否知道其集群ID？ [英] Does an EMR master node know its cluster id?

问题描述

我希望能够创建EMR群集，并让这些群集将消息发送回中央队列。为了这个工作，我需要在每个主节点上运行某种代理。这些代理中的每一个都必须在这个消息中标识它自己，以便接收者知道该消息是关于哪个集群的。

主节点知道它是否是id（j-的 * ** * * ）？如果没有，那么是否还有其他一些识别信息可以让消息接收者推断此ID？

我已经查看了/ home / hadoop / conf，我没有找到任何有用的东西。我在/mnt/var/log/instance-controller/instance-controller.log中发现了这个id，但看起来很难grep。我想知道哪里的实例控制器可能首先得到这个id。

解决方案

您可以看看<$ c主节点上的$ c> / mnt / var / lib / info / 以查找有关您的EMR群集设置的大量信息。更具体地说， /mnt/var/lib/info/job-flow.json 包含jobFlowId或ClusterID。

您可以使用预先安装的json解析器（ jq ）来获取作业流ID。

  cat /mnt/var/lib/info/job-flow.json | jq -r.jobFlowId
  

（根据@Marboni更新）

I want to be able to create EMR clusters and for those clusters to send messages back to some central queue. In order for this to work, I need to have some sort of agent running on each master node. Each one of those agents will have to identify itself in this message so that the recipient knows which cluster the message is about.

Does the master node know it's id (j-*****)? If not, then is there some other piece of identifying information that could allow the message recipient to infer this id?

I've taken a look through the config files in /home/hadoop/conf, and I haven't found anything useful. I found the id in /mnt/var/log/instance-controller/instance-controller.log, but it looks like it'll be difficult to grep for. I'm wondering where instance-controller might get that id from in the first place.

解决方案

You may look at /mnt/var/lib/info/ on Master node to find lot of info about your EMR cluster setup. More specifically /mnt/var/lib/info/job-flow.json contains the jobFlowId or ClusterID.

You can use the pre-installed json parser (jq) to get the jobflow id.

cat /mnt/var/lib/info/job-flow.json | jq -r ".jobFlowId"

(updated as per @Marboni)

这篇关于EMR主节点是否知道其集群ID？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！