EMR 主节点是否知道其集群 ID? [英] Does an EMR master node know its cluster ID?

问题描述

我希望能够创建 EMR 集群，并让这些集群将消息发送回某个中央队列.为了使其工作，我需要在每个主节点上运行某种代理.这些代理中的每一个都必须在此消息中标识自己，以便接收者知道该消息是关于哪个集群的.

I want to be able to create EMR clusters, and for those clusters to send messages back to some central queue. In order for this to work, I need to have some sort of agent running on each master node. Each one of those agents will have to identify itself in this message so that the recipient knows which cluster the message is about.

主节点是否知道自己的ID(j-****************)?如果不是，那么是否还有其他一些识别信息可以让消息接收者推断出此 ID?

Does the master node know its ID (j-*************)? If not, then is there some other piece of identifying information that could allow the message recipient to infer this ID?

我查看了 /home/hadoop/conf 中的配置文件，但没有发现任何有用的信息.我在 /mnt/var/log/instance-controller/instance-controller.log 中找到了 ID，但它看起来很难 grep.我想知道实例控制器首先可以从哪里获得该 ID.

I've taken a look through the config files in /home/hadoop/conf, and I haven't found anything useful. I found the ID in /mnt/var/log/instance-controller/instance-controller.log, but it looks like it'll be difficult to grep for. I'm wondering where instance-controller might get that ID from in the first place.

推荐答案

您可以查看主节点上的 /mnt/var/lib/info/ 以查找有关 EMR 集群设置的大量信息.更具体地说，/mnt/var/lib/info/job-flow.json 包含 jobFlowId 或 ClusterID.

You may look at /mnt/var/lib/info/ on Master node to find lot of info about your EMR cluster setup. More specifically /mnt/var/lib/info/job-flow.json contains the jobFlowId or ClusterID.

您可以使用预先安装的 json 解析器 (jq) 来获取作业流 ID.

You can use the pre-installed json parser (jq) to get the jobflow id.

cat /mnt/var/lib/info/job-flow.json | jq -r ".jobFlowId"

(根据@Marboni 更新)

(updated as per @Marboni)

这篇关于EMR 主节点是否知道其集群 ID?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！