无法初始化集群.请检查您的 mapreduce.framework.name 配置和相应的服务器地址 - 提交 job2remoteClustr [英] Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses-submiting job2remoteClustr

问题描述

我最近将集群从 Apache Hadoop1.0 升级到了 CDH4.4.0.我在另一台机器上有一个 weblogic 服务器，我通过 mapreduce 客户端将作业提交到这个远程集群.我仍然想使用 MR1 而不是 Yarn.我已经针对 CDH 安装 (/usr/lib/hadoop/client/*) 中的客户端 jar 编译了我的客户端代码

I recently upgraded my cluster from Apache Hadoop1.0 to CDH4.4.0. I have a weblogic server in another machine from where i submit jobs to this remote cluster via mapreduce client. I still want to use MR1 and not Yarn. I have compiled my client code against the client jars in the CDH installtion (/usr/lib/hadoop/client/*)

创建 JobClient 实例时出现以下错误.有很多帖子与同一问题相关，但所有解决方案都是指将作业提交到本地集群而不是远程，特别是在我的情况下从 wls 容器提交作业的场景.

Am getting the below error when creating a JobClient instance. There are many posts related to the same issue but all the solutions refer to the scenario of submitting the job to a local cluster and not to remote and specifically in my case from a wls container.

JobClient jc = new JobClient(conf);

无法初始化集群.请检查您的 mapreduce.framework.name 配置和对应的服务器地址.

但是从集群上的命令提示符运行完全正常.

But running from the command prompt on the cluster works perfectly fine.

感谢您的及时帮助！

推荐答案

您的应用程序可能正在查看旧的 Hadoop 1.x 配置文件.也许您的应用程序硬编码了一些配置?此错误往往表明您正在使用新的客户端库，但它们没有看到新样式的配置.

It's likely that your app is looking at your old Hadoop 1.x configuration files. Maybe your app hard-codes some config? This error tends to indicate you are using the new client libraries but that they are not seeing new-style configuration.

它必须存在，因为命令行工具可以很好地看到它们.检查您的 HADOOP_HOME 或 HADOOP_CONF_DIR 环境变量，尽管这是命令行工具倾向于选择的内容，并且它们可以工作.

It must exist since the command-line tools see them fine. Check your HADOOP_HOME or HADOOP_CONF_DIR env variables too although that's what the command line tools tend to pick up, and they work.

请注意，您需要在 CDH 4.4 中安装mapreduce"服务而不是yarn"，以使其与 MR1 客户端兼容.另请参阅 Maven 中的...-mr1-..."工件.

Note that you need to install the 'mapreduce' service and not 'yarn' in CDH 4.4 to make it compatible with MR1 clients. See also the '...-mr1-...' artifacts in Maven.

这篇关于无法初始化集群.请检查您的 mapreduce.framework.name 配置和相应的服务器地址 - 提交 job2remoteClustr的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！