无法初始化群集。请检查您的配置mapreduce.framework.name和相应的服务器地址 - 提交job2remoteClustr [英] Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses-submiting job2remoteClustr

问题描述

我最近将我的集群从Apache Hadoop1.0升级到CDH4.4.0。我在另一台机器上有一台weblogic服务器，通过mapreduce客户机将作业提交到此远程群集。我仍然想使用MR1而不是纱线。我已经在CDH installtion中编译了我的客户端代码（客户端代码为/ usr / lib / hadoop / client / *）

创建JobClient时出现以下错误实例。 有许多帖子涉及同一个问题，但所有的解决方案都是指将作业提交到本地集群而不是远程的，特别是在我的情况下，从wls容器。

JobClient jc =新的JobClient（conf）;

无法初始化群集。请检查您的配置mapreduce.framework.name和相应的服务器地址。

但从集群上的命令提示符运行完美无缺。

感谢您的及时帮助！

解决方案

应用程序正在查看您的旧Hadoop 1.x配置文件。也许你的应用程序硬编码一些配置？这个错误往往表明你正在使用新的客户端库，但他们没有看到新的配置。

它必须存在，因为命令行工具可以看到它们。请检查您的 HADOOP_HOME 或 HADOOP_CONF_DIR env变量，尽管这是命令行工具倾向于选择的功能，并且它们可以正常工作。注意你需要在CDH 4.4中安装'mapreduce'服务而不是'yarn'，以使它与MR1客户端兼容。另请参阅Maven中的'... -mr1 -...'构件。

I recently upgraded my cluster from Apache Hadoop1.0 to CDH4.4.0. I have a weblogic server in another machine from where i submit jobs to this remote cluster via mapreduce client. I still want to use MR1 and not Yarn. I have compiled my client code against the client jars in the CDH installtion (/usr/lib/hadoop/client/*)

Am getting the below error when creating a JobClient instance. There are many posts related to the same issue but all the solutions refer to the scenario of submitting the job to a local cluster and not to remote and specifically in my case from a wls container.

JobClient jc = new JobClient(conf);

Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses.

But running from the command prompt on the cluster works perfectly fine.

Appreciate your timely help!

解决方案

It's likely that your app is looking at your old Hadoop 1.x configuration files. Maybe your app hard-codes some config? This error tends to indicate you are using the new client libraries but that they are not seeing new-style configuration.

It must exist since the command-line tools see them fine. Check your HADOOP_HOME or HADOOP_CONF_DIR env variables too although that's what the command line tools tend to pick up, and they work.

Note that you need to install the 'mapreduce' service and not 'yarn' in CDH 4.4 to make it compatible with MR1 clients. See also the '...-mr1-...' artifacts in Maven.

这篇关于无法初始化群集。请检查您的配置mapreduce.framework.name和相应的服务器地址 - 提交job2remoteClustr的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！