Hadoop映射作业失败，并显示com.datastax.driver.core.exceptions.NoHostAvailableException [英] Hadoop map jobs fail with com.datastax.driver.core.exceptions.NoHostAvailableException

问题描述

我试图使用hadoop map-reduce运行分析，存储在cassandra中的数据。为此，我使用通过maven依赖 cassandra-all 可用的类 CqlInputFormat 。目前我们在我们的生产环境中一直使用2.0.10版本的这种依赖。此外，我们使用 caassandra-driver-core 版本2.1.1。

I am trying to run analytics using hadoop map-reduce over data stored inside cassandra. For this, I am using the class CqlInputFormat available through the maven dependency cassandra-all. Currently we have been using 2.0.10 version of this dependency in our production environment. Also, we are using caassandra-driver-core having version 2.1.1.

另一个需要注意的重要事情是，如果我使用 CqlPagingInputFormat 在 cassandra-all 的2.0.6中可用，一切正常。但此输入格式已在后续版本中删除。

Another important thing to note here is that, if I use CqlPagingInputFormat available in 2.0.6 of cassandra-all, everything works fine. But this input format has been removed in the later versions.

com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: 10.40.242.174:9042 (com.datastax.driver.core.exceptions.DriverException: Timeout while trying to acquire available connection (you may want to increase the driver number of per-host connections)))
    at com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:65)
    at com.datastax.driver.core.DefaultResultSetFuture.extractCauseFromExecutionException(DefaultResultSetFuture.java:258)
    at com.datastax.driver.core.Cluster.connect(Cluster.java:267)
    at org.apache.cassandra.hadoop.cql3.CqlRecordReader.initialize(CqlRecordReader.java:137)
    at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:521)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
    at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: 10.40.242.174:9042 (com.datastax.driver.core.exceptions.DriverException: Timeout while trying to acquire available connection (you may want to increase the driver number of per-host connections)))
    at com.datastax.driver.core.RequestHandler.sendRequest(RequestHandler.java:103)
    at com.datastax.driver.core.SessionManager.execute(SessionManager.java:446)
    at com.datastax.driver.core.SessionManager.executeQuery(SessionManager.java:482)
    at com.datastax.driver.core.SessionManager.executeAsync(SessionManager.java:88)
    at com.datastax.driver.core.AbstractSession.executeAsync(AbstractSession.java:60)
    at com.datastax.driver.core.Cluster.connect(Cluster.java:260)
    ... 9 more

任何输入都是最受欢迎的。

Any inputs are most welcome.

推荐答案

我使用 cassandra-all-2.0.10 时遇到了完全相同的问题，但问题已通过使用版本 cassandra-all-2.0.12 。

I have had the exactly same problem when using cassandra-all-2.0.10, but the problem has been resolved by using the version cassandra-all-2.0.12.

我认为更正是在这个提交：
在CqlConfigHelper中可能使用非本地主机

I think the correction is in this commit: Potentially use non-local hosts in CqlConfigHelper

这篇关于Hadoop映射作业失败，并显示com.datastax.driver.core.exceptions.NoHostAvailableException的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！