如何解决“无法分配请求的地址:16次重试后服务'sparkDriver'失败"在运行Spark代码时? [英] How to solve "Can't assign requested address: Service 'sparkDriver' failed after 16 retries" when running spark code?
本文介绍了如何解决“无法分配请求的地址:16次重试后服务'sparkDriver'失败"在运行Spark代码时?的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
我正在用intelliJ学习spark + scala,从下面的一小段代码开始
I am learning spark + scala with intelliJ , started with below small piece of code
import org.apache.spark.{SparkConf, SparkContext}
object ActionsTransformations {
def main(args: Array[String]): Unit = {
//Create a SparkContext to initialize Spark
val conf = new SparkConf()
conf.setMaster("local")
conf.setAppName("Word Count")
val sc = new SparkContext(conf)
val numbersList = sc.parallelize(1.to(10000).toList)
println(numbersList)
}
}
在尝试运行时,低于异常
when trying to run , getting below exception
Exception in thread "main" java.net.BindException: Can't assign requested address: Service 'sparkDriver' failed after 16 retries (on a random free port)! Consider explicitly setting the appropriate binding address for the service 'sparkDriver' (for example spark.driver.bindAddress for SparkDriver) to the correct binding address.
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:127)
at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:501)
at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1218)
at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:496)
at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:481)
at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:965)
at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:210)
at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:353)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:399)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:446)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
at java.lang.Thread.run(Thread.java:745)
Process finished with exit code 1
任何人都可以建议该怎么做.
can any one suggest what to do .
推荐答案
似乎您使用了旧版本的spark.在您的情况下,请尝试添加以下行:
Seems like you've used some old version of spark. In your case try to add this line:
conf.set("spark.driver.bindAddress", "127.0.0.1")
如果您将使用Spark 2.0+,则应遵循以下技巧:
If you will use spark 2.0+ folowing should do the trick:
val spark: SparkSession = SparkSession.builder()
.appName("Word Count")
.master("local[*]")
.config("spark.driver.bindAddress", "127.0.0.1")
.getOrCreate()
这篇关于如何解决“无法分配请求的地址:16次重试后服务'sparkDriver'失败"在运行Spark代码时?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文