本地主机上的执行器驱动程序丢失:执行器心跳超时 [英] Lost executor driver on localhost: Executor heartbeat timed out

问题描述

我正在调试本地模式下的Spark应用程序.禁用超时以避免调试会话期间出现火花崩溃而没有不利影响是否可行?

I am debugging a spark application in local mode. Is it feasible to disable timeouts to avoid spark crashing in the middle of a debug session, without adverse effects?

您会建议更改哪些与超时相关的参数，以免发生各种超时而不会完全导致火花崩溃?

Which timeout related parameters would you suggest changing for avoiding various timeouts without crashing spark altogether?

目前，在IntelliJ中调试时遇到了这个问题:

Currently I'm encountering this one, while debugging in IntelliJ:

本地主机上的执行器驱动程序丢失:执行器心跳在以下时间后超时 129006 ms

Lost executor driver on localhost: Executor heartbeat timed out after 129006 ms

推荐答案

将这两个添加到混合中:

Add these two into the mix:

 val spark = SparkSession.builder()
  .config("spark.network.timeout", "10000001")
  .config("spark.executor.heartbeatInterval", "10000000")

这篇关于本地主机上的执行器驱动程序丢失:执行器心跳超时的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！