java.lang.NoSuchMethodError：akka.actor.ActorCell.addFunctionRef [英] java.lang.NoSuchMethodError: akka.actor.ActorCell.addFunctionRef

问题描述

我正在尝试建立一个简单的akka​​-http 2.4.2项目以对其进行测试，但是我没有这样做。

I am trying to setup a simple akka-http 2.4.2 project to test it out, but I am failing to do so.

我的build.sbt：

My built.sbt:

import NativePackagerHelper._

lazy val akkaVersion = "2.4.2"

lazy val root = (project in file(".")).
settings(
name := "akkTest",
version := "0.1",
scalaVersion := "2.11.7")

libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-actor" % akkaVersion,
"com.typesafe.akka" %% "akka-http-spray-json-experimental" % akkaVersion
)

enablePlugins(JavaServerAppPackaging)

我的代码Main.scala中的代码段

my code snippet in Main.scala

import akka.http.scaladsl.Http
import akka.stream.ActorMaterializer
import akka.stream.scaladsl._
import akka.actor.ActorSystem


object Main extends App {

implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
implicit val ec = system.dispatcher

val serverSource =
    Http().bind(interface = "localhost", port = 8080)
val bindingFuture =
    serverSource.to(Sink.foreach { connection => // foreach materializes the source
    println("Accepted new connection from " + connection.remoteAddress)
    }).run()

}

执行错误引发：

Uncaught error from thread [default-akka.actor.default-dispatcher-2] shutting down JVM since 'akka.jvm-exit-on-fatal-error' is enabled for ActorSystem[default]
java.lang.NoSuchMethodError: akka.actor.ActorCell.addFunctionRef(Lscala/Function2;)Lakka/actor/FunctionRef;
        at akka.stream.stage.GraphStageLogic$StageActor.<init>(GraphStage.scala:143)
        at akka.stream.stage.GraphStageLogic.getStageActor(GraphStage.scala:904)
        at akka.stream.impl.io.ConnectionSourceStage$$anon$1.preStart(TcpStages.scala:56)
        at akka.stream.impl.fusing.GraphInterpreter.init(GraphInterpreter.scala:468)
        at akka.stream.impl.fusing.GraphInterpreterShell.init(ActorGraphInterpreter.scala:363)
        at akka.stream.impl.fusing.ActorGraphInterpreter.tryInit(ActorGraphInterpreter.scala:502)
        at akka.stream.impl.fusing.ActorGraphInterpreter.preStart(ActorGraphInterpreter.scala:539)
        at akka.actor.Actor$class.aroundPreStart(Actor.scala:472)
        at akka.stream.impl.fusing.ActorGraphInterpreter.aroundPreStart(ActorGraphInterpreter.scala:493)
        at akka.actor.ActorCell.create(ActorCell.scala:580)
        at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:456)
        at akka.actor.ActorCell.systemInvoke(ActorCell.scala:478)
        at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:279)
        at akka.dispatch.Mailbox.run(Mailbox.scala:220)
        at akka.dispatch.Mailbox.exec(Mailbox.scala:231)
        at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
        at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
        at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
        at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

这一定是我所处的环境，但是我不知道如何跟踪问题。我正在使用jdk 1.8u71

This must be something on my environment but I dont know how to track the issue. I am using jdk 1.8u71

[info] Done updating.
[info] Including from cache: ssl-config-akka_2.11-0.1.3.jar
[info] Including from cache: reactive-streams-1.0.0.jar
[info] Including from cache: akka-http-spray-json-experimental_2.11-2.4.2.jar
[info] Including from cache: config-1.3.0.jar
[info] Including from cache: spray-json_2.11-1.3.2.jar
[info] Including from cache: ssl-config-core_2.11-0.1.3.jar
[info] Including from cache: scala-parser-combinators_2.11-1.0.4.jar
[info] Including from cache: scala-java8-compat_2.11-0.7.0.jar
[info] Including from cache: akka-parsing_2.11-2.4.2.jar
[info] Including from cache: akka-http-experimental_2.11-2.4.2.jar
[info] Including from cache: akka-actor_2.11-2.4.2.jar
[info] Including from cache: akka-http-core_2.11-2.4.2.jar
[info] Including from cache: akka-stream_2.11-2.4.2.jar
[info] Including from cache: scala-library-2.11.7.jar

我只指向相同的Akka ve的依赖项rsion

Take in mind I only point to dependencies of the same akka version

该程序在使用 sbt run 时可以正常运行，但是在将装配好的jar与我自己的scala启动器一起使用时会失败

This program works fine when using sbt run but fails when using the assembled jar with my own scala launcher

推荐答案

问题是Spark内部使用Akka。只要您将Spark Job作为独立的应用程序运行（例如 sbt run ），这就不会出现问题，因为将使用您自己的Akka版本。但是，一旦将应用程序提交到具有 spark-submit 的群集中，情况就会发生变化。然后，Spark类加载器将通过 sparkJob.jar 中捆绑的Akka实现来选择Akka的内部版本。因此，上面的 NoSuchMethodError 来自 akka-stream_2.11-2.4.2 调用在Spark中使用的akka​​-actor_2.11-2.3.x.jar ，而不是 akka-actor_2.11-2.4.2.jar 捆绑在您的工作中。实际上，方法 addFunctionRef 是非常最近添加，并且在Akka的早期版本中不存在。您可以通过在发生异常的位置设置断点来验证这一点（或使用异常断点）。随着应用程序在 GraphStage 中有问题的位置挂起，评估

The problem is that Spark uses Akka internally. As long as you run your Spark Job as a self contained application (e.g. sbt run), this is not a problem, since your own version of Akka will be used. Things change however as soon as you submit your application to a cluster with spark-submit. The Spark classloader will then pick the internal version of Akka over the Akka implementation bundled in your sparkJob.jar. The NoSuchMethodError from above therefore comes from akka-stream_2.11-2.4.2 calling into akka-actor_2.11-2.3.x.jar which is used in Spark, instead of akka-actor_2.11-2.4.2.jar which is bundled in your job. The method addFunctionRef is in fact a very recent addition and is not present in earlier versions of Akka. You can verify this by setting a breakpoint at the place where the exception occurs (or use an exception breakpoint). With the application suspend at the problematic location in GraphStage, evaluate

materializer.supervsor.getClass().getResource("ActorCell.class")

这将打印出位置的类文件 ActorCell 的加载源。

This will print out the location of the class file ActorCell was loaded from.

为了确保您与Spark的Akka版本隔离在内部使用，您可以使用 spark-submit 的-driver-class-path 选项，例如

To make sure that you are isolated from the Akka version that Spark uses internally, you can use the --driver-class-path option of spark-submit, like

spark-submit --class MyJob \
  --driver-class-path akka-actor_2.11-2.4.2.jar \
  --master spark://master:7077 \
  sparkJob.jar

如果这样做，我还建议将 akka-actor 设置为提供的测试 在您的 build.sbt 中，这样您就不会在中也包含 akka-actor sparkJob.jar 。

If you do this, I also recommend setting akka-actor to "test, provided" in your build.sbt, so that you don't also include akka-actor in sparkJob.jar.

这篇关于java.lang.NoSuchMethodError：akka.actor.ActorCell.addFunctionRef的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！