Scala 脚本无法在 Ubuntu 上运行 [英] Scala script doesn't run on Ubuntu
问题描述
我有一个以前工作的 Scala 脚本，当我尝试在新 PC 上运行它时，编译失败.
所以我做了一个简单的脚本来测试:
#!/bin/shexec scala -J-Xmx2g "$0" "$@"!#println("测试")并尝试运行它我得到:
test.scala错误:编译服务器遇到致命条件:java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;在 scala.tools.nsc.io.SourceReader.read(SourceReader.scala:61)在 scala.tools.nsc.io.SourceReader.read(SourceReader.scala:40)在 scala.tools.nsc.io.SourceReader.read(SourceReader.scala:49)在 scala.tools.nsc.Global.getSourceFile(Global.scala:395)在 scala.tools.nsc.Global.getSourceFile(Global.scala:401)在 scala.tools.nsc.Global$Run$$anonfun$30.apply(Global.scala:1607)在 scala.tools.nsc.Global$Run$$anonfun$30.apply(Global.scala:1607)在 scala.collection.immutable.List.map(List.scala:284)在 scala.tools.nsc.Global$Run.compile(Global.scala:1607)在 scala.tools.nsc.StandardCompileServer.session(CompileServer.scala:151)在 scala.tools.util.SocketServer$$anonfun$doSession$1$$anonfun$apply$1.apply(SocketServer.scala:74)在 scala.tools.util.SocketServer$$anonfun$doSession$1$$anonfun$apply$1.apply(SocketServer.scala:74)在 scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)在 scala.Console$.withOut(Console.scala:65)在 scala.tools.util.SocketServer$$anonfun$doSession$1.apply(SocketServer.scala:74)在 scala.tools.util.SocketServer$$anonfun$doSession$1.apply(SocketServer.scala:69)在 scala.tools.nsc.io.Socket.applyReaderAndWriter(Socket.scala:49)在 scala.tools.util.SocketServer.doSession(SocketServer.scala:69)在 scala.tools.util.SocketServer.loop$1(SocketServer.scala:85)在 scala.tools.util.SocketServer.run(SocketServer.scala:97)在 scala.tools.nsc.CompileServer$$anonfun$execute$2$$anonfun$apply$mcZ$sp$1.apply$mcZ$sp(CompileServer.scala:218)在 scala.tools.nsc.CompileServer$$anonfun$execute$2$$anonfun$apply$mcZ$sp$1.apply(CompileServer.scala:213)在 scala.tools.nsc.CompileServer$$anonfun$execute$2$$anonfun$apply$mcZ$sp$1.apply(CompileServer.scala:213)在 scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)在 scala.Console$.withOut(Console.scala:53)在 scala.tools.nsc.CompileServer$$anonfun$execute$2.apply$mcZ$sp(CompileServer.scala:213)在 scala.tools.nsc.CompileServer$$anonfun$execute$2.apply(CompileServer.scala:213)在 scala.tools.nsc.CompileServer$$anonfun$execute$2.apply(CompileServer.scala:213)在 scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)在 scala.Console$.withErr(Console.scala:80)在 scala.tools.nsc.CompileServer$.execute(CompileServer.scala:212)在 scala.tools.nsc.CompileServer$.main(CompileServer.scala:180)在 scala.tools.nsc.CompileServer.main(CompileServer.scala)似乎 Scala 正在我的脚本附近编译一些东西，但我不太知道如何调试和修复它.
TL;DR
Ubuntu 的 Scala 软件包曾经与 Java 8 不兼容(这已在 2.11.12-4 中修复).解决方案是卸载 Ubuntu 的 Scala 包并安装 官方 Scala 包之一.这一次您可能仍然想要这样做，不是因为与 Java 不兼容，而是因为 Ubuntu 的最新打包 Scala 版本仍然是 2.11，而 Scala 的最新版本目前是 2.13.
sudo apt remove scala-library scalawget https://downloads.lightbend.com/scala/2.13.4/scala-2.13.4.deb须藤 dpkg -i scala-2.13.4.deb因为很多人都在问这个问题背后的原因，我也很好奇是什么原因造成的，所以我做了一些挖掘......
问题的根源
在 Java 9 中，Buffer 子类(包括 ByteBuffer)被更改为覆盖超类中返回 Buffer 的方法以返回相应的子类型.
错误:https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-4774077
提交: https://github.com/AdoptOpenJDK/openjdk-jdk9/commit/d9d7e875470bf478110b849315b4fff55b4c35cf
此更改不是二进制向后兼容的.如果某些在 Buffer 的子类之一中直接调用这些方法的 Java 代码是使用 JDK9+ 编译的，则生成的字节码将不会在 JRE8 中运行(即使根本不使用返回值).发生这种情况是因为调用时方法的签名将被编译为 java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer 这在 JRE8 中不存在.但是，如果用JDK8编译，编译成字节码的签名将是java/nio/ByteBuffer.clear:()Ljava/nio/Buffer，它存在于JRE8和JRE9+的Buffer类中.>
Scala 哪里出错了?(或者做到了?)
Scala 编译器确实使用了一些受上述更改影响的方法.特别是，在 SourceReader 类，其中发生 OP 问题中的错误.
查看 Scala 的兼容性矩阵，它说我们至少需要 Scala 2.11.12 才能使用 JDK11，但它并没有明确说明兼容性的相反方向.虽然它确实说Scala 2.12+ 绝对不能在 JDK 6 或 7 上运行"，所以我们可以预期 2.12+ 仍然与 JDK8 兼容，甚至更兼容 Scala 2.11.
那他们为什么要破坏兼容性?他们不能用旧的 JDK 版本编译 Scala 的源代码吗?他们没有，而且他们可以，所以他们仍然这样做.
如果我们下载官方 Scala 软件包之一并检查 scala-compiler.jar 的清单文件，我们会发现:
Scala 2.11.12:
捆绑包名称:Scala 编译器Bundle-RequiredExecutionEnvironment:JavaSE-1.6、JavaSE-1.7Bundle-SymbolicName: org.scala-lang.scala-compiler捆绑版本:2.11.12.v20171031-225310-b8155a5502创建者:1.6.0_45(Sun Microsystems Inc.)Scala 2.13.4:
捆绑包名称:Scala 编译器Bundle-RequiredExecutionEnvironment:JavaSE-1.8Bundle-SymbolicName: org.scala-lang.scala-compiler捆绑版本:2.13.4.v20201117-181115-VFINAL-39148e4创建者:1.8.0_275(采用OpenJDK)所以似乎 Scala 2.11 仍在使用 JDK6 编译，而 Scala 2.13 仍在使用 JDK8 编译.这不应该意味着它们都与 JRE8 兼容吗?是的，而且确实如此.那么错误来自哪里?
Ubuntu 哪里出了问题?
与大多数其他 Linux 发行版一样，Ubuntu 喜欢构建自己的包，这些包通过其包管理器提供.这样做是为了确保操作系统生态系统内的一切正常运行，这通常意味着修补上游项目的源代码.
特别是关于 Scala 包，Ubuntu 决定放弃用于编译 Scala 源代码的 JDK 版本的上游选择和 一段时间以来一直在使用较新的 JDK 版本来编译 Ubuntu 的 Scala 包.
如果我们检查 Ubuntu 的 Scala 2.11.12-4 中 scala-compiler.jar 的清单文件，我们可以看到它是用 JDK11 编译的:
创建者:11.0.2+9-Ubuntu-3ubuntu1(甲骨文公司)捆绑包名称:Scala 发行版Bundle-SymbolicName: org.scala-ide.scala.compiler;singleton:=true捆绑版本:2.11.12你不是说2.11.12-4的问题已经解决了吗?是的，我做到了.
Ubuntu 对这个问题的解决方案不是用 JDK8 编译 Scala，而是对 Scala 的源代码进行打补丁，以避免直接在子类中调用有问题的方法.这是通过在调用这些方法之前将 ByteBuffer(和 CharBuffer)转换为其超类 Buffer 来实现的.实际上，这意味着将 Scala 的源代码从 bytes.clear() 更改为 bytes.asInstanceOf[Buffer].clear().asInstanceOf[ByteBuffer](不知道为什么当 clear() 的结果似乎根本没有被使用时，将其转换回 ByteBuffer).这是 Ubuntu 的补丁.
Ubuntu 的方法似乎有点危险，因为其他不兼容的来源可能会被忽视，并且在某些非常特定的情况下仍然等待发生.此外，他们自己的设置与 Scala 官方版本不同，这意味着不需要整个 Scala 社区在实际场景中测试这些更改.
I have a previously working Scala script that when I try to run it on a new PC, the compilation fails.
So I made simple script to test:
#!/bin/sh
exec scala -J-Xmx2g "$0" "$@"
!#
println("test")
And trying to run it I get:
test.scala
error: Compile server encountered fatal condition: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
at scala.tools.nsc.io.SourceReader.read(SourceReader.scala:61)
at scala.tools.nsc.io.SourceReader.read(SourceReader.scala:40)
at scala.tools.nsc.io.SourceReader.read(SourceReader.scala:49)
at scala.tools.nsc.Global.getSourceFile(Global.scala:395)
at scala.tools.nsc.Global.getSourceFile(Global.scala:401)
at scala.tools.nsc.Global$Run$$anonfun$30.apply(Global.scala:1607)
at scala.tools.nsc.Global$Run$$anonfun$30.apply(Global.scala:1607)
at scala.collection.immutable.List.map(List.scala:284)
at scala.tools.nsc.Global$Run.compile(Global.scala:1607)
at scala.tools.nsc.StandardCompileServer.session(CompileServer.scala:151)
at scala.tools.util.SocketServer$$anonfun$doSession$1$$anonfun$apply$1.apply(SocketServer.scala:74)
at scala.tools.util.SocketServer$$anonfun$doSession$1$$anonfun$apply$1.apply(SocketServer.scala:74)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at scala.Console$.withOut(Console.scala:65)
at scala.tools.util.SocketServer$$anonfun$doSession$1.apply(SocketServer.scala:74)
at scala.tools.util.SocketServer$$anonfun$doSession$1.apply(SocketServer.scala:69)
at scala.tools.nsc.io.Socket.applyReaderAndWriter(Socket.scala:49)
at scala.tools.util.SocketServer.doSession(SocketServer.scala:69)
at scala.tools.util.SocketServer.loop$1(SocketServer.scala:85)
at scala.tools.util.SocketServer.run(SocketServer.scala:97)
at scala.tools.nsc.CompileServer$$anonfun$execute$2$$anonfun$apply$mcZ$sp$1.apply$mcZ$sp(CompileServer.scala:218)
at scala.tools.nsc.CompileServer$$anonfun$execute$2$$anonfun$apply$mcZ$sp$1.apply(CompileServer.scala:213)
at scala.tools.nsc.CompileServer$$anonfun$execute$2$$anonfun$apply$mcZ$sp$1.apply(CompileServer.scala:213)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at scala.Console$.withOut(Console.scala:53)
at scala.tools.nsc.CompileServer$$anonfun$execute$2.apply$mcZ$sp(CompileServer.scala:213)
at scala.tools.nsc.CompileServer$$anonfun$execute$2.apply(CompileServer.scala:213)
at scala.tools.nsc.CompileServer$$anonfun$execute$2.apply(CompileServer.scala:213)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at scala.Console$.withErr(Console.scala:80)
at scala.tools.nsc.CompileServer$.execute(CompileServer.scala:212)
at scala.tools.nsc.CompileServer$.main(CompileServer.scala:180)
at scala.tools.nsc.CompileServer.main(CompileServer.scala)
It seems like Scala is compiling something near my script, but I don't quite know how to debug it and fix it.
TL;DR
Ubuntu's Scala package used to be incompatible with Java 8 (this has been fixed in 2.11.12-4). The solution was to uninstall Ubuntu's Scala package and install one of the official Scala packages. You might still want to do that, this time around, not due to incompatibility with Java, but because Ubuntu's latest packaged Scala version is still 2.11, while Scala's latest version is currently 2.13.
sudo apt remove scala-library scala
wget https://downloads.lightbend.com/scala/2.13.4/scala-2.13.4.deb
sudo dpkg -i scala-2.13.4.deb
Since many people were asking for the reason behind this issue and I was also curious about what caused it, I did some digging...
The root of the problem
In Java 9, Buffer subclasses (including ByteBuffer) were changed to override methods that in the superclass return Buffer to return the respective subtype.
Bug: https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-4774077
Commit: https://github.com/AdoptOpenJDK/openjdk-jdk9/commit/d9d7e875470bf478110b849315b4fff55b4c35cf
This change is not binary backward compatible. If some Java code which calls one these methods directly in one of Buffer's subclasses is compiled with JDK9+, the generated bytecode will not run in JRE8 (even if the returned value is not used at all). This happens because the signature of the method when called will be compiled as java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer which doesn't exist in JRE8. However, if compiled with JDK8, the signature compiled into bytecode would be java/nio/ByteBuffer.clear:()Ljava/nio/Buffer which exists in the Buffer calss in both JRE8 and JRE9+.
Where did Scala go wrong? (Or did it?)
Scala compiler does use some of the methods affected by the changes above. Particularly, in the SourceReader class where the error in OP's question happened.
Looking at Scala's compatibility matrix, it says that we need at least Scala 2.11.12 to use JDK11, but it doesn't say much explicitly about the opposite direction of compatibility. It does say though that "Scala 2.12+ definitely doesn't work at all on JDK 6 or 7", so we could expect that 2.12+ is still compatible with JDK8, and even more so Scala 2.11.
Why did they break the compatibility then? Couldn't they just compile Scala's source code with an older JDK version? They didn't and they could, so much, that they still do it.
If we download one of the official Scala packages and check the manifest file for scala-compiler.jar, this is what we find:
Scala 2.11.12:
Bundle-Name: Scala Compiler
Bundle-RequiredExecutionEnvironment: JavaSE-1.6, JavaSE-1.7
Bundle-SymbolicName: org.scala-lang.scala-compiler
Bundle-Version: 2.11.12.v20171031-225310-b8155a5502
Created-By: 1.6.0_45 (Sun Microsystems Inc.)
Scala 2.13.4:
Bundle-Name: Scala Compiler
Bundle-RequiredExecutionEnvironment: JavaSE-1.8
Bundle-SymbolicName: org.scala-lang.scala-compiler
Bundle-Version: 2.13.4.v20201117-181115-VFINAL-39148e4
Created-By: 1.8.0_275 (AdoptOpenJDK)
So it seems Scala 2.11 is still being compiled with JDK6 and Scala 2.13 is still being compiled with JDK8. Shouldn't that mean that they are both compatible with JRE8? Yes and indeed they are. Where's the error coming from then?
Where did Ubuntu go wrong?
Ubuntu, as most other Linux distributions do, likes to build its own packages that are made available through its package manager. This is done to ensure that everything works properly within the OS ecosystem, and that often means patching the source code of upstream projects.
Regarding the Scala package in particular, Ubuntu decided to ditch the upstream choices of JDK versions used to compile the Scala source code and has been using newer JDK versions to compile Ubuntu's Scala package for a while.
If we check the manifest file for scala-compiler.jar in Ubuntu's Scala 2.11.12-4, we can see that is was compiled with JDK11:
Created-By: 11.0.2+9-Ubuntu-3ubuntu1 (Oracle Corporation)
Bundle-Name: Scala Distribution
Bundle-SymbolicName: org.scala-ide.scala.compiler;singleton:=true
Bundle-Version: 2.11.12
Didn't you say the issue was resolved in 2.11.12-4? Yes, I did.
Ubuntu's solution for this problem was not to compile Scala with JDK8, but rather to patch Scala's source code to avoid calling the problematic methods directly in the subclasses. This was achieved by casting ByteBuffer (and CharBuffer) to its superclass Buffer before calling these methods. In practice, that meant changing Scala's source code from bytes.clear() to bytes.asInstanceOf[Buffer].clear().asInstanceOf[ByteBuffer] (not sure why they cast it back to ByteBuffer when the result from clear() doesn't seem to be used at all). Here is Ubuntu's patch.
Ubuntu's approach seems a bit dangerous, because other sources of incompatibility could have gone unnoticed and still be there waiting to happen in some very specific situation. Also having their own setup different from the official Scala releases means not having the whole Scala community testing these changes in real-case scenarios.
这篇关于Scala 脚本无法在 Ubuntu 上运行的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
