在管道中间关闭流 [英] Closing streams in the middle of pipelines
问题描述
当我执行此代码时，会在流管道中打开大量文件：
When I execute this code which opens a lot of files during a stream pipeline:
public static void main(String[] args) throws IOException {
Files.find(Paths.get("JAVA_DOCS_DIR/docs/api/"),
100, (path, attr) -> path.toString().endsWith(".html"))
.map(file -> runtimizeException(() -> Files.lines(file, StandardCharsets.ISO_8859_1)))
.map(Stream::count)
.forEachOrdered(System.out::println);
}
我得到一个例外：
java.nio.file.FileSystemException: /long/file/name: Too many open files
问题是 Stream.count 在遍历流时不会关闭流。但我不明白为什么它不应该，因为它是一个终端操作。对于其他终端操作也是如此，例如 reduce 和 forEach 。另一方面， flatMap 关闭它所包含的流。
The problem is that Stream.count does not close the stream when it is done traversing it. But I don't see why it shouldn't, given that it is a terminal operation. The same holds for other terminal operations such as reduce and forEach. flatMap on the other hand closes the streams it consists of.
文档告诉我使用try-with -resouces-statement，如有必要，关闭流。在我的情况下，我可以用这样的东西替换 count 行：
The documentation tells me to use a try-with-resouces-statement to close streams if necessary. In my case I could replace the count line with something like this:
.map(s -> { long c = s.count(); s.close(); return c; } )
但这种情况很嘈杂，在某些情况下可能会给大型复杂的管道带来真正的不便。
But that is noisy and ugly and could be a real inconvenience in some cases with big, complex pipelines.
所以我的问题如下：
- 为什么流的设计不是为了让终端操作关闭他们正在处理的流？这将使它们在IO流中更好地工作。
- 关闭流水线中IO流的最佳解决方案是什么？
runtimizeException 是一个包装已检查异常的方法 RuntimeException s。
runtimizeException is a method that wraps checked exception in RuntimeExceptions.
推荐答案
这里有两个问题：处理已检查的异常，例如 IOException ，及时关闭资源。
There are two issues here: handling of checked exceptions such as IOException, and timely closing of resources.
没有任何预定义的功能接口声明任何已检查的异常，这意味着它们必须在lambda中处理，或包装在未经检查的异常中并重新抛出。看起来你的 runtimizeException 函数就是这样做的。您可能还必须为它声明自己的功能接口。正如您可能已经发现的那样，这是一种痛苦。
None of the predefined functional interfaces declare any checked exceptions, which means that they have to be handled within the lambda, or wrapped in an unchecked exception and rethrown. It looks like your runtimizeException function does that. You probably also had to declare your own functional interface for it. As you've probably discovered, this is a pain.
在关闭文件等资源时，有一些关于在结束时自动关闭流的调查流达到了。这很方便，但是它不会在抛出异常时处理关闭。在流中没有神奇的做正确的机制。
On the closing of resources like files, there was some investigation of having streams be closed automatically when the end of the stream was reached. This would be convenient, but it doesn't deal with closing when an exception is thrown. There's no magic do-the-right-thing mechanism for this in streams.
我们留下了处理资源关闭的标准Java技术，即 Java 7中引入的try-with-resources 构造.TWR确实希望在打开调用堆栈时将资源关闭到同一级别。 打开它的人必须关闭它的原则适用。 TWR还处理异常处理，通常可以方便地在同一个地方处理异常处理和资源关闭。
We're left with the standard Java techniques of dealing with resource closure, namely the try-with-resources construct introduced in Java 7. TWR really wants to have resources be closed at the same level in the call stack as they were opened. The principle of "whoever opens it has to close it" applies. TWR also deals with exception handling, which usually makes it convenient to deal with exception handling and resource closing in the same place.
在这个例子中，流有点不寻常它将 Stream< Path> 映射到 Stream< Stream< String>> 。这些嵌套流是未关闭的流，当系统用完打开的文件描述符时会导致最终异常。使这很困难的是文件通过一个流操作打开然后传递到下游;这使得无法使用TWR。
In this example, the stream is somewhat unusual in that it maps a Stream<Path> to a Stream<Stream<String>>. These nested streams are the ones that aren't closed, resulting in the eventual exception when the system runs out of open file descriptors. What makes this difficult is that files are opened by one stream operation and then passed downstream; this makes it impossible to use TWR.
构建此管道的另一种方法如下。
An alternative approach to structuring this pipeline is as follows.
Files.lines call是打开文件的那个，所以这必须是TWR语句中的资源。这个文件的处理是抛出（某些） IOExceptions ，所以我们可以在同一个TWR语句中进行异常包装。这表明有一个简单的函数将路径映射到行数，同时处理资源关闭和异常包装：
The Files.lines call is the one that opens the file, so this has to be the resource in the TWR statement. The processing of this file is where (some) IOExceptions get thrown, so we can do the exception wrapping in the same TWR statement. This suggests having a simple function that maps the path to a line count, while handling resource closing and exception wrapping:
long lineCount(Path path) {
try (Stream<String> s = Files.lines(path, StandardCharsets.ISO_8859_1)) {
return s.count();
} catch (IOException ioe) {
throw new UncheckedIOException(ioe);
}
}
一旦你有了这个辅助函数，主管道看起来像这样：
Once you have this helper function, the main pipeline looks like this:
Files.find(Paths.get("JAVA_DOCS_DIR/docs/api/"),
100, (path, attr) -> path.toString().endsWith(".html"))
.mapToLong(this::lineCount)
.forEachOrdered(System.out::println);
这篇关于在管道中间关闭流的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
