在Java 8中逐行读取Spring Multipartfile的最佳方法 [英] Best way to read Spring Multipartfile line by line in Java 8

问题描述

csv Spring Multipartfile的最佳处理方法是什么? 我以前曾经用过这样的东西:

What is the best handle a csv Spring Multipartfile? I have used something like this before:

public void handleFile(MultipartFile multipartFile){
    try{
        InputStream inputStream = multipartFile.getInputStream();
        IOUtils.readLines(inputStream, StandardCharsets.UTF_8)
                .stream()
                .forEach(this::handleLine);
    } catch (IOException e) {
        // handle exception
    }
}

private void handleLine(String s) {
    // do stuff per line
}

据我所知，这首先将整个文件加载到内存中的列表中，然后再处理它，这可能需要花费相当多的时间来处理几万行的文件.

As far as I know, this first loads the whole file into a list in memory before processing it, which will probably take quite some time for files with tens of thousends of lines.

有没有一种方法可以逐行处理它，而无需手动执行迭代(即使用诸如read()，hasNext(之类的东西)的开销?)? 我正在为文件系统中的文件寻找类似于此示例的简洁内容:

Is there a way to handle it line by line without the overhead of implementing the iteration by hand (i.e. using stuff like read(), hasNext(), ...)? I am looking for something concise similar to this example for files from the file system:

try (Stream<String> stream = Files.lines(Paths.get("file.csv"))) {
        stream.forEach(this::handleLine);
} catch (IOException e) {
    // handle exception
}

推荐答案

在拥有InputStream的情况下，可以使用以下代码:

In cases when you have InputStream you can use this one:

InputStream inputStream = multipartFile.getInputStream();
new BufferedReader(new InputStreamReader(inputStream, StandardCharsets.UTF_8))
                    .lines()
                    .forEach(this::handleLine);

在其他情况下:

无论是多部分文件还是有多个独立文件，在Java 8中都有许多使用Stream API的方法来实现:

No matter whether it is multipart file or you have multiple independent files, there are many approaches to do it in Java 8 using Stream API:

解决方案1:

如果文件位于其他目录中，则可以通过以下方式进行操作:

If your files are in different directories you can do it this way:

假设您的List为String，其中包含文件的路径，如下所示:

Imagine you have a List of String which contains paths of your files like below:

List<String> files = Arrays.asList(
                "/test/test.txt",
                "/test2/test2.txt");

然后您可以阅读上述文件的所有行，如下所示:

Then you can read all lines of above files as below:

files.stream().map(Paths::get)
        .flatMap(path -> {
            try {
                return Files.lines(path);
            } catch (IOException e) {
                e.printStackTrace();
            }
            return Stream.empty();
        }).forEach(System.out::println);

解决方案2:

您还可以按以下方式使用Files.walk读取/test/ehsan目录中存在的文件的所有行:

You can also read all lines of files that exist in /test/ehsan directory using Files.walk in the following way:

try (Stream<Path> stream = Files.walk(Paths.get("/test/ehsan"), 1)) {
    stream.filter(Files::isRegularFile)
            .flatMap(path -> {
                try {
                    return Files.lines(path);
                } catch (IOException e) {
                    e.printStackTrace();
                }
                return Stream.empty();
            })
            .forEach(System.out::println);
} catch (IOException e) {
    e.printStackTrace();
}

如果要递归地读取/test/ehsan目录中的所有文件行，则可以通过以下方式做到这一点:

And if you want to read all lines of files in /test/ehsan directory recursively you can do it this way:

try (Stream<Path> stream = Files.walk(Paths.get("/test/ehsan"))) {
    stream.filter(Files::isRegularFile)
            .flatMap(path -> {
                try {
                    return Files.lines(path);
                } catch (IOException e) {
                    e.printStackTrace();
                }
                return Stream.empty();
            })
            .forEach(System.out::println);
} catch (IOException e) {
    e.printStackTrace();
}

如您所见，Files.walk的第二个参数指定要访问的最大目录级别数，如果不通过，则将使用默认值Integer.MAX_VALUE.

As you can see the second parameter to Files.walk specifies the maximum number of directory levels to visit and if you don't pass it the default will be used which is Integer.MAX_VALUE.

解决方案3:

不要在这里停下来，我们可以走得更远.如果我们要读取文件的所有行都位于两个完全不同的目录(例如/test/ehsan和/test2/ehsan1)中怎么办?

Lets not stop here, we can go further. what if we wanted to read all lines of files exist in two completely different directories for example /test/ehsan and /test2/ehsan1?

我们可以做到，但是我们应该谨慎，Stream不应太长(因为这会降低程序的可读性)，最好将它们拆分为单独的方法，但是，因为不可能编写多个我将在这里写一个方法来做到这一点:

We can do it but we should be cautious, Stream should not be so long( because it reduces readability of our program) it will be better to break them in separate methods, However because it is not possible to write multiple methods here I will write in one place how to do that:

想象一下，您的List为String，其中包含目录的路径，如下所示:

Imagine you have a List of String which contains paths of your directories like below

list<String> dirs = Arrays.asList(
                "/test/ehsan",
                "/test2/ehsan1");

然后我们可以这样:

dirs.stream()
        .map(Paths::get)
        .flatMap(path -> {
            try {
                return Files.walk(path);
            } catch (IOException e) {
                e.printStackTrace();
            }
            return Stream.empty();
        })
        .filter(Files::isRegularFile)
        .flatMap(path -> { 
            try {
                return Files.lines(path);
            } catch (IOException e) {
                e.printStackTrace();
            }
            return Stream.empty();
        })
        .forEach(System.out::println);

这篇关于在Java 8中逐行读取Spring Multipartfile的最佳方法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！