在Java Spring Rest服务中下载并阅读一个大型的csv文件 [英] Download and read on a large csv file in java Spring Rest service

问题描述

我有一个春天休息服务，每10分钟后定期运行。服务知道存储在远程服务器上的文件的路径。该文件的大小是2GB左右。服务提取文件，然后读取文件的内容，然后根据文件内容操作数据库。春天休息服务获取大文件并对其内容进行操作的最佳方式是什么？使用Apache文件utils，

http://commons.apache.org/proper/commons-io/apidocs/org/apache/commons/io/FileUtils.html#copyURLToFile(java.net.URLrel =nofollow> http：// commons。 apache.org/proper/commons-io/apidocs/org/apache/commons/io/FileUtils.html#copyURLToFile(java.net.URL ，java.io.File，int，int）

下载1 GB文件的示例代码如下所示：

  package com.jai.dow​​nload; 
 
 import java.io.File; 
 import java.net.URL; 
 
 import org.apache.commons.io.FileUtils ; 
 
 public class FileDownload {
 
 public static void main（String [] args）{
 
 File file = new File（c：/ jj / 1gb.zip）; 
 
 URL url; 
 try {
 url = new URL（http：//download.thinkbroa dband.com/1GB.zip）; 
 long start = System.currentTimeMillis（）; 
 System.out.println（Downloading ....）; 
 FileUtils.copyURLToFile（url，file）; 
 long end = System.currentTimeMillis（）; 
 System.out.println（Completed .... in ms：+（end  -  start））; 
} catch（Exception e）{
 e.printStackTrace（）; 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 <我的互联网速度为100 Mbps，它在94秒内下载了这个1GB的文件。一旦你有这样的CSV文件下载，你可以阅读它的内容，你做业务逻辑检查，并采取适当的行动。
 
I have a Spring rest service which runs periodically after every 10 min. Service knows the path to a file stored on remote server. Size of the file is around 2GB. Service fetches the file and then reads contents of the file and then manipulates the database based on file contents. What is  the best way for spring rest service to fetch large file and operate upon its contents. 
解决方案
Use Apache File utils,

http://commons.apache.org/proper/commons-io/apidocs/org/apache/commons/io/FileUtils.html#copyURLToFile(java.net.URL, java.io.File, int, int)

An example code to download 1 GB file from online is as below,
            package com.jai.download;

            import java.io.File;
            import java.net.URL;

            import org.apache.commons.io.FileUtils;

            public class FileDownload {

                public static void main(String[] args) {

                    File file = new File("c:/jj/1gb.zip");

                    URL url;
                    try {
                        url = new URL("http://download.thinkbroadband.com/1GB.zip");
                        long start = System.currentTimeMillis();
                        System.out.println("Downloading....");
                        FileUtils.copyURLToFile(url, file);
                        long end = System.currentTimeMillis();
                        System.out.println("Completed....in ms : " + (end - start));
                    } catch (Exception e) {
                        e.printStackTrace();
                    }

                }

            }
With my internet speed of 100 Mbps, it downloaded this 1GB file in 94 secs. Once you have your csv file like this downloaded you could read its contents to do you business logic check and make appropriate action.

                        
这篇关于在Java Spring Rest服务中下载并阅读一个大型的csv文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！