大型 Excel 文件 - 上传和导入 - Java [英] Large Excel File - Upload and Import - Java

问题描述

全部

我想将一个超过一百万条记录的大型 Excel 文件上传并导入到我的 Java 程序中.

I want to upload and import a large excel files having more than one million records in to my Java program.

我可以使用 Apache POI 轻松地将小文件导入到我的系统中，但是当我开始处理大文件时，应用程序会抛出内存不足错误，

I can easily import small files using Apache POI in to my system , but when i starts with large files application throws and out of memory error,

我在谷歌上搜索并找到了很多主题，所以我尝试了一切，但无法解决这个问题.

任何人都可以为我的特定问题提供解决方案，导入时间对我来说不是问题，现在我也可以忍受性能问题，

can anybody give me solution for my particular problem, import time is not an issue for me, right now also i can bear with performance issue as well,

只想将这些数据导入到我现有的系统中，而不会出现 oem 错误.

just want to import this data in to my existing system without oem error.

我的系统配置非常好，java 有足够的内存可以使用，所以硬件不是问题.

谢谢

推荐答案

您需要流式传输数据，这样您就不需要一次将所有记录存储在内存中.POI 确实支持流式传输(请参阅 XSSF 和 SAX 事件 API).当您读取数据时，将其发送到您需要的任何地方(数据库或任何您未指定的地方)——使用流式 API，您不应在处理数据之前将所有数据读入内存.

You'll want to stream the data so that you don't need to store all the records in memory at once. POI does support streaming (see XSSF and SAX event API). As you read the data, ship it off to wherever you need to (database or wherever, you did not specify) -- with the streaming API you should not read all the data into memory before processing it.

您也可以将数据导出到 CSV 文件，然后使用常规 FileInputStream 读取文件并在读取时处理每条记录.

You could also export the data to a CSV file, and then use a regular FileInputStream to read the file and process each record as it is read.

这篇关于大型 Excel 文件 - 上传和导入 - Java的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！