从大表中检索所有记录时如何避免OOM（Out of memory）错误？ [英] How to avoid OOM (Out of memory) error when retrieving all records from huge table?

问题描述

我有一项任务是将一个巨大的表转换为自定义XML文件。我将使用Java来完成这项工作。

I am given a task to convert a huge table to custom XML file. I will be using Java for this job.

如果我只是发出SELECT * FROM customer，它可能会返回最终导致OOM的大量数据。我想知道，有没有办法可以立即处理记录，并在sql检索过程中从内存中删除记录？

If I simply issue a "SELECT * FROM customer", it may return huge amount of data that eventually causing OOM. I wonder, is there a way i can process the record immediately once it become available, and remove the record from memory after that during sql retrieving process?

---编辑2009年7月13日

--- edited on 13 Jul 2009

让我详细说明我的问题。我有1个db服务器和1个应用服务器。
当我在应用程序中发出选择查询时，数据将从数据库服务器传送到应用服务器。

Let me elaborate my question. I have 1 db server and 1 application server. When I issue a select query in application, the data will travel from db server to app server.

我相信（如果我错了，请纠正我）ResultSet将需要等到接收查询中的所有记录。即使我们将获取大小设置为4，对于1000记录表，我们仍然最终在app服务器的堆内存中有1000条记录，这是正确的吗？获取大小仅影响从/向数据库服务器的往返次数。

I believe (correct me if I am wrong) ResultSet will need to wait until receiving all records in the query. Even if we set fetch size as 4, for a 1000-record table, we still end up having 1000 records in heap memory of app server, is it correct? Fetch size only affect the number of round trip from/to db server.

我的问题是，如何在它之后立即开始处理该4（或任何数字）记录到达应用服务器，并将其处置以释放应用服务器中的内存？

My question is, how to start processing on that 4 (or any number) records immediately after it's arrival to app server, and dispose it to free up memory in app server?

推荐答案

我认为您可以使用相同的解决方案这一个。可滚动的结果集。

I think you could use the same solution as this one. A scrollable resultset.

这篇关于从大表中检索所有记录时如何避免OOM（Out of memory）错误？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！