Spring JDBC支持和大型数据集 [英] Spring JDBC support and large dataset

问题描述

当使用各种JDBC模板方法之一时，我对如何迭代/滚动大型结果集（不适合内存）感到困惑。即使没有直接暴露Iterable接口，我至少也会期望RowCallbackHandler的实例被调用，而查询正在执行，而不是在它完成之后（或堆溢出）。

When using one of the various JDBC template methods I am confused on how to iterate/scroll over large result sets (which won't fit into memory). Even without a direct exposure of an Iterable interface I would at least expect instances of RowCallbackHandler to get called while the query is executing not after it's finished (or the heap overfloats).

我确实看过这个 （尽管在堆栈溢出方面与这篇文章的精神相似，但对我没有任何改变）并在春季论坛这篇帖子中。后者似乎建议在游标获取数据时确实应该调用回调处理程序。然而，我的测试显示没有这样的行为。

I did have a look a at this (which changed nothing for me despite being similar in spirit to this post on stack overflow) and at this post in the spring forums. The latter seems to suggest that the callback handler should indeed get called while the cursor is fetching data. My tests however show no such behaviour.

数据库是Oracle10g。我使用的是11.1.0.7.0-Production驱动程序和Spring 2.5.6.SEC01。任何想法如何迭代结果集的想法，最好是保留RowMapper的映射逻辑等？

The database is an Oracle10g. I am using the 11.1.0.7.0-Production driver and Spring 2.5.6.SEC01. Any ideas anyone how to iterate over result sets, preferably while keeping the mapping logic of RowMapper etc.?

推荐答案

Oracle JDBC驱动程序对 java.sql.Statement 上的 setFetchSize（）方法有适当的支持，它允许你控制多少行驱动程序将一次性获取。

The Oracle JDBC driver has proper support for the setFetchSize() method on java.sql.Statement, which allows you to control how many rows the driver will fetch in one go.

然而，Spring使用的 RowMapper 通过将每行读入内存来工作，获取 RowMapper 将其转换为对象，并将每行的对象存储在一个大列表中。如果你的结果集很大，那么无论JDBC如何获取行数据，这个列表都会变大。

However, RowMapper as used by Spring works by reading each row into memory, getting the RowMapper to translate it into an object, and storing each row's object in one big list. If your result set is huge, then this list will get big, regardless of how JDBC fetches the row data.

如果你需要处理大的结果集，那么RowMapper不是不可扩展。您可以考虑使用 RowCallbackHandler ，以及JdbcTemplate上的相应方法。 RowCallbackHandler 没有规定结果的存储方式，由你来存储它们。

If you need to handle large result sets, then RowMapper isn't scaleable. You might consider using RowCallbackHandler instead, along with the corresponding methods on JdbcTemplate. RowCallbackHandler doesn't dictate how the results are stored, leaving it up to you to store them.

这篇关于Spring JDBC支持和大型数据集的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！