骆驼 jdbc 内存不足异常 [英] camel jdbc out of memory exception

问题描述

我正在尝试将数据从 postgres 摄取到另一个数据库，并且我正在使用 camel-jdbc 组件来执行此操作.我有一个大表，所以我想一次读取几行而不是整个表.所以我的路线如下所示(仅用于测试目的)from(fromUri).setBody("select * from table limit 10").to("jdbc://myDataSource?resetAutoCommit=false&statement.fetchSize=2").split(body()).streaming().process(测试)

I am trying to ingest data from postgres to another DB and I am using camel-jdbc component to do it. I have a large table so I want to read few rows at a time instead of the whole table altogether. so my route looks like below (only for testing purpose) from(fromUri).setBody("select * from table limit 10").to("jdbc://myDataSource?resetAutoCommit=false&statement.fetchSize=2").split(body()).streaming().process(test)

如上所示，出于测试目的，我一次仅获取 10 行，并且我已将 fetchSize 设置为 2 以一次仅接收 2 行.但是，我仍然总共收到所有 10 行.当我从查询中删除限制 10"时，我在 split 命令之前收到内存不足错误，该错误告诉我它试图将整个结果集加载到内存中.

As shown above, I am only getting 10 rows at a time for testing purpose and I have set fetchSize to 2 to only receive 2 rows at a time. However, I am still receiving all 10 rows altogether. When I remove the "limit 10" from the query I get Out of Memory error just before the split command which tells me that its trying to load the entire result set in memory.

我在这里遗漏了什么或者我做错了什么?

What am I missing here or what am I doing wrong?

感谢您的帮助.

推荐答案

我认为 fetchSize 更像是对 JDBC 驱动程序的提示.您可以使用 maxRows 选项来真正限制服务器端，例如 statement.maxRows=2.您可以在 JDBC javadoc 文档中阅读有关这些选项的更多信息.

I think fetchSize is more of a hint to the JDBC driver. You can use the maxRows option to really limit on the server side, eg statement.maxRows=2. You can read more about these options on the JDBC javadoc documentation.

https://docs.oracle.com/javase/7/docs/api/java/sql/Statement.html

这篇关于骆驼 jdbc 内存不足异常的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！