打印数百万行/JAVA堆空间 [英] printing millions of rows / JAVA heap space
问题描述
我正在使用某些数据库,因此需要将每个数据库打印为文件. 我的服务器有5 GB的RAM,而我为雄猫服务器分配了4GB.
I'm working with some databases and I need to print each database in files. My Server has 5 GB of RAM and I assigned 4GB just for tomcat server.
我制作了一个简单的结果集,该结果集查询了每个数据库的所有行. 然后,我将每行打印到文件(.dat)中(很明显,我是分开做的)
I made a simple resultset that consult all the rows of each database. Then I print each row in a file (.dat) (Obviously I do that separately)
我可以处理具有5738095行的整个结果集
I can work with an entire result set that has 5738095 rows
但是,如果我想使用具有超过700万行的数据库,则会出现错误"Java堆空间",那么我从3GB和2GB的tomcat开始,但是当我开始处理超过300万的数据库时,我需要向服务器添加更多的RAM行,所以这里的问题是,将更多的RAM放入服务器是否好,或者如何在不出现错误"java堆空间"的情况下对结果集进行划分并打印百万分之一.
But if I want to work with a database with more than 7 millions of rows I got the error "Java heap space", I've started with 3GB and 2GB for tomcat but when I started working with more than 3 millions of rows I needed to add more RAM to my server, so my question here is, is it good to put more RAM to my server or how can I divide the result set and print million per million without get the error " java heap space".
我一直在考虑对整个数据库进行计数,并做出一些限制0偏移1000000,限制1000000偏移1000001,但是我真心迷失了.感谢您的阅读和帮助,对不起我的英语.
I've been thinking in count the entire database and make something like limit 0 offset 1000000, limit 1000000 offset 1000001, but sincerely I'm lost. Thanks for reading and helping and sorry for my english.
有一些代码
//numero columnas is the total of columns
rs_datos = StDatos.executeQuery("select * from table");
while(rs_datos.next())
{
for(int i = 0; i < numeroColumnas; i++)
{
if(i+1 == numeroColumnas)
{
pw.print(rs_datos.getString(i+1));
}
else
pw.print(rs_datos.getString(i+1) + "|");
}
pw.println("");
}
pw.close();
推荐答案
您可以从应用程序或cron执行命令,并将其用于以csv格式创建文档 采用CSV格式的Mysqldump
You can execute command from application or cron and use this for create a document in format csv Mysqldump in CSV format
这篇关于打印数百万行/JAVA堆空间的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!