打印数百万行/JAVA堆空间 [英] printing millions of rows / JAVA heap space

问题描述

我正在使用某些数据库，因此需要将每个数据库打印为文件. 我的服务器有5 GB的RAM，而我为雄猫服务器分配了4GB.

I'm working with some databases and I need to print each database in files. My Server has 5 GB of RAM and I assigned 4GB just for tomcat server.

我制作了一个简单的结果集，该结果集查询了每个数据库的所有行. 然后，我将每行打印到文件(.dat)中(很明显，我是分开做的)

I made a simple resultset that consult all the rows of each database. Then I print each row in a file (.dat) (Obviously I do that separately)

我可以处理具有5738095行的整个结果集

I can work with an entire result set that has 5738095 rows

但是，如果我想使用具有超过700万行的数据库，则会出现错误"Java堆空间"，那么我从3GB和2GB的tomcat开始，但是当我开始处理超过300万的数据库时，我需要向服务器添加更多的RAM行，所以这里的问题是，将更多的RAM放入服务器是否好，或者如何在不出现错误"java堆空间"的情况下对结果集进行划分并打印百万分之一.

But if I want to work with a database with more than 7 millions of rows I got the error "Java heap space", I've started with 3GB and 2GB for tomcat but when I started working with more than 3 millions of rows I needed to add more RAM to my server, so my question here is, is it good to put more RAM to my server or how can I divide the result set and print million per million without get the error " java heap space".

我一直在考虑对整个数据库进行计数，并做出一些限制0偏移1000000，限制1000000偏移1000001，但是我真心迷失了.感谢您的阅读和帮助，对不起我的英语.

I've been thinking in count the entire database and make something like limit 0 offset 1000000, limit 1000000 offset 1000001, but sincerely I'm lost. Thanks for reading and helping and sorry for my english.

有一些代码

//numero columnas is the total of columns
 rs_datos =  StDatos.executeQuery("select * from table");
  while(rs_datos.next())
  {

    for(int i = 0; i < numeroColumnas; i++)
    {
      if(i+1 == numeroColumnas)
      {
       pw.print(rs_datos.getString(i+1));
      }
      else
      pw.print(rs_datos.getString(i+1) + "|");
    }

     pw.println("");
  }

  pw.close();

推荐答案

您可以从应用程序或cron执行命令，并将其用于以csv格式创建文档 采用CSV格式的Mysqldump

You can execute command from application or cron and use this for create a document in format csv Mysqldump in CSV format

这篇关于打印数百万行/JAVA堆空间的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！