Pentaho文本文件输入步骤崩溃(内存不足) [英] Pentaho text file input step crashing (out of memory)

问题描述

我正在使用Pentaho来读取非常大的文件. 11GB.

I am using Pentaho for reading a very large file. 11GB.

该进程有时会因内存不足异常而崩溃，有时它会说进程已终止.

The process is sometime crashing with out of memory exception, and sometimes it will just say process killed.

我正在12 GB的计算机上运行该作业，并为该进程分配了8 GB的空间.

I am running the job on a machine with 12GB, and giving the process 8 GB.

是否可以通过一些配置来运行文本文件输入步骤以使用更少的内存?也许更多地使用磁盘?

Is there a way to run the Text File Input step with some configuration to use less memory? maybe use the disk more?

谢谢！

推荐答案

打开spoon.sh/bat或pan/kettle .sh或.bat并更改-Xmx数字.搜索JAVAMAXMEM即使您有多余的内存，除非允许Java使用它，否则它将无法正常工作.为了公平起见，尽管在上面的示例中，我仍然无法真正理解为什么/如何会占用大量内存！

Open up spoon.sh/bat or pan/kettle .sh or .bat and change the -Xmx figure. Search for JAVAMAXMEM Even though you have spare memory unless java is allowed to use it it wont work. although to be fair in your example above i can't really see why/how it would be consuming much memory anyway!

这篇关于Pentaho文本文件输入步骤崩溃(内存不足)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！