解析大文件时出现内存错误-Python [英] Memory Error When Parsing Large File - Python

问题描述

有人问过有关Python中的内存错误的问题，但我想问一个更具体的情况.我是编程和Python的新手.

There have been questions asked about memory errors in Python, but I want to ask one more specific to my situation. I am new to programming and Python.

解析大型文本文件(〜8GB)时，该行

When parsing a large text file (~8GB), the line

mylist = [line.strip('\n').split('|') for line in f]

导致"MemoryError".

resulted in "MemoryError".

我在具有12GB RAM的Windows XP 64位上运行Python的64位[MSC v.1500 64位(AMD64)].除了安装更多的RAM，我该如何处理此内存错误?

I am running the 64-bit of Python [MSC v.1500 64 bit (AMD64)] on Windows XP 64-bit with 12GB of RAM. How can I handle this Memory Error other than installing more RAM?

推荐答案

出现内存错误是因为您试图将整个文件存储在列表(位于内存中)中.因此，尝试在每一行上工作而不是存储它:

The memory error is coming because you're trying to store your whole file in a list(which is in memory). So, try to work on each line instead of storing it:

for line in f:
   data = line.strip('\n').split('|')
   #do something here with data

这篇关于解析大文件时出现内存错误-Python的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！