Python Pandas:如何仅读取CSV文件的前n行? [英] Python Pandas: How to read only first n rows of CSV files in?

问题描述

我有一个非常大的数据集，我无法读取其中的整个数据集.因此，我正在考虑只读取其中的一个数据块进行训练，但是我不知道该怎么做.任何想法将不胜感激.

I have a very large data set and I can't afford to read the entire data set in. So, I'm thinking of reading only one chunk of it to train but I have no idea how to do it. Any thought will be appreciated.

推荐答案

如果您只想读取前999,999行(非标题):

If you only want to read the first 999,999 (non-header) rows:

read_csv(..., nrows=999999)

如果您只想读取1,000,000 ... 1,999,999行

If you only want to read rows 1,000,000 ... 1,999,999

read_csv(..., skiprows=1000000, nrows=999999)

行 :int，默认值无读取文件的行数.对...有用 读取大文件*

nrows : int, default None Number of rows of file to read. Useful for reading pieces of large files*

skiprows :类似列表或整数 文件开头要跳过的行号(索引为0)或要跳过的行数(整数)

skiprows : list-like or integer Row numbers to skip (0-indexed) or number of rows to skip (int) at the start of the file

对于大型文件，您可能还需要使用chunksize:

and for large files, you'll probably also want to use chunksize:

chunksize :整数，默认值无 返回TextFileReader对象进行迭代

chunksize : int, default None Return TextFileReader object for iteration

pandas.io.parsers.read_csv文档

这篇关于Python Pandas:如何仅读取CSV文件的前n行?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！