pandas :对DataFrame进行采样 [英] Pandas: Sampling a DataFrame
问题描述
我正在尝试使用Pandas读取一个相当大的CSV文件,并将其分成两个随机的块,其中一个占数据的10%,另一个占90%.
I'm trying to read a fairly large CSV file with Pandas and split it up into two random chunks, one of which being 10% of the data and the other being 90%.
这是我目前的尝试:
rows = data.index
row_count = len(rows)
random.shuffle(list(rows))
data.reindex(rows)
training_data = data[row_count // 10:]
testing_data = data[:row_count // 10]
由于某些原因,当我尝试在SVM分类器中使用这些结果DataFrame对象之一时,sklearn
引发此错误:
For some reason, sklearn
throws this error when I try to use one of these resulting DataFrame objects inside of a SVM classifier:
IndexError: each subindex must be either a slice, an integer, Ellipsis, or newaxis
我认为我做错了.有更好的方法吗?
I think I'm doing it wrong. Is there a better way to do this?
推荐答案
您使用的是哪个版本的熊猫?对我来说,您的代码工作正常(我在git master上使用).
What version of pandas are you using? For me your code works fine (i`m on git master).
另一种方法可能是:
In [117]: import pandas
In [118]: import random
In [119]: df = pandas.DataFrame(np.random.randn(100, 4), columns=list('ABCD'))
In [120]: rows = random.sample(df.index, 10)
In [121]: df_10 = df.ix[rows]
In [122]: df_90 = df.drop(rows)
较新的版本(从0.16.1开始)直接支持此功能: http://pandas.pydata.org/pandas-docs /stable/generation/pandas.DataFrame.sample.html
Newer version (from 0.16.1 on) supports this directly: http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.sample.html
这篇关于 pandas :对DataFrame进行采样的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!