预处理要与tflearn一起使用的csv文件 [英] Preprocessing csv files to use with tflearn
问题描述
我的问题是关于在将csv文件输入到神经网络之前对其进行预处理.
My question is about preprocessing csv files before inputing them into a neural network.
我想在python 3中使用tflearn为著名的虹膜数据集建立一个深度神经网络.
I want to build a deep neural network for the famous iris dataset using tflearn in python 3.
数据集: http://archive.ics .uci.edu/ml/machine-learning-databases/iris/iris.data
我正在使用tflearn加载csv文件.但是,我的数据集的类别"列中有诸如鸢尾花,鸢尾花,鸢尾花等词.
I'm using tflearn to load the csv file. However, the classes column of my data set has words such as iris-setosa, iris-versicolor, iris-virginica.
神经网络仅适用于数字.因此,我必须找到一种方法来将类从单词更改为数字.由于它是一个非常小的数据集,因此我可以使用Excel/文本编辑器手动进行操作.我为不同的班级手动分配了数字.
Nueral networks work only with numbers. So, I have to find a way to change the classes from words to numbers. Since it is a very small dataset, I can do it manually using Excel/text editor. I manually assigned numbers for different classes.
但是,我不可能为我处理的每个数据集都做到这一点.因此,我尝试使用熊猫执行一次热编码.
But, I can't possibly do it for every dataset I work with. So, I tried using pandas to perform one hot encoding.
preprocess_data = pd.read_csv("F:\Gautam\.....\Dataset\iris_data.csv")
preprocess_data = pd.get_dummies(preprocess_data)
但是现在,我无法使用这段代码:
But now, I can't use this piece of code:
data, labels = load_csv('filepath', categorical_labels=True,
n_classes=3)
"filepath"应仅是csv文件的目录,而不是诸如preprocess_data之类的任何变量.
'filepath' should only be a directory to the csv file, not any variable like preprocess_data.
原始数据集:
Sepal Length Sepal Width Petal Length Petal Width Class
89 5.5 2.5 4.0 1.3 iris-versicolor
85 6.0 3.4 4.5 1.6 iris-versicolor
31 5.4 3.4 1.5 0.4 iris-setosa
52 6.9 3.1 4.9 1.5 iris-versicolor
111 6.4 2.7 5.3 1.9 iris-virginica
手动修改的数据集:
Sepal Length Sepal Width Petal Length Petal Width Class
89 5.5 2.5 4.0 1.3 1
85 6.0 3.4 4.5 1.6 1
31 5.4 3.4 1.5 0.4 0
52 6.9 3.1 4.9 1.5 1
111 6.4 2.7 5.3 1.9 2
这是我的代码,可以很好地运行,但是,我手动修改了数据集.
Here's my code which runs perfectly, but, I have modified the dataset manually.
import numpy as np
import pandas as pd
import tflearn
from tflearn.layers.core import input_data, fully_connected
from tflearn.layers.estimator import regression
from tflearn.data_utils import load_csv
data_source = 'F:\Gautam\.....\Dataset\iris_data.csv'
data, labels = load_csv(data_source, categorical_labels=True,
n_classes=3)
network = input_data(shape=[None, 4], name='InputLayer')
network = fully_connected(network, 9, activation='sigmoid', name='Hidden_Layer_1')
network = fully_connected(network, 3, activation='softmax', name='Output_Layer')
network = regression(network, batch_size=1, optimizer='sgd', learning_rate=0.2)
model = tflearn.DNN(network)
model.fit(data, labels, show_metric=True, run_id='iris_dataset', validation_set=0.1, n_epoch=2000)
我想知道tflearn中是否有其他内置函数(或其他任何模块),可以用来将类的值从单词更改为数字.我认为手动修改数据集不会产生效果.
I want to know if there's any other built-in function in tflearn (or in any other module, for that matter) that I can use to modify the value of my classes from words to numbers. I don't think manually modifying the datasets would be productive.
我也是tflearn和神经网络的初学者.任何帮助,将不胜感激.谢谢.
I'm a beginner in tflearn and neural networks also. Any help would be appreciated. Thanks.
推荐答案
使用sklearn
库中的标签编码器:
Use label encoder from sklearn
library:
from sklearn.preprocessing import LabelEncoder,OneHotEncoder
df = pd.read_csv('iris_data.csv',header=None)
df.columns=[Sepal Length,Sepal Width,Petal Length,Petal Width,Class]
enc=LabelEncoder()
df['Class']=enc.fit_transform(df['Class'])
print df.head(5)
如果要One-hot encoding
,则首先需要labelEncode,然后执行OneHotEncoding:
if you want One-hot encoding
then first you need to labelEncode then do OneHotEncoding :
enc=LabelEncoder()
enc_1=OneHotEncoder()
df['Class']=enc.fit_transform(df['Class'])
df['Class']=enc_1.fit_transform([df['Class']]).toarray()
print df.head(5)
这些编码器首先按字母顺序对单词进行排序,然后为其分配标签.如果要查看将哪个标签分配给哪个类,请执行以下操作:
These encoders first sort the words in alphabetical order then assign them labels. If you want to see which label is assigned to which class, do:
for k in list(enc.classes_) :
print 'name ::{}, label ::{}'.format(k,enc.transform([k]))
如果要将此数据帧另存为csv文件,请执行以下操作:
If you want to save this dataframe as a csv file, do:
df.to_csv('Processed_Irisdataset.csv',sep=',')
这篇关于预处理要与tflearn一起使用的csv文件的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!