感知器在线训练(scikit-learn) [英] Perceptron online training (scikit-learn)

问题描述

我写了一个简单的程序来对一组线性可分的二维随机点进行分类.我使用了一个感知器，并用 fit 方法对其进行了训练.现在我想一次训练一个感知器，每次使用更新的权重绘制超平面(在这种情况下是一条线).我想要获得的是一个动画，它显示了线条如何变得越来越精确地划分集合.fit 方法取了整个训练集，partial_fit 呢?我可以做一个循环，每次我用一对新的输入/输出输入方法，并连续读取 coef_ 和intercept_ 吗?

i wrote a simple program to classify a set of linearly separable 2D random points. I used a perceptron and i trained it with the fit method. Now i'd like to train the perceptron one point at time, plotting each time the hyperplane (a line in this case) using the updated weights. What i want to obtain is an animation which shows how the line become more and more precise dividing the sets. The fit method takes the entire training set, what about the partial_fit? Can i make a loop where i feed the method each time with a new single couple of input/output, and read continuously the coef_ and intercept_?

我在这里阅读了文档http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html 但我对如何实现它有一些疑问.

I read the documentation here http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html but i have some doubts on how to implement it.

感谢 Vivek Kumar，我在我的代码中实现了 partial_fit 方法.该程序创建 2 组坐标，并为每对坐标产生一个输出，如果点在一条线上，则为 1，如果点在这条线上，则为 -1.该代码适用于 fit 方法，但此版本在数据形状方面存在一些问题.我尝试对 X 数据使用 reshape，但没有任何改进.

Thanks to Vivek Kumar, i implemented the partial_fit method to my code. The program creates 2 sets of coordinates, and for each couple produces an output which is 1 if the point is over a line and -1 if it is under the line. The code works with the fit method, but this versions gives some problems with the data shape. I tried to use reashape for the X data without any improvement.

import numpy as np
import matplotlib.pyplot as plt

def createLinearSet(nCamp, mTest, qTest):

    y_ = []
    X_ = np.random.rand(nCamp, 2)*20-10

    for n in range(nCamp):

        if X_[n][1] >= mTest*X_[n][0]+qTest :
            y_.append(1)
        else:
            y_.append(-1)

    return X_, y_

########################################################################
# VARIABLES
iterazioni = 100
eta = 0.6
y = []
error = []

########################################################################
# CREATING DATA SET
m_test = -2
q_test = 3
n_camp = 100

X, y = createLinearSet(n_camp, m_test, q_test)

########################################################################
# 70 % training data and 30 % test data
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.3,           random_state = 0)

########################################################################
# Data normalization
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
sc.fit(X_train)                       # Calcola la media dei campioni e la deviazione standard
X_train_std = sc.transform(X_train)   # Normalizza i dati di test e di addestramento
X_test_std = sc.transform(X_test)     # NB. uso media e deviazione dei dati di add. per entrambi,  
                                      #     così sono confrontabili
########################################################################
# Perceptron initialization
from sklearn.linear_model import Perceptron
ppn = Perceptron(n_iter = iterazioni, eta0 = eta, random_state = 0)

########################################################################
# Online training
num_samples = X_train_std.shape[0]
classes_y =  np.unique(y_train)

X_train_std = X_train_std.reshape(-1, 2)

for i in range(num_samples):
    ppn.partial_fit(X_train_std[i], y_train[i], classes = classes_y )

########################################################################
# Using test data for evaluation
y_pred = ppn.predict(X_test_std)

########################################################################
# Previsions accuracy
from sklearn.metrics import accuracy_score
accuracy = accuracy_score(y_test, y_pred) * 100

print("Accuracy: {} %".format(round(accuracy,2)))

print(ppn.coef_, ppn.intercept_) 

如您所见，问题出在在线培训"部分.错误是:

As you can see, the problem is in the "Online training" section. The error is:

/usr/local/lib/python3.5/dist-packages/sklearn/utils/validation.py:395: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.

从文档中，X 必须是:X : {array-like, sparse matrix}, shape (n_samples, n_features)

From documentation, X must be: X : {array-like, sparse matrix}, shape (n_samples, n_features)

如果我打印 X 的单个样本，输出为:[-0.25547959 -1.4763508]

If i print a single sample of X, the output is: [-0.25547959 -1.4763508 ]

错误在哪里?

将行 X_train_std[i].reshape(1,-1) 放入循环中，它给了我以下消息:回溯(最近一次调用最后一次):文件Perceptron_Retta_Online.py"，第 57 行，在 <module> 中ppn.partial_fit(X_train_std[i].reshape(1,-1), y_train[i], classes = classes_y )文件/usr/local/lib/python3.5/dist-packages/sklearn/linear_model/stochastic_gradient.py"，第512行，在partial_fit中coef_init=None，intercept_init=None)文件/usr/local/lib/python3.5/dist-packages/sklearn/linear_model/stochastic_gradient.py"，第344行，_partial_fitX, y = check_X_y(X, y, 'csr', dtype=np.float64, order="C")文件/usr/local/lib/python3.5/dist-packages/sklearn/utils/validation.py"，第526行，在check_X_yy = column_or_1d(y, 警告=真)文件/usr/local/lib/python3.5/dist-packages/sklearn/utils/validation.py"，第 562 行，在 column_or_1draise ValueError("错误的输入形状{0}".format(shape))ValueError: 错误的输入形状 ()

Putting the line X_train_std[i].reshape(1,-1) in the loop it gives me this message: Traceback (most recent call last): File "Perceptron_Retta_Online.py", line 57, in <module> ppn.partial_fit(X_train_std[i].reshape(1,-1), y_train[i], classes = classes_y ) File "/usr/local/lib/python3.5/dist-packages/sklearn/linear_model/stochastic_gradient.py", line 512, in partial_fit coef_init=None, intercept_init=None) File "/usr/local/lib/python3.5/dist-packages/sklearn/linear_model/stochastic_gradient.py", line 344, in _partial_fit X, y = check_X_y(X, y, 'csr', dtype=np.float64, order="C") File "/usr/local/lib/python3.5/dist-packages/sklearn/utils/validation.py", line 526, in check_X_y y = column_or_1d(y, warn=True) File "/usr/local/lib/python3.5/dist-packages/sklearn/utils/validation.py", line 562, in column_or_1d raise ValueError("bad input shape {0}".format(shape)) ValueError: bad input shape ()

推荐答案

大多数 scikit 估算器一次处理完整数据，即.当您调用 fit() 方法时，旧的训练(权重、系数等)将丢失，模型仅适用于新数据.

Most scikit estimators work on the full data at once, ie. when you call the fit() method, the older training (weights, coefficients etc) are lost, and model is fit on the new data only.

所以要像你说的那样实现它，你是正确的，我做一个循环，我每次都用一对新的输入/输出来提供方法，并连续读取 coef_ 和intercept_"，但有轻微的改变.您需要提供所有以前的值和新的单个值，而不是提供新的一对输入/输出.

So to implement it like you said, you are correct that "i make a loop where i feed the method each time with a new single couple of input/output, and read continuously the coef_ and intercept_", but with a slight change. Instead of feeding the new single couple of input/ output, you need to feed all previous values and the new single values.

为了说明我的观点，您可以执行以下操作(这有点伪代码，因此请阅读评论):

To illustrate my point, what you can do is following (This is somewhat pseudo coded, so read along the comments):

X = your_input
y = your_output
num_samples = X.shape[0]
for i in range(num_samples):

    # 1) This is necessary to initialize a new estimator everytime
    numpy.random.seed

    # 2) This is necessary to initialize a new estimator everytime
    estimator = MLPClassifier() OR SGDClassifier() OR ...

    # 3) Call fit on data from 0 to current index i
    estimator.fit(X[:i+1], y[:i+1])

    # Read the following values and do what you want
    estimator.coeff_ OR estimator.intercept_

说明:1)在循环内部设置随机种子参数，因此所有循环中权重的初始化都相同(到相同的值).见第 2 点.

Explanation: 1) Setting the random seed parameter inside the loop, so the initialization of weights in same (to same values) in all loops. See point 2.

2) 每次都初始化新对象，而不是使用相同的对象(在循环外初始化)，因此分配给它的初始权重是随机的(并且由于第 1 点而固定).

2) Initialization of new object each time, instead of using same object (initialized out of the loop), so that the initial weights assigned to it are random (and fixed due to point 1).

3) 根据需要使用从 0 到当前索引的所有值来学习.对于第一个循环，它只有第一个值(索引 0)，对于第二个循环，它的第一个和第二个值，依此类推...

3) Use all the values from 0 to current index to learning is as you want. For first loop its only 1st values (index 0), for 2nd loop its 1st and 2nd values, and so on ...

这种方法适用于所有 scikit 估计器，但由于重复训练相同的数据点而需要大量时间(如果数据非常大).

This approach will work for all scikit-estimators but takes a lot of time (if the data is very large) because of repeated training of same data points.

替代方案:当您链接 SGDClassifier 时，我假设您可能对在线学习或核外学习感兴趣，这完全符合您的要求.但并非所有估算器都能做到这一点.只有在官方文档中列出的估算器才能执行此操作.

ALTERNATIVE: As you linked the SGDClassifier, so I assume you may be interested in Online-learning or Out-of-core learning, which serves your requirement exactly. But not all estimators are able to do this. Only estimators which are listed in the official documentation are capable of doing this.

他们实现了一个 partial_fit() 方法，这正是您想要的.所以对他们来说，代码将是这样的:

They implement a partial_fit() method, which is what you want. So for them the code will be like this:

X = your_input
y = your_output
num_samples = X.shape[0]

estimator = SGDClassifier() OR ... (which implements partial_fit())

for i in range(num_samples):

    estimator.fit(X[i], y[i])

    estimator.coeff_ OR estimator.intercept_

希望它清除并为您提供方法.如果需要帮助，请随时询问.

Hope it clears up and give you a way to do it. Feel free to ask if need help.

这篇关于感知器在线训练(scikit-learn)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！