使用 scikit learn (sklearn) 批量梯度下降 [英] Batch gradient descent with scikit learn (sklearn)

问题描述

我正在使用 Scikit-Learn (sklearn) 使用一对多逻辑回归分类器.我有一个很大的数据集，一次运行太慢；我还想随着培训的进行研究学习曲线.

I'm playing with a one-vs-all Logistic Regression classifier using Scikit-Learn (sklearn). I have a large dataset that is too slow to run all at one go; also I would like to study the learning curve as the training proceeds.

我想使用批量梯度下降来批量训练我的分类器，比如 500 个样本.有什么方法可以使用 sklearn 来做到这一点，还是应该放弃 sklearn 并自己动手"?

I would like to use batch gradient descent to train my classifier in batches of, say, 500 samples. Is there some way of using sklearn to do this, or should I abandon sklearn and "roll my own"?

这是我目前所拥有的:

from sklearn.linear_model import LogisticRegression
from sklearn.multiclass import OneVsRestClassifier

# xs are subsets of my training data, ys are ground truth for same; I have more 
# data available for further training and cross-validation:
xs.shape, ys.shape
# => ((500, 784), (500))
lr = OneVsRestClassifier(LogisticRegression())
lr.fit(xs, ys)
lr.predict(xs[0,:])
# => [ 1.]
ys[0]
# => 1.0

即它正确识别了一个训练样本(是的，我意识到用新数据对其进行评估会更好——这只是一个快速的冒烟测试).

I.e. it correctly identifies a training sample (yes, I realize it would be better to evaluate it with new data -- this is just a quick smoke-test).

R.e.批量梯度下降:我还没有创建学习曲线，但是可以简单地在训练数据的后续子集上重复运行 fit 吗?或者还有其他一些功能可以批量训练吗?文档和谷歌对此事保持沉默.谢谢！

R.e. batch gradient descent: I haven't gotten as far as creating learning curves, but can one simply run fit repeatedly on subsequent subsets of the training data? Or is there some other function to train in batches? The documentation and Google are fairly silent on the matter. Thanks!

推荐答案

你要的不是批量梯度下降，而是随机梯度下降；批量学习意味着一次性学习整个训练集，而您所描述的正确称为小批量学习.这是在 sklearn.linear_model.SGDClassifier 中实现的，如果你给它选项 loss="log"，它适合逻辑回归模型.

What you want is not batch gradient descent, but stochastic gradient descent; batch learning means learning on the entire training set in one go, while what you describe is properly called minibatch learning. That's implemented in sklearn.linear_model.SGDClassifier, which fits a logistic regression model if you give it the option loss="log".

使用 SGDClassifier，就像使用 LogisticRegression 一样，不需要将估算器包装在 OneVsRestClassifier 中——两者都是一对一的开箱即用的培训.

With SGDClassifier, like with LogisticRegression, there's no need to wrap the estimator in a OneVsRestClassifier -- both do one-vs-all training out of the box.

# you'll have to set a few other options to get good estimates,
# in particular n_iterations, but this should get you going
lr = SGDClassifier(loss="log")

然后，要对小批量进行训练，请使用 partial_fit 方法而不是 fit.第一次，您必须向它提供一个类列表，因为并非每个小批量中都可能存在所有类:

Then, to train on minibatches, use the partial_fit method instead of fit. The first time around, you have to feed it a list of classes because not all classes may be present in each minibatch:

import numpy as np
classes = np.unique(["ham", "spam", "eggs"])

for xs, ys in minibatches:
    lr.partial_fit(xs, ys, classes=classes)

(在这里，我为每个 minibatch 传递了 classes，这不是必需的，但也不会造成伤害，并使代码更短.)

(Here, I'm passing classes for each minibatch, which isn't necessary but doesn't hurt either and makes the code shorter.)

这篇关于使用 scikit learn (sklearn) 批量梯度下降的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！