缩放数据时，为什么训练数据集使用'fit'和'transform'，而测试数据集只使用'transform'? [英] When scale the data, why the train dataset use 'fit' and 'transform', but the test dataset only use 'transform'?

问题描述

缩放数据时，为什么训练数据集使用'fit'和'transform'，而测试数据集只使用'transform'?

When scale the data, why the train dataset use 'fit' and 'transform', but the test dataset only use 'transform'?

SAMPLE_COUNT = 5000
TEST_COUNT = 20000
seed(0)
sample = list()
test_sample = list()
for index, line in enumerate(open('covtype.data','rb')):
    if index < SAMPLE_COUNT:
        sample.append(line)
    else:
        r = randint(0,index)
        if r < SAMPLE_COUNT:
            sample[r] = line
        else:
            k = randint(0,index)
            if k < TEST_COUNT:
                if len(test_sample) < TEST_COUNT:
                    test_sample.append(line)
                else:
                    test_sample[k] = line
from sklearn.preprocessing import StandardScaler
for n, line in enumerate(sample):
sample[n] = map(float, line.strip().split(','))
y = np.array(sample)[:,-1]
scaling = StandardScaler()

X = scaling.fit_transform(np.array(sample)[:,:-1]) ##here use fit and transform

for n,line in enumerate(test_sample):
test_sample[n] = map(float,line.strip().split(','))
yt = np.array(test_sample)[:,-1]

Xt = scaling.transform(np.array(test_sample)[:,:-1])##why here only use transform

正如注解所说，为什么 Xt 只使用变换而不使用拟合?

As the annotation says, why Xt only use transform but no fit?

推荐答案

我们在训练数据上使用 fit_transform() 以便我们在训练数据上学习缩放参数，同时我们缩放列车数据.我们只在测试数据上使用 transform() 因为我们使用在训练数据上学习的缩放参数来缩放测试数据.

We use fit_transform() on the train data so that we learn the parameters of scaling on the train data and in the same time we scale the train data. We only use transform() on the test data because we use the scaling paramaters learned on the train data to scale the test data.

这是缩放的标准程序.你总是在火车上学习你的缩放参数，然后在测试中使用它们.这是一篇很好地解释它的文章:https://sebastianraschka.com/faq/docs/scale-training-test.html

This is the standart procedure to scale. You always learn your scaling parameters on the train and then use them on the test. Here is an article that explane it very well : https://sebastianraschka.com/faq/docs/scale-training-test.html

这篇关于缩放数据时，为什么训练数据集使用'fit'和'transform'，而测试数据集只使用'transform'?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！