如何将sklearn fit_transform与pandas一起使用并返回数据框而不是numpy数组? [英] How to use sklearn fit_transform with pandas and return dataframe instead of numpy array?

问题描述

我想将缩放比例(使用来自sklearn.preprocessing的StandardScaler())应用于熊猫数据框.以下代码返回一个numpy数组，因此我丢失了所有列名和索引.这不是我想要的.

I want to apply scaling (using StandardScaler() from sklearn.preprocessing) to a pandas dataframe. The following code returns a numpy array, so I lose all the column names and indeces. This is not what I want.

features = df[["col1", "col2", "col3", "col4"]]
autoscaler = StandardScaler()
features = autoscaler.fit_transform(features)

我在网上找到的解决方案"是:

A "solution" I found online is:

features = features.apply(lambda x: autoscaler.fit_transform(x))

它似乎可以工作，但会导致弃用警告:

It appears to work, but leads to a deprecationwarning:

/usr/lib/python3.5/site-packages/sklearn/preprocessing/data.py:583: DeprecationWarning:在0.17中弃用数据时传递一维数组 并将在0.19中引发ValueError.使用以下方法重塑数据 X.reshape(-1，1)(如果您的数据具有单个功能)或X.reshape(1，-1) 如果它包含一个样本.

/usr/lib/python3.5/site-packages/sklearn/preprocessing/data.py:583: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.

我因此尝试:

features = features.apply(lambda x: autoscaler.fit_transform(x.reshape(-1, 1)))

但这给出了:

回溯(最近一次通话最后一次):文件"./analyse.py"，第91行，在 features = features.apply(lambda x:autoscaler.fit_transform(x.reshape(-1，1)))文件 "/usr/lib/python3.5/site-packages/pandas/core/frame.py"，第3972行，在 申请 返回self._apply_standard(f，axis，reduce = reduce)文件"/usr/lib/python3.5/site-packages/pandas/core/frame.py"，第4081行，在 _apply_standard 结果= self._constructor(数据=结果，索引=索引)文件"/usr/lib/python3.5/site-packages/pandas/core/frame.py"，第226行，在 初始化 mgr = self._init_dict(数据，索引，列，dtype = dtype)文件"/usr/lib/python3.5/site-packages/pandas/core/frame.py"，第363行，在 _init_dict dtype = dtype)文件"/usr/lib/python3.5/site-packages/pandas/core/frame.py"，第5163行，在 _arrays_to_mgr 数组= _homogenize(数组，索引，dtype)文件"/usr/lib/python3.5/site-packages/pandas/core/frame.py"，第5477行，在 _均质 (raise_cast_failure = False)文件"/usr/lib/python3.5/site-packages/pandas/core/series.py"，第2885行， 在_sanitize_array中 引发异常(数据必须是一维的")异常:数据必须是一维的

Traceback (most recent call last): File "./analyse.py", line 91, in features = features.apply(lambda x: autoscaler.fit_transform(x.reshape(-1, 1))) File "/usr/lib/python3.5/site-packages/pandas/core/frame.py", line 3972, in apply return self._apply_standard(f, axis, reduce=reduce) File "/usr/lib/python3.5/site-packages/pandas/core/frame.py", line 4081, in _apply_standard result = self._constructor(data=results, index=index) File "/usr/lib/python3.5/site-packages/pandas/core/frame.py", line 226, in init mgr = self._init_dict(data, index, columns, dtype=dtype) File "/usr/lib/python3.5/site-packages/pandas/core/frame.py", line 363, in _init_dict dtype=dtype) File "/usr/lib/python3.5/site-packages/pandas/core/frame.py", line 5163, in _arrays_to_mgr arrays = _homogenize(arrays, index, dtype) File "/usr/lib/python3.5/site-packages/pandas/core/frame.py", line 5477, in _homogenize raise_cast_failure=False) File "/usr/lib/python3.5/site-packages/pandas/core/series.py", line 2885, in _sanitize_array raise Exception('Data must be 1-dimensional') Exception: Data must be 1-dimensional

如何将缩放应用于熊猫数据框，而使数据框完整无缺?尽可能不复制数据.

How do I apply scaling to the pandas dataframe, leaving the dataframe intact? Without copying the data if possible.

推荐答案

您可以使用

You could convert the DataFrame as a numpy array using as_matrix(). Example on a random dataset:

根据上述as_matrix()文档的最后一句话将as_matrix()更改为values(不会更改结果):

Changing as_matrix() to values, (it doesn't change the result) per the last sentence of the as_matrix() docs above:

通常，建议使用".values".

Generally, it is recommended to use ‘.values’.

import pandas as pd
import numpy as np #for the random integer example
df = pd.DataFrame(np.random.randint(0.0,100.0,size=(10,4)),
              index=range(10,20),
              columns=['col1','col2','col3','col4'],
              dtype='float64')

注意，索引为10-19:

Note, indices are 10-19:

In [14]: df.head(3)
Out[14]:
    col1    col2    col3    col4
    10  3   38  86  65
    11  98  3   66  68
    12  88  46  35  68

现在fit_transform DataFrame以获得scaled_features array:

Now fit_transform the DataFrame to get the scaled_features array:

from sklearn.preprocessing import StandardScaler
scaled_features = StandardScaler().fit_transform(df.values)

In [15]: scaled_features[:3,:] #lost the indices
Out[15]:
array([[-1.89007341,  0.05636005,  1.74514417,  0.46669562],
       [ 1.26558518, -1.35264122,  0.82178747,  0.59282958],
       [ 0.93341059,  0.37841748, -0.60941542,  0.59282958]])

将缩放后的数据分配给DataFrame(注意:使用index和columns关键字参数来保留原始索引和列名:

Assign the scaled data to a DataFrame (Note: use the index and columns keyword arguments to keep your original indices and column names:

scaled_features_df = pd.DataFrame(scaled_features, index=df.index, columns=df.columns)

In [17]:  scaled_features_df.head(3)
Out[17]:
    col1    col2    col3    col4
10  -1.890073   0.056360    1.745144    0.466696
11  1.265585    -1.352641   0.821787    0.592830
12  0.933411    0.378417    -0.609415   0.592830

---

遍历 sklearn-pandas 包.它致力于使scikit-learn更易于与熊猫一起使用.当您需要将一种以上类型的转换应用于DataFrame的列子集(一种更常见的情况)时，sklearn-pandas特别有用.它已记录在案，但这是您实现我们刚刚执行的转换的方式.

Came across the sklearn-pandas package. It's focused on making scikit-learn easier to use with pandas. sklearn-pandas is especially useful when you need to apply more than one type of transformation to column subsets of the DataFrame, a more common scenario. It's documented, but this is how you'd achieve the transformation we just performed.

from sklearn_pandas import DataFrameMapper

mapper = DataFrameMapper([(df.columns, StandardScaler())])
scaled_features = mapper.fit_transform(df.copy(), 4)
scaled_features_df = pd.DataFrame(scaled_features, index=df.index, columns=df.columns)

这篇关于如何将sklearn fit_transform与pandas一起使用并返回数据框而不是numpy数组?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！