使用KNN在Python中缺少值插补 [英] Missing value imputation in python using KNN
问题描述
我有一个看起来像这样的数据集
I have a dataset that looks like this
1908 January 5.0 -1.4
1908 February 7.3 1.9
1908 March 6.2 0.3
1908 April NaN 2.1
1908 May NaN 7.7
1908 June 17.7 8.7
1908 July NaN 11.0
1908 August 17.5 9.7
1908 September 16.3 8.4
1908 October 14.6 8.0
1908 November 9.6 3.4
1908 December 5.8 NaN
1909 January 5.0 0.1
1909 February 5.5 -0.3
1909 March 5.6 -0.3
1909 April 12.2 3.3
1909 May 14.7 4.8
1909 June 15.0 7.5
1909 July 17.3 10.8
1909 August 18.8 10.7
我想使用KNN作为方法替换NaN
.我查找了sklearn
的Imputer
类,但它仅支持均值,中位数和众数插补.在此处有功能请求,但我认为这没有实现截至目前.关于如何使用KNN替换最后两列中的NaN
的任何想法?
I want to replace the NaN
s using KNN as the method. I looked up sklearn
s Imputer
class but it supports only mean, median and mode imputation. There is a feature request here but I don't think thats been implemented as of now. Any ideas on how to replace the NaN
s from the last two columns using KNN?
修改: 由于我需要在另一个环境上运行代码,因此我没有安装软件包的奢侈之处. sklearn,pandas,numpy和其他标准软件包是我唯一可以使用的软件包.
Edit: Since I need to run codes on another environment, I don't have the luxury of installing packages. sklearn, pandas, numpy and other standard packages are the only ones I can use.
推荐答案
fancyimpute软件包支持此类使用以下API进行归类:
fancyimpute package supports such kind of imputation, using the following API:
from fancyimpute import KNN
# X is the complete data matrix
# X_incomplete has the same values as X except a subset have been replace with NaN
# Use 3 nearest rows which have a feature to fill in each row's missing features
X_filled_knn = KNN(k=3).complete(X_incomplete)
以下是此程序包支持的插补:
Here are the imputations supported by this package:
•SimpleFill:将缺失的条目替换为每个条目的均值或中位数 列.
•SimpleFill: Replaces missing entries with the mean or median of each column.
•KNN:最近邻归因法,使用 两行都具有的特征的均方差 观察到的数据.
•KNN: Nearest neighbor imputations which weights samples using the mean squared difference on features for which two rows both have observed data.
•SoftImpute:通过对SVD进行迭代软阈值处理来完成矩阵 分解.受到R的softImpute软件包的启发,该软件包是 谱正则化算法的大型学习算法 Mazumder等人的《不完全矩阵》.
•SoftImpute: Matrix completion by iterative soft thresholding of SVD decompositions. Inspired by the softImpute package for R, which is based on Spectral Regularization Algorithms for Learning Large Incomplete Matrices by Mazumder et. al.
•IterativeSVD:迭代低阶SVD的矩阵完成 分解.值应与SVDimpute类似 Troyanskaya等人的DNA微阵列估计方法
•IterativeSVD: Matrix completion by iterative low-rank SVD decomposition. Should be similar to SVDimpute from Missing value estimation methods for DNA microarrays by Troyanskaya et. al.
•MICE:通过链式方程重新实现多重插补.
•MICE: Reimplementation of Multiple Imputation by Chained Equations.
•MatrixFactorization:不完整矩阵的直接因式分解 分为低阶的U和V,对的元素进行L1稀疏性惩罚 U和V的元素上的L2罚分.通过梯度下降解决.
•MatrixFactorization: Direct factorization of the incomplete matrix into low-rank U and V, with an L1 sparsity penalty on the elements of U and an L2 penalty on the elements of V. Solved by gradient descent.
•NuclearNormMinimization:精确矩阵的简单实现 伊曼纽尔·坎德斯(Emmanuel Candes)和本杰明(Benjamin)通过凸优化完成 使用cvxpy的Recht.对于大型矩阵来说太慢了.
•NuclearNormMinimization: Simple implementation of Exact Matrix Completion via Convex Optimization by Emmanuel Candes and Benjamin Recht using cvxpy. Too slow for large matrices.
•BiScaler:行/列均值和标准的迭代估计 得到双归一化矩阵的偏差.不保证收敛 但在实践中效果很好.取自矩阵完成和低排名 通过快速交替最小二乘的SVD.
•BiScaler: Iterative estimation of row/column means and standard deviations to get doubly normalized matrix. Not guaranteed to converge but works well in practice. Taken from Matrix Completion and Low-Rank SVD via Fast Alternating Least Squares.
这篇关于使用KNN在Python中缺少值插补的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!