为什么此示例会导致NaN? [英] Why does this example result in NaN?

问题描述

我正在查看PySpark中Statistics.corr的文档:

I'm looking at the documentation for Statistics.corr in PySpark: https://spark.apache.org/docs/1.1.0/api/python/pyspark.mllib.stat.Statistics-class.html#corr.

为什么这里的相关性导致NaN?

Why does the correlation here result in NaN?

>>> rdd = sc.parallelize([Vectors.dense([1, 0, 0, -2]), Vectors.dense([4, 5, 0, 3]),
...                       Vectors.dense([6, 7, 0,  8]), Vectors.dense([9, 0, 0, 1])])
>>> pearsonCorr = Statistics.corr(rdd)
>>> print str(pearsonCorr).replace('nan', 'NaN')
[[ 1.          0.05564149         NaN  0.40047142]
 [ 0.05564149  1.                 NaN  0.91359586]
 [        NaN         NaN  1.                 NaN]
 [ 0.40047142  0.91359586         NaN  1.        ]]

推荐答案

这很简单.皮尔森相关系数定义如下:

It is pretty simple.Pearson correlation coefficient is defined as follows:

由于第二列([0, 0, 0, 0])的标准偏差等于0，因此整个方程式将得出NaN.

Since standard deviation for the second column ([0, 0, 0, 0]) is equal 0, whole equation results in NaN.

这篇关于为什么此示例会导致NaN?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！