如何在Spark的Logistic回归中计算p值? [英] How to calculate p-values in Spark's Logistic Regression?

问题描述

我们正在使用LogisticRegressionWithSGD，并想弄清楚我们的哪些变量可以预测以及具有什么意义.一些统计数据包(StatsModels)为每个术语返回p值.低的p值(＜0.05)表明对该模型有有意义的附加.

We are using LogisticRegressionWithSGD and would like to figure out which of our variables predict and with what significance. Some stats packages (StatsModels) return p-values for each term. A low p-value (< 0.05) indicates a meaningful addition to the model.

我们如何从LogisticRegressionWithSGD模型获取/计算p值?

How can we get/calculate p-values from LogisticRegressionWithSGD model?

对此有任何帮助.

推荐答案

这是一个非常老的问题，但对于迟到的人们来说，一些指导可能会很有价值.

This is a very old question, but some guidance for people coming to it late might be valuable.

LogisticRegressionWithSGD为不推荐使用.在该版本中，模型本身未提供真正的摘要"信息集.如果您无法访问pyspark的最新版本，则必须自己计算每个功能的P值. 这里是通过手工"进行操作的很好的介绍./a>

LogisticRegressionWithSGD is deprecated. In that version, no true set of "summary" information was provided with the model itself. If you cannot get access to an up-to-date version of pyspark, you will have to calculate the P-values for each of your features yourself. Here is a nice intro to doing that by "hand".

如果您可以获取当前版本的pyspark，则需要使用: pyspark.mllib.classification.LogisticRegressionWithLBFGS (docs 此处)

If you can get the current version of pyspark, then you will want to use: pyspark.mllib.classification.LogisticRegressionWithLBFGS (docs here)

这篇关于如何在Spark的Logistic回归中计算p值?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！