如何向高管证明预测模型的可靠性? [英] How to prove the reliability of a predictive model to executives?

问题描述

我训练了500台设备中的数据以预测其性能.然后，我将训练有素的模型应用于另外500个设备的测试数据集，并显示出相当不错的预测结果.现在，我的高管希望我证明这种模型不仅可以在500台设备上运行，而且可以在一百万台设备上运行.显然，我们没有一百万台设备的数据.如果模型不可靠，他们希望我发现所需的火车数据量，以便对一百万个设备进行可靠的预测.我该如何应对这些没有统计分析和建模背景的高管?有什么建议?谢谢

I trained data from 500 devices to predict their performance. Then I applied my trained model to a test data set for another 500 devices and show pretty good prediction results. Now my executives want me to prove this model will work well on one million devices not only on 500. Obviously we don't have data for one million devices. And if the model is not reliable, they want me to discover the required amount of train data in order to make a reliable prediction on one million devices. How should I deal with these executives who don't have a background in statistical analysis and modeling? Any suggestions? Thanks

推荐答案

我建议@cep写下他的评论作为答案-包括提供variance和bias计算.无论如何，都可以添加

I have suggested to @cep to write up his comment as an answer - including providing the variance and bias calculations. In any case it could be added

"不要急于认为Exec在以下方面基本上无能为力: 技术或数学概念"

"Do not be quick to assume Execs are essentially incapable in terms of technical or mathematical concepts"

虽然那里可能有Dilbert位经理..某处我本人很少见到他们.管理人员更经常通过辛勤工作来担任自己的职位.他们可能会生锈-但是能力仍然存在.

While there may be Dilbert managers out there .. somewhere I have seen few of them myself. More often managers get to their positions through hard work. They are likely to be rusty - but the abilities are likely still there.

在这种情况下，他们是否具有统计分析和建模背景"，他们都在运用常识.

In this case whether or not they have a "background in statistical analysis and modeling" they are applying common sense.

您可能要做的第一件事是提供适当的上下文和术语. @cel提到了其中的一些:为:提供具体值:

The first thing you might do is to provide the proper context and terminology. @cel has mentioned some of it: providing concrete values for :

• 假设
  • 您对数据有什么假设.
  • 考虑有限数据的推断依据是什么
  • 为什么应该相信外推结果可应用于99.5％的未经测试的数据
  • assumptions
    • what assumptions are you making about the data.
    • What basis is there to consider extrapolation of the limited data
    • why should said extrapoated results be trusted to apply to the 99.5% of untested data
    • 基本描述统计
    • 您先验数据分布.证明选择原因的理由
    • 考虑了哪些模型/方法以及原因
    • 您实际选择的型号以及原因
    • 您是如何得出超参数的?
    • 您如何训练模型
    • 拟合度和错误率的统计量

    这篇关于如何向高管证明预测模型的可靠性?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！