如何解释神经网络层的权重分布 [英] How to interpret weight distributions of neural net layers

问题描述

我设计了一个三层神经网络，其输入是来自CNN和RNN的串联特征.网络学习的权重取很小的值.对此的合理解释是什么?以及如何解释Tensorflow中的权重直方图和分布?有什么好的资源吗?

这是使用张量板可视化的3层神经网络的第一个隐藏层的权重分布.如何解释呢?所有权重都占零值?

这是3层神经网络的第二个隐藏层的权重分布:

解决方案

如何解释Tensorflow中的权重直方图和分布?

嗯，您可能没有意识到这一点，但是您只是问了ML& 100万美元的问题. AI ...

模型的可解释性是当前研究的一个活跃和过热的领域(认为是圣杯之类的东西)，最近提出来的原因不仅仅在于(通常是巨大的)深度学习模型在各种任务中的成功；这些型号目前仅是黑匣子，我们对此自然感到不舒服...

有什么好的资源吗?

可能您所想的资源可能不完全相同，我们在这里还没有一个非常适合的主题，但是由于您提出的问题……:

• 《科学》杂志上最近(2017年7月)的一篇文章很好地概述了当前状态&研究: AI侦探如何破解深度学习的黑匣子(没有文本链接，但使用谷歌搜索的名称和术语会奏效)

• DARPA本身当前正在可解释人工智能(XAI)上运行程序

• NIPS 2016上的一个研讨会位于项目页面， TF纤薄包装器)

• FairML:审核黑匣子预测模型，由Fast Forward Labs(纸张，代码)

• Geoff Hinton的最新论文(2017年11月)，将神经网络提炼为软决策树，具有独立的 PyTorch实现

• SHAP:解释模型预测的统一方法(论文，作者的代码)

对于初学者来说，这些就足够了，并可以让您大致了解您所询问的主题...

更新(2018年10月):我在对问题

I have designed a 3 layer neural network whose inputs are the concatenated features from a CNN and RNN. The weights learned by network take very small values. What is the reasonable explanation for this? and how to interpret the weight histograms and distributions in Tensorflow? Any good resource for it?

This is the weight distribution of the first hidden layer of a 3 layer neural network visualized using tensorboard. How to interpret this? all the weights are taking up zero value?

This is the weight distribution of the second hidden layer of a 3 layer neural:

解决方案

how to interpret the weight histograms and distributions in Tensorflow?

Well, you probably didn't realize it, but you have just asked the 1 million dollar question in ML & AI...

Model interpretability is a hyper-active and hyper-hot area of current research (think of holy grail, or something), which has been brought forward lately not least due to the (often tremendous) success of deep learning models in various tasks; these models are currently only black boxes, and we naturally feel uncomfortable about it...

Any good resource for it?

Probably not exactly the kind of resources you were thinking of, and we are well off a SO-appropriate topic here, but since you asked...:

• A recent (July 2017) article in Science provides a nice overview of the current status & research: How AI detectives are cracking open the black box of deep learning (no in-text links, but googling names & terms will pay off)

• DARPA itself is currently running a program on Explainable Artificial Intelligence (XAI)

• There was a workshop in NIPS 2016 on Interpretable Machine Learning for Complex Systems

On a more practical level:

• The Layer-wise Relevance Propagation (LRP) toolbox for neural networks (paper, project page, code, TF Slim wrapper)

• FairML: Auditing Black-Box Predictive Models, by Fast Forward Labs (blog post, paper, code)

• A very recent (November 2017) paper by Geoff Hinton, Distilling a Neural Network Into a Soft Decision Tree, with an independent PyTorch implementation

• SHAP: A Unified Approach to Interpreting Model Predictions (paper, authors' code)

These should be enough for starters, and to give you a general idea of the subject about which you asked...

UPDATE (Oct 2018): I have put up a much more detailed list of practical resources in my answer to the question Predictive Analytics - "Why" factor?

这篇关于如何解释神经网络层的权重分布的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！