熵的计算机科学定义是什么? [英] What is the computer science definition of entropy?
问题描述
我最近在我的大学开始了一个数据压缩课程.但是，我发现术语熵"在计算机科学中的应用相当模糊.据我所知，它大致可以解释为系统或结构的随机性".
I've recently started a course on data compression at my university. However, I find the use of the term "entropy" as it applies to computer science rather ambiguous. As far as I can tell, it roughly translates to the "randomness" of a system or structure.
计算机科学熵"的正确定义是什么?
What is the proper definition of computer science "entropy"?
推荐答案
熵可以表示不同的含义:
Entropy can mean different things:
在计算中，熵是 操作人员收集的随机性 用于的系统或应用程序 密码术或其他用途 需要随机数据.这种随机性 通常是从硬件收集的 资料来源，例如既有的资料来源 作为鼠标移动或专门 提供了随机性发生器.
In computing, entropy is the randomness collected by an operating system or application for use in cryptography or other uses that require random data. This randomness is often collected from hardware sources, either pre-existing ones such as mouse movements or specially provided randomness generators.
在信息论中，熵是一个 测量不确定性 与随机变量.该词由 在这种情况下，它本身通常是指 香农熵 在某种意义上量化 期望值，信息 包含在邮件中，通常在 位等单位.等效地， 香农熵是衡量 平均信息含量之一是 当一个人不知道的时候失踪了 随机变量的值
In information theory, entropy is a measure of the uncertainty associated with a random variable. The term by itself in this context usually refers to the Shannon entropy, which quantifies, in the sense of an expected value, the information contained in a message, usually in units such as bits. Equivalently, the Shannon entropy is a measure of the average information content one is missing when one does not know the value of the random variable
数据压缩中的熵
数据压缩中的熵可能表示您输入到压缩算法中的数据的随机性.熵越大，压缩率越小.这意味着文本越随机，压缩的内容就越少.
Entropy in data compression may denote the randomness of the data that you are inputing to the compression algorithm. The more the entropy, the lesser the compression ratio. That means the more random the text is, the lesser you can compress it.
香农的熵代表一个 尽最大可能的绝对极限 任何无损压缩 沟通:将消息视为 编码为独立序列 和均匀分布的随机 变量，Shannon的源代码 定理表明，在极限情况下， 最短的平均长度 可能的表示形式来编码 给定字母中的消息是他们的 熵除以对数 目标中的符号数 字母.
Shannon's entropy represents an absolute limit on the best possible lossless compression of any communication: treating messages to be encoded as a sequence of independent and identically-distributed random variables, Shannon's source coding theorem shows that, in the limit, the average length of the shortest possible representation to encode the messages in a given alphabet is their entropy divided by the logarithm of the number of symbols in the target alphabet.
这篇关于熵的计算机科学定义是什么?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
