Average code length and entropy

Question

Ghady Hajj 2018-11-16

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/430352-average-code-length-and-entropy

回答： Anshuman 2024-8-24，18:16

Hello,

I have an uin16 vector and I got the Huffman code from the built-in functions in MATLAB. The thing is that the entropy of this file is different from the average code length that I've got from the Huffman code. Isn't the average code length supposed to be equal to the entropy of the file?

Thanks.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Anshuman 2024-8-24，18:16

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/430352-average-code-length-and-entropy#answer_1504549

Hi,

That's not always true, Huffman coding does not always achieve the entropy limit because it works with integer code lengths. Since Huffman codes must be whole numbers of bits, the average length can exceed the entropy slightly. Entropy represents the ideal case which might involve fractional bits. Also, If the source has a non-uniform distribution or symbols with probabilities that are not powers of two, the Huffman code's average length will typically be greater than the entropy.