Hartley function

The Hartley function is a measure of uncertainty, introduced by Ralph Hartley in 1928. If a sample from a finite setA uniformly at random is picked, the amount of information revealed after the outcome is known is given by the Hartley function

${\displaystyle H_{0}(A):=\mathrm {log} _{b}\vert A\vert ,}$

where |A| denotes the cardinality of A.

If the base of the logarithm is 2, then the unit of uncertainty is the shannon (more commonly known as bit). If it is the natural logarithm, then the unit is the nat. Hartley used a base-ten logarithm, and with this base, the unit of information is called the hartley (aka ban or dit) in his honor. It is also known as the Hartley entropy or max-entropy.

The Hartley function coincides with the Shannon entropy (as well as with the Rényi entropies of all orders) in the case of a uniform probability distribution. It is a special case of the Rényi entropy since:

${\displaystyle H_{0}(X)={\frac {1}{1-0}}\log \sum _{i=1}^{|{\mathcal {X}}|}p_{i}^{0}=\log |{\mathcal {X}}|.}$

But it can also be viewed as a primitive construction, since, as emphasized by Kolmogorov and Rényi, the Hartley function can be defined without introducing any notions of probability (see Uncertainty and information by George J. Klir, p. 423).

The Hartley function only depends on the number of elements in a set, and hence can be viewed as a function on natural numbers. Rényi showed that the Hartley function in base 2 is the only function mapping natural numbers to real numbers that satisfies

1. ${\displaystyle H(mn)=H(m)+H(n)}$ (additivity)
2. ${\displaystyle H(m)\leq H(m+1)}$ (monotonicity)
3. ${\displaystyle H(2)=1}$ (normalization)

Condition 1 says that the uncertainty of the Cartesian product of two finite sets A and B is the sum of uncertainties of A and B. Condition 2 says that a larger set has larger uncertainty.

We want to show that the Hartley function, log₂(n), is the only function mapping natural numbers to real numbers that satisfies

1. ${\displaystyle H(mn)=H(m)+H(n)\,}$ (additivity)
2. ${\displaystyle H(m)\leq H(m+1)\,}$ (monotonicity)
3. ${\displaystyle H(2)=1\,}$ (normalization)

Let f be a function on positive integers that satisfies the above three properties. From the additive property, we can show that for any integer n and k,

${\displaystyle f(n^{k})=kf(n).\,}$

ليكن a و b و t أي أعداد صحيحة موجبة. يوجد عدد صحيح وحيد s يتم تحديده بواسطة

${\displaystyle a^{s}\leq b^{t}\leq a^{s+1}.\qquad (1)}$

لذلك،

${\displaystyle s\log _{2}a\leq t\log _{2}b\leq (s+1)\log _{2}a\,}$

و

${\displaystyle {\frac {s}{t}}\leq {\frac {\log _{2}b}{\log _{2}a}}\leq {\frac {s+1}{t}}.}$

من ناحية أخرى، من خلال الرتابة،

${\displaystyle f(a^{s})\leq f(b^{t})\leq f(a^{s+1}).\,}$

باستخدام المعادلة (1)، نحصل على

${\displaystyle sf(a)\leq tf(b)\leq (s+1)f(a),\,}$

و

${\displaystyle {\frac {s}{t}}\leq {\frac {f(b)}{f(a)}}\leq {\frac {s+1}{t}}.}$

لذلك،

${\displaystyle \left\vert {\frac {f(b)}{f(a)}}-{\frac {\log _{2}(b)}{\log _{2}(a)}}\right\vert \leq {\frac {1}{t}}.}$

بما أن قيمة t يمكن أن تكون كبيرة بشكل تعسفي، فإن الفرق على الجانب الأيسر من المتباينة أعلاه يجب أن يكون صفراً.

${\displaystyle {\frac {f(b)}{f(a)}}={\frac {\log _{2}(b)}{\log _{2}(a)}}.}$

لذا،

${\displaystyle f(a)=\mu \log _{2}(a)\,}$

بالنسبة لثابت ما μ ، والذي يجب أن يساوي 1 وفقًا لخاصية التوحيد.

• إنتروبيا ريني
• الحد الأدنى من الإنتروبيا

• تتضمن هذه المقالة مواد من دالة هارتلي على موقع PlanetMath ، وهي مرخصة بموجب رخصة Creative Commons Attribution/Share-Alike .
• تتضمن هذه المقالة مواد من اشتقاق دالة هارتلي على موقع PlanetMath ، وهو مرخص بموجب رخصة Creative Commons Attribution/Share-Alike .