Theil index

The Theil index, derived by econometrician Henri Theil, is a statistic used to measure economic inequality.

Mathematics

The formula is

<math>

T=\frac{1}{N}\sum_{i=1}^N \left( \frac{x_i}{\overline{x}} \cdot \ln{\frac{x_i}{\overline{x}}} \right) </math>

where <math>x_i</math> is the income of the <math>i</math>th person, <math>\overline{x}= \frac{1}{N} \sum_{i=1}^N x_i </math> is the mean income, and <math>N</math> is the number of people. The first term inside the sum can be considered the individual's share of aggregate income, and the second term is that person's income relative to the mean. If everyone has the same (i.e., mean) income, then the index is 0. If one person has all the income, then the index is ln N.

The Theil index is derived from Shannon's measure of information entropy. Letting T be the Theil index and S be Shannon's information entropy measure,

<math> T=\ln(N)-S. \, </math>

Shannon derived his entropy measure in terms of the probability of an event occurring. This can be interpreted in the Theil index as the probability a dollar drawn at random from the population came from a specific individual. This is the same as the first term, the individual's share of aggregate income.

With reference to information theory, Theil's measure is a redundancy rather than an entropy. The redundancy of a system at a given time is the difference between its maximum entropy and its present entropy at that time.

Decomposability

One of the advantages of the Theil index is that it is a weighted average of inequality within subgroups, plus inequality among those subgroups. For example, inequality within the United States is the average inequality within each state, weighted by state income, plus the inequality among states.

If the population is divided into <math>m</math> certain subgroups and <math>s_k</math> is the income share of group <math>k</math>, <math>T_k</math> is the Theil index for that subgroup, and <math>\overline{x}_k</math> is the average income in group <math>k</math>, then the Theil index is

<math>

T = \sum_{k=1}^m s_k T_k + \sum_{k=1}^m s_k \ln{\frac{\overline{x}_k}{\overline{x}}}. </math>

Another, more popular, measure of inequality is the Gini coefficient. The Gini coefficient is more intuitive to many people since it is based on the Lorenz curve. However, it is not easily decomposable like the Theil.

Application of the Theil index

Theil's index takes an equal distribution for reference which is similar to distributions in statistical physics. An index for an actual system is an actual redundancy, that is, the difference between maximum entropy and actual entropy of that system.

Theil's measure can be converted into one of the indexes of Anthony Barnes Atkinson. The result of the conversion also is called normalized Theil index. James E. Foster used such a measure to replace the Gini coefficient in Amartya Sen's welfare function W=f(income,inequality). The income e.g. is the average income for individuals in a group of income earners. Thus, Foster's welfare function can be computed directly from the Theil index T, if the conversion is included into the computation of the average per capita welfare function:

<math>W = \overline\text{income} \times {e^{-T}}.\,</math>

Map of economic inequality in the United States using the Theil Index. A high positive theil index indicates more income than population while a negative value shows more population than income. A value of zero shows equality between population and income.

Map of economic inequality in the United States using the Theil Index. A high positive theil index indicates more income than population while a negative value shows more population than income. A value of zero shows equality between population and income.

Note: This image is not the Theil Index in each area of the United States, but of contributions to the US Theil Index by each area (the Theil Index is always positive, individual contributions to the Theil Index may be negative or positive).

Pareto principle

For resource distributions described by only two quantiles, the Theil index is 0 for 50:50 distributions and reaches 1 at 82:18, which is very close to a distribution often referred to as "Pareto Principle". Higher inequities yield Theil indices above 1.

Theil index and Hoover index

File:SymmetricTheil minus Hoover.jpg

For the income distributions provided by the The World Income Inequality Database (2007-05) the difference between their symmetrized Theil indices and their Hoover indices are plotted over their respective Gini indices. The difference illustrates the impact of the different inequalities on the information generated by them. Negative values occur for Theil indices, which are smaller than the respective Hoover indices.

A comparison of the Hoover index (also called Robin Hood index) and the Theil index shows the meaning of both indices:

• For the Hoover index, the relative deviations in each quantile are summed up. Each deviation is weighted by its own sign (+1 or −1). Thus, the Hoover index is the most simple inequality measure. It has no normative foundations and does not refer to any models from physics or information theory.
• For the symmetrized Theil index, the relative deviations in each quantile are summed up as well. But each deviation is weighted by its relative information weight. Thus, the Theil index is an indicator not only for the plain relative inequality, it also attempts to indicate how much attention inequality can get.

The following formulas illustrate that difference in the categories symmetry and percevability. For the formulas, a notation is used, where the amount <math>N</math> of quantiles only appears as upper border of summations. Thus, inequities can be computed for quantiles with different widths <math>A_i</math>. For example, <math>E_i</math> could be the income in the quantile #i and <math>A_i</math> could be the amount (absolute or relative) of earners in the quantile #i. <math>E_\text{total}</math> then would be the sum of incomes of all <math>N</math> quantiles and <math>A_\text{total}</math> would be the sum of the income earners in all <math>N</math> quantiles.

Computation of the (asymmetric) Theil index T :

<math>

T = \ln{\frac{{A}_\text{total}}{{E}_\text{total}}} - \frac{\sum_{i=1}^N {{E}_i} \ln{\frac{{A}_i}{{E}_i}}}{{E}_\text{total}}. </math>

With normalized data, <math>{{E}'_i}=E_i/E_\text{total}</math> and <math>{{A}'_i}=A_i/A_\text{total}</math> would apply. This would simplify the formula:
<math> \color{Gray} T = 0 - \frac{\sum_{i=1}^N {{E}'_i} \ln{\frac{{A}'_i}{{E}'_i}}}{1} = \sum_{i=1}^N {{E}'_i} \ln{\frac{{E}'_i}{{A}'_i}} </math>

Computation of the symmetrized Theil index <math>T_s</math>:

<math>

T_s = \frac{1}{2} \left( \ln{\frac{{A}_\text{total}}{{E}_\text{total}}} - \frac{\sum_{i=1}^N {{E}_i} \ln{\frac{{A}_i}{{E}_i}}}{{E}_\text{total}} + \ln{\frac{{E}_\text{total}}{{A}_\text{total}}} - \frac{\sum_{i=1}^N {{A}_i} \ln{\frac{{E}_i}{{A}_i}}}{{A}_\text{total}} \right). </math>

This leads to:

<math>

T_s = {\frac{1}{2}} \sum_{i=1}^N \color{OliveGreen} \ln{\frac{{E}_i}{{A}_i}} \color{Black} \left( {\frac{{E}_i}{{E}_\text{total}}} - {\frac{{A}_i}{{A}_\text{total}}} \right). </math>

For comparison, the Hoover index <math>H</math>:

<math>

H = {\frac{1}{2}} \sum_{i=1}^N \color{Blue} \color{Black} \left| {\frac{{E}_i}{{E}_\text{total}}} - {\frac{{A}_i}{{A}_\text{total}}} \right|. </math>