View source for Dirichlet distribution

[[Image:Dirichlet distributions.png|thumb|right|250px|Several images of the probability density of the Dirichlet distribution when ''K''=3 for various parameter vectors ''&alpha;''.  Clockwise from top left: ''&alpha;''=(6,&nbsp;2,&nbsp;2), (3,&nbsp;7,&nbsp;5), (6,&nbsp;2,&nbsp;6), (2,&nbsp;3,&nbsp;4).]]

In [[probability]] and [[statistics]], the '''Dirichlet distribution''' (after [[Johann Peter Gustav Lejeune Dirichlet]]), often denoted Dir(''&alpha;''), is a family of [[Continuous probability distribution|continuous]] [[multivariate random variable|multivariate]] [[probability distribution]]s parametrized by the vector ''&alpha;'' of positive [[real number|reals]]. It is the multivariate generalization of the [[beta distribution]], and [[conjugate prior]] of the [[multinomial distribution]] in [[Bayesian]] statistics.  That is, its probability density function returns the belief that the probabilities of ''K'' rival events are <math>x_i</math> given that each event has been observed <math>\alpha_i-1</math> times.

==Probability density function==


[[Image:LogDirichletDensity-alpha 0.3 to alpha 2.0.gif|thumb|right|250px|Illustrating how the log of the density function changes when ''K''=3 as we change the vector ''&alpha;'' from ''&alpha;''=(0.3,&nbsp;0.3,&nbsp;0.3) to (2.0,&nbsp;2.0,&nbsp;2.0), keeping all the individual <math>\alpha_i</math>'s equal to each other.]]

The Dirichlet distribution of order ''K''&nbsp;≥&nbsp;2 with parameters ''&alpha;''<sub>1</sub>, ..., ''&alpha;''<sub>''K''</sub> >&nbsp;0 has a [[probability density function]] with respect to [[Lebesgue measure]] on the [[Euclidean space]] '''R'''<sup>''K''–1</sup> given by

:<math>f(x_1,\dots, x_{K-1}; \alpha_1,\dots, \alpha_K) = \frac{1}{\mathrm{B}(\alpha)} \prod_{i=1}^K x_i^{\alpha_i - 1} </math>

for all ''x''<sub>1</sub>, ..., ''x''<sub>''K''–1</sub> >&nbsp;0 satisfying ''x''<sub>1</sub> + ... + ''x''<sub>''K''–1</sub> <&nbsp;1, where ''x''<sub>''K''</sub> is an abbreviation for 1 – ''x''<sub>1</sub> – ... – ''x''<sub>''K''–1</sub>. The density is zero outside this [[open set|open]] (''K''&nbsp;&minus;&nbsp;1)-dimensional [[simplex]].

The [[normalizing constant]] is the multinomial [[beta function]], which can be expressed in terms of the [[gamma function]]:

:<math>\mathrm{B}(\alpha) = \frac{\prod_{i=1}^K \Gamma(\alpha_i)}{\Gamma\bigl(\sum_{i=1}^K \alpha_i\bigr)},\qquad\alpha=(\alpha_1,\dots,\alpha_K).</math>

==Properties==
Let <math>X = (X_1, \ldots, X_K)\sim\operatorname{Dir}(\alpha)</math>, meaning that the first ''K''&nbsp;–&nbsp;1 components have the above density and

:<math>X_K=1-X_1-\cdots-X_{K-1}.</math>

Define <math>\textstyle\alpha_0 = \sum_{i=1}^K\alpha_i</math>. Then

:<math> \mathrm{E}[X_i] = \frac{\alpha_i}{\alpha_0},</math>

:<math>\mathrm{Var}[X_i] = \frac{\alpha_i (\alpha_0-\alpha_i)}{\alpha_0^2 (\alpha_0+1)},</math>

in fact, the marginals are [[Beta distribution| Beta distributions]]:

:<math>X_i \sim \operatorname{Beta}(\alpha_i, \alpha_0 - \alpha_i). </math>

Furthermore,

:<math>\mathrm{Cov}[X_iX_j] = \frac{- \alpha_i \alpha_j}{\alpha_0^2 (\alpha_0+1)}.</math>

The mode of the distribution is the vector (''x''<sub>1</sub>, ..., ''x''<sub>K</sub>) with

:<math> x_i = \frac{\alpha_i - 1}{\alpha_0 - K}, \quad \alpha_i > 1. </math>

The Dirichlet distribution is conjugate to the [[multinomial distribution]] in the following sense: if
 
:<math>\beta \mid X=(\beta_1, \ldots, \beta_{K}) \mid X \sim \operatorname{Mult}(X),</math>

where ''&beta;''<sub>''i''</sub> is the number of occurrences of ''i'' in a sample of ''n'' points from the discrete distribution on {1, ..., ''K''} defined by ''X'', then

:<math>X \mid \beta \sim \operatorname{Dir}(\alpha + \beta).</math>

This relationship is used in [[Bayesian]] statistics to estimate the hidden parameters, ''X'', of a discrete probability distribution given a collection of ''n'' samples. Intuitively, if the [[prior distribution|prior]] is represented as Dir(''&alpha;''), then Dir(''&alpha; + &beta;'') is the [[posterior distribution|posterior]] following a sequence of observations with [[histogram]] ''&beta;''.


===Neutrality===

(main article:  [[neutral vector]]).

If <math>X = (X_1, \ldots, X_K)\sim\operatorname{Dir}(\alpha)</math>, then the vector~<math>X</math> is said to be ''neutral''<ref>R. J. Connor and J. E. Mosiman 1969.  ''Concepts of independence for proportions with a generalization of the Dirichlet distibution''.  Journal of the American Statistical Association, volume 64, pp194--206</ref> in the sense that <math>X_1</math> is independent of <math>X_2/(1-X_1),X_3/(1-X_1),\ldots,X_K/(1-X_1)</math> and similarly for <math>X_2,\ldots,X_{K-1}</math>.

Observe that any permutation of <math>X</math> is also neutral (a property not possessed by samples drawn from a [[generalized Dirichlet distribution]]).

==Related distributions==
* If, for <math>\scriptstyle i\in\{1,2,\ldots,K\},</math>
:: <math>Y_i\sim\operatorname{Gamma}(\textrm{shape}=\alpha_i,\textrm{scale}=1)\text{ independently},</math>
: then
::<math>V=\sum_{i=1}^K Y_i\sim\operatorname{Gamma}(\textrm{shape}=\sum_{i=1}^K\alpha_i,\textrm{scale}=1),</math>
: and
:: <math>(X_1,\ldots,X_K) = (Y_1/V,\ldots,Y_K/V)\sim \operatorname{Dir}(\alpha_1,\ldots,\alpha_K).</math>

:Though the ''X''<sub>''i''</sub>s are not independent from one another, they can be seen to be generated from a set of <math>K</math> independent [[Gamma distribution|gamma]] random variables. Unfortunately, since the sum <math>V</math> is lost in the process of forming ''X'' = (''X''<sub>1</sub>, ..., ''X''<sub>''K''</sub>), it is not possible to recover the original gamma random variables from these values alone. Nevertheless, because independent random variables are simpler to work with, this reparametrization can still be useful for proofs about properties of the Dirichlet distribution.
*Multinomial opinions in [[subjective logic]] are equivalent to Dirichlet distributions.

==Random number generation==

A method to sample a random vector <math>x=(x_1, \ldots, x_K)</math> from the ''K''-dimensional Dirichlet distribution with parameters <math>(\alpha_1, \ldots, \alpha_K)</math> follows immediately from this connection.  First, draw ''K'' independent random samples <math>y_1, \ldots, y_K</math> from [[gamma distribution]]s each with density 

:<math> \frac{y_i^{\alpha_i-1} \; e^{-y_i}}{\Gamma (\alpha_i)}, \!</math>

and then set 

:<math>x_i = y_i/\sum_{j=1}^K y_j. \!</math>

==Intuitive interpretation of the parameters==
One example use of the Dirichlet distribution is if one wanted to cut strings (each of initial length 1.0) into K pieces with different lengths, where each piece had, on average, a designated average length, but allowing some variation in the relative sizes of the pieces.  The ''&alpha;''/''&alpha;''<sub>0</sub> values specify the mean lengths of the cut pieces of string resulting from the distribution.  The variance around this mean varies inversely with ''&alpha;''<sub>0</sub>.

==See also==

* [[beta distribution]]
* [[binomial distribution]]
* [[categorical distribution]]
* [[generalized Dirichlet distribution]]
* [[latent Dirichlet allocation]]
* [[multinomial distribution]]
* [[multivariate Polya distribution]]

==External links==

* [http://www.cis.hut.fi/ahonkela/dippa/node95.html Dirichlet Distribution]
* [http://research.microsoft.com/~minka/papers/dirichlet/minka-dirichlet.pdf Estimating the parameters of the Dirichlet distribution]

==References==

{{reflist|2}}

'''Non-Uniform Random Variate Generation''', by Luc Devroye http://cg.scs.carleton.ca/~luc/rnbookindex.html

{{ProbDistributions|multivariate}}

[[Category:Continuous distributions]]

[[it:Variabile casuale di Dirichlet]]
[[pt:Distribuição de Dirichlet]]
[[ru:Распределение Дирихле]]
[[de:Dirichlet-Verteilung]]