View source for Kernel density estimation

{{SI}}


==Overview==
In [[statistics]], ''kernel density estimation''' (or '''Parzen window''' method, named after Emanuel Parzen) is a way of [[Density estimation|estimating]] the [[probability density function]] of a [[random variable]]. As an illustration, given some data about a [[statistical sample|sample]] of a [[population]], kernel density estimation makes it possible to [[extrapolation|extrapolate]] the data to the entire population. 

==Definition==
If ''x''<sub>1</sub>, ''x''<sub>2</sub>, ..., ''x''<sub>''N''</sub> ~ f is a [[Independent and identically-distributed random variables|IID]] sample of a random variable, then the kernel density approximation of its probability density function is

:<math>\widehat{f}_h(x)=\frac{1}{Nh}\sum_{i=1}^N K\left(\frac{x-x_i}{h}\right)</math>

where ''K'' is some [[kernel (statistics)|kernel]] and ''h'' is the bandwidth ([[smoothing]] parameter).  Quite often ''K'' is taken to be a standard [[Gaussian function]] with [[expected value|mean]] zero and [[variance]] 1:

:<math>K(x) = {1 \over \sqrt{2\pi} }\,e^{-\frac{1}{2}x^2}.</math>

==Intuition==
Although less smooth density estimators such as the histogram density estimator can be made to be asymptotically consistent, others are often either discontinuous or converge at slower rates than the kernel density estimator. Rather than grouping observations together in bins, the kernel density estimator can be thought to place small "bumps" at each observation, determined by the kernel function. The estimator consists of a "sum of bumps" and is clearly smoother as a result (see below image).

==Properties==
Let <math>R(f,\hat f(x))</math> be the L<sup>2</sup> [[risk function]] for f. Under weak assumptions on f and K,

:<math>R(f,\hat f(x)) \approx \frac{1}{4}\sigma_k^4h^4\int(f''(x))^2dx + \frac{\int K^2(x)dx}{nh}</math> where <math>\sigma_K^2 = \int x^2K(x)dx</math>.

By minimizing the theoretical [[risk function]], it can be shown that the optimal bandwidth is
:<math>h^* = \frac{c_1^{-2/5}c_2^{1/5}c_3^{-1/5}}{n^{1/5}}</math>
where
:<math>c_1 = \int x^2K(x)dx</math>
:<math>c_2 = \int K(x)^2dx</math>
:<math>c_3 = \int (f''(x))^2dx</math>

When the optimal choice of bandwidth is chosen, the risk function is <math>R(f, \hat f(x)) \approx \frac{c_4}{n^{4/5}}</math> for some constant c<sub>4</sub> &gt; 0. It can be shown that, under weak assumptions, there cannot exist a non-parametric estimator that converges at a faster rate than the kernel estimator. Note that the n<sup>-4/5</sup> rate is slower than the typical n<sup>-1</sup> convergence rate of parametric methods.

==Statistical implementation==
* In Stata, it is implemented through <code>kdensity</code>; for example <code>histogram x, kdensity</code>.
* In R, it is implemented through the <code>density</code> function.

==See also==
*[[kernel (statistics)]] 
*[[density estimation]]
*[[smoothing]]

==References==
* Parzen E. (1962). ''On estimation of a probability density function and [[Mode (statistics)|mode]]'', Ann. Math. Stat. '''33''', pp. 1065-1076.
* Duda, R. and Hart, P. (1973). ''Pattern Classification and Scene Analysis''. John Wiley & Sons. ISBN 0-471-22361-1.
* Wasserman, L. (2005). ''All of Statistics: A Concise Course in Statistical Inference'', Springer Texts in Statistics.

== External links ==
* [http://www.maths.uwa.edu.au/~duongt/seminars/intro2kde/ Introduction to kernel density estimation] 
* [http://www.wessa.net/rwasp_density.wasp Free Online Software (Calculator)] computes the Kernel Density Estimation for any data series according to the following Kernels: Gaussian, Epanechnikov, Rectangular, Triangular, Biweight, Cosine, and Optcosine.
[[Category:Probability and statistics]]
[[Category:non-parametric statistics]]

[[de:Kerndichteschätzer]]
{{WikiDoc Help Menu}}
{{WikiDoc Sources}}