Lecture 12 
1.(Regular) Exponential Family
The density function of a regular exponential family is:
Example. Poisson(θ)
Example. Normal.   (both unknown).
2.Theorem (Exponential family & sufficient Statistic). Let be a random sample from the regular exponential family.
Then
is sufficient for
Example. Poisson(θ)
Let be a random sample from Poisson(θ)
Then
is sufficient for
Example. Normal.   (both unknown).
Let be a random sample from
Then
is sufficient for
Exercise.
Apply the general exponential family result to all the standard families discussed above such as binomial, Poisson, normal, exponential, gamma.
A Non-Exponential Family Example.
Discrete uniform.
is a positive integer.
Another Non-exponential Example.
iid
Universal Cases.
are iid with density .
• The original data are always sufficient for .
(They are trivial statistics, since they do not lead any data reduction)
• Order statistics are always sufficient for .
( The dimension of order statistics is , the same as the dimension of the data.  Still this is a nontrivial reduction as ! different values of data corresponds to one value of . )
3. Theorem (Rao-Blackwell)
Let be a random sample from the population with pdf  truncated normal distribution. Let be a sufficient statistic for θ, and be any unbiased estimator of θ.
Let , then
(1)is an unbiased estimator of  
(2) is a function of T, 
(3) for every , and for some unless with probability 1 .
Rao-Blackwell theorem tells us that in searching for an unbiased estimator with the smallest possible variance (i.e., the best estimator, also called the uniformly minimum variance unbiased estimator – UMVUE, which is also referred to as simply the MVUE), we can restrict our search to only unbiased functions of the sufficient statistic T(X).
4. Transformation of Sufficient Statistics
1. If is sufficient for and a mathematical function of some other statistic, then is also sufficient.
2. If is sufficient for , and with being one-to-one, then is also sufficient.
Remark:  When one statistic is a function of the other statistic and vise verse, then they carry exactly the same amount of information.
Examples:
• If   is sufficient, so is  .
• If (  are sufficient, so is  .
• If   is sufficient, so is  is sufficient, and so is .
Examples of non-sufficiency.
Ex. iid Poisson(). is not sufficient.
Ex. iid pmf . is not sufficient.
5.    Minimal Sufficient Statistics
It is seen that different sufficient statistics are possible.  Which one is the "best"? Naturally, the one with the maximum reduction.
• For ,    is a better sufficient statistic for than
Definition:
is a minimal sufficient statistic if, given any other sufficient statistic, there is a function such that .
Equivalently, is minimal sufficient if, given any other sufficient statistic whenever and are two data values such that , then .
Partition Interpretation for Minimal Sufficient Statistics:
• Any sufficient statistic introduces a partition on the sample space.
• The partition of a minimal sufficient statistic is the coarsest.
• Minimal sufficient statistic has the smallest dimension among possible sufficient statistics. Often the dimension is equal to the number of free parameters (exceptions do exist).
Theorem (How to check minimal sufficiency).
A statistic T is minimal sufficient if the following property holds: For any two sample points x and y does not depend on (i.e. is a constant function of ) if and only if
6. Exponential Families & Minimal Sufficient Statistic:
For a random sample from the regular exponential family with  probability density ], where is k dimensional, the statistic

版权声明：本站内容均来自互联网，仅供演示用，请勿用于商业和其他非法用途。如果侵犯了您的权益请与我们联系QQ:729038198，我们将在24小时内删除。