ECMT1020 代写、代写 R 编程设计

日期：2023-03-11 12:12

ECMT1020 Introduction to Econometrics Week 2, 2023S1

Lecture 2: Distributions, Samples, and Estimators

Instructor: Ye Lu

Please read Chapter R.5每R.8 of the textbook.

Contents

1 Four Important Probability Distributions 1

1.1 Normal distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 t distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.3 Chi-squared distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.4 F distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 Samples and Estimators 4

2.1 Sampling and double structure of a sampled random variable . . . . . . . . . 4

2.2 Estimators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.3 Bias and Variance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.4 Loss functions and mean squared error . . . . . . . . . . . . . . . . . . . . . . 8

3 Exercises 10

1 Four Important Probability Distributions

In the previous lecture, we discussed discrete and continuous random variables and their

probability distributions. Before today＊s review on sampling, estimators, and hypothesis

testing, we first review/introduce four probability distributions, all continuous, that turns

out to be important in the statistical inference that we will use. They are the normal

distribution, the t distribution, the F distribution, and the chi-squared (聿2) distribution.

1.1 Normal distribution

The normal distribution is the most commonly-used distribution in econometrics. The prob-

ability density function (pdf) of the normal distribution is symmetric and beautifully bell-

shaped. It is fully determined by the mean/expectation 米 ﹋ R and variance 考2 > 0 of the

distribution, and has the form. (1)

The structure of the normal distribution is shown in Figure 1. Think about below questions:

What are the mean, median, and mode of the normal distribution?

How will the shape of the normal pdf change when 考2 becomes larger or smaller?

1

Figure 1: Structure of the normal distribution (Figure R.12 in the textbook)

When 米 = 0 and 考 = 1 the normal distribution is called the standard normal distribution,

and the pdf of the standard normal distribution is usually denoted as ?(x):

where f(x|米, 考2) is the normal pdf defined in (1). Note that we have the relationship

f(x|米, 考2) = 1.

Therefore, every normal pdf can be considered as derived from the standard normal pdf by

the following three steps:

1. relocate the center of standard normal pdf from 0 to 米;

2. stretch/scale the whole domain of the standard normal pdf by a factor of 考;

3. multiply the pdf function by 1/考. (This is to ensure the pdf integrates to 1.)

For this reason (the steps 1 and 2 above in specific), 米 is called the location parameter and

考 is called the scale parameter of the normal distribution.

We call a random variable, say X, as a ＆normal random variable＊ or ＆normally distributed＊

if it follows a normal distribution. Given the location parameter 米 and scale parameter 考,

we write

X ‵ N(米, 考2).

It is clear that E(X) = 米 and Var(X) = 考2.

1.2 t distribution

The t distribution, or Student＊s t distribution, arises in statistics when estimating the pop-

ulation mean of a normally distributed random variable in situations where the sample size

is small and the population variance is unknown. The Student＊s t distribution was named

after English statistician William Sealy Gosset under the pseudonym of ＆Student＊.

2

The pdf of t distribution is1,

where 忙(﹞) is the gamma function, and the parameter 糸 > 0 is called the ＆degrees of freedom＊

of the t distribution.

The pdf of t distribution is also symmetric and bell-shaped, however, it has ＆fatter tails＊

than normal distribution. Note there are two special cases of t distribution:

When 糸 = 1, the t distribution becomes the well-known Cauchy distribution which

does not have a well-defined (because it is infinity) expectation/mean.

When 糸 ↙﹢, the t distribution becomes the standard normal distribution2.

The notation of a random variable X following t distribution with degrees of freedom 糸

is

X ‵ t(糸) or X ‵ t糸 .

1.3 Chi-squared distribution

The chi-squared (or sometimes chi-square or 聿2) distribution with k ＆degrees of freedom＊,

denoted as 聿2(k) or 聿2k, is the distribution of a sum of the squares of k independent standard

normal random variables. Here the parameter k is a positive integer.

In other words, if Z1, . . . , Zk are independent, standard normal random variables, then

the random variable

X :=

k﹉

i=1

Z2i = Z

2

1 + ﹞ ﹞ ﹞+ Z2k

follows the chi-squared distribution with k degrees of freedom, and the notation is

X ‵ 聿2(k) or X ‵ 聿2k.

Clearly, a chi-squared random variable can only take values on [0,﹢). The chi-squared

distribution is commonly used for critical values for ＆asymptotic＊ tests3.

1You don＊t need to remember this, but I have it here for completion.

2This is because lim糸↙﹢ f(x|糸) = ?(x) where ?(x) is the pdf of the standard normal distribution defined

in (2). For your curiosity of why this is the case, first note below two results on the limits of functions:

每 As one way of defining exponential function, we have lim糸↙﹢(1 + x/糸)糸 = ex for any x.

每 By using the Stirling＊s approximation of gamma functions, we have lim糸↙﹢

忙((糸+1)/2)﹟

糸忙(糸/2)

= 1/

﹟

2.

Then we have, as 糸 ↙﹢,

f(x|糸) = 忙((糸 + 1)/2)﹟.

3The ＆asymptotic＊ tests are the hypothesis tests conducted when the sample size is large enough to be

approximated as infinity.

3

1.4 F distribution

The F distribution with two parameters, 糸1 and 糸2, indicating ＆degrees of freedom＊ is the

distribution of a random variable X defined as

X :=

Q1/糸1

Q2/糸2

,

where

Q1 ‵ 聿2(糸1) and Q2 ‵ 聿2(糸2);

Q1 and Q2 are independent.

The notation is

X ‵ F (糸1, 糸2).

The F distribution was tabulated by a 1934 paper by Snedecor who introduced the notation

F as the distribution is related to Sir Ronald Fisher＊s work on the analysis of variance.

2 Samples and Estimators

The unifying methodology of modern econometrics was articulated by Trygve Haavelmo

in his seminar paper ※The probability approach in econometrics§ (1944, [link]). In this

paper, Haavelmo argued that quantitative economic models must necessarily be ＆probability

models＊ instead of deterministic models, because the latter are blatently inconsistent with

observational economic quantities. Once we acknowledge that

1. an economic model is a probability model, and

2. observational4 economic data are ＆realizations＊ of some random variables whose popu-

lation distributions are not fully known,

it follows naturally that an appropriate way to quantify, estimate, and conduct inferences

about the economic phenomena should be through the powerful theory of mathematical

statistics.

2.1 Sampling and double structure of a sampled random variable

In a certain application, to infer some population characteristics of a random variable or

to infer the relationship among a set of random variables, an econometrician uses a set of

repeated measurements on these variables. For example, in a labor application the variables

could include weekly earnings, years of education, age, gender, among others. We call these

measurements the data, dataset, or sample. We use the term observations to refer to distinct

repeated measures on the variables.

An individual observation may corresponds to a specific economic unit, such as the

income of a person, household, firm, city, country, etc. ↙ cross-sectional observations

4Most economic data are ＆observational＊ instead of ＆experimental＊ which is more common in natural science.

This is because conducting experiments in social and economic studies is oftentimes condemned as immoral

or simply impossible. The constraint of having only observational data makes the inference of ＆causality＊

particularly hard in econometrics.

4

An individual observation may also corresponds to a measurement at a point in time,

such as quarterly GDP or a daily stock price. ↙ time-series observations

Now let＊s formulate things mathematically. Let X be the random variable we are inter-

ested in, and we want take to a sample of n observations to infer, say the population mean

of X. A subtle but important point here is how we understand these n observations in our

sample.

Before (Pre) the sample is generated, the n potential observations of X are considered

as a set of n random variables which follow the same distribution as that of X. Fol-

lowing our convention of using upper case Roman letters to denote random variables,

we denote the n observations of X in a sample as

{X1, X2, . . . , Xn}.

In particular, we call {Xi : i = 1, . . . , n} a random sample if they are (1) mutually

independent, and (2) identically distributed (i.i.d.) across i = 1, . . . , n. In the following,

unless mentioned otherwise, the samples we will discuss are random samples.

After (Post) the sample is generated, the observations of X become n specific numbers.

We denote these numbers as

{x1, x2, . . . , xn},

using lower case letters. A statistician would refer to {x1, . . . , xn} as a realization of

the random variable X.

Understanding such ＆double structure＊ of a sampled random variable before and after the

sample is generated is crucial for understanding the (pre-sample) analysis of the properties

of estimators and the procedure of hypothesis testing.

2.2 Estimators

An estimator can be, in general, considered as a function of the sample. It takes all the

observations in the sample, X1, . . . , Xn, as inputs, and produce an output quantity (based

on a particular rule) to estimate a certain population characteristic of the random variable

X. For example, suppose we have a random sample {X1, . . . , Xn} of X and a random sample

{Y1, . . . , Yn} of Y :

If we want to estimate the (population) mean 米X of X, then we may consider the

＆sample mean＊ If we want to estimate the (population) variance 考2X of X, then we may consider the

＆sample variance＊

If we want to estimate the (population) covariance 考XY of X and Y , then we may

consider the ＆sample covariance＊

Question: why do we divide by n 1 rather than n in the formulas of 考?2X and 考?2XY ?

(Read textbook R.7, pages 33-34)

If we want to estimate the (population) correlation coefficient 老XY of X and Y , then

we may consider the ＆sample correlation coefficient＊.

Note that all the quantities above, namelyX, 考?2X , 考XY , and 老XY are estimators. Two questions:

1. Can we talk about the probability distributions of these estimators? Why?

2. If yes, then what are the mean and variance of X for example? How do the mean and

variance of X depend on the sample size n?

The key here is, again, the distinction between the potential distribution of the estimator

before the sample is generated, and the actual realization after the sample is generated.

Before the sample is generated, an estimator is a function of the observations in the

sample(s) which are considered as random variables. Therefore, an estimator is also a

random variable in the pre-sample analysis.

After the sample is generated, the random variables (Xi or Yi) in the formula of the

estimator can be replaced by their actual realizations (xi or yi). We describe the

realized value of the estimator as the estimate, which is just a specific number.

See Figure 2 for an illustration of this ＆double structure＊ of estimator inherited from the

double structure of a sampled random variable.

Having fixed the idea that an estimator is a random variable which follows certain

probability distribution in the pre-sample analysis, we can now talk in general about the

mean/expectation and the variance of an estimator. The study of these two characteristics

in the distribution of an estimator leads us to the analysis of the two important properties

of an estimator, namely unbiasedness and efficiency.

2.3 Bias and Variance

Let＊s adopt some generic notations. Let Z = Z(X1, . . . , Xn) be an estimator for the value of

a population characteristic (say mean, variance, etc.), denoted as 牟. We say Z is an unbiased

estimator if

E(Z) = E[Z(X1, . . . , Xn)] = 牟. (3)

If equation (3) does not hold, then we say Z is a biased estimator, and the bias is E(Z)? 牟.

6

Figure 2: Sample and estimator (Table R.5 in the textbook)

If the bias is negative, or E(Z) < 牟, then there is an under-estimation bias.

If the bias is positive, or E(Z) > 牟, then there is an over-estimation bias.

Taking sample mean as an example. Suppose X = 1n

﹉n

i=1Xi is used to estimate the

unknown population mean 米X of X. We say X is unbiased if.

Questions:

1. Can you show that X is unbiased for 米X? What if the observations X1, . . . , Xn are

not mutually independent? Does it matter?

2. Is X the only unbiased estimator for 米X?

If there are more than one unbiased estimators, how do we compare them? See Figure 3.

The idea of efficiency comparison5 is that we prefer the estimator to have as high probability

as possible of giving a close estimation of the population characteristic. ↙ pdf as concen-

trated as possible around the true value. Another way to put it is that we want the variance

of the estimator to be as small as possible.

5Note that efficiency is a comparative concept, and you should use the term only when comparing different

estimators rather than summarizing changes in the variance of a single estimator.

7

Figure 3: Two unbiased estimators and efficiency comparison (Figure R.8 in the textbook)

Mathematically, suppose we have two unbiased6 estimators Z1 = Z1(X1, . . . , Xn) and

Z2 = Z2(X1, . . . , Xn) for the population characteristic 牟; we say Z1 is more efficient than Z2

if

Var(Z1) < Var(Z2),

and vice versa. Note that in the definition, we requite Z1 and Z2 to use the same amount

information: X1, . . . , Xn as observations on random variable X. This is for fair comparison.

In the textbook (R.45)每(R.49), it shows that the sample mean is the most efficient es-

timator for the population among all the estimators of the weighted average kind, with a

simple illustration where the sample size n = 2.

2.4 Loss functions and mean squared error

Clearly, both unbiasedness and mininum variance are desirable properties of an estimator.

But sometimes there can be conflicts between these two properties when we choose among

estimators. See Figure 4.

There is not a sure answer to the question of which estimator to choose. It all depends

on the circumstances and what criterion one would like to use. In the decision theory of

statistics, a ＆loss function＊ denoted as ?(Z, 牟) is often introduced to quantify the cost of using

an estimator, say Z, to estimate a target parameter 牟. The loss function can be very general

as long as it satisfies:

? ?(Z, 牟) ≡ 0 for any Z.

? ?(Z, 牟) = 0 if Z = 牟.

Given the loss function, the ＆optimal＊ estimator is considered as the one which minimizes the

expected loss E[?(Z, 牟)].

Just to name a few examples of the loss function:

6In the textbook, the comparisons of efficiency are mostly confined to unbiased estimators.

8

Figure 4: Which estimator to choose? (Figure R.9 in the textbook)

quadratic/squared loss: ?(Z, 牟) = (Z ? 牟)2

linear/absolute loss: ?(Z, 牟) = |Z ? 牟|

Hubor loss: quadratic for small values of |Z ? 牟| and linear for large values of |Z ? 牟|

In particular, the quadratic loss function is the most commonly used. The expected loss

when quadratic loss function is used is known as the mean squared error (MSE):

MSE of estimator Z = E[(Z ? 牟)2].

So the estimator that minimizes the expected loss when quadratic loss function is used is the

estimator that has the smallest mean squared error.

Next we show a useful decomposition of MSE:

MSE of an estimator := Variance of the estimator + Bias of the estimator squared . (4)

In mathematical form, let Z be the estimator for 牟, and let 米Z and 考

2

Z denote the mean and

variance of Z, respectively. We decompose the MSE of Z as follows:

MSE(Z) = E[(Z ? 牟)2]

= E[(Z ? 米Z + 米Z ? 牟)2]

= E[(Z ? 米Z)2 + (米Z ? 牟)2 + 2(Z ? 米Z)(米Z ? 牟)]

= E[(Z ? 米Z)2] + E[(米Z ? 牟)2伴 佞佞 伴

constant

] + 2E[(Z ? 米Z) (米Z ? 牟)伴 佞佞 伴

constant

]

= E[(Z ? 米Z)2]伴 佞佞 伴

考2Z

+( 米Z ? 牟伴 佞佞 伴

=E(Z)?牟

)2 + 2(米Z ? 牟) E(Z ? 米Z)伴 佞佞 伴

=E(Z)?米Z=0

= Var(Z) + Bias2(Z).

Because of this decomposition, the MSE is sometimes used to generalize the concept of

9

efficiency to cover comparisons of biased as well as unbiased estimators.

Example: MSE of the sample variance as the estimator of population variance and the

idea of shrinkage in statistics.

3 Exercises

The below questions are from Exercises R.15, R.19-23, R.30-33 in the textbook. Note that

R.23 and R.30-33 are related to hypothesis testing that you are supposed to have learned in

a pre-requisite course. You should take these exercises as a review together with reading the

corresponding textbook sections (Chapter R.9每R.13).

R.15 For the special case 考2X = 1 and a sample of two observations X1 and X2, calculate the

variance of the generalized estimator, Z = 竹1X1+ 竹2X2 with 竹1+ 竹2 = 1, of the population

mean. Using the fact that

竹21 + 竹

2

2 = 竹

2

1 + (1? 竹1)2 = 2竹21 ? 2竹1 + 1,

obtain the variance of Z with values of 竹1 from 0 to 1 at steps of 0.1, and plot it in a diagram.

Is it important that the weights 竹1 and 竹2 should be exactly equal?

R.19? In general, the variance of the distribution of an estimator decreases when the sample

size is increased. Is it correct to describe the estimator as becoming more efficient?

R.20 If you have two estimators of an unknown population parameter, is the one with the

smaller variance necessarily more efficient?

R.21? Suppose that you have observations on three variables X,Y , and Z, and suppose

that Y is an exact linear function of Z:

Y = 竹+ 米Z

where 竹 and 米 are positive constants. Show that 老?XZ = 老?XY . (This is the counterpart of

Exercise R.14.)

R.22 A scalar multiple of a normally-distributed random variable also has a normal dis-

tribution. A random variable X has a normal distribution with mean 5 and variance 10.

Sketch the distribution of Z = X/2.

R.23 Suppose that a random variable with hypothetical mean 10 may be assumed to have

a normal distribution with variance 25. Given a sample of 100 observations, derive the

acceptance and rejection regions for X, (a) using a 5 percent significance test, (b) using a 1

percent test.

R.30 A drug company asserts that its course of treatment will, on average, reduce a per-

son＊s cholesterol level by 0.8 mmol/L. A researcher undertakes a trial with a sample of 30

individuals with the objective of evaluating the claim of the drug company. What should he

report if he obtains the following results:

(a) a mean decrease of 0.6 units, with standard error 0.2 units;

(b) a mean decrease of 0.4 units, with standard error 0.2 units;

(c) a mean increase of 0.4 units, with standard error 0.2 units?

10

R.31 When a local sales tax was abolished, a survey of 20 households showed that mean

household expenditure increased by $160 and the standard error of the increase was $60.

What＊s the 99 percent confidence interval for the effect of the sales?

R.32 Determine the 95 percent confidence interval for the effect of an increase in the min-

imum wage on employment, given the data in Exercise R.29, for each part of the exercise.

How do these confidence intervals relate to the results of the t tests in that exercise?

R.33? Demonstrate that the 95 percent confidence interval defined by equation

X ? tcrit,2.5% ℅ s.e.(X) ≒ 米 ≒ X + tcrit,2.5% ℅ s.e.(X)

has a 95 percent probability of capturing 米0 if H0 : 米 = 米0 is true.