Multinomial,
Exponential and Gamma distributions
Multinomial
Distribution: This
distribution can be regarded as a generalization of Binomial distribution.
When
there are more than two exclusive outcomes of a trial, the observation leads to
Multinomial distribution.
Suppose $E_1, E_2,......, E_k$ are mutually exclusive
and exhaustive outcomes of a trial with respective probabilities $p_1, p_2, .....,p_k$. The probability that, $E_1$ occurs $x_1$ times, $E_2$ occurs $x_2$ times,………, $E_k$ occurs $x_k$ times in $n$ independent observations,
is given by
$$P[x_1, x_2,......, x_k]=C. p_1^{x_1} p_2^{x_2}....... p_k^{x_k}, ~ where, \sum x_i =n$$
and $C$ is the number of permutation of the events $E_1, E_2,.....,E_k$ with $C=\frac{n!}{x_1!x_2!.....x_k!}$
Therefore, $P[x_1,x_2,....,x_k]=\frac{n!}{\prod_{i=1}^k x_i!} \prod_{i=1}^k p_i^{x_i}, 0\leq x_i \leq n$
Also $(p_1+p_2+.......+p_k)^n=1$ as $\sum p_i =1$.
Example
1. There are given a bag of marbles. Inside
the bag, there are 5 red marbles, 4 white marbles and 3 blue marbles. Calculate
the probability that with 6 trials, we have to choose 3 marbles that are red, 1
marble that is white and 2 marbles that are blue, replacing each marble after
it is chosen.
Solution: Here total number of marbles are 12. Therefore
probabilities of selecting red, white and blue marbles are $p_r=\frac{5}{12}, p_w=\frac{4}{12}$ and $p_b=\frac{3}{12}$ respectively.
Let $E_r, E_w$ and $E_b$ denote the events of choosing 3 red, 1 white and
2 blue marbles, then the number of marbles to be chose $x_r=3, x_w=1$ and $x_b=2$ and number of trials $n=6=x_r+x_w+x_b$.
Now the number of permutation of the events $E_r, E_w$ and $E_b$ is $C=\frac{6!}{3!1!2!}$
Therefore, probability that $E_r, E_w$ and $E_b$ occur 3 times, 1 time and 2 times, is
$P[x_r=3, x_w=1, x_b=2]=C. p_r^{x_r} p_w^{x_w} p_b^{x_b}$
$P[x_r=3, x_w=1, x_b=2]=\frac{6!}{3!1!2!} \left( \frac{5}{12}\right)^3 \left(\frac{1}{12}\right)^1 \left(\frac{3}{12}\right)^2$
$$P[x_r=3, x_w=1, x_b=2]=0.0899$$
Exercise: We are randomly drawing cards from an
ordinary deck of cards. Every-time we pick one, we place it back in the deck.
We do this 5 times. What is the probability of drawing 1 heart, 1 space, 1 club
and 2 diamonds? Ans. 0.0586
Exponential
and Gamma distributions
Exponential distribution: A
random variable X is said to have an exponential distribution with parameter $\lambda>0$, if its density function
is given by $f_X(x)=\lambda e^{-\lambda x}$ for all $x\geq 0$ and $\lambda>0$
We write it as $X \sim Expo(\lambda)$
Gamma distribution: A random variable X is said to have a gamma
distribution if its density function is given by $f_X(x)=\frac{\lambda}{\Gamma(r)} (\lambda x)^{r-1}e^{-\lambda x}$ for all $x\geq 0$,
Where $r>0$ and $\lambda >0$ are called the parameters of the gamma distribution.
We write it as $X \sim gam(\lambda; r)$ or $X\sim G(\lambda ; r)$
Remark 1. $\Gamma(r) $ is called gamma function and is defined as $\Gamma(r)=\int_0^{\infty} x^{r-1}e^{-x}dx$
It is easy to verify that $\Gamma(1)=1, \Gamma(a+1)=a\Gamma(a), \Gamma(n)=(n-1)!, n$ is a positive integer.
Remark 2. Taking $r=1$, we see that gamma density function becomes exponential density
function.
Theorem: If X has an exponential distribution,
then
$$E[X]=\frac{1}{\lambda}, var[X]=\frac{1}{\lambda^2}~ and~ M_X(t)=\frac{\lambda}{\lambda-t} ~for~ t< \lambda$$
Proof: We have $E[X]=\int_0^{\infty} xf_X(x) dx = \int_0^{\infty}x \lambda e^{-\lambda x}dx, \lambda>0$
Integrating by parts, we get
$$E[X]=\lambda \left| x \left(\frac{e^{-\lambda x}}{-\lambda}\right)\right|_0^{\infty}+\int_0^{\infty} 1. e^{-\lambda x} dx = 0-\frac{1}{\lambda}\left| e^{-\lambda x}\right|_0^{\infty}=-\frac{1}{\lambda}(0-1)$$
$$E[x]=\frac{1}{\lambda} ....... (1)$$
Now $E[x^2]=\int_0^{\infty} x^2 f_X(x) dx =\int_0^{\infty} x^2 \lambda e^{-\lambda x} dx$
Integrating by parts, we get
$$E[X^2]=\lambda \left| x^2 \left( \frac{e^{-\lambda x}}{-\lambda}\right)\right|_0^{\infty}+ 2 \int_0^{\infty} x. e^{-\lambda x} dx=0+0+\frac{2}{\lambda} \int_0^{\infty} e^{-\lambda x}dx$$
$$E[X^2]=\frac{2}{\lambda}.\frac{1}{\lambda}=\frac{2}{\lambda^2}$$
Therefore, $Var[X]=E[X^2]-(E[X])^2=\frac{2}{\lambda^2}-\frac{1}{\lambda^2}=\frac{1}{\lambda^2}$
The m.g.f. of X is given by
$$ M_X(t) = E[e^{tX}]=\int_0^{\infty} e^{tx}f_X(x) dx=\int_0^{\infty} e^{tx}\left(\lambda e^{-\lambda x}\right)dx $$
$$ M_X(t)=\lambda \int_0^{\infty} e^{-(\lambda-t)x}dx,~~~ \lambda >t$$
$$ M_X(t)=- \frac{\lambda}{\lambda-t}\left| e^{-(\lambda-t)x}\right|_0^{\infty}=-\frac{\lambda}{\lambda-t}(0-1)$$
Hence, $$ M_X(t)=\frac{\lambda}{\lambda - t}, ~ for ~\lambda > t$$.
Theorem: If X has gamma distribution with
parameters $r$ and $\lambda$, then
$$E[X]=\frac{r}{\lambda},~ var[X]=\frac{r}{\lambda^2}~ and~ M_X(t)=\left(\frac{\lambda}{\lambda-t}\right)^r~ for ~t<\lambda$$
Theorem: Show that the sum of independent gamma variates
is also a gamma variate
Hint:
Let $x_i \sim G(\lambda; r_i)$ for $i=1, 2, ......., n;$
Example 1. If
X has exponential distribution with mean 2, find $P[X<1|X>2]$
Solution: We are given that $\frac{1}{\lambda}=2 \Rightarrow \lambda=\frac{1}{2}$
Now $P[X<1|X<2]=\frac{P[(X<1)\cap (X<2)]}{P[X<2]}=\frac{P[X<1]}{P[X<2]}=\frac{\int_0^1 \lambda e^{-\lambda x}dx}{\int_0^2 \lambda e^{-\lambda x}dx}$
$P[X<1|X<2]=\frac{-\frac{1}{\lambda}|e^{-\lambda x}|_0^{\infty}}{\frac{1}{\lambda}|e^{-\lambda x}|_0^2}= \frac{1-e^{-\lambda}}{1-e^{-2\lambda}}=\frac{1}{1+e^{-\lambda}}$
Hence, $P[X<1|X<2]=\frac{1}{1+e^{-\frac{1}{2}}}$
Exercises:
1. If X has exponential distribution with $P[X\leq 1]=P[X>1],$ then find $var[x]
2. Find
the median of the exponential distribution.
3. If $X\sim Expo (\lambda)$ find the value $k$ such that $\frac{P[X>k]}{P[X\leq k]}=a.$