Unit-1: Random Variables and Probability Distributions

1. Random Variables

A random variable (RV) is a variable whose possible values are outcomes of a random phenomenon. It can be classified as either discrete or continuous.

Discrete Random Variable: Takes on a countable number of distinct values (e.g., number of heads in coin tosses).
Continuous Random Variable: Takes on an infinite number of possible values within a range (e.g., height, weight).

Cumulative Distribution Function (CDF)

The CDF of a random variable $X$ is defined as:

F (x) = P (X \leq x) = \sum_{t \leq x} f (t)

Properties of CDF:

$0 \leq F (x) \leq 1$
$F (x)$ is non-decreasing.
$\lim_{x \to - \infty} F (x) = 0$
$\lim_{x \to \infty} F (x) = 1$

2. Probability Distributions

Probability Mass Function (PMF)

For a discrete random variable, the PMF $P (X = x)$ satisfies:

0 \leq P (X = x) \leq 1

\sum_{x} P (X = x) = 1

f (x) \geq 0

\sum_{x} f (x) = 1

Probability Density Function (PDF)

For a continuous random variable, the PDF $f (x)$ satisfies:

f (x) \geq 0

\int_{- \infty}^{\infty} f (x) d x = 1

P (a < X < b) = \int_{a}^{b} f (x) d x

Note: For continuous RVs, $P (X = x) = 0$ .

Relation Between PDF and CDF:

F' (x) = f (x)

Properties of CDF

Property	Discrete Random Variable	Continuous Random Variable
Definition	$F (x) = \sum_{x_{i} \leq x} P (X = x_{i})$	$F (x) = \int_{- \infty}^{x} f (t) d t$
Range	$0 \leq F (x) \leq 1$	$0 \leq F (x) \leq 1$
Monotonicity	Non-decreasing	Non-decreasing
Right-Continuity	Right-continuous	Continuous
Limits	$\lim_{x \to - \infty} F (x) = 0$ $\lim_{x \to \infty} F (x) = 1$	$\lim_{x \to - \infty} F (x) = 0$ $\lim_{x \to \infty} F (x) = 1$
Jump Discontinuities	Has jumps at each value of $x$ where $P (X = x) > 0$	No jumps (smooth curve)
Probability at a Point	$P (X = x) = F (x) - F (x^{-})$	$P (X = x) = 0$

3. Joint Probability Distributions

Joint PMF

For two discrete random variables $X$ and $Y$ , the joint PMF $P (X = x, Y = y)$ satisfies:

0 \leq P (X = x, Y = y) \leq 1

\sum_{x} \sum_{y} P (X = x, Y = y) = 1

Joint PDF

For two continuous random variables $X$ and $Y$ , the joint PDF $f_{X, Y} (x, y)$ satisfies:

f_{X, Y} (x, y) \geq 0

\int_{- \infty}^{\infty} \int_{- \infty}^{\infty} f_{X, Y} (x, y) d x d y = 1

4. Marginal Probability Distribution

The marginal probability distribution is the probability distribution of a subset of random variables, obtained by summing or integrating over the other variables.

For Discrete Random Variables

Given two discrete random variables $X$ and $Y$ , the marginal PMF of $X$ is:

P (X = x) = g (x) = \sum_{y} f (x, y)

Similarly, the marginal PMF of $Y$ is:

P (Y = y) = h (y) = \sum_{x} f (x, y)

For Continuous Random Variables

Given two continuous random variables $X$ and $Y$ , the marginal PDF of $X$ is:

f_{X} (x) = \int_{- \infty}^{\infty} f_{X, Y} (x, y) d y

Similarly, the marginal PDF of $Y$ is:

f_{Y} (y) = \int_{- \infty}^{\infty} f_{X, Y} (x, y) d x

4. Conditional Probability Distribution

The conditional probability distribution describes the probability of one random variable given the value of another random variable.

For Discrete Random Variables

The conditional PMF of $X$ given $Y = y$ is:

P (X = x | Y = y) = \frac{P (X = x, Y = y)}{P (Y = y)}

Similarly, the conditional PMF of $Y$ given $X = x$ is:

P (Y = y | X = x) = \frac{P (X = x, Y = y)}{P (X = x)}

For Continuous Random Variables

The conditional PDF of $X$ given $Y = y$ is:

f_{X | Y} (x | y) = \frac{f_{X, Y} (x, y)}{f_{Y} (y)}

Similarly, the conditional PDF of $Y$ given $X = x$ is:

f_{Y | X} (y | x) = \frac{f_{X, Y} (x, y)}{f_{X} (x)}

5. Mean, Variance, and Covariance

Mean (Expected Value)

For a discrete random variable:

μ = E (X) = \sum_{x} x \cdot f (x)

μ_{g (x)} = E [g (x)] = \sum_{x} g (x) \cdot f (x)

For a continuous random variable:

μ = E (X) = \int_{- \infty}^{\infty} x \cdot f (x) d x

μ_{g (x)} = E [g (x)] = \int_{- \infty}^{\infty} g (x) \cdot f (x) d x

Linearity of Expectation:

E (a X + b Y) = a E (X) + b E (Y)

Variance

For any random variable:

Var (X) = E (X^{2}) - {[E (X)]}^{2}

Properties:

$Var (a X + b) = a^{2} Var (X)$
$Var (X + Y) = Var (X) + Var (Y) + 2 Cov (X, Y)$

Covariance

For two random variables $X$ and $Y$ :

Cov (X, Y) = E (X Y) - E (X) E (Y)

Properties:

$Cov (X, X) = Var (X)$
$Cov (X, Y) = Cov (Y, X)$
$Cov (a X, b Y) = a b Cov (X, Y)$

6. Chebyshev's Theorem

For any random variable $X$ with mean $μ$ and variance $σ^{2}$ :

P (| X - μ | \leq k σ) \geq 1 - \frac{1}{k^{2}}

Complementary Form:

P (| X - μ | > k σ) \leq \frac{1}{k^{2}}

Key Features:

Applies to any distribution with finite mean and variance.
Provides a lower bound on the probability.

MCQ Questions

1. Which of the following is true for a probability mass function (PMF)?

a) ( P(X = x) ≥ 0 )
b) ( ∑_x P(X = x) = 1 )
c) Both a) and b)
d) None of the above

Answer: c) Both a) and b)

2. For ( k = 2 ), Chebyshev's Theorem states that the probability ( P(|X - μ| ≤ 2σ) ) is at least:

a) 0.50
b) 0.75
c) 0.95
d) 0.99

Answer: b) 0.75