Random variables, probability mass and density…

Core facts

Random variable (r.v.) X: a real-valued function defined on the sample space.
Discrete r.v.: takes countable values (number of heads, number of defects).
Continuous r.v.: takes any value in an interval (height, time, weight).
Probability mass function (pmf) p(x) = P(X = x): p(x) ≥ 0 and Σ p(x) = 1.
Probability density function (pdf) f(x): f(x) ≥ 0 and ∫ f(x) dx = 1 over the whole range.
Continuous case: P(a ≤ X ≤ b) = ∫ from a to b of f(x) dx, and P(X = a) = 0 for every single point a.
Distribution function F(x) = P(X ≤ x). It is non-decreasing, F(−∞) = 0, F(+∞) = 1, and it is right-continuous.
Discrete: F is a step function. Continuous: f(x) = dF(x)/dx.
Expectation: E(X) = Σ x p(x) (discrete) or ∫ x f(x) dx (continuous).
Variance: V(X) = E[X − E(X)]² = E(X²) − [E(X)]². SD = √V(X).
E(aX + b) = aE(X) + b; V(aX + b) = a² V(X). Change of origin does not change variance.
E(constant) = constant; V(constant) = 0.
Addition theorem: E(X + Y) = E(X) + E(Y), for any X and Y.
Multiplication theorem: E(XY) = E(X) E(Y), when X and Y are independent.
Covariance: Cov(X, Y) = E(XY) − E(X)E(Y). Independent ⇒ Cov = 0.
V(X ± Y) = V(X) + V(Y) ± 2 Cov(X, Y); for independent X, Y: V(X ± Y) = V(X) + V(Y) (plus in both cases).
Moment generating function: M_X(t) = E(e^(tX)). The r-th raw moment is the coefficient of t^r/r! in it.

Explained

1. Discrete case.

Example 1. A fair die is thrown. Find E(X) and V(X), where X is the number shown. Solution. E(X) = (1 + 2 + … + 6)/6 = 21/6 = 7/2. E(X²) = (1 + 4 + 9 + 16 + 25 + 36)/6 = 91/6. V(X) = 91/6 − 49/4 = 182/12 − 147/12. Answer: E(X) = 3.5; V(X) = 35/12

Example 2. X takes the values 1, 2, 3, 4 with p(x) = kx. Find k, E(X), V(X) and F(2). Solution. Σ p(x) = k(1 + 2 + 3 + 4) = 10k = 1, so k = 1/10. E(X) = Σ x · x/10 = (1 + 4 + 9 + 16)/10 = 3. E(X²) = (1 + 8 + 27 + 64)/10 = 10, so V(X) = 10 − 9 = 1. F(2) = p(1) + p(2) = 3/10. Answer: k = 1/10; E(X) = 3; V(X) = 1; F(2) = 0.3

Example 3. A player wins ₹10 if a coin shows head and loses ₹5 if it shows tail. Find the expected gain. Solution. E = 10 × 1/2 + (−5) × 1/2 = 5 − 2.5. Answer: ₹2.50 per game A game is called fair when the expected gain is 0.

2. Continuous case. Probability is an area under f(x), not the height of f(x). The value f(x) itself can be more than 1.

Example 4. f(x) = 2x for 0 < x < 1 (and 0 elsewhere). Check that it is a pdf. Find P(X ≤ 1/2), E(X) and V(X). Solution. ∫₀¹ 2x dx = [x²]₀¹ = 1, and f ≥ 0, so it is a pdf. P(X ≤ 1/2) = [x²] from 0 to 1/2 = 1/4. E(X) = ∫₀¹ 2x² dx = 2/3. E(X²) = ∫₀¹ 2x³ dx = 1/2. V(X) = 1/2 − 4/9 = 1/18. Answer: P = 1/4; E(X) = 2/3; V(X) = 1/18

3. Linear changes.

Example 5. E(X) = 5 and V(X) = 4. Find E(3X − 2), V(3X − 2) and the SD of 3X − 2. Solution. E = 3 × 5 − 2 = 13. V = 3² × 4 = 36 (the −2 has no effect). SD = 6. Answer: 13, 36, 6

Example 6. X and Y are independent with V(X) = 4 and V(Y) = 9. Find V(X − Y). Solution. For independent variables the variances add, even for a difference. 4 + 9. Answer: 13

Deeper

1. Discrete versus continuous.

Point	Discrete	Continuous
Function	pmf p(x) = P(X = x)	pdf f(x); P(X = x) = 0
Total	Σ p(x) = 1	∫ f(x) dx = 1
E(X)	Σ x p(x)	∫ x f(x) dx
F(x)	step function	continuous; F′(x) = f(x)

Because P(X = a) = 0 for a continuous r.v., P(a < X < b) = P(a ≤ X ≤ b). For a discrete r.v. the end points matter.

2. Why V(X) = E(X²) − [E(X)]². Write μ = E(X). Then E(X − μ)² = E(X²) − 2μE(X) + μ² = E(X²) − μ². Since V(X) ≥ 0, we always have E(X²) ≥ [E(X)]².

3. Why V(aX + b) = a²V(X). aX + b − E(aX + b) = a(X − μ). Squaring gives a²(X − μ)², and its expectation is a²V(X). The constant b cancels: variance is independent of the change of origin but not of scale.

4. Covariance cautions.

Independent ⇒ Cov(X, Y) = 0. The converse is false: zero covariance does not prove independence.
Example: X takes −1, 0, 1 each with probability 1/3, and Y = X². Then E(X) = 0 and E(XY) = E(X³) = 0, so Cov = 0. But Y is fully decided by X.
Cov(aX, bY) = ab Cov(X, Y). Cov(X, X) = V(X).

5. Number of heads in three tosses. X = 0, 1, 2, 3 with probabilities 1/8, 3/8, 3/8, 1/8. E(X) = 12/8 = 3/2. E(X²) = 24/8 = 3. V(X) = 3 − 9/4 = 3/4. This matches the binomial results np and npq (STA-PD-03).

6. Moment generating function (mgf).

M_X(t) = E(e^(tX)) = 1 + t μ′₁ + (t²/2!) μ′₂ + …
μ′ᵣ = r-th derivative of M_X(t) at t = 0.
M_(aX+b)(t) = e^(bt) M_X(at).
For independent X and Y, M_(X+Y)(t) = M_X(t) M_Y(t). This is the key tool for additive properties.
The mgf, when it exists, decides the distribution uniquely.

7. Chebyshev's inequality (a moment inequality). For any r.v. with mean μ and SD σ, and any k > 0: P(|X − μ| < kσ) ≥ 1 − 1/k², or P(|X − μ| ≥ kσ) ≤ 1/k².

k = 2: at least 3/4 of the probability lies within μ ± 2σ.
k = 3: at least 8/9 lies within μ ± 3σ.
It holds for every distribution, so it is a rough bound. For the normal curve the true figures are much higher (95.45% and 99.73%).

Common traps

✗ "For a continuous r.v., P(X = 2) = f(2)." ✓ P(X = 2) = 0; f(2) is a density, not a probability.
✗ "A pdf value cannot exceed 1." ✓ It can (f(x) = 2x gives f(0.9) = 1.8); only the total area must be 1.
✗ "V(aX + b) = aV(X) + b." ✓ V(aX + b) = a²V(X).
✗ "V(X − Y) = V(X) − V(Y) for independent X, Y." ✓ V(X) + V(Y).
✗ "E(XY) = E(X)E(Y) always." ✓ Only for independent (more precisely, uncorrelated) X and Y. E(X + Y) = E(X) + E(Y) is always true.
✗ "Cov(X, Y) = 0 means X and Y are independent." ✓ Zero covariance does not imply independence.
✗ "V(X) = E(X²) − E(X)." ✓ V(X) = E(X²) − [E(X)]².
✗ "F(x) can decrease." ✓ A distribution function is non-decreasing, from 0 to 1.

Quick revision

pmf: p(x) ≥ 0, Σ p(x) = 1; pdf: f(x) ≥ 0, ∫ f = 1.
Continuous: P(X = a) = 0; f = F′.
F(x) = P(X ≤ x): non-decreasing, 0 to 1.
E(X) = Σ x p(x) or ∫ x f(x) dx.
V(X) = E(X²) − [E(X)]².
E(aX + b) = aE(X) + b; V(aX + b) = a²V(X).
E(X + Y) = E(X) + E(Y) always; E(XY) = E(X)E(Y) if independent.
Independent: V(X ± Y) = V(X) + V(Y).
Cov = 0 does not imply independence.
Chebyshev: P(|X − μ| < kσ) ≥ 1 − 1/k².

Medha AI writes its own notes and checks them against the sources listed. Medha AI is an independent preparation service. It is not affiliated with, endorsed by or connected to APPSC, TGPSC (formerly TSPSC), MPSC, UPSC or any other commission or government body.

4.1.1 Random variables, probability mass and density functions, mathematical expectation and variance

Download this PDF

Core facts

Explained

Deeper

Common traps

Quick revision