Business Statistics for Entrepreneurs

Covariance

Covariance quantifies the direction of a linear relationship between two quantitative variables. It indicates whether an increase in one variable tends to be associated with an increase (positive) or decrease (negative) in the other.

For a sample with $n$ observation pairs $(x_1, y_1), (x_2, y_2), \dots, (x_n, y_n)$, the sample covariance $s_{xy}$ is:

$s_{xy} = \frac{\sum_{i=1}^{n} (x_i - \bar{x})(y_i - \bar{y})}{n - 1}$

where $\bar{x}$ and $\bar{y}$ are sample means. The division by $n-1$ (rather than $n$) follows the same degrees‑of‑freedom reasoning used for sample variance.

Intuition via quadrants

Plot the data with vertical/horizontal lines at $\bar{x}$ and $\bar{y}$:

• Quadrant I ($x_i > \bar{x},\; y_i > \bar{y}$) → product $(x_i-\bar{x})(y_i-\bar{y}) > 0$
• Quadrant III ($x_i < \bar{x},\; y_i < \bar{y}$) → product $> 0$
• Quadrant II ($x_i < \bar{x},\; y_i > \bar{y}$) → product $< 0$
• Quadrant IV ($x_i > \bar{x},\; y_i < \bar{y}$) → product $< 0$

flowchart LR
    A[Point location] --> B{Product sign}
    B -->|Quad I or III| C[Positive contribution]
    B -->|Quad II or IV| D[Negative contribution]

If most influential points lie in Quadrants I and III → $s_{xy} > 0$ (positive linear association).
If they lie in Quadrants II and IV → $s_{xy} < 0$ (negative linear association).
If points are evenly spread → $s_{xy} \approx 0$ (no linear association).

Relation to variance

When $x = y$, covariance reduces to variance of $x$:

$s_{xx} = \frac{\sum (x_i - \bar{x})^2}{n - 1} = s_x^2$

Thus covariance generalises variance to pairs of variables.

Unit dependence – a major flaw

Covariance changes with the units of measurement. For example, measuring income in rupees instead of lakhs scales $(x_i - \bar{x})$ by $10^5$, inflating $s_{xy}$ even though the underlying relationship is unchanged. This makes covariance unsuitable for comparing the strength of association across different datasets.

Example: Hanumantha Pai’s credit card data

Variable pair	$s_{xy}$	Interpretation
Monthly spend vs. annual income	16,109	Positive association
Monthly spend vs. household size	1,610	Positive association (smaller magnitude)

The numerical values are not comparable because units differ (lakhs vs. number of people).

Exam tip: Never compare covariance values across different pairs to judge which relationship is “stronger”. Use the correlation coefficient instead.

Key takeaways – Covariance

• Measures the direction (positive/negative) but not the strength of linear association.
• Formula: $s_{xy} = \frac{\sum (x_i-\bar{x})(y_i-\bar{y})}{n-1}$.
• Positive → x and y tend to move together; negative → they move opposite.
• Unit‑dependent – not useful for comparing strength across different variable pairs.

---

Correlation

The Pearson product‑moment correlation coefficient (sample correlation coefficient) overcomes covariance’s unit dependence by standardising:

$r_{xy} = \frac{s_{xy}}{s_x \cdot s_y}$

where $s_x$ and $s_y$ are the sample standard deviations of $x$ and $y$.

Properties

• $-1 \leq r_{xy} \leq 1$
• $r_{xy} = +1$ iff all points lie on a positively‑sloped straight line (perfect positive linear).
• $r_{xy} = -1$ iff all points lie on a negatively‑sloped straight line (perfect negative linear).
• $r_{xy} = 0$ indicates no linear relationship (non‑linear relationships may still exist).
• Closer to $\pm 1$ → stronger linear association.

Worked example – Hanumantha Pai

Income vs. Monthly spend
$s_{xy} = 16{,}109$, $s_x = 13.74$ lakhs, $s_y = 1{,}454$ rupees.

$r = \frac{16{,}109}{13.74 \times 1{,}454} = 0.81$

→ Strong positive linear association.

Household size vs. Monthly spend
$s_{xy} = 1{,}610$, $s_x = 1.76$, $s_y = 1{,}454$.

$r = \frac{1{,}610}{1.76 \times 1{,}454} = 0.63$

→ Moderate positive linear association (weaker than income).

Correlation	Strength	Example value
$\pm 1$	Perfect	—
$> 0.8$	Strong	0.81 (income)
$0.5$ – $0.8$	Moderate	0.63 (household)
$< 0.5$	Weak	—
$0$	None (linear)	—

Correlation is NOT causation

A high correlation does not prove that a change in one variable causes a change in the other. In the credit‑card example, higher income does not force higher spending – other factors (preferences, debt) could be at play.

Exam tip: The mantra “correlation does not imply causation” is a frequently tested concept. Always remember that a third (lurking) variable or reverse causality could explain the association.

Limitation – linear only

The Pearson correlation only measures linear relationships. A near‑zero $r$ can occur even when a strong non‑linear pattern exists (e.g., U‑shaped relationship between utility spending and outdoor temperature). Always inspect a scatter diagram alongside the correlation coefficient.

When variables are not quantitative

If one or both variables are nominal or ordinal, other measures (e.g., Spearman’s rank correlation) are used – these are discussed later.

Key takeaways – Correlation

• $r_{xy} = \frac{s_{xy}}{s_x s_y}$, dimensionless and scale‑invariant.
• Lies between $-1$ (perfect negative) and $+1$ (perfect positive).
• $0$ means no linear relationship – non‑linear patterns may still exist.
• Correlation is not causation.
• Always check a scatter plot to avoid being misled by a near‑zero $r$ from a non‑linear relationship.

Covariance and Correlation

Covariance measures the direction of the linear relationship between two variables: positive → both move together; negative → one goes up as the other goes down. Correlation standardises covariance onto a $[-1, 1]$ scale, giving a unitless measure of linear association strength.

Computing in Excel

• Sample covariance (divides by $n-1$): =COVARIANCE.S(array1, array2)
• Sample correlation (Pearson’s $r$): =CORREL(array1, array2)

Applied to Hanumantha’s credit card data:

Variables	Covariance	Correlation
Monthly spend & household income	16,109	0.806
Monthly spend & household size	1,610	0.629

The Pearson correlation measures only linear association

$r = \frac{s_{xy}}{s_x s_y}, \qquad s_{xy} = \frac{1}{n-1}\sum_{i=1}^n (x_i - \bar{x})(y_i - \bar{y})$

Worked example (nonlinear relationship):
$x = \{3, 2, 1, -1, -2, -3\}$, $y = 1/x^2 = \{\frac{1}{9}, \frac{1}{4}, 1, 1, \frac{1}{4}, \frac{1}{9}\}$.
Excel gives $s_{xy} = 0$, $r = 0$. But $y = 1/x^2$ is a perfect nonlinear relationship.

Exam tip: $r = 0$ does not mean “no association” – it means no linear association. Always inspect a scatter plot.

Spurious vs. Explainable Correlation

• Explainable correlation: logical reason connects the variables.
• Spurious correlation: apparent relationship due to a hidden (latent) variable or pure coincidence.

Example	Observed correlation	Hidden cause
Ice cream sales ↔ crime rate	Positive	Temperature (summer → more ice cream + more vacant homes)
Number of doctors ↔ number of deaths in villages	Positive	Poor public health → more doctors needed + more deaths
Divorce rate in Maine ↔ margarine consumption	$r \approx 0.99$ (1999–2009)	No known hidden variable – purely spurious
Skirt length ↔ economic activity (Scotland theory)	Skirts shorter in boom, longer in bust	Proposed reason: affordability of stockings; widely considered spurious

Key takeaways

• Covariance and Pearson correlation quantify linear association only.
• $r = 0$ does not imply independence; check for nonlinear patterns.
• Spurious correlation can arise from hidden variables or random chance – never assume causation.
• Sample correlation is a point estimate; a hypothesis test (using a $t$-distribution) can assess if the population correlation is nonzero.

---

Simple Linear Regression – I

Simple linear regression (SLR) predicts a continuous dependent (response) variable $y$ using one independent (predictor) variable $x$. It assumes the relationship can be approximated by a straight line.

Origin and terminology

• Francis Galton (1886) studied heights of 905 children from 205 families. He multiplied female heights by 1.08 to make them male-equivalent. He observed regression toward the mean: tall parents had slightly shorter children, short parents had slightly taller children. He coined the term regression.
• $y$ = dependent / response / outcome variable
• $x$ = independent / predictor / explanatory variable / feature

The model

$y_i = \beta_0 + \beta_1 x_i + \varepsilon_i, \quad i = 1,2,\dots,n$

• $\beta_0$ (intercept), $\beta_1$ (slope) – regression coefficients (parameters)
• $\beta_0 + \beta_1 x_i$ = predicted value (conditional expected value of $y$ given $x_i$)
• $\varepsilon_i$ = random error (residual/unexplained variation)

What “linear” really means

Linearity is defined with respect to the parameters $\beta_0, \beta_1$, not the variable $x$.

• $y_i = \beta_0 + \beta_1 x_i^2 + \varepsilon_i$ → linear regression (parameters are linear)
• $y_i = \beta_0 + \beta_1 \log(x_i) + \varepsilon_i$ → linear regression
• $y_i = \beta_0 + e^{\beta_1} x_i + \varepsilon_i$ → nonlinear regression (parameter $\beta_1$ appears in exponent)

Exam tip: A model is linear if it can be written as a linear combination of the parameters. Transformations of $x$ are allowed; transformations of $\beta$ are not (unless the model can be re‑linearised).

Examples of SLR applications

• Monthly credit card spend vs. household income or household size
• Government policy impact on wages
• Restaurant waiting time vs. popularity rating
• Hospital treatment cost vs. patient weight
• Website visits vs. e‑commerce sales revenue
• Unemployment rate vs. bank non‑performing assets

Nine steps of building a regression model

flowchart TD
  A[1. Collect & extract data] --> B[2. Preprocess data]
  B --> C[3. Split into training (≈75%) & validation]
  C --> D[4. Compute descriptive statistics + visualise]
  D --> E[5. Define functional form]
  E --> F[6. Estimate parameters via OLS]
  F --> G[7. Perform model diagnostics]
  G --> H[8. Validate on validation set]
  H --> I[9. Deploy if performance is acceptable]

• Data preprocessing: handle outliers, missing values; derive new features (e.g., Galton’s 1.08 factor).
• Training/validation split: random sampling; typical 75% training, 25% validation.
• Ordinary Least Squares (OLS): finds $\beta_0, \beta_1$ that minimise $\sum (y_i - (\beta_0 + \beta_1 x_i))^2$. OLS gives the best linear unbiased estimates (BLUE).
• Diagnostics: check assumptions (linearity, normality of errors, homoscedasticity, independence).
• Overfitting: model performs well on training data but poorly on validation data. An ideal model has low bias and low variance.

Key takeaways

• SLR models a continuous $y$ using one $x$; parameters are linear in $\beta_0, \beta_1$.
• “Linear” refers to coefficients, not the shape of $x$.
• OLS minimises sum of squared residuals.
• Model building is iterative: collect → preprocess → split → describe → specify → estimate → diagnose → validate → deploy.
• Spurious correlation does not imply regression causation; always use domain logic and diagnostics.

The Regression Equation and Its Interpretation

The regression equation for a sample is written as

$y_i = \beta_0 + \beta_1 x_i + \varepsilon_i$

where

• $y_i$: observed value of the dependent variable for the $i$th observation,
• $x_i$: observed value of the independent variable for the $i$th observation,
• $\varepsilon_i$: random error (also called residual or unexplained variation),
• $\beta_0, \beta_1$: regression parameters (coefficients).

What the equation really means.
The population can be viewed as a collection of subpopulations — one for each distinct $x$ value. For example, among Hanumanthappa’s bank customers, all customers with an annual income of ₹20 lakh form one subpopulation, those with ₹30 lakh another, and so on. Each subpopulation has its own distribution of monthly spend $y$ and its own expected value $E(y)$. The regression equation describes how this expected value depends on $x$:

$E(y|x) = \beta_0 + \beta_1 x$

The error term disappears when taking the expectation because $E(\varepsilon_i)=0$.

Estimation Using Sample Data

Population parameters $\beta_0$ and $\beta_1$ are unknown. They are estimated from a sample (the training data) by sample statistics $b_0$ and $b_1$. Substituting these estimates gives the estimated regression equation:

$\hat{y} = b_0 + b_1 x$

where $\hat{y}$ is the point estimator of:

• the mean value of $y$ for a given $x$ (e.g., mean monthly spend for all customers with ₹20 lakh income), and
• the predicted value of $y$ for an individual observation at that $x$ (e.g., predicted spend for one particular customer with ₹20 lakh income).

Exam tip: The same formula $\hat{y}=b_0+b_1x$ serves both for estimating a conditional mean and for predicting an individual outcome. The difference matters when constructing confidence/prediction intervals (later).

Ordinary Least Squares (OLS) Estimation

The ordinary least squares (OLS) method finds $b_0$ and $b_1$ that minimize the sum of squared deviations between observed $y_i$ and predicted $\hat{y}_i$:

$\text{Minimize } \sum_{i=1}^n (y_i - \hat{y}_i)^2$

Using calculus, the solution is:

$b_1 = \frac{\sum_{i=1}^n (x_i - \bar{x})(y_i - \bar{y})}{\sum_{i=1}^n (x_i - \bar{x})^2}$

$b_0 = \bar{y} - b_1 \bar{x}$

• $\bar{x}$, $\bar{y}$ are sample means of $x$ and $y$.
• The slope $b_1$ estimates the expected change in $y$ for a one-unit increase in $x$.
• The intercept $b_0$ estimates $E(y)$ when $x=0$ (meaningful only if $x=0$ is within the data range).

Caution: Predictions using $x$ values outside the range of the training data are unreliable — the linear relationship may not hold beyond the observed data.

Worked Example: Hanumanthappa’s Bank Customers

Data: monthly spend ($y$) vs. annual income in lakhs ($x$) for 55 customers.

Statistic	Value
$\bar{x}$	32.89 lakhs
$\bar{y}$	₹3,852
$b_1$	85.27
$b_0$	1,047.37

Estimated regression equation:

$\hat{y} = 1,\!047.37 + 85.27\,x$

Interpretation of slope: An increase of ₹1 lakh in annual income is associated with an increase of ₹85.27 in monthly spend.

Prediction: For a customer with annual income ₹35 lakhs:

$\hat{y} = 1,\!047.37 + 85.27 \times 35 = 4,\!032$

Predicted monthly spend: ₹4,032.

Computing OLS in Excel

• Create a scatter plot (y vs. x).
• Right-click a data point → Add Trendline → Linear.
• Check “Display Equation on chart” and “Display R-squared value on chart”.

Excel outputs the same $b_0$, $b_1$, and also $R^2$.

Coefficient of Determination ($R^2$)

$R^2$ (or coefficient of determination) measures the goodness of fit of the estimated regression equation. It is the percentage of the total variation in $y$ that is explained by the linear relationship with $x$:

$R^2 = \frac{\text{variation explained by model}}{\text{total variation in } y}$

• Ranges from 0 to 1 (or 0% to 100%).
• Higher $R^2$ indicates a better fit.
• In simple linear regression, $R^2$ equals the square of the Pearson correlation $r_{xy}$.

Exam tip: $R^2$ tells you how much of y’s variability the model explains, not whether the model is correct. Always inspect residual plots before trusting $R^2$.

Key takeaways

• Regression models $E(y|x) = \beta_0 + \beta_1 x$; estimated by $\hat{y} = b_0 + b_1 x$.
• OLS minimizes $\sum (y_i - \hat{y}_i)^2$; formulas: $b_1 = \frac{\sum (x_i-\bar{x})(y_i-\bar{y})}{\sum (x_i-\bar{x})^2}$, $b_0 = \bar{y} - b_1\bar{x}$.
• Slope interpretation: $b_1$ = estimated change in $y$ per one‑unit increase in $x$.
• Do not extrapolate beyond the range of $x$ in the sample.
• $R^2$ = proportion of variation in $y$ explained by the model; closer to 1 → better linear fit.

Residuals and Sums of Squares

For the (i)-th observation, the residual is the difference between the observed value (y_i) and the predicted value (\hat{y}_i):

[ \text{residual}_i = y_i - \hat{y}_i ]

Residuals capture the error in using the regression line to predict the sample. The least‑squares method minimises the sum of squared residuals, called the sum of squares due to error (SSE):

[ \text{SSE} = \sum_{i=1}^{n} (y_i - \hat{y}_i)^2 ]

SSE measures the unexplained variation in (y) after fitting the model.

If no predictor (x) were available, the best guess for any (y_i) is the sample mean (\bar{y}). The total sum of squares (SST) measures the total variation in (y):

[ \text{SST} = \sum_{i=1}^{n} (y_i - \bar{y})^2 ]

The sum of squares due to regression (SSR) measures how much the regression predictions (\hat{y}_i) deviate from (\bar{y}):

[ \text{SSR} = \sum_{i=1}^{n} (\hat{y}_i - \bar{y})^2 ]

These three quantities are linked by a fundamental identity:

[ \text{SST} = \text{SSR} + \text{SSE} ]

Total variation = explained variation + unexplained variation.

Worked example (Hanumantha’s data, (n=55)): [ \text{SSE} = 40,017,408,\quad \text{SSR} = 74,180,516,\quad \text{SST} = 114,197,923 ] Check: (74,180,516 + 40,017,408 = 114,197,923).

Coefficient of Determination ( (R^2) )

The coefficient of determination (R^2) is the proportion of total variation explained by the regression:

[ R^2 = \frac{\text{SSR}}{\text{SST}} ]

• (R^2 = 1) when (\text{SSE}=0) (perfect fit).
• (R^2 = 0) when (\text{SSR}=0) (model explains nothing).
• (0 \le R^2 \le 1).

Interpreted as a percentage, (R^2) tells us the percentage of variability in (y) accounted for by the linear relationship with (x). For Hanumantha’s data:

[ R^2 = \frac{74,180,516}{114,197,923} \approx 0.650 \quad\Rightarrow\quad 65% ]

Thus, 65% of the variation in monthly spend is explained by annual income.

Exam tip: A high (R^2) indicates good fit in the sample, but does not guarantee the model is appropriate – assumptions must still be checked.

Key takeaways (decomposition & (R^2))

• (\text{SST} = \text{SSR} + \text{SSE}) – total variation splits into explained and unexplained parts.
• (R^2 = \text{SSR}/\text{SST}) – fraction of variation explained by the regression.
• Higher (R^2) implies a better linear fit, but (R^2) alone does not validate the model.

Correlation Coefficient vs. Coefficient of Determination

The sample correlation coefficient (r) measures the strength and direction of a linear relationship between (x) and (y) ((-1 \le r \le 1)). It relates to (R^2) in simple linear regression:

[ r = \sqrt{R^2} \times \text{sign}(b_1) ]

where (b_1) is the estimated slope from the regression.

• If (b_1 > 0), (r = +\sqrt{R^2}).
• If (b_1 < 0), (r = -\sqrt{R^2}).

In Hanumantha’s example, (b_1 = 85.27 > 0) and (R^2 = 0.65), so:

[ r = \sqrt{0.65} \approx 0.806 ]

This indicates a strong positive linear association.

Exam tip: Unlike (r), (R^2) can be used for nonlinear models and multiple regression – it is a more general measure of fit.

Assumptions of the Error Term

For valid inference in simple linear regression, four key assumptions are made about the error term (\varepsilon):

1. Zero mean: (E(\varepsilon) = 0) for all (x). Hence (E(y) = \beta_0 + \beta_1 x).
2. Constant variance: (\text{Var}(\varepsilon) = \sigma^2) for all (x) (homoscedasticity).
3. Independence: The errors are independent; no autocorrelation.
4. Normality: (\varepsilon) is normally distributed for every (x).

Together, these imply that for any given (x), (y) is normally distributed with mean (\beta_0 + \beta_1 x) and constant variance (\sigma^2).

Test for Significance of the Linear Relationship

We test whether the slope (\beta_1) is different from zero (i.e., whether a linear relationship exists).

[ H_0: \beta_1 = 0,\quad H_a: \beta_1 \neq 0 ]

Estimating (\sigma^2)

The mean square error (MSE) provides an unbiased estimate of (\sigma^2):

[ \text{MSE} = s^2 = \frac{\text{SSE}}{n-2} ]

With (n-2) degrees of freedom (two parameters estimated: (\beta_0, \beta_1)). The standard error of the estimate is (s = \sqrt{\text{MSE}}).

Hanumantha’s data: (n=55), (\text{SSE} = 40,017,408), so

[ \text{MSE} = \frac{40,017,408}{53} \approx 755,045,\quad s = \sqrt{755,045} \approx 868.9 ]

t‑Test

The standard error of the slope (b_1) is:

[ \text{se}(b_1) = \frac{s}{\sqrt{\sum (x_i - \bar{x})^2}} ]

For Hanumantha’s data, (\sum (x_i - \bar{x})^2 = 10,201), so

[ \text{se}(b_1) = \frac{868.9}{\sqrt{10,201}} \approx 8.603 ]

The test statistic follows a t distribution with (n-2) degrees of freedom:

[ t = \frac{b_1 - \beta_1}{\text{se}(b_1)} = \frac{85.27 - 0}{8.603} \approx 9.91 ]

The corresponding p‑value (two‑tailed) is extremely small ((\approx 1.1 \times 10^{-13})), providing strong evidence to reject (H_0). We conclude that a significant linear relationship exists.

Exam tip: For simple linear regression, the t‑test and an F‑test (using SSR/MSR vs MSE) are equivalent. In multiple regression they diverge.

Key takeaways (assumptions & significance test)

• Error term assumptions: zero mean, constant variance, independence, normality.
• (\text{MSE} = \text{SSE}/(n-2)) estimates (\sigma^2); (s = \sqrt{\text{MSE}}) is the standard error of the estimate.
• Test (H_0: \beta_1=0) using (t = b_1/\text{se}(b_1)) with (n-2) df.
• A very small p‑value (e.g., (<0.05)) indicates a statistically significant linear relationship.

SLR - Summary Statistics

The Excel regression output for Hanumantha's data provides a compact summary of all key statistics. The R square value is 0.65, meaning 65% of the variation in monthly spend is explained by annual income. The correlation between the two variables is the square root of R square: $r = \sqrt{0.65} = 0.806$, indicating a strong positive linear relationship.

The standard error $s = 868.9$ is the estimate of $\sigma$ (the standard deviation of the error term $\epsilon$). Its square $s^2$ estimates $\sigma^2$, the variance of $\epsilon$.

ANOVA Table

Source	SS	df	MS	F	p-value
Regression (SSR)	74,180,516	1	74,180,516	98.25	$\approx 0$
Residual (SSE)	40,017,408	53	755,045
Total	114,197,924	54

• Sum of squares regression SSR = 74,180,516.
• Sum of squares error SSE = 40,017,408.
• Mean square regression MSR = SSR / 1 = 74,180,516.
• Mean square error MSE = SSE / 53 = 755,045.
• F statistic = MSR / MSE = 98.25.
• The p-value (Significance F) is near zero, so we reject $H_0: \beta_1 = 0$ and conclude the linear relationship is statistically significant.

Coefficients and Estimated Regression Equation

Coefficient	Estimate	Standard Error	t Statistic	p-value
Intercept ($b_0$)	10,470.37	—	—	—
Annual Income ($b_1$)	85.27	8.6	9.91	$\approx 0$

Estimated regression equation:
$\hat{y} = 10\,470.37 + 85.27\,x$

• $b_1$ standard error: $s_{b_1} = \frac{s}{\sqrt{\sum (x_i - \bar{x})^2}} = \frac{868.9}{\sqrt{10\,201}} = 8.603 \approx 8.6$.
• t test for $\beta_1 = 0$: $t = \frac{b_1}{s_{b_1}} = \frac{85.27}{8.6} = 9.91$, with p-value $\approx 0$ → reject $H_0$.

Exam tip: Excel’s regression tool outputs all these values at once. The key interpretation steps: check R square, then overall F test, then individual coefficient t tests. The F and t tests yield the same conclusion for simple linear regression (since $t^2 = F$).

Key takeaways

• R square = 0.65, correlation = 0.806.
• Standard error $s = 868.9$ estimates $\sigma$.
• F test ($F=98.25$, p ≈ 0) confirms a significant linear relationship.
• Estimated equation: $\hat{y} = 10\,470.37 + 85.27\,x$.
• Slope’s t test ($t=9.91$, p ≈ 0) matches F test result.

---

SLR - Residual Analysis

Residual for observation $i$: $e_i = y_i - \hat{y}_i$ (observed minus predicted). Residuals are the best empirical proxy for the error term $\epsilon$. Three model assumptions about $\epsilon$ must be checked:

1. Expected value $E(\epsilon) = 0$ (automatically satisfied by least squares).
2. Constant variance — homoscedasticity — $\text{Var}(\epsilon) = \sigma^2$ for all $x$.
3. Independence of errors.
4. Normality — $\epsilon \sim N(0,\sigma^2)$.

If assumptions fail, hypothesis tests (t and F) may be invalid.

Residual Plots

Three diagnostic plots are commonly examined:

1. Residual vs. independent variable $x$ (provided by Excel).
2. Residual vs. predicted values $\hat{y}$ (can be created from output).
3. Standardized residual plot (also provided by Excel).

Standardized residual: $e_i^* = \frac{e_i}{s_{e_i}}$ (mean = 0, standard deviation ≈ 1). For normally distributed errors, about 95% of standardized residuals lie between −2 and +2.

Interpreting Residual Plot Patterns

flowchart TD
    A[Residual plot shape] --> B{Pattern?}
    B -->|Horizontal band| C[Assumptions hold: constant variance, linear model adequate]
    B -->|Funnel (spread increases with x)| D[Violation: heteroscedasticity]
    B -->|U-shaped (curvilinear)| E[Model inadequate: consider polynomial or multiple regression]

• Panel A (horizontal band): no obvious violation → model assumptions appear satisfied.
• Panel B (funnel): variance increases with $x$ → homoscedasticity assumption violated.
• Panel C (U-shape): systematic curvature → linear model not appropriate.

For Hanumantha’s data, the residual plot shows a horizontal band pattern (like Panel A), supporting the assumptions. The same pattern appears in the residual vs. $\hat{y}$ plot (for simple linear regression, both plots give identical information; for multiple regression, the $\hat{y}$ plot is preferred).

Checking Normality with Standardized Residuals

Excel provides standardized residuals. In Hanumantha’s dataset, nearly all standardized residuals lie within ±2, consistent with a standard normal distribution. Thus the normality assumption is not violated.

Exam tip: Residual analysis is qualitative. Look for clear deviations (funnel, U-shape, many points outside ±2). A few points near the boundaries do not automatically invalidate the model.

Key takeaways

• Residuals estimate $\epsilon$ and are used to validate model assumptions.
• Three key plots: residuals vs. $x$, residuals vs. $\hat{y}$, standardized residuals.
• Desired pattern: horizontal band (constant variance, linearity).
• About 95% of standardized residuals should fall in [−2, +2] for normality.
• Hanumantha’s data satisfies all assumptions (band pattern, no extreme standardized residuals).

---

Outliers

An outlier is a data point that does not follow the trend of the rest of the data. In simple linear regression, outliers can be detected via:

• Scatter plot of $y$ vs. $x$.
• Standardized residuals — any observation with $|e_i^*| > 2$ (roughly) is a potential outlier.

Handling Outliers

Situation	Action
Erroneous data (entry/recording error)	Correct the value and re-run regression; R square usually improves.
Violation of model assumptions	Consider a different model (e.g., curvilinear, transformation).
Genuine unusual value	Retain the observation — it may be a chance occurrence.

Influential Observations

An influential observation dramatically changes the estimated regression equation (slope, intercept, or both) when removed. Detection methods:

• Scatter diagram — look for points that are:
  • An outlier in $y$ (far from the trend line),
  • An extreme $x$ value (far from $\bar{x}$),
  • A combination of both.

Handling Influential Observations

1. Check for errors in data collection/recording — correct if possible.
2. If valid, treat the observation as providing additional insight. Consider collecting more data at intermediate $x$ values to better understand the relationship.

Exam tip: Outliers and influential points are not automatically bad. Always investigate the reason before discarding. In small datasets, a single influential point can dominate the regression — report it transparently.

Key takeaways

• Outliers: $|$standardized residual$| > 2$; may be erroneous, model violation, or genuine.
• Influential observation: extreme $x$, outlying $y$, or both; can shift the regression line substantially.
• First step for both: check data accuracy.
• Valid influential observations may motivate collecting additional data at intermediate $x$ values.

Test of Significance and Residual Analysis

A fitted regression line is not enough — we must ask whether the observed linear relationship is statistically significant or merely a fluke of sampling. Two hypothesis tests (t-test and F-test) answer this question. Both rely on assumptions about the error term $\epsilon$, which are checked through residual analysis.

The Regression Model and Estimation (Review)

The simple linear regression model is
$y = \beta_0 + \beta_1 x + \epsilon$

Using ordinary least squares (OLS), we obtain estimates $b_0$ and $b_1$, giving the estimated regression equation:

$\hat{y} = b_0 + b_1 x$

OLS minimises the sum of squares error (SSE):
$\text{SSE} = \sum (y_i - \hat{y}_i)^2$

Variation in $y$ is decomposed as

$\text{SST} = \text{SSR} + \text{SSE}$

where

• $\text{SST} = \sum (y_i - \bar{y})^2$ (total sum of squares)
• $\text{SSR} = \sum (\hat{y}_i - \bar{y})^2$ (regression sum of squares)

The coefficient of determination $R^2$ measures how well the model explains the data:

$R^2 = \frac{\text{SSR}}{\text{SST}} \quad (0 \le R^2 \le 1)$

The correlation between $y$ and $x$ is $\sqrt{R^2}$ with the sign of $b_1$.

Assumptions for Hypothesis Testing

The error term $\epsilon$ is assumed to be:

• Normally distributed (for exact p‑values)
• Mean zero, $E(\epsilon)=0$
• Constant variance $\sigma^2$ (homoscedasticity)
• Independent across observations

These assumptions are critical for the validity of the tests. The variance $\sigma^2$ is estimated by the mean square error (MSE):

$s^2 = \text{MSE} = \frac{\text{SSE}}{n-2}$

The standard error of $b_1$ is $\sqrt{\text{MSE} / \sum (x_i - \bar{x})^2}$.

t‑Test for $\beta_1$

• Null hypothesis $H_0$: $\beta_1 = 0$ (no linear relationship)
• Alternative $H_a$: $\beta_1 \neq 0$

Test statistic:

$t = \frac{b_1}{\text{se}(b_1)}$

Under $H_0$, $t$ follows a $t$-distribution with $n-2$ degrees of freedom. The p-value is computed; if small (e.g., $< 0.05$), reject $H_0$ and conclude the linear relationship is statistically significant.

F‑Test for Overall Significance

For simple linear regression, the F‑test tests the same null hypothesis ($\beta_1=0$).

$\text{MSR} = \frac{\text{SSR}}{1} \quad,\quad \text{MSE} = \frac{\text{SSE}}{n-2}$

Test statistic:

$F = \frac{\text{MSR}}{\text{MSE}}$

If $H_0$ is true, $F$ is approximately 1. If $\beta_1\neq 0$, $F > 1$. The $F$-statistic has 1 and $n-2$ degrees of freedom; a small p‑value leads to rejecting $H_0$.

Exam tip: In simple linear regression, the t‑test and F‑test are equivalent — they give the same p‑value. In multiple linear regression, the F‑test tests the overall model while t‑tests test individual coefficients.

Residual Analysis

Residuals $e_i = y_i - \hat{y}_i$ are used to check the assumptions on $\epsilon$.

• Standardised residuals $= e_i / s \cdot \sqrt{1 - h_{ii}}$ (or simpler $e_i / s$). Values outside $\pm 2$ suggest possible outliers.
• Plot residuals vs. $x$ — should be randomly scattered around zero with constant spread (no funnel shape, no curvilinear pattern).
• Plot residuals vs. $\hat{y}$ (or vs. $y$) — same expectation. A clear trend indicates that the model is missing relevant variables or that a linear fit is inadequate.

Key takeaways

• Both t‑test and F‑test test $H_0\!:\beta_1=0$; reject → linear relationship is significant.
• $\text{MSE}$ estimates error variance $\sigma^2$; $s = \sqrt{\text{MSE}}$ is the standard error of the estimate.
• Residual plots must be examined for violations of constant variance, independence, and linearity.
• Standardised residuals beyond ±2 may indicate outliers.

---

Application: Monthly Spend vs. Household Size

Using the same dataset (Hanumantha’s), we now model monthly spend ($y$) as a function of household size ($x$).

Estimated Regression Equation

From Excel’s regression output:

$\hat{y} = 2083.7 + 520.1 \times x$

Model Fit

Measure	Value
$R^2$	0.396
Correlation $r_{xy}$	$\sqrt{0.396} = 0.63$
Standard error $s$	1140.71

Only 39.6% of the variation in monthly spend is explained by household size — much weaker than the model using annual income ($R^2=0.65$).

ANOVA and F‑Test

Source	SS	df	MS	$F$	p‑value
Regression (SSR)	45,233,346	1	45,233,346	34.7	≈ 0
Error (SSE)	68,964,576	51	1,301,218
Total (SST)	114,197,922	52

$F = 34.7$ with a very small p‑value → reject $H_0$ → linear relationship between monthly spend and household size is statistically significant.

Coefficients and t‑Test

Coefficient	Value	Std. Error	$t$	p‑value
Intercept $b_0$	2083.7
Household size $b_1$	520.1	88.2	5.89	≈ 0

The t‑test confirms significance (same conclusion as F‑test).

Residual Diagnostics

• Standardised residuals: three values exceed +2 in absolute value → potential outliers.
• Residuals vs. $x$: no funnel or curvilinear shape; band is acceptable.
• Residuals vs. $y$: shows a slight trend (not a perfect random band). This suggests that other factors (e.g., income) also influence monthly spend — a multiple regression model may improve fit.

Exam tip: Always check residual plots, not just $R^2$ and p‑values. A trend in residuals vs. fitted values is a red flag that important predictors are missing.

Key takeaways

• $R^2 = 0.396$ means household size alone explains only 39.6% of spend variation.
• Despite low $R^2$, the linear relationship is statistically significant (p ≈ 0 for both t and F).
• Three observations are flagged as possible outliers (standardised residual > 2).
• Residual pattern suggests the model is incomplete; include other variables like income in multiple regression.

Simple Linear Regression — Applications: Examples II

Two worked examples demonstrate the full SLR workflow: check scatter plot, fit model, interpret R², evaluate significance via F-test and t-test, examine residuals for violations and missing predictors.

---

Example 1: Buzzer Rogers Customer Satisfaction

Context — 36 survey responses. Question: Is price a significant determinant of satisfaction score?

• Dependent variable $y$ = satisfaction score (0–100)
• Independent variable $x$ = price paid (₹ 1000s)

Step 1 — Scatter plot

Price on x‑axis, score on y‑axis. A linear trend is visible, though with spread.

Step 2 — Regression equation & R²

From the trend line (and later from the tool pack):

$$\hat{y} = 48.44 + 0.132\,x$$

• R² = 0.36 — only 36 % of score variation is explained by price. The plot showed a fair trend, so the low R² hints that other factors (brand, RAM, etc.) matter.
• Correlation $r = \sqrt{0.36} = 0.59$ (positive, moderate).

Step 3 — ANOVA and overall significance

Source	SS	df	MS	$F$	$p$
Regression (SSR)	510.8	1	510.8	= MSR/MSE = 18.9	≈ 0
Error (SSE)	914.8	34	26.9
Total	1425.6	35

• $F$ = 18.9, $p \approx 0$ $\Rightarrow$ reject $H_0:\beta_1=0$. The linear relationship is statistically significant.

Step 4 — Coefficient table

Predictor	Coefficient	Std. Error	$t$	$p$
Intercept ($b_0$)	48.44	—	—	—
Price ($b_1$)	0.132	0.03	4.36	≈ 0

• $t = 4.36$ matches $F = t^2$ ($4.36^2 \approx 19.0$; rounding). $p$ small → reject $H_0$.
• Interpretation: For every ₹ 1000 increase in price, the predicted satisfaction score rises by 0.132 points.

⚠️ Exam tip: This slope is counter‑intuitive (higher price → higher satisfaction). Regression describes the sample; the data may reflect that higher‑priced computers satisfy certain customers. Always check real‑world plausibility separately.

Step 5 — Residual diagnostics

• Standardized residuals: Two values below –2 (–2.48, –2.12) — possible outliers (points far below the regression line).
• Residuals vs. x: No funnel or curve, but not a perfect band. Some pattern suggests omitted variables (e.g., brand, features).
• Residuals vs. y: Shows a trend — again hints that more predictors are needed.

Key takeaways — Example 1

• R² = 0.36: price alone explains little of satisfaction.
• Relationship is significant despite low R² ($F$ test rejects $\beta_1=0$).
• Slope is positive — data says higher price was associated with higher satisfaction in this sample.
• Residual analysis flags outliers and missing predictors → multiple regression needed.

---

Example 2: Manjula Nayak — Household Debt (99 Acres)

Context — ~500 Bengaluru households. Question: Is first income a significant determinant of household debt?

• Dependent variable $y$ = household debt (₹)
• Independent variable $x$ = first income (₹)

Step 1 — Scatter plot

First income vs. debt. A linear trend is barely visible; large variation in debt for a given income.

Step 2 — Regression equation & R²

$$\hat{y} = 32\,976 + 0.04\,x$$

• R² = 0.31 — income only explains 31 % of debt variation.
• Correlation $r = \sqrt{0.31} \approx 0.56$ (positive, moderate).

Step 3 — ANOVA and overall significance

Source	SS	df	MS	$F$	$p$
Regression (SSR)	$1.86 \times 10^{11}$	1	$1.86 \times 10^{11}$	= 228	≈ 0
Error (SSE)	$4.08 \times 10^{11}$	498	$8.19 \times 10^8$
Total	$5.94 \times 10^{11}$	499

• $F$ = 228, $p \approx 0$ → reject $H_0$. Linear relationship is significant.

Step 4 — Coefficient table

Predictor	Coefficient	Std. Error	$t$	$p$
Intercept ($b_0$)	32 976	—	—	—
First income ($b_1$)	0.04	0.002	15.1	≈ 0

• $t = 15.1$ (very large). $p$ tiny → strong evidence against $\beta_1=0$.
• Interpretation: For every ₹ 1 increase in first income, debt rises by ₹ 0.04.

Step 5 — Residual diagnostics

• Standardized residuals: Few values outside ±2, but mostly within range.
• Residuals vs. x: Oval shape — not a perfect band, no funnel or curve.
• Residuals vs. y: Shows a trend, again hinting that other variables (ownership, monthly payments, etc.) affect debt.

Key takeaways — Example 2

• R² = 0.31: income alone explains little of debt.
• Relationship is highly significant ($F=228$, $t=15.1$).
• Slope 0.04 means a 1‑rupee increase in income is associated with 0.04 rupee more debt.
• Residual patterns signal that a multiple regression (next module) will better capture debt drivers.

---

Common workflow across both examples

flowchart LR
    A[Scatter plot: linear trend?] --> B[Fit OLS: get equation & R²]
    B --> C{Check overall F-test}
    C -->|p < α| D[Significant relationship]
    C -->|p ≥ α| E[Model not useful]
    D --> F[Inspect individual t-tests for coefficients]
    D --> G[Residual diagnostics: outliers, patterns]
    G --> H[If patterns remain → consider multiple regression]

Exam tip: When R² is low but the overall $F$‑test is significant, the model is still statistically useful but practically weak — other variables are needed. Residual plots that show trends (not random scatter) are the main clue.

SLR Applications: Monthly Payment vs. Debt

Intuition: Households with higher monthly payments (loans, credit cards, utilities) are expected to carry more debt. This second example uses monthly payment as the independent variable $x$ to predict debt $y$, comparing results with the earlier income-based model.

Estimated Regression Equation

Using the same 500-household dataset, the least‑squares regression yields:

$\hat{y} = b_0 + b_1 x = 28\,791 + 2.94\,x$

• $b_1 = 2.94$: for every one‑rupee increase in monthly payment, predicted debt increases by 2.94 rupees.
• Caution: With only one predictor, this coefficient captures both the direct effect and correlations with omitted variables; it is not a pure marginal effect.

Goodness‑of‑Fit: $R^2$ and Correlation

Measure	Value	Interpretation
$R^2$	0.37	Monthly payment explains 37% of the variability in debt.
$r = \sqrt{R^2}$	0.60	Positive correlation between monthly payment and debt.

Comparison with earlier model (Debt vs. First Income):

Model	$R^2$	$r$
Debt vs. Income	0.31	0.56
Debt vs. Monthly Payment	0.37	0.60

The monthly‑payment model is marginally better, but both leave >60% of variance unexplained – strong evidence that debt depends on multiple factors.

ANOVA and Hypothesis Tests

Null hypothesis: $H_0: \beta_1 = 0$ (no linear relationship).
Alternative: $H_1: \beta_1 \neq 0$.

Source	Sum of Squares	df	Mean Square	$F$	$p$-value
Regression (SSR)	$2.18 \times 10^{11}$	1	$2.18 \times 10^{11}$	288	$\approx 0$
Error (SSE)	$3.80 \times 10^{11}$	498	$757\,268\,785$
Total (SST)		499

• Standard error of estimate: $\sqrt{\text{MSE}} = \sqrt{757\,268\,785} \approx 27\,518$ rupees – typical prediction error.
• $F = 288$, $p \approx 0$ → reject $H_0$; the linear relationship is statistically significant.

Coefficient details for $b_1$:

	Coefficient	Std. Error	$t$ statistic	$p$-value
Monthly payment ($b_1$)	2.94	0.174	$t = 2.94 / 0.174 = 16.95$	$\approx 0$

• The $t$‑test confirms $\beta_1 \neq 0$, identical conclusion to the $F$‑test.

Residual Analysis

Standardized residuals: Several observations lie below −2 or above +2, suggesting potential outliers.

Residual vs. $x$ (monthly payment): Forms an oval shape – not a perfect random band, but no obvious funnel or curvature. This points to heteroscedasticity or non‑linearity that may be addressed by including additional predictors.

Residual vs. $\hat{y}$: Similar pattern reinforces that the model fails to capture all systematic variation.

Exam tip: An oval‑shaped residual plot (instead of a horizontal band) is a classic sign that a simple linear regression is misspecified – often resolved by moving to multiple regression.

Key takeaways

• Monthly payment explains 37% of debt variability ($R^2 = 0.37$, $r = 0.60$), slightly better than income (31%).
• Both slope and overall model are highly significant ($F=288$, $t=16.95$, $p \approx 0$).
• Residual analysis reveals several outliers and a non‑random pattern, indicating omitted variables.
• Coefficient $b_1 = 2.94$ cannot be interpreted as a pure marginal effect in simple regression.

---

Module Summary

This module introduced the fundamentals of measuring and modelling linear relationships between two variables.

Key concepts covered:

1. Covariance and correlation – quantify the direction and strength of linear association. $\text{Cov}(X,Y) = \frac{\sum (x_i - \bar{x})(y_i - \bar{y})}{n-1}, \quad r = \frac{\text{Cov}}{s_x s_y}$

   • Distinguish explainable vs. spurious correlation.

2. Simple linear regression model $Y = \beta_0 + \beta_1 X + \epsilon$ with $\mathbb{E}(Y) = \beta_0 + \beta_1 X$.

3. Least‑squares estimation – sample statistics $b_0$ and $b_1$ estimate population parameters.

4. Coefficient of determination $R^2$ – proportion of $Y$’s variability explained by $X$.

5. Assumptions about errors $\epsilon$ – independence, constant variance, normality – required for valid inference.

6. Hypothesis tests – $t$‑test for $\beta_1$ and $F$‑test for overall significance (equivalent in SLR).

7. Residual analysis – plots and standardized residuals used to check assumptions and detect outliers/influential points.

The next module extends this framework to multiple linear regression, where $Y$ depends on several $X$ variables.

Key takeaways

• Covariance and correlation measure association; regression models the relationship.
• $R^2$ is the key measure of fit; low $R^2$ indicates omitted variables.
• Hypothesis tests ($t$, $F$) assess statistical significance of the linear relationship.
• Residual plots are essential for model validation.

Key Concepts: Continuous vs. Discrete

A continuous random variable can take any value within an interval (finite or infinite), including fractional values. Examples: time between customer arrivals, fluid volume in a bottle, oven temperature. Unlike discrete variables (e.g., number of defects, number of customers), continuous variables have a probability density function (pdf) $f(x)$, not a probability mass function.

Feature	Discrete	Continuous
Possible values	Countable (integers)	Uncountable (any real in an interval)
Probability at a point	$P(X=x)=f(x)$ (the mass)	$P(X=x)=0$ (area under a point is zero)
Probability for an interval	Sum of $f(x)$ over values in $[a,b]$	Integral of $f(x)\,dx$ from $a$ to $b$
Total probability	$\sum f(x)=1$	$\int f(x)\,dx=1$

Probability Density Function (PDF) and Cumulative Distribution Function (CDF)

For a continuous random variable $X$:

• pdf: $f(x) \ge 0$ for all $x$, and $\int_{-\infty}^{\infty} f(x)\,dx = 1$.
• CDF: $F(x) = P(X \le x) = \int_{-\infty}^{x} f(t)\,dt$.
• Tail probability: $\bar{F}(x) = P(X > x) = 1 - F(x)$.

The probability that $X$ lies in $[a,b]$ is the area under $f(x)$ between $a$ and $b$:

$P(a \le X \le b) = \int_{a}^{b} f(x)\,dx = F(b) - F(a).$

Exam tip: For a continuous random variable, $P(X = \text{any single value}) = 0$. Always think in terms of intervals.

Expectation and Variance

• Expectation (mean): $E[X] = \int_{-\infty}^{\infty} x\,f(x)\,dx$ (replace sum with integral).
• Variance: $\text{Var}(X) = \int_{-\infty}^{\infty} (x - E[X])^2\,f(x)\,dx$.

Formulas mirror the discrete case, substituting summation with integration.

---

Uniform Distribution

A random variable is uniformly distributed over $[L, U]$ if every value in that interval is equally likely. The pdf is constant:

$f(x) = \frac{1}{U - L}, \quad L \le x \le U, \quad \text{and } f(x)=0 \text{ elsewhere.}$

Key formulas

Quantity	Expression
CDF $F(x)$	$\displaystyle \frac{x - L}{U - L}$
Tail probability $\bar{F}(x)$	$\displaystyle \frac{U - x}{U - L}$
Probability in $[a,b]$	$\displaystyle \frac{b - a}{U - L}$
Expectation $E[X]$	$\displaystyle \frac{L + U}{2}$
Variance $\text{Var}(X)$	$\displaystyle \frac{(U - L)^2}{12}$

Worked Example: Driving Time

Nilesh Shah’s driving time from Vadodara to Ahmedabad is uniformly distributed between $L=100$ minutes and $U=140$ minutes.

• Expected time: $E[X] = \frac{100+140}{2} = 120$ minutes.
• $P(X \le 130)$: $F(130) = \frac{130-100}{140-100} = \frac{30}{40} = 0.75$ (75% chance of 130 minutes or less).
• $P(X > 105)$: $\bar{F}(105) = \frac{140-105}{40} = \frac{35}{40} = 0.875$ (87.5% chance of more than 105 minutes).
• $P(X = 120)$: $=0$ (any single point has zero probability for a continuous variable).

Note: Probabilities for intervals (e.g., $P(105 < X < 130) = F(130)-F(105) = 0.75 - 0.125 = 0.625$) can be computed similarly.

---

Key takeaways

• Continuous random variables have a pdf; probabilities are areas under the curve.
• The CDF gives $P(X \le x)$; tail probability is $1-\text{CDF}$.
• For continuous variables, $P(X=a)=0$ for any specific $a$.
• Expectation and variance use integrals instead of sums.
• Uniform distribution: constant pdf over $[L,U]$; $E[X]=\frac{L+U}{2}$, $\text{Var}(X)=\frac{(U-L)^2}{12}$.
• In a uniform model, probability is proportional to interval length.

Uniform Distribution Example

The uniform distribution models a random variable that is equally likely to take any value within a given interval $[L, U]$. Intuitively: every value in that range has the same chance—there is no “peaked” area.

Worked Example: Battery Life of a Phone

Meena’s phone battery lasts between 8 and 12 hours before needing a charge. Let $X$ = time (in hours) until recharge. Model: $X \sim \text{Uniform}(L=8, U=12)$.

PDF (not needed for calculations here):

$$f(x) = \frac{1}{U-L} = \frac{1}{4}, \quad 8 \le x \le 12$$

CDF for any $x$ in $[L,U]$:

$$F(x) = P(X \le x) = \frac{x - L}{U - L}$$

---

1. Probability that recharge needed within 9 hours ($P(X \le 9)$)

$$F(9) = \frac{9-8}{12-8} = \frac{1}{4} = 0.25$$

25% chance the phone needs charging in the first 9 hours.

2. Probability that battery lasts at least 11 hours ($P(X \ge 11)$)

$$P(X \ge 11) = 1 - F(11) = \frac{12-11}{12-8} = 0.25$$

25% chance the battery lasts 11 hours or more.

3. 80th percentile – the value $x$ such that $P(X \le x) = 0.8$.

$$\frac{x - 8}{12 - 8} = 0.8 \quad \Rightarrow \quad x = 8 + (0.8)(4) = 11.2 \text{ hours}$$

There is an 80% chance the phone will need charging within 11.2 hours.

---

Exam tip: For uniform distributions, percentiles are a straight linear interpolation: $x = L + p \cdot (U-L)$.

Discrete Uniform – Reminder

The discrete uniform distribution assigns equal probability to a finite set of values (e.g., a fair die: $P(X=k)=1/6$ for $k=1,\dots,6$). The continuous version is its natural extension to an interval.

Key takeaways (Uniform)

• $X \sim \text{Uniform}(L,U)$: all values equally likely in $[L,U]$.
• CDF: $F(x) = \frac{x-L}{U-L}$ for $L \le x \le U$.
• Percentile $p$: $x = L + p \cdot (U-L)$.
• Used when only the minimum and maximum are known and nothing else.

---

Exponential Distribution

The exponential distribution models the time between random events that occur at a constant average rate $\lambda$ (events per unit time). Intuitively: if events happen “on average every 1/λ time units”, the waiting time until the next event follows an exponential distribution.

Formal Definition

Let $X$ = time between events. $X \ge 0$.

Probability density function (PDF):

$$f(x) = \lambda e^{-\lambda x}, \quad x \ge 0$$

Cumulative distribution function (CDF):

$$F(x) = P(X \le x) = 1 - e^{-\lambda x}$$

Tail probability (survival function):

$$\bar{F}(x) = P(X > x) = e^{-\lambda x}$$

Probability between $a$ and $b$ ($0 \le a < b$):

$$P(a < X \le b) = \bar{F}(a) - \bar{F}(b) = e^{-\lambda a} - e^{-\lambda b}$$

Mean and Variance

Property	Formula	Note
Mean $E[X]$	$\frac{1}{\lambda}$	Average waiting time
Variance $\text{Var}(X)$	$\frac{1}{\lambda^2}$
Standard deviation	$\frac{1}{\lambda}$	Equal to the mean – a unique property

The mean equals the standard deviation for the exponential distribution. This is a quick check: if the sample mean and sample SD are very different, the data may not be exponential.

Key Properties

• Skewed right – the PDF peaks at $x=0$ and decays; most probability lies near zero.
• Memoryless property – $P(X > s+t \mid X > s) = P(X > t)$. The future waiting time does not depend on how long you have already waited.
• Connection to Poisson: If events occur at rate $\lambda$ per unit time (Poisson process), the waiting time between consecutive events is exponential with mean $1/\lambda$, and the number of events in a fixed interval is Poisson with mean $\lambda t$.

Applications

• Time to complete a manual task (assembly, phone call)
• Lifetime of electronic components (bulbs, circuits)
• Time until equipment requires repair
• Waiting time in a queue (call center, customer service)

Worked Example: Call Center Waiting Time

Megha’s waiting time (in minutes) for a service desk agent is exponential with mean 2 minutes. So $\lambda = 1/2 = 0.5$ per minute.

1. Probability she waits less than 1 minute ($P(X < 1)$)

$$F(1) = 1 - e^{-0.5 \times 1} = 1 - e^{-0.5} \approx 0.393$$

2. Probability she waits more than 3 minutes ($P(X > 3)$)

$$\bar{F}(3) = e^{-0.5 \times 3} = e^{-1.5} \approx 0.223$$

3. Probability she waits between 1 and 3 minutes ($P(1 < X \le 3)$)

$$e^{-0.5 \times 1} - e^{-0.5 \times 3} = e^{-0.5} - e^{-1.5} \approx 0.6065 - 0.2231 = 0.383$$

4. 90th percentile – find $x$ such that $F(x) = 0.9$:

$$1 - e^{-0.5 x} = 0.9 \quad \Rightarrow \quad e^{-0.5 x} = 0.1 \quad \Rightarrow \quad -0.5 x = \ln(0.1) \quad \Rightarrow \quad x = -2 \ln(0.1) \approx 4.605 \text{ minutes}$$

90% of calls are answered within about 4.6 minutes.

Computation in Excel

Use the function =EXPON.DIST(x, λ, cumulative):

• cumulative = FALSE returns the PDF $f(x)$.
• cumulative = TRUE returns the CDF $F(x)$.

For inverse calculations (given probability, find $x$), Excel provides =EXPON.INV(probability, λ) or use the formula $x = -\frac{\ln(1-p)}{\lambda}$ for the $p$-th percentile.

flowchart LR
    A[Given: λ or mean] --> B{Calculate?}
    B --> C[Probability: P(X ≤ x) or P(X > x) or P(a<X≤b)]
    B --> D[Percentile: find x given p]
    C --> E[Use F(x)=1-e^{-λx} or tail e^{-λx}]
    D --> F[Solve x = -ln(1-p)/λ]

Key takeaways (Exponential)

• Models waiting time between random events at constant rate λ.
• PDF: $f(x) = \lambda e^{-\lambda x}$, CDF: $F(x) = 1 - e^{-\lambda x}$.
• Mean = Standard deviation = $1/\lambda$.
• Memoryless: $P(X > s+t \mid X > s) = P(X > t)$.
• Connected to Poisson: number of events per interval ~ Poisson(λt), waiting time ~ Exponential(λ).
• Use EXPON.DIST in Excel for probabilities; EXPON.INV for percentiles.

Memoryless Property of the Exponential Distribution

The memoryless property is a defining characteristic of the exponential distribution. Intuitively: if you are waiting for an event that follows an exponential distribution, the probability of waiting an additional $y$ units does not depend on how long you have already waited — the process "forgets" the elapsed time.

Formal definition

Let $X \sim \text{Exponential}(\lambda)$, with mean $1/\lambda$. For any $t_0 > 0$ and $y > 0$,

$$P(X > t_0 + y \mid X > t_0) = P(X > y).$$

Equivalently, the remaining time $Y = X - t_0$ conditional on $X > t_0$ has the same exponential distribution as $X$:

$$P(Y > y) = e^{-\lambda y}.$$

Derivation

The tail probability of an exponential is $P(X > x) = e^{-\lambda x}$.

$$\begin{aligned} P(Y > y) &= P(X > t_0 + y \mid X > t_0) \\ &= \frac{P(X > t_0 + y)}{P(X > t_0)} \\ &= \frac{e^{-\lambda (t_0 + y)}}{e^{-\lambda t_0}} \\ &= e^{-\lambda y}. \end{aligned}$$

Thus $Y$ has the same distribution as $X$.

Exam tip: The memoryless property is unique to the exponential distribution (and its discrete analogue, the geometric). It is the reason exponential waiting times are used in queueing models: the future waiting time is independent of the past.

Worked example — Shanti's bus wait

Shanti waits for a bus. Waiting time $X$ (minutes) is exponential with mean $20$ minutes.

$$\lambda = \frac{1}{20} = 0.05 ,\quad \text{tails: } P(X > x) = e^{-0.05 x}.$$

Question	Computation	Result
Probability she waits $\le 15$ min	$P(X \le 15) = 1 - e^{-0.05 \cdot 15}$	$0.528$
Probability she waits $> 30$ min	$P(X > 30) = e^{-0.05 \cdot 30}$	$0.223$
Probability she waits between $15$ and $30$ min	$P(15 < X \le 30) = e^{-0.05\cdot 15} - e^{-0.05 \cdot 30}$	$0.249$
Given she already waited $10$ min, probability she waits another $\ge 10$ min	$P(X > 20 \mid X > 10) = e^{-0.05 \cdot 10}$	$0.607$
Given she already waited $20$ min, probability she waits another $\ge 10$ min	$P(X > 30 \mid X > 20) = e^{-0.05 \cdot 10}$	$0.607$

The last two results are identical — the waiting time resets regardless of elapsed time, illustrating the memoryless property.

---

Key takeaways

• Memoryless property: $P(X > t_0 + y \mid X > t_0) = P(X > y)$.
• Equivalent to the remaining time having the same exponential distribution.
• Only the exponential (continuous) and geometric (discrete) have this property.
• Conditional waiting times are computed using $e^{-\lambda y}$ — no dependence on $t_0$.
• Always check that the random variable is exponential before applying memorylessness.

---

Relation Between Exponential and Poisson Distributions

The exponential distribution is the waiting time distribution that underlies a Poisson process. If events occur according to a Poisson process with rate $\lambda$ (average number of events per unit time or space), then:

• The number of events in a fixed interval of length $t$ is $\text{Poisson}(\lambda t)$.
• The time between consecutive events (interarrival times) is $\text{Exponential}(\lambda)$.

This duality allows us to switch between counting problems and waiting‑time problems.

Process	Interarrival times	Counting (number of events in $t$)
Deterministic	Fixed, constant	Fixed number $\lambda t$
Poisson	Exponential($\lambda$)	Poisson($\lambda t$)

Example — highway defects

Recall the Solapur highway example: defects occur as a Poisson process with average $3$ defects per kilometer.

$$\lambda = 3 \text{ defects/km}.$$

• Distance between defects $X$ is $\text{Exponential}(\lambda = 3)$ with mean $1/3 \approx 0.333$ km.
• Number of defects in $L$ km is $\text{Poisson}(3L)$.

Question	Computation	Result
Probability of at least $0.5$ km without a defect	$P(X > 0.5) = e^{-3 \cdot 0.5} = e^{-1.5}$	$0.223$
Probability of at least $2$ km without a defect	$P(X > 2) = e^{-3 \cdot 2} = e^{-6}$	$0.002$

The same result can be obtained from the Poisson: $P(\text{zero defects in }L\text{ km}) = e^{-\lambda L}$.

Exam tip: Whenever you see a problem about “time between events” or “distance between occurrences”, check whether the process is Poisson. If yes, the interarrival time is exponential — use the exponential tail formula $e^{-\lambda x}$ directly.

Key takeaways

• In a Poisson process, interarrival times are i.i.d. exponential with the same rate $\lambda$.
• Average interarrival time = $1/\lambda$; average number of events per unit time = $\lambda$.
• Switching between Poisson and exponential: $P(\text{zero events in }t) = e^{-\lambda t} = P(\text{interarrival time} > t)$.
• Memoryless property of the exponential explains the “lack of clustering” in a Poisson process — the hazard rate is constant.

Worked Example: Oil Change Service Time — Exponential & Poisson

Scenario: Rohit’s garage offers a 50% discount if an oil change takes more than 20 minutes. The average oil change time is 20 minutes. Assuming the time $X$ (in minutes) for one oil change follows an exponential distribution, we evaluate the risk of this promise. Later we shift to counting the number of oil changes $Y$ in a fixed time window, which follows a Poisson distribution.

1. Time for a single oil change: Exponential model

If the mean time is $\mu = 20$ minutes, the rate parameter is $\lambda = \frac{1}{\mu} = 0.05$ per minute.
The exponential CDF: $F(x) = P(X \le x) = 1 - e^{-\lambda x}$.

Key relationship: Exponential distribution models the time between events (here, the time to complete one oil change).

Probability that time < 20 minutes (promise kept)

$P(X < 20) = F(20) = 1 - e^{-0.05 \times 20} = 1 - e^{-1} \approx 0.632$

There is a 63.2% chance the oil change finishes within 20 minutes. Rohit will likely give many discounts.

Probability that time is between 15 and 30 minutes

$P(15 < X < 30) = F(30) - F(15) = (1 - e^{-0.05 \times 30}) - (1 - e^{-0.05 \times 15}) = e^{-0.75} - e^{-1.5} \approx 0.472 - 0.223 = 0.249$

There is a 24.9% chance the oil change takes between 15 and 30 minutes.

Event	Probability
$X < 20$	0.632
$15 < X < 30$	0.249

2. Number of oil changes in a time window: Poisson model

Number of oil changes $Y$ in a fixed time period $t$ follows a Poisson distribution with mean $\Lambda = \lambda t$, where $\lambda$ is the exponential rate (0.05 per minute). This connection holds because the exponential distribution models inter-arrival times of a Poisson process.

Four‑hour morning shift (240 minutes)

$\Lambda = 0.05 \times 240 = 12 \quad \Rightarrow \quad Y \sim \text{Poisson}(12)$

Probability of more than 15 oil changes in 4 hours:
$P(Y \ge 15) = 1 - P(Y \le 14)$. Using Poisson tables or software: $P(Y \ge 15) \approx 0.228$

Eight‑hour day (480 minutes)

$\Lambda = 0.05 \times 480 = 24 \quad \Rightarrow \quad Y \sim \text{Poisson}(24)$

Probability of more than 30 oil changes in 8 hours:
$P(Y \ge 30) \approx 0.132$

Time window	Average oil changes ($\Lambda$)	$P(Y \ge \text{threshold})$	Threshold
4 hours (morning)	12	0.228	15
8 hours (full day)	24	0.132	30

Relationship between Exponential and Poisson

flowchart LR
  A[Exponential: time per oil change<br>mean=20 min, λ=0.05] --> B[Poisson: number of oil changes<br>in time t, mean Λ=λ·t]
  B --> C[4-hour shift: Λ=12]
  B --> D[8-hour day: Λ=24]

The Poisson distribution counts events when the inter‑event times are i.i.d. exponential with the same rate $\lambda$.

Key Takeaways

• Exponential models the time until one event (oil change). Use $\lambda = 1/\text{mean}$.
• Poisson models the number of events in a fixed interval when events occur at a constant average rate.
• The two are linked: $\Lambda = \lambda \times t$.
• Rohit’s offer is risky: 63% chance of finishing on time (but 37% chance of discount). The probabilities of completing many oil changes per shift are modest (22.8% for >15 in 4 hours, 13.2% for >30 in 8 hours).

• Exam tip: When a problem gives an average time and asks for counts over a time period, switch from exponential to Poisson using $\Lambda = \text{rate} \times \text{time}$. Always check if the exponential inter‑event time assumption is justified (often stated as “random arrivals” or “memoryless” process).

The Normal Random Variable and its Distribution

The normal distribution is the most widely used continuous probability distribution. It models many naturally occurring phenomena—heights, weights, test scores, rainfall—and is central to statistical inference. Its probability density function (PDF) produces the familiar bell-shaped curve, symmetric about the mean.

$$f(x) = \frac{1}{\sigma\sqrt{2\pi}} \, e^{-\frac{(x-\mu)^2}{2\sigma^2}}, \quad -\infty < x < \infty$$

The entire family of normal distributions is differentiated by two parameters:

• Mean $\mu$ – determines the centre; can be any real number.
• Standard deviation $\sigma$ – determines the spread; $\sigma > 0$.

Properties of the Normal Curve

• Highest point at $x = \mu$, which is also the median and mode.
• Symmetric about $\mu$: left half is a mirror image of the right half. Skewness = 0.
• Tails extend to infinity in both directions, never touching the horizontal axis.
• Total area under curve = 1 (like all PDFs). Because of symmetry, area to the left of $\mu$ = 0.5; area to the right = 0.5.
• Spread determined by $\sigma$: larger $\sigma$ → wider, flatter curve; smaller $\sigma$ → taller, narrower curve.

The Empirical Rule (68–95–99.7 Rule)

For any normal random variable $X$, the percentage of values within $k$ standard deviations of the mean is fixed:

Interval	Percentage of values	Probability
$\mu \pm 1\sigma$	68.3%	$P(\mu - \sigma \leq X \leq \mu + \sigma) = 0.683$
$\mu \pm 2\sigma$	95.4%	$P(\mu - 2\sigma \leq X \leq \mu + 2\sigma) = 0.954$
$\mu \pm 3\sigma$	99.7%	$P(\mu - 3\sigma \leq X \leq \mu + 3\sigma) = 0.997$

Exam tip: The empirical rule provides quick probability approximations and is often tested directly.

The Standard Normal Distribution

A normal distribution with $\mu = 0$ and $\sigma = 1$ is called the standard normal distribution. Its random variable is denoted by $Z$ instead of $X$.

$$f(z) = \frac{1}{\sqrt{2\pi}} \, e^{-z^2/2}, \quad -\infty < z < \infty$$

The cumulative distribution function (CDF) of $Z$ is often written as $\Phi(z)$ (capital phi), and the PDF as $\phi(z)$. Probability tables for $\Phi(z)$ eliminate the need for integration.

Computing Normal Probabilities with Excel

Two built-in functions handle any normal distribution directly (no need to standardise):

• NORM.DIST$(x, \text{mean}, \text{standard\_dev}, \text{cumulative})$

  • cumulative = TRUE → returns $\Phi(x) = P(X \leq x)$ (CDF).
  • cumulative = FALSE → returns the PDF value at $x$.

• NORM.INV$(\text{probability}, \text{mean}, \text{standard\_dev})$

  • Returns $x$ such that $P(X \leq x) = \text{probability}$ (inverse CDF).

Template example (general normal: $\mu = 10$, $\sigma = 3$)

Input	Computed Probability	Result
$x = 13$	$P(X \leq 13)$	0.841
$x = 13$	$P(X \geq 13)$	0.159
$a = 7, b = 13$	$P(7 \leq X \leq 13)$	0.683

For inverse:

• Enter cumulative probability 0.841 → get $x = 13$.
• Enter interval probability 0.683 → get $a = 7$, $b = 13$.

Worked Examples: Standard Normal Probabilities

Set $\mu = 0$, $\sigma = 1$ in the template.

Example 1

Probability	Result
$P(Z \leq -1)$	0.159
$P(Z \geq -1)$	0.841
$P(Z \geq -1.5)$	0.933
$P(Z \geq -2.5)$	0.994
$P(-3 < Z \leq 0)$	0.499

Note: $P(Z = a) = 0$ for any continuous random variable, so $P(Z \leq a) = P(Z < a)$ and $P(Z \geq a) = P(Z > a)$.

Example 2

Probability	Result
$P(0 \leq Z \leq 0.8)$	0.288
$P(-1.5 \leq Z \leq 0)$	0.433
$P(Z \geq 0.4)$	0.345
$P(Z \geq -0.2)$	0.579
$P(Z \leq 1.2)$	0.885
$P(Z \leq -0.7)$	0.242

Example 3: Inverse Calculations

Condition	Range of $z$
Area to the left of $z$ is 0.21	$z \leq -0.806$ (i.e., $z = -0.806$)
Area between $-z$ and $+z$ is 0.9	$-1.64 \leq z \leq +1.64$
Area between $-z$ and $+z$ is 0.2	$-0.25 \leq z \leq +0.25$
Area to the left of $z$ is 0.955	$z \leq 2.58$ (i.e., $z = 2.58$)
Area to the right of $z$ is 0.692	$z \geq -0.502$ (i.e., $z = -0.502$)

Key Takeaways

• The normal distribution is bell-shaped, symmetric about $\mu$, with $\mu =$ median $=$ mode.
• It is parameterised by $\mu$ (centre) and $\sigma$ (spread).
• The empirical rule: ≈68% within 1σ, ≈95% within 2σ, ≈99.7% within 3σ.
• The standard normal $Z$ has $\mu = 0$, $\sigma = 1$; its CDF is $\Phi(z)$.
• Excel: NORM.DIST for cumulative/PDF, NORM.INV for inverse; both work for any normal.
• For continuous variables, point probabilities are zero; probabilities always refer to intervals.

Standard Normal Distribution

The standard normal distribution is the symmetric, bell-shaped distribution centered at 0 with a standard deviation of 1. Its random variable is denoted $Z$. This distribution serves as the universal reference for all normal distributions — any normal variable can be transformed into a standard normal.

An important concept for statistical inference is the critical value. For a standard normal, a critical value $z_{\alpha/2}$ is defined such that the tail probability to the right of it equals $\alpha/2$:

$P(Z > z_{\alpha/2}) = \alpha/2$

By symmetry, $-z_{\alpha/2}$ has a left‑tail probability of $\alpha/2$. Consequently, the central probability between $-z_{\alpha/2}$ and $+z_{\alpha/2}$ is $1-\alpha$:

$P(-z_{\alpha/2} \leq Z \leq z_{\alpha/2}) = 1-\alpha$

The familiar 68‑95‑99.7 rule is a special case: for $1-\alpha = 0.68$, $\alpha/2 = 0.16$, and $z_{0.16} = 1$.

Common critical values

$1-\alpha$ (confidence level)	$\alpha/2$	$z_{\alpha/2}$
0.80	0.10	1.28
0.90	0.05	1.64
0.95	0.025	1.96
0.99	0.005	2.58

Exam tip: Memorise the three critical values 1.64, 1.96, and 2.58 — they appear repeatedly in confidence intervals and hypothesis tests.

Two standard types of questions

1. Given $Z$, find probability — use the standard normal table or template.
2. Given probability, find $Z$ — inverse lookup. Sketching the bell curve and shading the relevant area always helps avoid sign errors.

Converting any normal to the standard normal

If $X \sim N(\mu, \sigma)$, then the transformed variable

$Z = \frac{X - \mu}{\sigma}$

follows a standard normal distribution. $Z$ measures how many standard deviations $X$ is away from the mean — positive above, negative below.

flowchart LR
  A[Original normal X] --> B[Compute Z = (X-µ)/σ]
  B --> C[Use standard normal table<br>to find probability]
  C --> D[Interpret result for X]

Worked example
Let $X \sim N(\mu=10,\;\sigma=2)$. Find $P(10 < X < 14)$.

1. Convert boundaries:
   $Z_1 = \frac{10-10}{2} = 0, \quad Z_2 = \frac{14-10}{2} = 2$
2. Then $P(10 < X < 14) = P(0 < Z < 2)$.
3. From the standard normal table, $P(0 < Z < 2) = 0.4772$.

The same result could be obtained by directly using a normal template with $\mu=10$, $\sigma=2$; the conversion simply standardises the problem.

Key takeaways

• Critical value $z_{\alpha/2}$: right‑tail area = $\alpha/2$; symmetric about 0.
• Central probability $1-\alpha$ lies between $-z_{\alpha/2}$ and $+z_{\alpha/2}$.
• Common $z_{\alpha/2}$ values: 1.28, 1.64, 1.96, 2.58 for confidence levels 80%, 90%, 95%, 99%.
• Any normal $X$ can be transformed via $Z = (X-\mu)/\sigma$ to a standard normal.
• $Z$ tells the distance from the mean in standard deviation units.
• Always sketch the curve and shade the area when solving probability problems.

Examples of Normal Distribution Applications

The normal distribution is a versatile model for many real-world variables. These worked examples illustrate how to compute probabilities for intervals, tails, and cumulative regions, and how to perform inverse calculations (percentiles) using given mean $\mu$ and standard deviation $\sigma$. The key tool is the z‑score: $z = (x - \mu)/\sigma$, which maps any $x$ to a standard normal $N(0,1)$.

flowchart LR
  A[What do you need?] --> B{Probability}
  A --> C{Inverse (percentile)}
  B --> D[Less than x?] --> E[Cumulative CDF: P(X < x)]
  B --> F[Greater than x?] --> G[Tail: P(X > x) = 1 - CDF]
  B --> H[Between a and b?] --> I[Interval: CDF(b) - CDF(a)]
  C --> J[Given probability p, find x] --> K[Inverse CDF (quantile)]

Exam tip: Always check units (e.g., minutes vs. hours). A small mistake in input changes the entire answer.

1. Stock Returns (Munaf)

A stock fund has a mean annual return of $\mu = 12\%$ and a standard deviation $\sigma = 3\%$. Returns are normally distributed.

• Unsatisfactory (return $< 5\%$):
  $z = \frac{5-12}{3} = -2.33$ → $P(X < 5\%) = 0.01$ (1% chance).
• Excellent (return $> 10\%$):
  $z = \frac{10-12}{3} = -0.667$ → $P(X > 10\%) = 0.748$ (74.8% chance).
• Moderate (return between 5% and 10%):
  $P(5\% < X < 10\%) = P(X < 10\%) - P(X < 5\%) = 0.748 - 0.01 = 0.243$ (24.3% chance).
• 90th percentile (inverse):
  • Cumulative: 90% of returns are less than 15.8%.
  • Tail: 90% of returns are greater than 8.1%.
  • Interval: 90% of returns lie between 7% and 17%.

Key mechanism: The normal distribution template directly uses $\mu$ and $\sigma$ to output these probabilities without manual z‑score calculation.

2. Screen Time (Bhavani)

Daily screen time of school kids: $\mu = 8.4$ hours, $\sigma = 2.5$ hours.

• More than 4 hours:
  $z = \frac{4-8.4}{2.5} = -1.76$ → $P(X > 4) = 0.96$ (96% of kids exceed 4 hours).
• Between 6 and 12 hours:
  $z_6 = \frac{6-8.4}{2.5} = -0.96$, $z_{12} = \frac{12-8.4}{2.5} = 1.44$ → $P(6 < X < 12) = 0.757$ (75.7% of kids).
• Top 20% (inverse):
  Need $P(X < x) = 0.8$ → $x = 10.5$ hours. A kid with >10.5 hours is in the top 20%.
• Bottom 20% (inverse):
  Need $P(X < x) = 0.2$ → $x = 6.3$ hours. A kid with <6.3 hours is in the bottom 20%.
• (Bonus) 80% interval: 80% of kids have screen time between 5.2 and 11.6 hours.

3. Exam Duration (Hirav)

Time to complete an exam: $\mu = 150$ minutes, $\sigma = 20$ minutes. 60 students.

• 2 hours or less (≤120 min):
  $z = \frac{120-150}{20} = -1.5$ → $P(X \leq 120) = 0.067$ → $0.067 \times 60 \approx 4$ students.
• Between 2 and 3 hours (120–180 min):
  $z_{120} = -1.5$, $z_{180} = +1.5$ → $P(120 < X < 180) = 0.866$ → $0.866 \times 60 \approx 52$ students.
• 3 hours or more (≥180 min):
  $P(X \geq 180) = 0.067$ → $0.067 \times 60 \approx 4$ students.

Exam tip: The symmetry of the normal distribution means probabilities at equal distances above and below the mean are identical. Here 120 and 180 min are both 1.5σ from the mean, giving the same tail probability (0.067).

4. Sugar Packet Weight (Naman)

Two filling machines; weights are normally distributed. Naman evaluates which machine is better.

Machine	$\mu$ (kg)	$\sigma$ (kg)	$P(\text{underweight} < 1$ kg)	$P(1 \leq X \leq 1.05)$	90% interval
1	1.01	0.02	0.309 (30.9%)	0.669 (66.9%)	[0.97, 1.04]
2	1.03	0.04	0.227 (22.7%)	0.465 (46.5%)	[0.96, 1.09]

• Machine 1 has fewer underweight? No, 30.9% > 22.7%, so Machine 2 produces fewer underweight packets.
• Machine 2 has a narrower range for 90% of packets? No, its interval is wider (0.96–1.09 vs. 0.97–1.04).
  Naman’s claim that 90% of packets lie between 0.99 and 1.01 is false for both machines.

Trade-off: Machine 2 reduces underweight risk but increases variability, making the middle‑90% range much broader.

5. Glass Thickness (Vibhor)

Glass sheet thickness: $\mu = 3$ mm, $\sigma = 0.12$ mm.

• Thickness < 2.9 mm (cumulative):
  $z = \frac{2.9-3}{0.12} \approx -0.833$ → $P(X < 2.9) = 0.202$ (20.2% of sheets too thin).
• Thickness > 3.2 mm (tail):
  $z = \frac{3.2-3}{0.12} \approx 1.667$ → $P(X > 3.2) = 0.048$ (4.8% too thick).
• 95% thickness range (inverse):
  For a central probability of 0.95, the endpoints are $3 \pm 1.96 \times 0.12$ → [2.76 mm, 3.24 mm].

Key takeaways

• All calculations rely on the z‑score transformation $z = (x - \mu)/\sigma$, linking any normal variable to the standard normal.
• Three standard probability queries: cumulative (less than), tail (greater than), and interval (between). Use the CDF differences.
• Inverse problems (percentiles) find the value $x$ for a given cumulative probability (e.g., top 20% → $p=0.8$).
• Always check units (hours vs. minutes, percentage vs. decimal) before plugging numbers into a template or formula.
• Symmetry simplifies: $P(X < \mu - k\sigma) = P(X > \mu + k\sigma)$.

Normal Approximation to the Binomial Distribution

When n is large, computing binomial probabilities directly (via factorials in $\binom{n}{x} p^x q^{n-x}$) becomes unwieldy. The normal distribution provides a remarkably accurate approximation because the binomial’s probability mass function is approximately bell‑shaped — especially when p is not too close to 0 or 1.

Conditions for a Good Approximation

The approximation works well when both:

$np \ge 5 \quad \text{and} \quad nq \ge 5 \quad (\text{where } q = 1-p)$

If these hold, match the mean and variance of the normal to those of the binomial:

$\mu = np \qquad \sigma^2 = npq$

So we use $Y \sim N(\mu = np,\; \sigma^2 = npq)$ to approximate $X \sim \text{Bin}(n,p)$.

The Continuity Correction

Because the binomial is discrete and the normal is continuous, a continuity correction of ±0.5 is applied to improve accuracy.

Desired binomial probability	Continuity‑corrected normal probability
$P(X \le a)$	$P(Y \le a + 0.5)$
$P(X \ge b)$	$P(Y \ge b - 0.5)$
$P(a \le X \le b)$	$P(a - 0.5 \le Y \le b + 0.5)$
$P(X = k)$	$P(k - 0.5 \le Y \le k + 0.5)$

Exam tip: The continuity correction is the most common mistake. Always add/subtract 0.5; never approximate a discrete probability using a single point from a continuous distribution (that probability would be 0).

Worked Examples

Example 1: Unbiased coin, 16 tosses
$n=16,\; p=0.5$ → $\mu=8,\; \sigma^2=4\;( \sigma=2)$.

• Exact binomial: $P(X \le 5) = 0.105$
  Normal approx: $P(Y \le 5.5) = 0.1056 \quad (\approx 0.105)$

• Exact binomial: $P(8 \le X \le 11) = 0.5598$
  Normal approx: $P(7.5 \le Y \le 11.5) = 0.5586 \quad (\approx 0.5598)$

Example 2: Invoices with errors (Kunal Poonawala, Junagadh)
$n=100,\; p=0.1$ → $\mu=10,\; \sigma^2=9\;(\sigma=3)$.

• Exact binomial: $P(X = 12) = 0.09$
  Normal approx: $P(11.5 \le Y \le 12.5) = 0.1052$ (close, though slightly off; approximation improves with larger $np$).

• Exact binomial: $P(X \le 13) = 0.876$
  Normal approx: $P(Y \le 13.5) = 0.879 \quad (\approx 0.876)$

The approximation is reliable, especially when $np$ and $nq$ are well above 5.

Connection to the Poisson Distribution

The Poisson distribution can also be approximated by the normal when its mean $\lambda$ is large. Moreover, for a binomial with small $q$ (failure probability), the variance $npq \approx np$, making the binomial similar to a Poisson. Hence, in certain cases the Poisson can be approximated via the binomial → normal chain.

---

Key Takeaways

• Normal approximation to the binomial is valid when $np \ge 5$ and $nq \ge 5$.
• Match mean $\mu = np$ and variance $\sigma^2 = npq$.
• Always apply a continuity correction (±0.5) because the binomial is discrete.
• The approximation is remarkably accurate and is often used by software internally.

---

Linear Combinations of Random Variables (Normal Case)

Any linear combination of normally distributed random variables is itself normally distributed – not just its mean and variance, but the entire distribution.

General Results (any distribution)

Given random variables $X_1, X_2, \dots, X_n$ and constants $a_1, a_2, \dots, a_n$, define

$Y = a_1 X_1 + a_2 X_2 + \cdots + a_n X_n$

Expectation (always linear):

$E[Y] = a_1 E[X_1] + a_2 E[X_2] + \cdots + a_n E[X_n]$

Variance:

• If the $X_i$ are independent:

  $\text{Var}(Y) = a_1^2 \text{Var}(X_1) + a_2^2 \text{Var}(X_2) + \cdots + a_n^2 \text{Var}(X_n)$

• If the $X_i$ are correlated (not independent), covariance terms appear:

  $\text{Var}(Y) = \sum_{i=1}^n a_i^2 \text{Var}(X_i) + 2\sum_{i<j} a_i a_j \text{Cov}(X_i, X_j)$

These formulas hold for any distributions of the $X_i$.

Special Case: Normally Distributed $X_i$

If each $X_i \sim N(\mu_i, \sigma_i^2)$ and they are jointly normal (or independent normal), then $Y$ is also normally distributed:

$Y \sim N\!\left( \sum a_i \mu_i,\; \sum a_i^2 \sigma_i^2 + 2\sum_{i<j} a_i a_j \text{Cov}(X_i, X_j) \right)$

This property is unique to the normal family and is crucial for statistical inference (e.g., sums of normal data remain normal).

Exam tip: When a problem states “$X_1, X_2, \dots$ are independent normal random variables,” any linear combination (like $\bar{X}$ or $X_1 - X_2$) is also normal. You only need to compute its mean and variance.

---

Key Takeaways

• For any random variables, $E[a_1 X_1 + \dots]$ is linear; variance adds with squares plus covariance terms if not independent.
• For normal random variables, the linear combination is also normal — a key result for later use (e.g., Central Limit Theorem).
• This property does not hold for most other distributions; it is special to the normal.

Linear Combinations of Normal Random Variables

A key property: any linear combination of independent normal random variables is itself normally distributed. This makes it possible to model sums, differences, and averages of normal data with a single normal distribution.

The Basic Result: (Y = aX + b)

If (X \sim N(\mu,\ \sigma^2)) and (Y = aX + b) (with constants (a) and (b)), then
[ Y \sim N\big(a\mu + b,\ a^2\sigma^2\big). ]

Intuition – Shifting and scaling a normal distribution preserves its bell shape; only the mean and variance change linearly.

Standardization as a Special Case

Set (a = \frac{1}{\sigma}) and (b = -\frac{\mu}{\sigma}):

[ Y = \frac{X - \mu}{\sigma} \sim N(0,1) ]

This is the standard normal random variable (Z).

Sum of Two Independent Normals

Let (X_1 \sim N(\mu_1, \sigma_1^2)) and (X_2 \sim N(\mu_2, \sigma_2^2)) be independent. Then
[ Y = X_1 + X_2 \sim N(\mu_1 + \mu_2,\ \sigma_1^2 + \sigma_2^2). ]

If the variables are not independent, the sum is still normal, but the variance includes covariance terms.

General Linear Combination

For (n) independent normal variables (X_i \sim N(\mu_i, \sigma_i^2)) and constants (a_1, \dots, a_n, b):

[ Y = a_1 X_1 + a_2 X_2 + \cdots + a_n X_n + b ]

has distribution

[ Y \sim N!\left( \sum_{i=1}^n a_i \mu_i + b,\ \sum_{i=1}^n a_i^2 \sigma_i^2 \right). ]

Important Special Cases: Sum and Sample Mean

Assume (X_1, X_2, \dots, X_n) are iid (independent and identically distributed) as (N(\mu, \sigma^2)).

Quantity	Definition	Distribution	Mean	Variance	Standard Deviation
Sum	(S_n = \sum_{i=1}^n X_i)	(N(n\mu,\ n\sigma^2))	(n\mu)	(n\sigma^2)	(\sqrt{n},\sigma)
Sample mean	(\bar{X} = \frac{1}{n}\sum_{i=1}^n X_i)	(N(\mu,\ \sigma^2/n))	(\mu)	(\frac{\sigma^2}{n})	(\frac{\sigma}{\sqrt{n}})

Exam tip: Standard deviations do not add up directly. For the sum, (\text{SD}(S_n) = \sqrt{n},\sigma); for the average, (\text{SD}(\bar{X}) = \sigma/\sqrt{n}). The variance reduces by a factor of (n) when averaging – this is the foundation of sampling precision.

---

Worked Examples

1. Piston–Cylinder Gap

Problem – Piston head radius (X_1 \sim N(30\ \text{mm},\ 0.0025\ \text{mm}^2)), cylinder inside radius (X_2 \sim N(30.25\ \text{mm},\ 0.0036\ \text{mm}^2)). The gap is (Y = X_2 - X_1). Find
(a) (P(Y \le 0)) (piston does not fit)
(b) (P(0.1 \le Y \le 0.35)) (optimal performance)

Solution
(Y) is normal because it is a linear combination of independent normals.

[ \begin{aligned} \mu_Y &= 30.25 - 30 = 0.25 \ \sigma_Y^2 &= 0.0025 + 0.0036 = 0.0061 \ \sigma_Y &= \sqrt{0.0061} \approx 0.078 \end{aligned} ]

(a)
[ P(Y \le 0) = \Phi!\left(\frac{0 - 0.25}{0.078}\right) \approx \Phi(-3.205) \approx 0.0007 ]

(b)
[ P(0.1 \le Y \le 0.35) = \Phi!\left(\frac{0.35-0.25}{0.078}\right) - \Phi!\left(\frac{0.1-0.25}{0.078}\right) \approx \Phi(1.282) - \Phi(-1.923) \approx 0.872 ]

Interpretation – Extremely low chance of misfit; ~87 % chance of optimal gap.

2. Average Height of Cotton Plants

Problem – Individual plant height after two weeks: (X_i \sim N(29.4\ \text{cm},\ 4.41\ \text{cm}^2)). For 20 plants, the sample mean (\bar{X}) is normal. Find the 95th percentile of (\bar{X}).

Solution
[ \bar{X} \sim N!\left(29.4,\ \frac{4.41}{20} = 0.2205\right),\quad \sigma_{\bar{X}} = \sqrt{0.2205} \approx 0.469 ]

The 95th percentile is
[ \mu + z_{0.05},\sigma_{\bar{X}} = 29.4 + 1.645 \times 0.469 \approx 30.17\ \text{cm}. ]

Notice that the standard deviation of the average (0.47 cm) is much smaller than that of an individual plant (2.1 cm) – averaging greatly reduces variability.

3. Stock Returns Comparison

Problem – Stock A: (X_A \sim N(8%,\ 2.25%^2)), Stock B: (X_B \sim N(9.5%,\ 4%^2)), independent.
Define (Y = X_B - X_A). Find:
(a) (P(X_A \text{ moderate})) meaning (5 \le X_A \le 10)
(b) (P(X_B \text{ excellent})) meaning (X_B \ge 10)
(c) (P(X_B > X_A)) i.e., (P(Y > 0))
(d) (P(Y \ge 2))

Solution

(a) (X_A \sim N(8, 2.25)).
[ P(5 \le X_A \le 10) = \Phi!\left(\frac{10-8}{1.5}\right) - \Phi!\left(\frac{5-8}{1.5}\right) \approx \Phi(1.333) - \Phi(-2) \approx 0.886 ]

(b) (X_B \sim N(9.5, 4)).
[ P(X_B \ge 10) = 1 - \Phi!\left(\frac{10-9.5}{2}\right) \approx 1 - \Phi(0.25) \approx 0.4013 ]

(c) (Y = X_B - X_A \sim N(1.5,\ 2.25+4=6.25)), so (\sigma_Y = 2.5).
[ P(Y > 0) = 1 - \Phi!\left(\frac{0-1.5}{2.5}\right) = 1 - \Phi(-0.6) \approx 0.726 ]

(d)
[ P(Y \ge 2) = 1 - \Phi!\left(\frac{2-1.5}{2.5}\right) = 1 - \Phi(0.2) \approx 0.421 ]

---

Key Takeaways

• Any linear combination of independent normal random variables is normally distributed.
• For (Y = \sum a_i X_i + b): mean = (\sum a_i \mu_i + b), variance = (\sum a_i^2 \sigma_i^2) (independence assumed).
• Sum of iid normals: mean (n\mu), variance (n\sigma^2).
• Sample mean of iid normals: mean (\mu), variance (\sigma^2/n) – precision improves with (n).
• Difference of two independent normals: variance adds; subtract the means.
• Use standardization to compute probabilities for any linear combination.

Chi-Square Distribution

The chi-square distribution arises naturally from squared standard normal variables. If $Z \sim N(0,1)$, then $X = Z^2$ follows a chi-square distribution with 1 degree of freedom. More generally, if $Z_1, Z_2, \dots, Z_n$ are independent standard normal random variables, then

$X = Z_1^2 + Z_2^2 + \cdots + Z_n^2$

follows a chi-square distribution with $n$ degrees of freedom (denoted $\chi^2_n$).

Intuitively, it measures the sum of squared deviations; because squares are always non‑negative, the distribution is defined only for positive values.

Properties

• Single parameter: the degrees of freedom ($k$).
• Mean: $\mu = k$.
• Variance: $\sigma^2 = 2k$.
• Shape: right‑skewed for small $k$; as $k$ increases, the skewness decreases and the density approaches a normal distribution (by the Central Limit Theorem).
• Support: $x > 0$.

Applications

• Non‑parametric hypothesis tests (e.g., goodness‑of‑fit).
• Feature selection and classification.
• Testing independence in contingency tables (logistic regression).

Critical Values

The chi‑square critical value $\chi^2_{\alpha, k}$ is the value such that the right‑tail probability equals $\alpha$:

$P(X \geq \chi^2_{\alpha, k}) = \alpha$

Unlike the standard normal, critical values depend on the degrees of freedom. The table below gives critical values for several $\alpha$ and $k$ (computed using Microsoft Excel’s CHISQ.DIST function).

$k$	$\alpha = 0.20$	$\alpha = 0.10$	$\alpha = 0.05$	$\alpha = 0.01$
5	7.3	9.2	11.1	15.1
10	13.4	16.0	18.3	23.2
15	19.3	22.3	25.0	37.6
20	25.0	28.4	31.4	37.6

Exam tip: As $k$ increases, critical values increase because the distribution spreads out (variance $=2k$). The values in the last column are the largest because the tail probability $\alpha$ is smallest.

Computing in Excel

Use CHISQ.DIST(x, k, cumulative):

• x: value of the chi‑square random variable.
• k: degrees of freedom.
• cumulative: TRUE returns the cumulative distribution function; FALSE returns the probability density function.

A template can automate mean, variance, probability calculations, and inverse (critical value) look‑up.

Key takeaways

• Chi‑square is the sum of $k$ independent squared standard normals.
• Mean = $k$, variance = $2k$.
• Always positive, right‑skewed, converges to normal as $k \to \infty$.
• Critical values increase with $k$; used in goodness‑of‑fit and independence tests.

---

Student’s t‑Distribution

The Student’s t‑distribution (or simply t‑distribution) is defined when a standard normal variable is divided by the square root of an independent chi‑square variable scaled by its degrees of freedom:

$T = \frac{Z}{\sqrt{X / k}}$

where $Z \sim N(0,1)$, $X \sim \chi^2_k$, and $Z$ and $X$ are independent. $T$ follows a t‑distribution with $k$ degrees of freedom.

Intuitively, the t‑distribution is used in place of the standard normal when the population variance is unknown and the sample size is small. It has fatter tails (more probability in the extremes) than the normal.

History

Discovered by William Sealy Gosset while working at the Guinness Brewery. He published under the pseudonym “Student” because his employer prohibited employees from publishing research. Hence the name Student’s t‑distribution.

Properties

• Symmetric and bell‑shaped, centered at 0.
• Mean: $\mu = 0$ (for $k > 1$).
• Variance: $\sigma^2 = \frac{k}{k-2}$ (for $k > 2$; undefined for $k \leq 2$).
• Shape: flatter (more spread) than the standard normal; as $k \to \infty$, the t‑distribution converges to $N(0,1)$.
• Support: $-\infty < t < \infty$.

Applications

• Hypothesis tests about population means (especially small samples).
• Comparing means of two populations.
• Diagnostics in linear regression (e.g., t‑tests for coefficients).

Critical Values

Because the t‑distribution is symmetric, critical values are often defined for two‑tailed regions. The value $t_{\alpha/2, k}$ is the positive number such that:

$P(T \geq t_{\alpha/2, k}) = \alpha/2$

By symmetry, $P(T \leq -t_{\alpha/2, k}) = \alpha/2$. Therefore $P(-t_{\alpha/2, k} \leq T \leq t_{\alpha/2, k}) = 1 - \alpha$.

The table below gives $t_{\alpha/2, k}$ for common $\alpha/2$ and degrees of freedom (computed using Excel’s T.DIST function).

$k$	$\alpha/2 = 0.10$	$\alpha/2 = 0.05$	$\alpha/2 = 0.025$	$\alpha/2 = 0.005$
5	1.48	2.02	2.57	4.03
10	1.37	1.81	2.23	3.17
15	1.34	1.75	2.13	2.95
20	1.33	1.73	2.09	2.85

Notice that as $k$ increases, the critical values decrease (the distribution becomes less flat) and approach the corresponding standard normal critical values (e.g., for $\alpha/2 = 0.025$, $z_{0.025} \approx 1.96$).

Computing in Excel

Use T.DIST(x, k, cumulative):

• x: value of the t random variable.
• k: degrees of freedom.
• cumulative: TRUE for cumulative probability, FALSE for PDF.

For critical values, the inverse function T.INV or a template can be used.

Exam tip: The t‑distribution is flatter than the normal, so its critical values are larger than the corresponding normal critical values for small $k$. As $k$ grows, they converge.

Key takeaways

• t‑distribution: $T = Z / \sqrt{\chi^2_k / k}$.
• Symmetric, zero‑mean, fatter tails than normal.
• Variance $= k/(k-2)$; converges to standard normal as $k \to \infty$.
• Critical values decrease with increasing $k$; used in mean hypothesis tests and regression diagnostics.

The F Distribution

The F distribution (Fisher–Snedecor distribution, after Ronald Fisher) models the ratio of two independent chi-square random variables, each divided by its own degrees of freedom.

$F(k_1, k_2) = \frac{\chi^2_{k_1} / k_1}{\chi^2_{k_2} / k_2}$

Here $k_1$ = numerator degrees of freedom and $k_2$ = denominator degrees of freedom. The order matters: $F(5,10) \neq F(10,5)$.

Properties

• Defined only for positive values of the random variable.
• Unimodal and right-skewed (long right tail).
• Mean: $\displaystyle \frac{k_2}{k_2 - 2}$ for $k_2 \geq 2$ (when $k_2 \leq 2$, the mean is undefined).
• Standard deviation decreases as $k_2$ increases.
• As both $k_1$ and $k_2$ become large, the PDF becomes sharply spiked around $1$.

Where it is used

• Analysis of variance (ANOVA)
• Feature selection in machine learning
• Testing overall fit of multiple linear regression models

Excel Function: F.DIST

In Excel, the function F.DIST(x, k1, k2, cumulative) computes probabilities.

Parameter	Meaning
x	Value of the F random variable
k1	Numerator degrees of freedom
k2	Denominator degrees of freedom
cumulative	TRUE → cumulative probability; FALSE → probability density

Inverse (critical values) can be obtained via F.INV(alpha, k1, k2) for right‑tail probability $\alpha$.

Worked Example: Critical Values (Right‑Tail)

For an F‑distribution with given $k_1, k_2$, the critical value $F_{\alpha, k_1, k_2}$ is the value such that the area to its right equals $\alpha$.

Using an F‑distribution template (or Excel), with $\alpha = 0.2, 0.1, 0.05, 0.01$:

$(k_1, k_2)$	$F_{0.20}$	$F_{0.10}$	$F_{0.05}$	$F_{0.01}$
(5, 5)	2.23	3.45	5.05	10.97
(5, 10)	1.80	2.52	3.33	5.64
(10, 5)	2.19	3.30	4.74	10.05
(20, 20)	1.47	1.79	2.21	2.94

Exam tip: Swapping $k_1$ and $k_2$ changes critical values — always double‑check which is numerator and which is denominator. When $k_2$ is large, critical values approach 1 because the distribution becomes concentrated near 1.

Key takeaways

• $F(k_1, k_2) = (\chi^2_{k_1}/k_1) / (\chi^2_{k_2}/k_2)$ — ratio of independent scaled chi‑squares.
• Positive, unimodal, right‑skewed; mean $= k_2/(k_2-2)$ for $k_2 \geq 2$.
• Order of degrees of freedom matters: $F(5,10) \neq F(10,5)$.
• As $k_2$ increases, variance decreases and distribution peaks nearer 1.
• Widely used in ANOVA, regression, and machine learning feature selection.
• Excel: F.DIST(x, k1, k2, TRUE/FALSE) for cumulative/density; F.INV for critical values.