Advanced Statistics for Business

Review of Simple Linear Regression

Simple linear regression models the relationship between one dependent variable (outcome) and one independent variable (predictor). The goal is to find a straight line that best describes how changes in the predictor affect the outcome – a best‑fit line.

$\hat{y} = \beta_0 + \beta_1 x$

where $\hat{y}$ is the predicted dependent variable, $x$ the independent variable, $\beta_0$ the intercept, and $\beta_1$ the slope (the estimated change in $y$ per unit change in $x$).

Intuition: Quantify and predict a one‑factor influence.
Business examples:

• Predict sales based on advertising spend.
• Forecast monthly revenue from foot traffic.
• Estimate fuel consumption from distance driven.

Multiple Linear Regression

Multiple linear regression extends the idea to multiple independent variables simultaneously:

$\hat{y} = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \cdots + \beta_k x_k$

Why it matters: Real‑world outcomes are rarely driven by a single factor. Multiple regression disentangles the joint effect of several predictors.

Business examples:

• Real estate: House price depends on location, bedrooms, amenities.
• Finance: Credit score assessment using income, employment status, debt level.
• Sales prediction: Price, economic conditions, number of competitors all matter.

Exam tip: Simple regression is a special case of multiple regression ($k=1$). All assumptions and diagnostic checks apply to both.

Comparison of Simple vs. Multiple Linear Regression

Aspect	Simple Linear Regression	Multiple Linear Regression
Number of predictors	One	Two or more
Model equation	$\hat{y} = \beta_0 + \beta_1 x$	$\hat{y} = \beta_0 + \sum \beta_i x_i$
Use case	Single‑factor influence	Multi‑factor influence
Business complexity	Low (e.g., ad spend → sales)	High (e.g., house pricing)

Key takeaways – Simple & Multiple Regression

• Both provide a structured way to quantify relationships and predict outcomes.
• Simple regression handles one predictor; multiple regression handles many.
• Effective for forecasting, resource allocation, and identifying key drivers.
• Reliability depends on assumptions (next section).

---

Key Assumptions of Linear Regression

Validity of linear regression rests on several assumptions. When any is violated, predictions and inferences become biased.

1. Linearity

The relationship between predictors and the dependent variable must be linear – a constant change in $x$ produces a constant change in $y$.

• Violation example: Advertising spend often shows diminishing returns; initial dollars yield steep sales increases, later dollars yield little. The true relationship is curved, not a straight line.

2. Independence of Residuals

Residuals (errors = observed – predicted) must be independent of each other.

• Violation example: Time‑series data – today’s stock price depends on yesterday’s, leading to correlated errors.

3. Homoscedasticity

The variance of residuals should be constant across all levels of the predictors.

• Violation (heteroscedasticity): As income increases, spending variability tends to rise – wealthier individuals have more diverse habits.

4. Normality of Error Terms

Residuals should follow a normal distribution (critical for confidence intervals and hypothesis tests).

• Violation: Many real‑world datasets are skewed or have outliers. For continuous non‑normal data, consider transformation; for binary outcomes, switch to logistic regression.

5. No Multicollinearity

Independent variables should not be highly correlated with each other.

• Violation example: Advertising spend and marketing spend are often strongly correlated, making it impossible to isolate each variable’s unique effect on sales.

Assumption	Description	Business violation example
Linearity	Predictor–outcome relationship is a straight line	Advertising with diminishing returns
Independence	Residuals not correlated over time	Stock prices; time‑series data
Homoscedasticity	Constant residual variance	Income vs. spending; richer people show more variability
Normality	Residuals are normally distributed	Skewed data or binary outcomes
No multicollinearity	Predictors not highly correlated	Advertising spend & marketing spend

Key takeaways – Assumptions

• Violations lead to biased estimates, invalid hypothesis tests, and poor predictions.
• Each assumption points to a specific advanced technique (e.g., logistic regression for binary outcomes; nonlinear regression for curvature).
• Always test assumptions before relying on linear regression results.

---

When Assumptions Fail: Introduction to Advanced Methods

When linear regression’s assumptions are not met, two common alternatives are introduced:

flowchart TD
    A[Regression Problem] --> B{Assumptions hold?}
    B -->|Yes| C[Use Linear Regression]
    B -->|No| D{Type of failure?}
    D -->|Binary outcome OR non‑normal errors| E[Logistic Regression]
    D -->|Non‑linear relationship| F[Non‑linear Regression]

Logistic Regression

Used when the dependent variable is categorical – often binary (yes/no, success/failure).

• Linear regression is inappropriate because it assumes a continuous outcome and can predict probabilities outside [0,1].
• Logistic regression models the probability of an event using a logistic (S‑shaped) function, ensuring predictions fall between 0 and 1.
• Business applications: predict customer purchase (yes/no), loan default, employee retention.

Non‑linear Regression

Used when the relationship between variables is inherently non‑linear.

• Examples: price–demand curves (small price changes cause large demand shifts at certain points), drug dosage–response (diminishing effects at high doses).
• Common forms: polynomial, exponential, logarithmic models.

Exam tip: If the dependent variable is binary, always choose logistic regression over linear regression – even if other assumptions hold. The linear model will produce nonsense probabilities.

Key takeaways – Advanced Methods

• Logistic regression handles binary outcomes; models probability via a logistic curve.
• Non‑linear regression captures curvilinear relationships that linear models cannot fit.
• These methods preserve the interpretability and predictive power that linear regression offers, but under more realistic conditions.

Linear Regression for Two-Sample Comparison of Means

Intuition: Comparing the means of two independent groups (e.g., before/after a campaign, with/without an accident) is typically done with a two-sample t-test. The same test can be performed using linear regression with a binary dummy variable that encodes group membership. This may seem roundabout, but it pays off: once the comparison is embedded in a regression framework, we can easily add control variables, extend to multiple groups, and link to more advanced models like fixed effects and interaction effects.

Setup

• Dependent variable $Y$: continuous outcome (e.g., satisfaction level).
• Independent variable $X$: binary (0 or 1) indicating group membership.
  • $X=0$: reference group (e.g., no accident at work).
  • $X=1$: comparison group (e.g., had an accident).

Model:

$Y = \beta_0 + \beta_1 X + \varepsilon$

Interpretation of Coefficients

$X$ value	Expected $Y$	Interpretation
$0$	$\beta_0$	Mean of reference group
$1$	$\beta_0 + \beta_1$	Mean of comparison group
Difference	$\beta_1$	Mean difference (comparison group minus reference group)

Thus:

• Testing whether $\beta_1$ is significantly different from zero is equivalent to testing whether the two group means differ.
• The sign of $\beta_1$ indicates which group has the higher mean.

Worked Example: Accidents and Job Satisfaction

Question: Do employees who experienced an accident at work have a different average satisfaction level than those who did not?

Data: HR dataset containing satisfaction_level (continuous) and any_accident (1 = yes, 0 = no).

Simple linear regression output (from Excel):

Coefficient	Estimate	Std. Error	p-value
Intercept ($\beta_0$)	0.607	...	<0.05
any_accident ($\beta_1$)	0.041	0.006	<0.05

• $\beta_0 = 0.607$: mean satisfaction of employees without an accident.
• $\beta_1 = 0.041$: employees with an accident have a mean satisfaction that is 0.041 (≈4%) higher.
• The coefficient is statistically significant → the mean difference is real.

Exam tip: A regression with a single binary predictor produces exactly the same p‑value as an independent-samples t-test for the mean difference. The regression approach becomes superior when you need to control for additional variables.

Why This Result Might Be Surprising

The positive sign (accident → higher satisfaction) is counter‑intuitive. A possible explanation is confounding: employees who have accidents may also work in different departments, have different workloads, or be treated differently by the company. The simple regression cannot separate the effect of the accident from these other factors.

Controlling for Confounders: Multiple Regression

Add covariates that capture workload and experience:

• number_projects and average_monthly_hours (workload)
• years_at_company and promotion_last_5years (experience)

Model:

$\text{satisfaction} = \beta_0 + \beta_1 \text{accident} + \beta_2 \text{projects} + \beta_3 \text{hours} + \beta_4 \text{years} + \beta_5 \text{promotion} + \varepsilon$

Result: The coefficient for any_accident remains about 0.041 and statistically significant. Even after controlling for workload and experience, the positive relationship persists. This strengthens the evidence that the accident effect is genuine (perhaps due to company support policies), though further investigation is warranted.

Key advantage over a t-test: The regression framework naturally controls for multiple covariates, reducing omitted‑variable bias and yielding a more reliable estimate of the causal effect (under appropriate assumptions).

Connections to Advanced Topics

• Fixed effects: Using dummy variables for each individual (or group) to control for all time‑invariant unobservable characteristics. The regression shown is a simple fixed‑effects model at the group level (accident vs. no accident). In panel data, each subject gets its own dummy, isolating within‑subject variation.
• Interaction effects: The effect of the binary variable may differ across levels of another variable (e.g., salary). For example, an accident might lower satisfaction for low‑paid employees but not for high‑paid ones. This is modelled by adding an interaction term (e.g., accident × salary) to the regression.

Key Takeaways

• A simple linear regression with a binary dummy variable performs a two‑sample comparison of means.
• $\beta_0$ = mean of reference group; $\beta_1$ = mean difference (comparison minus reference).
• A significant $\beta_1$ indicates a statistically significant difference between groups.
• The sign of $\beta_1$ reveals which group has the higher mean.
• Multiple regression allows controlling for confounders, a major advantage over a standard t-test.
• Dummy variable regression is the foundation for fixed effects models and interaction effects.

From Two Samples to Multiple Groups

A linear regression model can compare means across more than two independent groups, just as it does for two samples. Intuitively: instead of running a one-way ANOVA, we regress the outcome on a set of binary (dummy) variables that encode group membership, and test whether the group differences are statistically significant.

This approach treats the outcome as a continuous variable (unlike a chi‑square test of independence, which would require binning the continuous outcome into categories). The regression model directly estimates group means and their differences.

Dummy Variable Setup for $k$ Groups

• Create $k$ binary columns, one per group (e.g., Low, Medium, High).
• Choose one group as the baseline (reference) category – omit its dummy from the model.
• The intercept $\beta_0$ estimates the mean of the baseline group.
• Each other coefficient $\beta_j$ estimates the difference between the mean of group $j$ and the baseline mean.

Example: Salary Level and Satisfaction

A dataset of 14,999 employees has salary categorised as Low, Medium, or High. The satisfaction level (0–1) is the dependent variable. To test whether mean satisfaction differs by salary group:

1. Create dummy variables: Low (=1 if salary=Low), Medium, High.
2. Set baseline: e.g., Low is the reference group – omit it from the model.
3. Run regression: satisfaction ~ Medium + High (plus intercept).

Variable	Coefficient	Interpretation
Intercept $\beta_0$	$\approx 0.600$	Mean satisfaction of low‑salary employees ≈ 0.600 (i.e., 60% satisfied).
Medium $\beta_1$	$+0.021$ (significant)	Medium‑salary employees are on average 2.1% more satisfied than low‑salary.
High $\beta_2$	$+0.037$ (significant)	High‑salary employees are on average 3.7% more satisfied than low‑salary.

Thus:

• Low salary mean = 60.0%
• Medium salary mean = 60.0% + 2.1% = 62.1%
• High salary mean = 60.0% + 3.7% = 63.7%

A coefficient that is significantly different from zero (via its $t$‑statistic / $p$‑value) indicates that the group mean differs from the baseline. The sign tells the direction.

Exam tip: Always set the baseline to a meaningful group (e.g., the most common or a control). The interpretation of the intercept and all other coefficients depends on that choice.

Key Takeaways

• Dummy variables convert categorical membership into numeric predictors.
• One category is always omitted – it becomes the baseline (intercept).
• Coefficients on the included dummies represent mean differences relative to baseline.
• A significant coefficient implies the group’s mean is statistically different from baseline.
• The same logic extends the two‑sample $t$‑test to multiple groups (equivalent to one‑way ANOVA).

---

Interaction Effects: Does Salary’s Impact Vary by Department?

The transcript asks: Do employees from different departments place equal emphasis on salary when reporting satisfaction? To answer this, we include an interaction term – a variable that captures the joint effect of two categorical predictors.

Intuition

A simple additive model assumes the salary effect is the same across departments. An interaction model allows the slope (difference between salary groups) to differ by department. For example, sales employees might value salary more than R&D employees, for whom job security matters more.

How to Build an Interaction Model (in Excel)

1. Create dummy variables for every combination of the two categorical variables.
   • Example: 10 departments × 3 salary levels = 30 dummy columns (e.g., Sales_Low, Sales_Medium, Sales_High, HR_Low, …).
2. Choose a baseline combination – e.g., HR_Low – and omit it from the regression.
3. Run the regression with all remaining 29 dummy variables as predictors.

Practical Concern – Sparse Cells

If a combination has very few observations (e.g., only 2 management employees with low salary), its coefficient will be unreliable. A recommended workaround: drop dummy columns for those sparse interactions. You will still capture the overall effect for that department but sacrifice the ability to estimate the interaction precisely.

Interpreting the Output

• Each interaction coefficient measures the difference in mean satisfaction for that specific (department, salary) combination relative to the baseline combination.
• Joint significance test (e.g., $F$‑test on all interaction dummies) tells whether including them significantly improves the model.
• If the interaction effects are significant: salary’s influence on satisfaction depends on department.
• If not: salary has a similar impact across all departments – the additive model suffices.

Exam tip: Interaction terms answer the question “Does the effect of X on Y depend on Z?” When Z is categorical, you need dummies for every X‑by‑Z cell. Always check for sparse cells; drop them to maintain reliable estimates.

Key Takeaways

• Interaction effects capture how the relationship between the dependent variable and one categorical predictor changes across levels of another categorical predictor.
• Implementation: create dummy variables for every combination of the two categories, then regress on those dummies (minus one baseline).
• Sparse cells (few observations) produce unreliable coefficients – consider dropping those dummy columns.
• Significant interactions imply the effect of salary differs by department; non‑significant interactions imply uniform effect.

Logistic Regression Model - I

Logistic regression is a technique for modeling the probability of a binary outcome (e.g., yes/no, success/failure, leave/stay) as a function of one or more independent variables.
It is the natural choice when the dependent variable is categorical – unlike linear regression, which assumes a continuous response.

Why linear regression fails for binary outcomes

• A binary outcome (coded $0$ or $1$) follows a Bernoulli distribution.
• Linear regression would model $P(Y=1) = \beta_0 + \beta_1 x$. Because a line is unbounded, predicted probabilities can fall outside $[0,1]$ – nonsensical for a probability.
• Logistic regression fixes this by applying the logistic (sigmoid) function, which maps any real-valued input to $(0,1)$.

flowchart LR
  A[Binary outcome Y ∈ {0,1}] --> B[Linear regression<br>predicts continuous values]
  B --> C[Predictions outside 0-1]
  A --> D[Logistic regression<br>models P(Y=1) via sigmoid]
  D --> E[Predictions always in 0-1]

The logistic (sigmoid) function

$P(Y=1) = \frac{1}{1 + e^{-(\beta_0 + \beta_1 x)}}$

• The curve is S‑shaped: for very low or very high $x$, the probability flattens near $0$ or $1$.
• The sign of $\beta_1$ determines the direction: positive → increasing probability as $x$ rises; negative → decreasing.

Foundation: Bernoulli distribution

• A binary variable $Y$ follows a Bernoulli distribution: $P(Y=1) = p$, $P(Y=0) = 1-p$.
• Logistic regression estimates $p$ as a function of the independent variables, ensuring $0 \le p \le 1$.

Odds and log odds

Odds of success:

$\text{Odds} = \frac{p}{1-p}$

• If $p=0.5$, odds $=1$ (equal chance).
• If $p>0.5$, odds $>1$; if $p<0.5$, odds $<1$.

Log odds (logit):

$\log\!\left(\frac{p}{1-p}\right)$

• Spans the entire real line: $-\infty$ when $p\to 0$, $+\infty$ when $p\to 1$.
• Logistic regression models log odds as a linear function of the independent variables:

$\log\!\left(\frac{p}{1-p}\right) = \beta_0 + \beta_1 x$

Exam tip: Linear regression models the mean directly; logistic regression models the log odds linearly. This linearity in log odds is what makes coefficients interpretable as changes in log odds per unit predictor increase.

---

Worked example: Instagram users by gender

Data: 1069 survey respondents (537 men, 532 women).
328 women and 234 men have Instagram accounts.

Gender	Users	Total	Proportion $p$	Odds $\frac{p}{1-p}$	Log odds $\ln(\text{odds})$
Women	328	532	$328/532 \approx 0.6165$	$0.6165/0.3835 \approx 1.608$	$\ln(1.608) \approx 0.475$
Men	234	537	$234/537 \approx 0.4358$	$0.4358/0.5642 \approx 0.772$	$\ln(0.772) \approx -0.258$

Define $x = 1$ for women, $x = 0$ for men.
Logistic regression assumes:

$\log\!\left(\frac{p}{1-p}\right) = \beta_0 + \beta_1 x$

Plugging in the log odds:

• For men ($x=0$): $\beta_0 = -0.258$
• For women ($x=1$): $\beta_0 + \beta_1 = 0.475 \;\Rightarrow\; \beta_1 = 0.475 - (-0.258) = 0.733$

Estimated model:

$\log\!\left(\frac{p}{1-p}\right) = -0.258 + 0.733\,x$

Convert back to probability:

$p = \frac{1}{1 + e^{-(-0.258 + 0.733\,x)}}$

• For women ($x=1$): $p = \frac{1}{1+e^{0.258-0.733}} = \frac{1}{1+e^{-0.475}} \approx 0.6165$ (matches data).
• For men ($x=0$): $p = \frac{1}{1+e^{0.258}} \approx 0.4358$.

Exam tip: The coefficient $\beta_1$ is the difference in log odds between women and men. A positive $\beta_1$ means higher log odds (and thus higher probability) for the group coded $1$.

---

Business applications

• Marketing: Predict whether a customer buys a product (based on demographics, browsing data).
• Credit scoring: Classify loan applicants as likely to default or repay.
• HR: Model employee attrition (e.g., probability of leaving given satisfaction level).

In all cases, logistic regression outputs a probability between 0 and 1, which can be used for classification with a chosen threshold (e.g., >0.5 → “will leave”).

---

Key takeaways

• Logistic regression models binary outcomes using the logistic (sigmoid) function to keep predicted probabilities in $(0,1)$.
• It is built on the Bernoulli distribution.
• Instead of predicting $Y$ directly, it models the log odds linearly: $\log(p/(1-p)) = \beta_0 + \beta_1 x$.
• The odds are $p/(1-p)$; log odds are the natural log of that ratio.
• Coefficients $\beta_1$ represent the change in log odds for a one‑unit increase in $x$.
• Linear regression fails because it can produce probabilities outside $[0,1]$.
• The sigmoid curve ensures valid probabilities: S‑shaped, flat near 0 and 1.

Logistic Regression Model - II

Logistic regression predicts the probability of a binary outcome (success/failure, quit/stay, buy/not buy). Unlike linear regression, it models the log odds of the event as a linear function of predictors, ensuring predictions stay between 0 and 1.

From linear combination to log odds

For multiple predictors $x_1, x_2, \dots, x_k$, the logistic regression equation expresses the log odds:

$\ln\left(\frac{P}{1-P}\right) = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \dots + \beta_k x_k$

• $P$ = probability the event occurs (e.g., employee leaves)
• $\beta_0$ = intercept (log odds when all $x$ = 0)
• $\beta_1, \beta_2, \dots$ = coefficients measuring the effect of each predictor on log odds

Using log odds solves the range problem: probability $P$ is bounded $[0,1]$, but log odds can take any real value $(-\infty, +\infty)$, so the linear model can fit without constraints.

The logistic (sigmoid) function

Rearranging the equation gives probability directly:

$P = \frac{e^{\beta_0 + \beta_1 x_1 + \dots + \beta_k x_k}}{1 + e^{\beta_0 + \beta_1 x_1 + \dots + \beta_k x_k}}$

This is the logistic function (or sigmoid). It produces an S‑shaped curve that smoothly increases from 0 to 1 as the linear combination changes.

Definition: The logistic function maps any real input to a probability between 0 and 1 — essential for binary classification.

Interpreting coefficients

• A positive coefficient $\beta_j$ means an increase in $x_j$ increases the log odds (and thus the probability) of the outcome.
• A negative coefficient means an increase in $x_j$ decreases the log odds.
• Because the relationship between $x_j$ and $P$ is non‑linear, the change in probability per unit change in $x_j$ is not constant — it depends on the current value of $x_j$. However, the direction (positive/negative) is fixed.

Exam tip: Logistic regression coefficients refer to log odds, not probability. To get the effect on odds, exponentiate the coefficient: $e^{\beta_j}$ gives the odds ratio.

Estimating coefficients: Maximum Likelihood Estimation (MLE)

Linear regression uses least squares; logistic regression uses maximum likelihood estimation (MLE).

Intuition: The model tries many possible coefficient values. For each set, it computes the predicted probability $P_i$ for every observation. Then it compares these probabilities to the actual binary outcomes (0 or 1). The goal is to find the coefficient values that make the observed data most likely — i.e., that maximize the likelihood of seeing the actual outcomes.

Procedure:

1. Start with initial guesses for $\beta$s.
2. Compute predicted probabilities via the logistic function.
3. Calculate the likelihood (a measure of how well predictions match real outcomes).
4. Adjust coefficients iteratively to increase the likelihood.
5. Stop when improvement becomes negligible — the maximum likelihood estimates.

flowchart LR
  A[Choose initial β] --> B[Compute predicted probabilities]
  B --> C[Calculate likelihood]
  C --> D{Maximum reached?}
  D -- No --> E[Update β]
  E --> B
  D -- Yes --> F[Final β estimates]

This iterative search can be performed in Excel using the Solver add‑in, by setting up the likelihood function and maximizing it.

Implementation in Excel

For a dataset with one binary outcome and predictors:

• Use the logistic function to compute predicted probabilities for each row.
• Construct the log‑likelihood formula (sum of log probabilities for observed outcomes).
• Run Solver to maximize the log‑likelihood by changing the coefficient cells.

Exam tip: Solver finds the same coefficients that statistical software (R, Python, SPSS) produces — it's a good way to see MLE in action without coding.

Key takeaways

• Logistic regression models the log odds of a binary outcome.
• The sigmoid function turns the linear combination into a probability $[0,1]$.
• Coefficients indicate direction (positive/negative) on log odds; the effect on probability is non‑linear.
• Maximum likelihood estimation iteratively finds the best coefficients by maximizing the probability of observing the data.
• In Excel, Solver can be used to perform MLE for small problems.

Intuition and Problem Setup

Logistic regression models a binary outcome (e.g., leave / stay) as a function of one or more predictors. Here the goal is to predict employee attrition (1 = left, 0 = stayed) from satisfaction level (0 to 1). The relationship is non‑linear: probability of leaving $P$ is linked to the predictors via the logistic function:

$P = \frac{1}{1 + e^{-(\beta_0 + \beta_1 x)}}$

Equivalently, the log‑odds (logit) of the event is linear:

$\log\!\left(\frac{P}{1-P}\right) = \beta_0 + \beta_1 x$

The coefficients $\beta_0, \beta_1$ are estimated by maximum likelihood – an iterative search that finds the values making the observed data most probable.

Step‑by‑Step Estimation in Excel

The data (columns A, B) contain the binary outcome (A) and satisfaction (B). The Excel implementation proceeds as follows.

1. Compute Log‑Odds

Place initial guesses for $\beta_0$ and $\beta_1$ in cells (e.g., H2, I2). For each row $i$:

$\text{log‑odds}_i = \beta_0 + \beta_1 \cdot x_i$

In Excel: = $H$2 + $I$2 * B2 (copied down).

2. Convert Log‑Odds → Odds → Probability

$\text{odds}_i = e^{\text{log‑odds}_i}$

Then the predicted probability depends on the actual outcome:

• If $y_i = 1$: $P_i = \frac{\text{odds}_i}{1 + \text{odds}_i}$
• If $y_i = 0$: $P_i = \frac{1}{1 + \text{odds}_i}$

Excel: = IF(A2=1, D2/(1+D2), 1/(1+D2)).

3. Log‑Likelihood

Because probabilities multiply across independent observations (product → very small), we work with logs:

$\ell_i = \ln(P_i)$

Sum all $\ell_i$ to get the total log‑likelihood (cell, e.g., J2).

4. Optimization with Solver

Goal: maximize the total log‑likelihood by changing $\beta_0$ and $\beta_1$.

• Open Solver (Data → Solver).
• Set Objective: cell containing sum of log‑likelihood.
• To: Max.
• By Changing Variable Cells: the $\beta_0$, $\beta_1$ cells.
• Select Solving Method: GRG Nonlinear.
• Click Solve.

Excel iterates through coefficient combinations and returns the maximum‑likelihood estimates.

flowchart LR
    A[Data: y, x] --> B[Log-odds = β₀ + β₁x]
    B --> C[Odds = exp(log-odds)]
    C --> D{Outcome y?}
    D -->|y=1| E[P = odds/(1+odds)]
    D -->|y=0| F[P = 1/(1+odds)]
    E --> G[Log-likelihood = ln(P)]
    F --> G
    G --> H[Sum log-likelihood]
    H --> I[Maximize with Solver]
    I --> J[MLE coefficients β₀, β₁]

Results and Interpretation

Single Variable Model

The solver yields:

Coefficient	Estimate
$\beta_0$ (intercept)	0.974
$\beta_1$ (satisfaction)	–3.832

Thus the estimated model:

$\log\!\left(\frac{\hat{P}}{1-\hat{P}}\right) = 0.974 - 3.832 \cdot \text{satisfaction}$

Interpretation:

• Sign of $\beta_1$ is negative → higher satisfaction lowers the odds of quitting.

• Magnitude cannot be directly read as a change in probability (non‑linear). Instead compute predicted probability at key satisfaction values:

  Satisfaction	Predicted probability of leaving
  0 (not at all satisfied)	$\approx 0.72$ (72 %)
  1 (fully satisfied)	$\approx 0.04–0.05$ (4–5 %)

• The decrease is non‑linear: probability drops steeply at low satisfaction and flattens at high satisfaction.

Multiple Variable Model (Extension)

The same Solver approach extends to several predictors. For a model including:

• satisfaction level
• last performance evaluation rating
• average monthly hours
• years at company
• promotion in last 5 years (coded 1/0)

Variable	Coefficient (approx.)	Sign	Interpretation
Satisfaction	–3.7	–	More satisfied → less likely to leave
Promotion in last 5 years	large negative	–	Promoted employees much less likely to leave
Years at company	positive (largest magnitude)	+	Longer tenure → higher chance of leaving (retirement / better offers)
Average monthly hours	positive	+	Higher workload → higher chance of leaving
Last evaluation rating	positive	+	Higher rating → more likely to leave (counterintuitive – possible reasons: felt under‑rewarded, better external opportunities)

Exam tip: The sign of a logistic coefficient tells the direction of effect on the log‑odds of the outcome. To express effect on probability, compute predicted $P$ at different $x$ values. Never interpret a coefficient as a linear change in probability.

Key takeaways

• Logistic regression models binary outcomes via the logit link: $\log(\frac{P}{1-P}) = \beta_0 + \beta_1 x$.
• Estimation uses maximum likelihood – iterative maximization of the sum of log‑likelihoods.
• In Excel: compute log‑odds → odds → probability (conditional on outcome) → log‑likelihood → maximize with Solver (GRG Nonlinear).
• Coefficient sign indicates direction: negative → predictor reduces odds of event.
• For multiple predictors, each coefficient is interpreted holding others constant; magnitude comparison gives relative importance.
• Counterintuitive signs (e.g., positive coefficient for performance rating) can reveal hidden dynamics – use domain knowledge to hypothesise.

Logistic Regression – Using Excel with Dummy Variables

When a categorical predictor (e.g., salary level) has ( k ) categories, it enters a logistic regression as ( k-1 ) dummy variables. One category is chosen as the baseline; the coefficients on the dummies measure the change in log‑odds relative to that baseline.

Setting Up Dummy Variables

• Salary has three categories: low, medium, high.
• Set low as baseline → create two dummies:
  • ( x_{\text{medium}} = 1 ) if medium salary, else 0
  • ( x_{\text{high}} = 1 ) if high salary, else 0

The logistic regression then includes these dummies alongside continuous predictors.

Model Specification

Eight independent variables are used:

Variable	Type	Description
satisfaction_level	continuous (0–1)	Employee satisfaction
last_evaluation	continuous (0–1)	Last performance rating
average_monthly_hours	continuous	Workload (hours/month)
years_at_company	integer	Tenure
promotion	binary (0/1)	Received promotion in last year?
salary_medium	dummy	1 if medium salary
salary_high	dummy	1 if high salary
(plus intercept)	—	( \beta_0 )

Parameters: ( \beta_0, \beta_1, \dots, \beta_7 ).
Log‑odds for employee ( i ):

[ \text{logit}(p_i) = \ln\left(\frac{p_i}{1-p_i}\right) = \beta_0 + \beta_1 x_{i1} + \cdots + \beta_7 x_{i7} ]

The model is fitted by maximizing the sum of log‑likelihoods (using Excel Solver).

Interpretation of Coefficients (Transcript‑Based)

The exact values are not given, but the sign and relative magnitude are discussed.

Variable	Sign	Interpretation
satisfaction_level	negative, large magnitude	Higher satisfaction → much less likely to leave
promotion	negative, large magnitude	Promotion → much less likely to leave
salary_medium	negative	vs low‑salary baseline: medium‑salary employees are less likely to leave
salary_high	negative, larger magnitude than medium	High‑salary employees are even less likely to leave than medium
years_at_company	positive, substantial magnitude	Longer tenure → more likely to leave (perhaps for better prospects)
last_evaluation	diminished after including salary/promotion	Becomes secondary once satisfaction and salary are controlled
average_monthly_hours	diminished	Similarly secondary

Key insight: Satisfaction, salary, and promotion dominate the prediction; evaluation and hours matter less once these are accounted for.

Worked Example: What‑If Analysis

Employee profile:

Variable	Value
satisfaction_level	0.5 (50%)
last_evaluation	0.7
average_monthly_hours	250
years_at_company	1
promotion	0
salary	low → ( x_{\text{medium}}=0,; x_{\text{high}}=0 )

Prediction (using previously estimated coefficients):

Compute log‑odds:

[ \begin{aligned} \text{log‑odds} &= \beta_0 + \beta_1(0.5) + \beta_2(0.7) + \beta_3(250) + \beta_4(1) \ &\quad + \beta_5(0) + \beta_6(0) + \beta_7(0) \ &= -1.075 \end{aligned} ]

Convert to probability:

[ p = \frac{e^{-1.075}}{1+e^{-1.075}} \approx 0.254 \quad (25.4%) ]

Scenario Testing

Change one variable at a time and recompute ( p ) (hypothetical outcomes from transcript):

Scenario	Change	New ( p )	Interpretation
Raise salary to medium	( x_{\text{medium}}=1 )	16.8%	Quit risk drops ◄
Raise salary to high	( x_{\text{high}}=1 )	4.7%	Quit risk very low ◄◄
Give promotion	( x_{\text{promotion}}=1 )	(substantial reduction)	Promotion also strongly reduces leaving

Business use: the HR department can estimate the impact of each intervention and choose the most cost‑effective strategy.

flowchart LR
  A[Employee profile] --> B[Logistic model]
  B --> C[Baseline probability 25.4%]
  C --> D{Intervention?}
  D --> E[Increase salary to medium]
  D --> F[Increase salary to high]
  D --> G[Give promotion]
  E --> H[16.8%]
  F --> I[4.7%]
  G --> J[<25.4%]

Extensions: Interaction Effects & Scenario Forecasting

• Interaction between salary level and department can reveal differential effects (e.g., a salary hike may retain HR staff more than management).
• This method is called scenario forecasting — the organisation tests multiple hypothetical changes and picks the one that best reduces turnover risk.

Exam tip: When building logistic models with categorical predictors, always create ( k-1 ) dummies. The baseline category (e.g., “low salary”) is absorbed into the intercept. Changing a dummy to 1 shifts the log‑odds relative to that baseline.

Key takeaways

• Categorical variables enter logistic regression as dummy variables (baseline omitted).
• Coefficients on dummies represent change in log‑odds compared to the baseline category.
• In the worked HR example: low → medium salary cut quit probability from 25.4% to 16.8%; low → high cut it to 4.7%.
• Satisfaction and promotion have large negative effects; years at company increases leaving propensity.
• Once key factors (salary, satisfaction, promotion) are controlled, variables like work hours and evaluation ratings lose predictive power.
• Use the fitted model for scenario forecasting: change one predictor at a time and compute the new probability to guide decisions.

The Curse of Dummy Variables

Fixed effects and interaction effects quickly explode the number of features. In the employee‑retention example:

• Salary – 3 categories → 2 dummy variables
• Department – 10 categories → 9 dummy variables
• Salary × Department interaction – 30 unique combinations → 29 dummy variables

Total: over 30 dummy columns. Estimation becomes unstable, convergence may fail, and interpretation suffers.

Source	Categories	Dummy variables needed
Salary (fixed)	3	2
Department (fixed)	10	9
Salary × Department (interaction)	30	29
Total		≥ 30

One partial remedy: drop interaction terms for categories with very few observations (e.g., if a department has only low‑salary employees, ignore the other salary‑department combos). But even then the model may stay too complex.

Exam tip: Whenever a categorical feature has many levels, or you include interactions, always check the total dummy count. A model with >20–30 dummy variables on a modest dataset is a red flag for overfitting and non‑convergence.

Why Variable Selection Matters

Variable selection is the process of identifying the subset of features that are most important for predicting the outcome, keeping the model simple without sacrificing predictive or explanatory power.

Three core reasons:

1. Avoid overfitting – Adding more features always inflates $R^2$ (or pseudo‑$R^2$), eventually approaching 1 on training data. But an $R^2$ close to 1 signals that the model fits noise, not signal – it will generalise poorly to new data.
2. Interpretability – A model with 5–10 coefficients is far easier to understand than one with 50.
3. Computational efficiency – Fewer variables means faster training and lower memory use.

Forward Selection

Start with an empty model (only the intercept). At each step:

1. Test every candidate variable not yet in the model.
2. Add the one that most improves model fit (e.g., highest $R^2$ increase).
3. Use an F‑test (or deviance test in logistic regression) to check whether the improvement is significant.
4. Repeat until no remaining variable yields a significant improvement.

flowchart LR
    A[Empty model: intercept only] --> B{Test each candidate}
    B --> C[Add variable with best improvement]
    C --> D{Significant?}
    D -->|Yes| B
    D -->|No| E[Stop, keep current model]

Backward Elimination

Start with the full model (all variables). At each step:

1. Identify the least important variable (e.g., highest $p$‑value, smallest $R^2$ drop if removed).
2. Remove it.
3. Repeat until removing any remaining variable significantly hurts model fit.

flowchart LR
    A[Full model: all variables] --> B{Find least important}
    B --> C[Remove it]
    C --> D{Removal harms model?}
    D -->|No| B
    D -->|Yes| E[Stop, keep current model]

Stepwise Selection

A hybrid that combines forward and backward:

• Begin like forward selection: add the best variable.
• At every step, re‑evaluate all previously included variables. If any has become non‑significant (because the new variable took over its role), remove it.
• Continue until no variable can be added and none needs removal.

This handles the problem that some variables matter only in the presence of others – a variable that was significant at step 2 may become redundant after a later addition.

Ad‑hoc P‑Value Approach

A simpler, more manual method:

1. Fit the full model once.
2. Examine the $p$‑values of each coefficient.
3. Remove all variables with non‑significant $p$‑values.
4. (Optional) Refit and re‑check – significance can shift when variables are removed.

Caution: Because coefficients interact, a variable may appear non‑significant in the full model but become significant after dropping a correlated predictor. Always re‑evaluate after pruning.

Exam tip: The ad‑hoc p‑value method is fast but risky. It is acceptable only when you verify stability by fitting a few reduced models. The stepwise approach is more robust because it continuously checks for both additions and removals.

Key Takeaways

• Fixed/interaction effects with many categories generate dozens of dummy variables, causing overfitting, convergence issues, and poor interpretability.
• Variable selection finds a balance between model simplicity and accuracy.
• Common approaches: forward selection (add one by one), backward elimination (remove one by one), stepwise selection (add then re‑check removals), and ad‑hoc p‑value (prune non‑significant predictors).
• Always watch for overfitting: more features inflate $R^2$, but hurt generalisation.
• In practice, use software‑built routines; but understand the logic to set sensible stopping criteria (e.g., significance level for entry/removal).

Non-Linear Regression Model

Non-linear regression models relationships where the change in the dependent variable is not proportional to the change in the independent variable. Unlike linear regression, it can capture curved, S‑shaped, or saturating patterns. This flexibility is essential for many real-world business problems where the underlying dynamics are inherently complex.

Why non‑linear? Common business applications

• Advertising & sales – Early ad spend may boost sales sharply, but additional spend eventually yields diminishing returns. Non‑linear models capture this plateau.
• Pricing & demand – Lowering price may initially spike demand, but further cuts produce only marginal gains. Complex price–response curves are better fitted with non‑linear terms.
• Growth models – Revenue or product adoption often follows an S‑shaped (sigmoid) pattern: rapid early growth, then slowdown as market saturates.
• Manufacturing & cost – Output costs may fall due to economies of scale, then rise again from inefficiencies. A non‑linear model can identify the optimal production level.

Key non‑linear regression methods

Polynomial regression

Generalizes linear regression by adding higher‑degree terms of the predictor(s). A quadratic (2nd‑degree) model with one predictor $x$ is:

$$y = \beta_0 + \beta_1 x + \beta_2 x^2$$

Adding a cubic term ($\beta_3 x^3$) yields a cubic polynomial.

When to use – When the dependent variable rises/falls sharply at certain points and flattens elsewhere (e.g., product demand vs. price).
Caution – Higher degrees increase complexity and risk overfitting; quadratic or cubic usually suffice.

Exponential regression

Models rapid acceleration (or deceleration) where the dependent variable changes at a rate proportional to its current value:

$$y = \beta_0 e^{\beta_1 x}$$

When to use – Viral campaign spread, rumor propagation, or any scenario where growth accelerates quickly.

Logarithmic regression

Models quick initial growth that then levels off – the mirror of exponential:

$$y = \beta_0 + \beta_1 \ln(x)$$

When to use – Situations with diminishing returns, e.g., advertising spend vs. brand awareness.

Power regression

Models a relationship where the rate of change in $y$ is a constant proportion of the change in $x$:

$$y = \beta_0 x^{\beta_1}$$

When to use – Production efficiency vs. units produced (efficiency improves at a decreasing rate as scale increases).

Logistic regression (non‑linear form)

Uses the sigmoid (logistic) function to produce an S‑shaped curve:

$$y = \frac{1}{1 + e^{-(\beta_0 + \beta_1 x)}}$$

(Here $y$ can be a continuous proportion or a binary probability.)
When to use – New product adoption, market penetration, or any process that saturates.

Choosing the right method

flowchart TD
    A[Observe pattern of relationship] --> B{Is it linear?}
    B -->|Yes| C[Use linear regression]
    B -->|No| D[Identify shape]
    D -->|Diminishing returns, levels off| E[Logarithmic]
    D -->|Rapid acceleration / deceleration| F[Exponential]
    D -->|S‑shaped / saturation| G[Logistic]
    D -->|Parabolic / curved peak or trough| H[Polynomial (quadratic/cubic)]
    D -->|Constant proportional rate| I[Power]

Comparison of methods

Method	Pattern captured	Typical equation	Example use
Polynomial (quadratic)	Curved, single bend	$y = \beta_0 + \beta_1 x + \beta_2 x^2$	Price vs. demand
Exponential	Rapid growth/decay	$y = \beta_0 e^{\beta_1 x}$	Viral campaign spread
Logarithmic	Fast initial growth, then flattening	$y = \beta_0 + \beta_1 \ln x$	Ad spend vs. sales
Power	Constant proportion rate change	$y = \beta_0 x^{\beta_1}$	Production efficiency scaling
Logistic	S‑shaped saturation	$y = \frac{1}{1 + e^{-(\beta_0+\beta_1 x)}}$	Product adoption

Exam tip – Overfitting is a real danger with polynomial regression. Stick to quadratic or cubic unless domain knowledge strongly suggests a higher degree. Cross‑validate to check generalisability.

Key takeaways

• Non‑linear regression models curved, non‑proportional relationships between variables.
• Common methods: polynomial (parabolic), exponential (rapid change), logarithmic (diminishing returns), power (constant proportion), logistic (S‑shaped).
• Choice depends on the observed pattern in the data – mis‑specification leads to poor predictions.
• Applications include advertising response, pricing optimisation, growth modelling, and manufacturing cost analysis.
• Higher‑degree polynomials increase flexibility but also the risk of overfitting; keep models as simple as the data allow.

Case Study: Bangalore House Prices

A case study integrating nonlinear regression and categorical-variable techniques to model house prices in Bangalore. The dataset contains 4340 rows and 10 variables.

Data Overview

Variable	Type	Description
posted_by	Categorical	Owner, dealer, or builder
RERA_approval	Binary	1 if approved, 0 otherwise
bedrooms	Numeric	Number of bedrooms
total_sqft	Numeric	Total area in square feet
ready_to_move	Binary	1 if ready to occupy, 0 otherwise
resale	Binary	1 if resale property, 0 otherwise
address	Categorical (high-level)	General location
longitude	Numeric	Longitude coordinate
latitude	Numeric	Latitude coordinate
price	Numeric (lakhs of ₹)	House price

The response variable of interest is price, but due to its relationship with area and skewness, a transformation is needed.

Five Key Questions

The case study builds toward one main question through four preparatory ones:

1. How is price distributed across the city? (exploratory mapping)
2. Does price depend on specific streets/areas? (pattern identification)
3. Is the price distribution normal? (normality check for regression)
4. Does price differ by who posted (builder, owner, dealer)?
5. What factors impact price, and how? (main modelling question)

These questions are cumulative: Q1–Q4 inform the model specification for Q5.

Exploratory Data Analysis (EDA)

Price Distribution and Normality

A histogram of price shows positive skew: high frequency at low prices, a sharp drop, then a long right tail with few observations. The distribution is clearly not normal.

Plotting price per square foot ($\text{price} / \text{total\_sqft}$) reduces the effect of area but still shows positive skew – not symmetric.

Exam tip: In real data, variables like price, income, and rent almost always exhibit positive skew. The standard remedy is a log transformation.

Applying $\log(\text{price per sq ft})$ yields a near-symmetric histogram (some outliers on the left remain). Visually, the normality assumption becomes plausible. Formal confirmation would require a chi-square goodness-of-fit test.

Why Price per Sq Ft and Log Transformation

• Price per sq ft is the relevant metric for buyers because total price is proportional to area. A buyer compares cost per unit area.
• Log transformation is the appropriate fix for positive skew. It stabilises variance and makes the distribution more symmetric – both important for regression assumptions.

Nonlinear Regression Model

Model Specification

The regression is nonlinear because the response variable is transformed:

$$\text{Dependent variable} = \log(\text{price per sq ft})$$

Independent variables (predictors):

• RERA_approval (binary)
• bedrooms (numeric – initially treated as continuous)
• ready_to_move (binary)
• resale (binary)
• total_sqft (numeric)
• posted_by (categorical: owner, dealer, builder) – converted to dummy variables with builder as the baseline

This is a log-linear model: response is log-transformed, predictors remain untransformed.

Results and Interpretation

Predictor	Sign	Significance (5% level)	Interpretation
RERA_approval	Positive	Significant	Approval increases price per sq ft
bedrooms	Positive	Significant	More bedrooms → higher price per sq ft (holding area fixed)
total_sqft	Negative	Significant	Larger area → lower price per sq ft (economies of scale)
ready_to_move	—	Not significant	No price premium for readiness
resale	—	Not significant	Resale properties do not sell at a discount or premium
posted_by = owner	Positive	Significant	Higher price per sq ft than builder
posted_by = dealer	Positive (larger)	Significant	Even higher premium over builder

Exam tip: In a log-linear model, coefficients on binary predictors represent the expected change in $\log(Y)$; exponentiate to get the multiplicative effect on $Y$ (e.g., $e^{\beta}$ gives the factor change in price per sq ft).

Because the dependent variable is logged, each coefficient indicates the change in $\log(\text{price per sq ft})$ for a one-unit increase in the predictor. For posted_by, both owner and dealer command a premium relative to builder, with dealer being the costliest.

Model Improvements and Further Analysis

The lecture identifies several ways to upgrade the model:

1. Log-log model: Transform both response and total_sqft – i.e., model $\log(\text{price per sq ft})$ as a function of $\log(\text{total\_sqft})$. This captures proportional relationships better.
2. Bedrooms as categorical: The price increment per additional bedroom is likely nonlinear. Treating bedrooms as a categorical variable (1BR, 2BR, 3BR+) or applying a transformation (log, square root) may improve fit.
3. Interaction effects: resale and posted_by may interact – an owner selling a resale property might have different pricing than a builder selling a new one.
4. Location effect: Address/lat–lon data introduces many categories. Smart encoding (e.g., clustering neighbourhoods, using spatial coordinates, or creating a distance-from-center variable) is needed to avoid an explosion of dummy variables.
5. Cross-validation for prediction: To compare models on predictive performance (not just $R^2$), split the data (e.g., 4000 training, 340 test). Train on one part, predict the other, and evaluate accuracy. This concept is explored further in the next module on time series and predictive modelling.

Key takeaways

• The case study demonstrates a complete workflow: EDA → transformation → nonlinear regression → interpretation → refinement.
• Log transformation is essential for right-skewed price data; log-linear models are a standard nonlinear regression tool.
• Categorical predictors require dummy variables; interpretation of coefficients changes when the response is logged.
• Model improvement avenues include log-log form, categorical treatment of numeric variables, interactions, and spatial effects.
• Cross-validation separates model fitting from evaluation – critical for predictive modelling.

Linear Regression Recap

Linear regression models the relationship between a continuous dependent variable and one or more independent variables. The goal is to quantify the strength and direction of these relationships: how changes in a predictor translate into increases or decreases in the outcome.

Because of its simplicity and interpretability, linear regression is widely used when the relationship between variables can be assumed linear. For example, a company can model the relationship between various factors (salary, workload, work environment) and employee satisfaction to identify what keeps employees happy.

Exam tip: Linear regression predicts a continuous outcome. The coefficients are directly interpretable: a one-unit increase in $X$ is associated with a $\beta$ change in $Y$, holding other predictors constant.

Key takeaways

• Linear regression models continuous dependent variables.
• Assumes a straight-line relationship.
• Provides clear, actionable insights in business contexts.
• Example: employee satisfaction analysis.

Logistic Regression Recap

Logistic regression is used when the dependent variable is binary (two possible outcomes). It models the probability of an event occurring, ensuring predicted values fall between 0 and 1.

Unlike linear regression, logistic regression uses the logit link function:

$\text{logit}(p) = \ln\left(\frac{p}{1-p}\right) = \beta_0 + \beta_1 X_1 + \dots + \beta_k X_k$

where $p$ is the probability of the event.

Business applications

• Employee attrition: Predict likelihood of an employee leaving based on job satisfaction, salary, workload, tenure. Enables proactive retention (promotions, salary increases, development).
• Customer segmentation & marketing: Predict whether a customer will purchase based on browsing behavior, purchase history, demographics. Helps target campaigns.
• Credit scoring & risk management: Predict probability of loan default using credit history, income, employment status. Informs approval/denial decisions.

Key takeaways

• Logistic regression for binary outcomes (0/1).
• Predicted values are probabilities bounded by 0 and 1.
• Used in HR analytics, marketing, financial risk.

Non‑Linear Regression Recap

Non‑linear regression is needed when the relationship between variables cannot be adequately captured by a straight line. Many real‑world business situations show curved patterns.

Example: House price analysis – the dependent variable (price) may not be normally distributed. Applying a logarithmic transformation (e.g., $\ln(\text{price})$) creates a non‑linear model that often improves performance.

Common scenarios

• Diminishing returns in marketing: Sales increase with ad spend, but after a point each additional dollar yields smaller sales increments. Non‑linear models capture this saturation effect, helping allocate budgets efficiently.
• Demand forecasting: The relationship between price and demand often follows a curve – small price changes may have large effects on demand in some ranges, little effect in others.

Exam tip: Non‑linear does not necessarily mean “complicated.” A log transformation of either the dependent or independent variable is a simple form of non‑linear regression that can linearise a curved relationship.

Key takeaways

• Non‑linear regression for curved relationships.
• Logarithmic transformations are a common tool.
• Business examples: marketing diminishing returns, price–demand curves.

Categorical predictors

Many business variables are categorical – gender, region, product type. These are incorporated into linear, logistic, or non‑linear regression using dummy variables (0/1 coding). This allows comparison of the impact of different categories on the outcome.

Interaction effects

Interaction effects occur when the relationship between one independent variable and the dependent variable changes depending on the value of another independent variable. For example:

• The effect of advertising on sales may depend on the pricing strategy.
• The impact of employee training on performance may vary by department.

Interaction terms are added to the regression model, often as the product of the involved variables (e.g., $X_1 \times X_2$). Including them leads to a more precise model that captures how combinations of factors influence outcomes.

flowchart LR
  A[Factor 1] --> C[Outcome]
  B[Factor 2] --> C
  A -.->|moderates| B
  B -.->|moderates| A
  A & B --> D[Interaction Term]
  D --> C

Key takeaways

• Dummy variables enable categorical predictors in regression.
• Interaction effects capture dependency between predictors.
• Including interactions improves model accuracy for business decisions (e.g., attrition, house prices).

Parametric and Nonparametric Methods

Statistical methods answer questions about populations using sample data: summarizing, testing relationships, making predictions. The choice between parametric and nonparametric methods depends entirely on what we assume about the underlying data distribution.

What is a parametric method?

The term comes from parameters — numbers that summarize a population (e.g., population mean $\mu$, population proportion $p$, population variance $\sigma^2$). Parametric methods assume the data follow a specific probability distribution completely defined by these parameters.

• Example: Binary data → Bernoulli distribution, defined by $p$.
• Example: Continuous data → Normal distribution, defined by $\mu$ and $\sigma^2$.

Key idea: Parametric methods require a “blueprint” — the data must come from a known distribution (e.g., normality for a $t$-test). When the assumption holds, these methods are efficient and powerful.

What is a nonparametric method?

Nonparametric methods make no assumption about the underlying distribution. They are also called distribution-free methods. They work with ranks, relative positions, or counts rather than raw values and parameters.

• Data can be skewed, multimodal, have outliers — nonparametric methods remain valid.
• They are more flexible and robust when assumptions fail.

Core problems addressed by nonparametric methods in this module

1. Independence between two variables (e.g., product type vs. satisfaction).
2. Goodness of fit — does the data follow a specified distribution?
3. Two-sample distribution comparison — do two samples come from the same distribution?

---

Worked example: Customer satisfaction scores (Product A vs. Product B)

A company collects satisfaction scores (1–10) for two products. The goal: determine whether a significant difference exists.

Parametric approach (two-sample $t$-test)

• Null hypothesis: $\mu_A = \mu_B$ (population means equal).
• Alternative: $\mu_A \neq \mu_B$.
• Assumptions: scores are normally distributed in both groups; often equal variances assumed.
• Output: estimate of mean difference, $p$-value, confidence interval.

Nonparametric approach (Mann‑Whitney U test / Wilcoxon rank sum test)

• Null hypothesis: No difference in the distribution of scores between groups.
• Alternative: distributions differ.
• Method: Ranks all observations together; uses relative ranks to compute test statistic.
• No normality assumption; robust to skewness, outliers, bimodality.
• Output: a $p$-value indicating whether one group tends to have higher/lower scores, but no direct estimate of effect size (e.g., mean difference).

---

Comparison: parametric vs. nonparametric

Aspect	Parametric	Nonparametric
Assumptions	Data follows a specific distribution (e.g., normal)	None (distribution‑free)
Parameters estimated	$\mu, \sigma^2, p$, etc.	Not directly; uses ranks
Efficiency (given assumptions hold)	High – fewer data needed, more powerful	Lower – larger samples needed for same precision
Robustness to violations	Low – results can be misleading	High – valid even with skewness, outliers
Interpretation	Direct: mean difference, regression coefficients	Indirect: only direction or existence of difference
Example test	Two‑sample $t$-test	Mann‑Whitney U test

flowchart TD
    A[Data collected] --> B{Can we assume normality?}
    B -->|Yes| C[Use parametric method - e.g., t-test]
    B -->|No / Unsure| D[Use nonparametric - e.g., Wilcoxon rank sum test]
    C --> E{Check assumptions?}
    E -->|Violated| D

Exam tip: Parametric tests are more powerful only when their assumptions are met. If data is skewed or has outliers, a nonparametric test is the safer choice — it may require a larger sample but avoids invalid conclusions.

Advantages and disadvantages

Parametric

• ✅ Simpler, fewer parameters to estimate.
• ✅ Powerful and precise when assumptions hold.
• ❌ Results can be completely wrong if assumptions violated.

Nonparametric

• ✅ Flexible, works with any distribution.
• ✅ Robust to outliers and skewed data.
• ❌ Requires larger sample sizes for equivalent power.
• ❌ Provides less detailed inference (e.g., no mean difference).

Practical workflow

1. Start with parametric methods if assumptions seem reasonable.
2. Always check assumptions (e.g., using goodness‑of‑fit tests, Q‑Q plots).
3. If assumptions are violated, switch to a nonparametric alternative.

Key takeaways

• Parametric methods assume a known distribution (e.g., normal) and estimate population parameters.
• Nonparametric methods (distribution‑free) use ranks or counts and require no distributional assumption.
• Same goal — hypothesis testing or inference — but different trade‑offs: efficiency vs. robustness.
• The Mann‑Whitney U test is the nonparametric counterpart to the two‑sample $t$-test.
• Nonparametric tests often cannot quantify effect size beyond “different or not”.

Intuition and Definition

Independence between two events means the occurrence or non‑occurrence of one does not influence the occurrence of the other. Intuitively: knowing whether one event happened gives zero information about whether the other happened.
Example: Tossing two coins — the outcome of the first toss tells you nothing about the outcome of the second.

Formal definition – events $A$ and $B$ are independent iff
$P(A \cap B) = P(A) \cdot P(B)$
This is the multiplication rule for independent events. If the equality holds, the chance that both events occur is simply the product of their individual probabilities; no “adjustment” for influence.

Business Example: Shoe and Phone Case

Retail manager analyzing shopping behaviour:

• Event A: customer buys a particular brand of shoes
• Event B: customer buys a mobile phone case

Suppose $P(A) = 0.3$ and $P(B) = 0.2$.
If the two purchases are independent, then
$P(A \cap B) = 0.3 \times 0.2 = 0.06$
i.e., about 6% of customers would buy both.

Dependence can be positive or negative:

• Positive: customers who buy shoes are more likely to buy socks → $P(A \cap B) > P(A)P(B)$
• Negative: customers who buy shoes are less likely to buy slippers → $P(A \cap B) < P(A)P(B)$

For any two dependent events, the multiplication rule does not hold.

Dependent Events: Age and Gaming Console

• Event A: customer aged 18‑25
• Event B: customer purchases a gaming console

Intuition says younger people are more likely to buy a console → the events are dependent.
In this case:
$P(A \cap B) > P(A)P(B)$
A larger joint probability than the independence assumption would predict.

Independent Random Variables

Independence extends naturally to random variables $X$ and $Y$:
$X$ and $Y$ are independent iff for every pair of values $(x, y)$,
$P(X = x,\; Y = y) = P(X = x) \cdot P(Y = y)$
Knowing the value of $X$ gives no information about $Y$ (and vice versa).

Business context – an online platform tracks two variables per customer per month:

• $X$ = number of digital products purchased
• $Y$ = number of physical goods purchased

If $X$ and $Y$ are independent, marketing strategies for digital and physical products can be developed separately.

Testing Independence with Data: Toy Example

The company samples 100 customers and observes:

Marginal distribution of $X$ (digital products):

$x$	Frequency	$P(X=x)$
0	10	0.10
1	50	0.50
2	30	0.30
3	10	0.10
Sum	100	1.00

Marginal distribution of $Y$ (physical goods):

$y$	Frequency	$P(Y=y)$
0	5	0.05
1	40	0.40
2	40	0.40
3	15	0.15
Sum	100	1.00

If $X$ and $Y$ are independent, the expected joint probability for any combination is the product of the marginals:

• $P(X=0, Y=0) = 0.10 \times 0.05 = 0.005$ → expected 0.5 customers
• $P(X=1, Y=1) = 0.50 \times 0.40 = 0.20$ → expected 20 customers

Similarly, the full independence‑implied joint distribution would be computed for all $(x,y)$ pairs.

Key insight: In real data, even when independence holds, the observed counts will never exactly match the products due to sampling variability. A non‑parametric test (e.g., chi‑square test of independence) determines whether the deviation is statistically significant or just random noise.

In the toy example, we expect 0.5 customers with $X=0,Y=0$, but we could observe 1 or 0. If observed = 1, the difference is small relative to sampling error; if observed = 10, it would strongly suggest dependence.

The goal: detect whether, for all $(x,y)$,
$P(X=x,Y=y) \approx P(X=x)\,P(Y=y)$
“Approximately” because sampling variation is accounted for.

Key takeaways

• Independence means $P(A \cap B) = P(A)P(B)$; knowing one event gives no info about the other.
• For random variables: equality of joint and product of marginals must hold for all values.
• Dependence can be positive (increases joint probability) or negative (decreases it).
• In practice, we compare observed joint frequencies to the product of marginal frequencies; sampling variability prevents exact match.
• A non‑parametric method (covered next) tests whether the gap is statistically significant.

Chi-square Test of Independence

A chi-square test of independence determines whether two categorical variables are associated. Intuitively, if the variables are independent, the observed pattern in a contingency table should match what we expect by chance; large mismatches signal dependence.

Hypotheses

• Null $H_0$: The two variables are independent.
• Alternative $H_1$: The two variables are not independent (dependent).

Contingency Table (Observed Frequencies)

Data are arranged in an $r \times c$ table, where $r$ = number of rows (categories of variable 1) and $c$ = number of columns (categories of variable 2). Each cell contains the observed frequency $O_{ij}$.

Expected Frequencies

Under $H_0$, the expected frequency for cell $(i,j)$ is derived from the multiplication rule of probability:

$E_{ij} = \frac{(\text{Row } i \text{ total}) \times (\text{Column } j \text{ total})}{\text{Grand total } n}$

Test Statistic

The chi-square test statistic measures total squared deviation between observed and expected, standardized by expected:

$\chi^2 = \sum_{i=1}^{r} \sum_{j=1}^{c} \frac{(O_{ij} - E_{ij})^2}{E_{ij}}$

Degrees of Freedom

Under $H_0$, $\chi^2$ follows a chi-square distribution with:

$df = (r-1)(c-1)$

Decision via p-value

Compute the p-value = $P(\chi^2_{df} \geq \text{calculated } \chi^2)$. If p-value $< \alpha$ (commonly 0.05), reject $H_0$ and conclude the variables are dependent.

---

Worked Example: Bloomberg Businessweek

A survey asked business travelers about the type of ticket (first class, business class, economy) and trip type (domestic, international).

Observed frequencies (partial):

Ticket type	Domestic	International	Row total
First class	29	22	51
Others (business, economy)	not specified	not specified	(remainder)
Column total	642	?	n = 920

(Only the first‑class row was explicitly given; the domestic column total is 642.)

Expected frequency for first class / domestic under independence:

$E_{11} = \frac{51 \times 642}{920} \approx 35.59$

Contribution of this cell to $\chi^2$:

$\frac{(29 - 35.59)^2}{35.59} \approx 1.22$

Summing over all six cells gives:

$\chi^2 = 100.43$

Degrees of freedom: $df = (3-1)(2-1) = 2$.

p-value: Using Excel’s CHISQ.DIST.RT(100.43, 2), the p-value is approximately $10^{-22}$ (essentially 0).

Conclusion: Since p-value $< 0.05$, reject $H_0$. The type of ticket purchased is not independent of the type of trip; a significant association exists.

---

flowchart LR
  A[Observed & Expected frequencies] --> B[Compute χ² = Σ (O-E)²/E]
  B --> C[Compare to χ² distribution with df = (r-1)(c-1)]
  C --> D{p-value < α?}
  D -->|Yes| E[Reject H₀: variables are dependent]
  D -->|No| F[Fail to reject H₀: no evidence of dependence]

Exam tip: The test requires that expected frequencies are at least 5 for most cells (commonly 80% of cells). If violated, combine categories or use Fisher’s exact test. The test only detects existence of association, not strength or direction.

Key takeaways

• Chi-square test of independence assesses association between two categorical variables.
• $H_0$: independence; $H_1$: dependence.
• Expected frequency per cell = (row total × column total) / grand total.
• Test statistic: $\chi^2 = \sum (O-E)^2/E$.
• Degrees of freedom = $(r-1)(c-1)$.
• A small p-value (e.g., < 0.05) leads to rejection of independence.

Chi-Square Test of Independence (Categorical Variables)

The chi-square test of independence determines whether two categorical variables are related or independent. Intuitively, if the variables are independent, the distribution of one variable should look roughly the same across all levels of the other variable. If they are dependent, some categories will have systematically different proportions.

Example: Department vs. Salary in an HR Dataset

• Dataset: 14,999 employees, 10 variables. Focus on two categorical variables:
  • Department (10 categories): Sales, Accounting, HR, Technical, Support, Management, IT, Product Management, Marketing, R&D.
  • Salary (3 categories): Low, Medium, High.
• Research question: Is salary independent of department? (Equivalently: Does salary distribution vary across departments?)

Hypotheses

• $H_0$: Department and salary are independent.
• $H_1$: Department and salary are not independent (i.e., there is an association).

Steps of the Test

Step 1 – Contingency Table of Observed Frequencies

Create a two-way table using COUNTIFS in Excel. Rows = departments, columns = salary levels.

Department	Low	Medium	High	Row total
Sales	2099	...	...	7316
Accounting	358	...	...	...
HR	...	...	...	...
Technical	...	...	...	...
Support	...	...	...	...
Management	...	...	...	...
IT	...	...	...	...
Product Mgmt	...	...	...	...
Marketing	...	...	...	...
R&D	...	...	...	...
Column total	4140	...	...	14999

(Exact counts not fully listed in the lecture; the computed χ² uses full 30-cell table.)

Step 2 – Expected Frequencies under $H_0$

Under independence, the expected count for cell $(i,j)$ is:

$$E_{ij} = \frac{(\text{Row total}_i) \times (\text{Column total}_j)}{N}$$

Example for Sales × Low:

$$E_{\text{Sales,Low}} = \frac{7316 \times 4140}{14999} \approx 2019.35$$

Compute all expected frequencies; verify that row and column totals match the observed marginals.

Step 3 – Chi-Square Test Statistic

$$\chi^2 = \sum_{i=1}^{r} \sum_{j=1}^{c} \frac{(O_{ij} - E_{ij})^2}{E_{ij}}$$

For Sales × Low:

$$\frac{(2099 - 2019.35)^2}{2019.35} \approx 3.14$$

Summing over all 30 cells yields:

$$\chi^2 = 700.92$$

Step 4 – Degrees of Freedom

$$df = (r-1)(c-1) = (10-1)(3-1) = 18$$

Step 5 – p-value

The p-value is the right-tail probability of obtaining a $\chi^2$ value at least as extreme as 700.92 under a $\chi^2_{18}$ distribution.

In Excel: =CHISQ.DIST.RT(700.92, 18) → $p \approx 1 \times 10^{-137}$ (essentially zero).

Step 6 – Conclusion

Since the p-value is far below any common significance level (e.g., 0.05), we reject $H_0$. There is strong evidence that salary and department are not independent – salary distributions differ across departments.

Understanding which categories drive the association

The largest contributions to $\chi^2$ come from the Management department:

• Deviations: (3 values) sum to ≈ 636.77 out of 700.92.
• This suggests that Management’s salary pattern is markedly different from the average.

Further analysis could test independence after removing Management (or Management, Sales, HR, Support) to see if the remaining departments show independence.

flowchart TD
    A[State H₀: Independence] --> B[Build contingency table of observed counts]
    B --> C[Compute expected counts under H₀]
    C --> D[Calculate χ² statistic]
    D --> E[Find degrees of freedom (r-1)(c-1)]
    E --> F[Obtain p-value (right-tail)]
    F --> G{ p < α? }
    G -->|Yes| H[Reject H₀ → Variables are dependent]
    G -->|No| I[Fail to reject H₀ → No evidence of association]

Exam tip: The chi-square test only tells you whether there is an association, not how strong it is. Large cell contributions (like Management’s) can hint at which categories differ, but follow-up tests are needed to confirm.

Key Takeaways

• The chi-square test of independence compares observed frequencies to expected frequencies under independence.
• Expected frequencies are calculated using the product of marginal totals divided by the total sample size.
• Test statistic: $\chi^2 = \sum \frac{(O - E)^2}{E}$.
• Degrees of freedom = $(\#\text{rows} - 1) \times (\#\text{cols} - 1)$.
• A very small p-value leads to rejection of independence.
• The test is non-parametric – it makes no distributional assumptions about the underlying population, only that the data are counts.

Chi-Square Test of Independence for Continuous Variables

Pearson’s correlation coefficient measures linear association between two numerical variables, but it has two critical limitations:

1. Normality assumption – conventional inference requires the data to be normally distributed, a condition often violated in practice.
2. Linear focus only – it cannot capture non‑linear relationships. Two variables may be strongly dependent yet have near‑zero correlation.

The chi‑square test of independence overcomes both problems when applied to continuous data. It requires no distributional assumptions, detects any form of association (linear or non‑linear), and is robust to data quality issues (e.g., outliers, entry errors).

How it works: discretize then test

The core idea is to convert continuous variables into categorical bins (discretize), then apply the standard chi‑square test of independence on the resulting contingency table.

Example: Satisfaction vs. Average Monthly Hours (HR dataset)

• Satisfaction level (continuous, 0–1) → discretised into 5 bins: [0,0.2), [0.2,0.4), [0.4,0.6), [0.6,0.8), [0.8,1].
• Average monthly hours (continuous, 96–310) → discretised into 4 bins: ≤150, (150,200], (200,250], >250.

These choices are a judgment call – guided by descriptive statistics (range, mean, median).
Rules of thumb:

• Not too few or too many bins.
• Each cell of the table should have sufficient (≥5) observations; avoid highly disproportionate counts.

The discretised data produce a joint frequency table (rows = satisfaction bins, columns = hours bins):

Satisfaction \ Hours	≤150	150–200	200–250	>250	Row total
[0.0 – 0.2)	...	...	...	...	...
[0.2 – 0.4)	...	...	...	...	...
[0.4 – 0.6)	...	...	...	...	...
[0.6 – 0.8)	...	...	...	...	...
[0.8 – 1.0]	...	...	...	...	...
Column total	...	...	...	...	N

Exam tip: The chi‑square test is performed exactly as for categorical data – compute expected frequencies under independence, calculate $\chi^2 = \sum \frac{(O_{ij} - E_{ij})^2}{E_{ij}}$ and compare to the critical value with $(r-1)(c-1)$ degrees of freedom.

Hypotheses

• $H_0$: Satisfaction level is independent of levels of average monthly hours (i.e., the row and column categories are independent).
• $H_a$: Satisfaction level is not independent of levels of average monthly hours (dependence exists).

Connection to Pearson correlation

In an ideal setting, the conclusions from the chi‑square test should align with the sign and strength of the Pearson correlation coefficient. For this dataset, the (linear) correlation between satisfaction and average monthly hours is mildly negative – employees who work more tend to be less satisfied. If the chi‑square test also rejects independence, it reinforces that conclusion. A discrepancy would suggest a non‑linear relationship that correlation missed.

Aspect	Pearson correlation	Chi‑square test (discretised)
Assumptions	Bivariate normality	No distributional assumptions
Relationship detected	Linear only	Any form (linear or non‑linear)
Sensitivity to outliers	High	Low (after binning)
Output	$r$ (‑1 to +1)	$\chi^2$ + p‑value

Key takeaways

• Discretise continuous variables into bins (judgment call based on range and sample size) and then apply the standard chi‑square test of independence.
• The test works without normality, captures non‑linear associations, and is robust to data quality issues.
• Interpret results in the context of the bins – dependence means that the distribution of one variable differs across categories of the other.
• Compare findings with the Pearson correlation to check for consistency and to identify possible non‑linearity.

Why Goodness of Fit?

Many standard parametric methods — the t‑test, ANOVA, linear regression — assume the data follow a particular distribution (most often the normal). Other common assumptions include the exponential distribution for product lifetimes or the Poisson distribution for count data (e.g., number of defects). Because the validity of these methods hinges on the assumption being true, an objective way to check it is essential. The goodness of fit test provides that check: it determines whether a sample of observed data matches a specified theoretical distribution.

Core Idea: Observed vs. Expected Frequencies

The test compares the observed frequencies (how often each outcome actually occurred in the sample) with the expected frequencies derived from the hypothesised distribution. The key question: are the deviations between observed and expected large enough to conclude the data does not follow the assumed distribution, or can they be attributed to random chance?

This logic is the same as that of the χ² test of independence (covered earlier in the module). Both tests use a chi‑square statistic to evaluate the gap between observed and expected counts.

Assumed distribution (e.g., normal, Poisson)
       ↓
Compute expected frequencies for each category
       ↓
Collect observed frequencies from the sample
       ↓
Test: Are deviations (observed – expected) significantly large?
       ↓
If yes → reject assumption | If no → data consistent with the distribution

Exam tip: The goodness of fit test is a nonparametric method — it makes no assumption about the distribution of the test statistic itself, only about the data being tested.

Common Distributions Tested in Business Contexts

Variable type	Example distributions	Typical application
Discrete	Binomial, Poisson	Number of defects in a batch (Poisson); number of successes in trials (binomial)
Continuous	Normal, Exponential	Customer purchase amounts (normal); product lifetime (exponential)

Selecting the correct distribution for a given business problem is a critical first step; the goodness of fit test then validates that choice.

Business Applications

The transcript details numerous real‑world uses. They all share the same pattern: an assumed distribution is embedded in a decision model, and the test confirms or rejects that assumption.

Domain	Specific use	Distribution typically assumed
Marketing & consumer behaviour	Check if customers choose brands equally (uniform) or if preferences follow a normal distribution.	Uniform, normal
Retail & sales	Validate that product demand follows a seasonal or normal pattern; check if sales across store locations follow a similar distribution (identify outlier stores).	Normal, seasonal patterns
Product demand forecasting	Assess whether past sales conform to the distribution used for future forecasts.	Varies (e.g., normal, exponential)
Quality control	Test if number of defects follows a Poisson distribution (stable process); verify that product weights/dimensions match the expected distribution.	Poisson, normal
Risk management	Check whether actual default rates match the binomial or normal model used for credit pricing.	Binomial, normal
HR analytics	Evaluate if employee attrition matches historical turnover distribution to guide workforce planning.	Historical distribution (any)
Supply chain & inventory	Validate demand patterns for inventory policy decisions.	Various

Why It Matters

Goodness of fit tests act as a gatekeeper for downstream analytics. If the distributional assumption is wrong, any conclusions drawn from parametric methods (confidence intervals, hypothesis tests, forecasts) may be unreliable. By confirming or rejecting the assumption, the test ensures that business decisions are based on accurate and trustworthy models.

Key takeaways

• Goodness of fit test checks whether observed data follows a specific theoretical distribution (normal, Poisson, exponential, etc.).
• It compares observed frequencies to expected frequencies from the hypothesised distribution.
• The test is nonparametric and conceptually similar to the χ² test of independence.
• Widely used across marketing, retail, quality control, risk management, HR analytics, and supply chain to validate assumptions.
• A significant result (large deviations) indicates the data does not match the assumed distribution; a non‑significant result means the data is consistent with it.
• Always first identify the appropriate distribution for the business context, then use the test to verify.

General Principles of Goodness of Fit Test

A goodness of fit test checks whether a sample of data follows a specified probability distribution. Intuitively: does the observed data "look like" it came from the assumed distribution (uniform, normal, Poisson, etc.), or is the difference too large to be chance? The test quantifies the discrepancy between what we expect under the hypothesis and what we observe in the sample, using the chi‑square distribution.

Procedure: step‑by‑step

flowchart TD
  A[Define H₀ and H₁] --> B[Construct intervals / categories]
  B --> C[Compute observed frequencies O_i]
  C --> D[Compute expected frequencies E_i under H₀ using CDF]
  D --> E[Calculate chi-square statistic: χ² = Σ (Oᵢ – Eᵢ)² / Eᵢ]
  E --> F[Determine degrees of freedom df = k – 1 – m]
  F --> G[Find p‑value from χ² distribution with df]
  G --> H{Reject H₀?}
  H -->|p‑value small| I[Conclude data does NOT follow assumed distribution]
  H -->|p‑value large| J[Fail to reject H₀ – data consistent with distribution]

1. Formulate hypotheses

• Null hypothesis ($H_0$): The data follow a specific distribution (e.g., uniform, normal, Poisson).
• Alternative hypothesis ($H_1$): The data do not follow that distribution. (No claim about what other distribution holds.)

2. Build intervals (for continuous variables)

• Divide the range into $k$ bins or categories.
• Bins need not be equal in width, but expected frequencies should be balanced (neither too large nor too small). For uniform distributions, equal‑width bins are natural.
• Example: For satisfaction level (range 0–1), use 10 bins of width 0.1.

3. Obtain observed frequencies ($O_i$)

Count how many sample observations fall into each bin.

4. Compute expected frequencies ($E_i$) under $H_0$

• Use the cumulative distribution function (CDF) of the assumed distribution.
• For any bin, $E_i = N \times P(\text{observation in bin})$, where $N$ is the total sample size.
• For a uniform distribution: $P(\text{bin}) = \frac{\text{bin length}}{\text{total range}}$.
• For other distributions (normal, binomial, Poisson), you must first estimate the parameters from the data (e.g., $\mu,\sigma^2$ for normal, $\lambda$ for Poisson). Then use the CDF (e.g., NORM.DIST in Excel).

5. Calculate the chi‑square test statistic

$\chi^2 = \sum_{i=1}^{k} \frac{(O_i - E_i)^2}{E_i}$

A larger value indicates greater deviation from $H_0$.

6. Degrees of freedom (df)

$\text{df} = k - 1 - m$

• $k$ = number of intervals
• $m$ = number of parameters estimated from the data (e.g., 2 for normal, 0 for uniform, 1 for Poisson)

7. Decision

• Compute the p‑value using the chi‑square distribution with the appropriate df (e.g., CHISQ.DIST.RT in Excel).
• If p‑value < significance level (typically 0.05), reject $H_0$ – the data do not fit the distribution.

Exam tip: The goodness of fit test only detects departure from $H_0$; it does not tell you what the true distribution is. Also, each estimated parameter reduces df by 1 – forgetting this is a common error.

---

Worked example: Testing uniform distribution of employee satisfaction

Data: Satisfaction level (0–1) of $N = 14,\!999$ employees.

Hypotheses:

• $H_0$: Satisfaction is uniformly distributed over [0,1].
• $H_1$: Satisfaction is not uniformly distributed.

Step 2 – intervals: 10 equal‑width bins: [0,0.1], [0.1,0.2], …, [0.9,1.0].

Step 3 – observed frequencies ($O_i$):

Interval	Observed ($O_i$)
[0, 0.1)	553
[0.1, 0.2)	…
…	…
[0.9, 1.0]	…
Total	14,999

(Only the first interval’s count is given in the lecture; the others sum to 14,999.)

Step 4 – expected frequencies: Under uniform distribution, each bin has probability 0.1, so

$E_i = 14,\!999 \times 0.1 = 1,\!499.9 \quad \text{for all } i=1,\dots,10.$

Step 5 – chi‑square statistic: The contribution for the first bin:

$\frac{(553 - 1,\!499.9)^2}{1,\!499.9} \approx 597.79$

Summing over all 10 bins gives:

$\chi^2 = 2,\!710.25$

Step 6 – degrees of freedom: $k = 10$, $m = 0$ (uniform requires no estimated parameters), so

$\text{df} = 10 - 1 - 0 = 9.$

Step 7 – p‑value: Using a chi‑square distribution with 9 df, the p‑value is effectively 0 (extremely small). Therefore, reject $H_0$ – satisfaction level is not uniformly distributed. The company should investigate factors (department, workload, etc.) driving this non‑uniform pattern.

---

Degrees of freedom – general rule (examples)

Distribution	Parameters estimated ($m$)	Example with $k=8$ intervals
Uniform	0	$\text{df} = 8 - 1 - 0 = 7$
Normal	2 ($\mu,\sigma^2$)	$\text{df} = 8 - 1 - 2 = 5$
Poisson	1 ($\lambda$)	$\text{df} = 8 - 1 - 1 = 6$
Binomial	1 ($p$)	$\text{df} = 8 - 1 - 1 = 6$

Exam tip: Always count how many parameters were estimated from the sample to compute the expected frequencies. This directly affects df and the critical value.

---

Key takeaways

• Goodness of fit tests whether sample data come from a specified distribution.
• Use the chi‑square statistic: $\chi^2 = \sum (O_i - E_i)^2 / E_i$, comparing observed vs. expected counts.
• For continuous distributions, bin the data and compute expected frequencies via the CDF.
• Degrees of freedom: $\text{df} = k - 1 - m$ (intervals minus parameters estimated).
• A very small p‑value rejects $H_0$; the data do not fit the assumed distribution.
• The test is widely applicable – uniform, normal, Poisson, binomial – but requires careful interval construction and parameter estimation.

Goodness of Fit Test for Poisson Distribution

Intuition: The Poisson distribution models the count of rare, independent events over a fixed interval (e.g., customer arrivals, equipment failures). A goodness of fit test checks whether observed frequency counts (e.g., number of projects per employee) plausibly come from a Poisson process. If the data fit, the events appear random and independent at a constant average rate. If not, systematic factors (e.g., uneven workload) may be present.

Application context: In the HR dataset (14,999 employees), the only integer-valued variable suitable for Poisson modelling is number of projects (satisfaction level is continuous; years/hours are continuous even if reported as integers). A goodness of fit test here informs whether project distribution is random — or whether some employees are overburdened.

Step-by-step procedure

1. Hypotheses

$$H_0: \text{Number of projects follows a Poisson distribution}$$ $$H_1: \text{Number of projects does NOT follow a Poisson distribution}$$

2. Observed frequencies

• Raw counts show zero employees with 0 or 1 projects, and none with >7.
• Cell merging (required to avoid zero expected frequencies): aggregate into six intervals.
• Final observed frequencies (after merging):

Interval (projects)	Observed frequency $O_i$
$\leq 2$	2,388
$3$	(computed in Excel)
$4$	(computed in Excel)
$5$	(computed in Excel)
$6$	(computed in Excel)
$\geq 7$	(computed in Excel)
Total	14,999

Exam tip: Always merge cells so that no expected count is < 1 and at most 20% of cells have expected counts < 5. For Poisson, low-probability tails (0,1,7+) are typical candidates for merging.

3. Expected frequencies under $H_0$

The Poisson probability mass function:

$$P(X = k) = \frac{e^{-\lambda} \lambda^k}{k!}, \quad k = 0,1,2,\dots$$

• Estimate $\lambda$ using the sample mean: $\hat{\lambda} = \bar{x} = 3.8$ (from 14,999 observations).
• Compute probabilities for each merged interval using the Poisson distribution with $\lambda = 3.8$.
• Multiply each probability by $n = 14,999$ to get expected frequencies $E_i$.

Example – first cell ($\leq 2$ projects):

$$P(X \leq 2) = \text{POISSON.DIST}(2, 3.8, \text{TRUE}) \approx 0.27$$ $$E_1 = 0.27 \times 14,999 \approx 4,025.79$$

In Excel use:

• POISSON.DIST(x, mean, cumulative)
  • For exact probability: FALSE
  • For cumulative probability: TRUE
• Last cell ($\geq 7$): $P(X \geq 7) = 1 - \text{POISSON.DIST}(6, 3.8, \text{TRUE})$

Resulting table (partial):

Interval	$O_i$	$E_i$
$\leq 2$	2,388	4,025.79
$3$	...	...
$4$	...	...
$5$	...	...
$6$	...	...
$\geq 7$	...	...
Total	14,999	14,999.00

4. Chi-square test statistic

$$\chi^2 = \sum_{i=1}^{6} \frac{(O_i - E_i)^2}{E_i}$$

For this dataset, the sum of all six contributions yields:

$$\chi^2 = 2780.09$$

5. Degrees of freedom and p-value

$$\text{df} = (\text{number of cells}) - 1 - (\text{number of estimated parameters}) = 6 - 1 - 1 = 4$$

Using CHISQ.DIST.RT(2780.09, 4) in Excel gives a p-value ≈ 0 (extremely small).

6. Inference

Because the p-value is virtually zero, we reject $H_0$. There is overwhelming evidence that the number of projects does not follow a Poisson distribution. The company should investigate why some employees are overburdened while others have too few projects.

Brief extension: Testing normality

The same logic applies to continuous variables. For a goodness of fit test for normality:

• $H_0$: Data follow a normal distribution.
• Estimate both parameters: $\mu$ and $\sigma$ from the sample.
• Create intervals (bins) over the continuous range.
• Compute expected frequencies using NORM.DIST(x, mean, stdev, cumulative).
• Calculate $\chi^2$ and compare to $\chi^2_{\text{df}}$ with $\text{df} = (\#\text{bins}) - 1 - 2$.

Example variables from the HR dataset to test: satisfaction level, last evaluation, average monthly hours.

Key takeaways

• Goodness of fit tests whether observed frequencies match a specified theoretical distribution (Poisson, normal, etc.).
• For Poisson: estimate $\lambda$ from the sample mean; merge low-frequency cells to satisfy expected count conditions.
• Degrees of freedom = (#cells) – 1 – (#estimated parameters).
• A very large $\chi^2$ (e.g., 2780 on 4 df) leads to rejection of the Poisson assumption.
• The same framework extends to any distribution (normal, binomial, etc.) by using the appropriate probability function.
• Real business use: validating randomness of events (arrivals, defects) or detecting systematic imbalances (workload, resource allocation).

Wilcoxon Signed Rank Test

The Wilcoxon signed rank test is a non‑parametric procedure that relies solely on the relative ordering (ranks) of observations rather than their raw values. Unlike a parametric t-test, it makes no assumption about the underlying distribution — making it ideal for small samples, skewed data, or when normality fails.

Core idea

• One‑sample: Are the data systematically higher or lower than a specified median?
• Two‑sample (paired): Is the median of the paired differences significantly different from zero?

The test ranks the absolute deviations (or differences) and then examines the signs of those deviations. If the positive deviations are consistently larger than the negative ones (or vice versa), the test infers a significant shift away from the hypothesised median.

When to use it

Scenario	Parametric alternative	Why Wilcoxon wins
Small sample size, normality unclear	One‑sample t-test / paired t-test	No normality required; robust to outliers
Data heavily skewed (e.g., financial returns, delivery times)	Same	Ranks neutralise extreme values
Before‑after intervention (same subjects)	Paired t-test	Works even if effect magnitude is irregular
Matched pairs (treatment vs control)	Paired t-test	Handles unknown distribution of differences

One‑sample signed rank test

Intuition: A company sets a target delivery time of 5 days. Observed times deviate above or below that target. The test checks whether the deviations are balanced around zero (i.e., typical delivery = target) or are systematically positive/negative (i.e., median delivery ≠ 5 days).

Procedure (conceptual – details follow in later lectures):

1. Compute $d_i = x_i - \text{hypothesised median}$ for each observation.
2. Rank the absolute differences $|d_i|$ (ignore signs).
3. Assign the original signs (+ or –) to the ranks.
4. Sum the ranks of the positive differences ($W^+$) and negative differences ($W^-$).
5. Compare the smaller sum to a critical value.

Because ranks are used, a few extreme values do not dominate the result — the test focuses on the ordering of deviations.

Exam tip: The one‑sample signed rank test does not test the mean; it tests the median. This is a common source of confusion in exams.

Two‑sample (paired) signed rank test

When observations come in pairs — either from the same subject before/after, or from matched subjects (treatment vs control) — the test examines whether the median of the within‑pair differences is zero.

Example: A pharmaceutical company gives a drug to one group and a placebo to a matched control group. The paired difference (drug outcome – placebo outcome) is computed for each pair. If the drug has no effect, the differences should centre around zero. A signed rank test can detect if the median difference is significantly positive (drug better) or negative.

Business applications:

• Marketing: Compare sales before and after a campaign.
• Quality control: Product weight vs. a standard weight.
• Finance: Stock returns before vs. after a market event, or returns of two portfolios over the same period.
• Operations: Production efficiency before and after a process change.

Why it is attractive

• No distributional assumptions — works when a goodness‑of‑fit test rejects normality.
• Robust to outliers — ranks cap the influence of extreme values.
• Handles small samples — where parametric tests lose power or cannot be validated.
• Applicable to paired data — common in business experiments and A/B testing.

flowchart TD
    A[Sample data available] --> B{Normality plausible?}
    B -->|Yes, large sample| C[Use parametric t‑test]
    B -->|No / small / skewed| D[Use Wilcoxon signed rank test]
    D --> E{One sample?}
    E -->|Compare to benchmark median| F[One‑sample signed rank]
    E -->|Paired/matched observations| G[Two‑sample signed rank<br/>(paired differences)]

Key takeaways

• Wilcoxon signed rank test is a non‑parametric alternative to the one‑sample and paired t-tests.
• It tests the median (not mean) of differences from a hypothesised value or of paired differences.
• Ranks are based on absolute deviations; signs determine the direction of deviation.
• No normality assumption — safe for skewed data, small samples, and outlier‑prone data.
• Common in business for before/after comparisons, quality control, and financial analysis.

Wilcoxon Signed Rank Test – II

The Wilcoxon signed rank test (also called the one-sample Wilcoxon rank sum test) is a non-parametric procedure for determining whether the median of a single sample differs significantly from a hypothesized value $M_0$. It is an alternative to the one-sample t-test when the normality assumption is violated or when the sample size is too small for parametric methods. Because the test works on ranks rather than raw values, it is robust to skewness and outliers.

Intuition: Instead of asking whether the mean differs from a target, we ask whether the typical observation (the median) is systematically higher or lower than a claimed value. By ranking the deviations and summing the ranks for positive deviations, we can detect a consistent directional shift.

When to use

• Data are not normally distributed (e.g., skewed, heavy tails).
• Sample size is very small (making normality check unreliable).
• Interest lies in the population median, not the mean.
• Presence of outliers would distort a t-test.

Test procedure

1. Compute differences: For each observation $x_i$, calculate $d_i = x_i - M_0$, where $M_0$ is the hypothesized median.

2. Rank absolute differences: Rank $|d_i|$ from smallest to largest (ignore sign). If ties occur, assign the average rank to tied values.

3. Assign signs: Attach the sign (+ or –) of $d_i$ to each rank. Ranks of zero differences ($d_i = 0$) are discarded.

4. Sum positive signed ranks: $W^+ = \sum \text{(ranks with positive } d_i\text{)}$ Similarly, define $W^-$ as the sum of negative signed ranks. Always: $W^+ + W^- = \frac{n(n+1)}{2}$ where $n$ is the number of non‑zero differences.

5. Test statistic: For large samples ($n \ge 30$), use the z‑approximation: $z = \frac{W^+ - \mu_{W^+}}{\sigma_{W^+}}$ with $\mu_{W^+} = \frac{n(n+1)}{4}, \quad \sigma_{W^+} = \sqrt{ \frac{n(n+1)(2n+1)}{24} }$ Under the null hypothesis (population median $= M_0$), $W^+$ has this known mean and standard error.

   Exam tip: The approximation works because $W^+$ is the sum of $n$ independent (under $H_0$ equally likely +/–) signed ranks; by the Central Limit Theorem it becomes normal as $n$ grows.

   For small samples ($n < 30$), exact critical values from the Wilcoxon signed rank distribution are used (not covered in this lecture).

6. Decision rule: Compare $|z|$ to a critical value from the standard normal distribution.

   • At $\alpha = 0.05$: reject $H_0$ if $|z| > 1.96$.
   • At $\alpha = 0.01$: reject $H_0$ if $|z| > 2.58$. Alternatively, compute the p‑value: $\text{p-value} = 2 \times P(Z > |z|)$ using Excel NORM.DIST or similar.

flowchart TD
  A["Compute W⁺"] --> B["Calculate z = (W⁺ – μ) / σ"]
  B --> C["|z| > critical value?"]
  C -->|Yes| D["Reject H₀: median ≠ M₀"]
  C -->|No| E["Fail to reject H₀"]

Worked example: Average monthly hours

Data: HR analytics dataset with $n = 14,999$ employees.
Hypothesis: $H_0: \text{median} = 200$ hours (a believed norm) vs. $H_1: \text{median} \neq 200$.

Steps:

1. Differences: For employee 1, $x_1 = 157$, so $d_1 = 157-200 = -43$.
2. Rank absolute differences: $|d_1| = 43$. Using RANK.AVG, ties produce average ranks (e.g., 7134.5).
3. Signs: Negative differences get sign –1; positive get +1; exact 200 get 0 (discarded).
4. Positive signed ranks: Extract ranks for all positive signs.
5. $W^+$: Sum of positive signed ranks = 57,539,873.
6. Z‑score: $\mu_{W^+} = \frac{14999 \times 15000}{4} = 56,246,250$ $\sigma_{W^+} = \sqrt{ \frac{14999 \times 15000 \times 29999}{24} } \approx 1,180,625$ $z = \frac{57,539,873 - 56,246,250}{1,180,625} \approx 2.44$

Decision:

• At $\alpha = 0.05$, critical value = 1.96 → $|2.44| > 1.96$, reject $H_0$. The median is significantly different from 200 hours.
• At $\alpha = 0.01$, critical value = 2.58 → $|2.44| < 2.58$, fail to reject $H_0$. The evidence is not strong enough at the 1% level.

Further exploration: Testing medians 199 and 201 yields:

• 199: significant evidence of deviation.
• 201: no evidence of deviation.

Thus the organization’s median lies between 200 and 201 hours.

Comparison with one‑sample t-test

The lecture suggests running a one‑sample t-test on the same data (testing mean = 200) and comparing results. Because the data likely violate normality, the t-test may give a different conclusion. Always accompany such tests with exploratory plots (e.g., histogram of monthly hours) to build a complete picture.

Exam tip: The Wilcoxon signed rank test is not a test of the mean; it is a test of the median. When the population is symmetric and normally distributed, the t-test is more powerful. When normality fails, the Wilcoxon test is more reliable.

Key takeaways

• The Wilcoxon signed rank test assesses whether the population median differs from a hypothesized value.
• It is a non‑parametric alternative to the one‑sample t-test, robust to non‑normality and outliers.
• Procedure: compute differences, rank absolute differences, assign signs, sum positive ranks ($W^+$).
• For $n \ge 30$, use the z‑approximation: $z = (W^+ - n(n+1)/4) / \sqrt{n(n+1)(2n+1)/24}$.
• Reject $H_0$ if $|z|$ exceeds the normal critical value (1.96 at 5%).
• Always report exploratory analysis (histograms, boxplots) alongside the test.

Paired Wilcoxon Signed‑Rank Test

The paired Wilcoxon signed‑rank test is a non‑parametric alternative to the paired t‑test. It answers: Is the median difference between two related samples significantly different from zero?
Use it when data come in pairs (same subject before/after, matched pairs) and the normality assumption fails.

Intuition

We don’t care about the absolute values of the two groups; we care only about the direction and magnitude of the difference within each pair. By ranking the absolute differences and then restoring the sign, we test whether positive and negative differences are balanced – if they are, the median difference is zero.

Hypothesis

For a two‑tailed test (is there any difference?):

• $H_0$: median difference between paired observations = 0
• $H_1$: median difference $\neq 0$

For a one‑tailed test (e.g., did the campaign increase visits?), change $H_1$ to “median difference $>0$” or “$<0$”.

Procedure

1. Compute differences for each pair: $d_i = \text{after}_i - \text{before}_i$.
2. Ignore signs and rank the absolute values $|d_i|$. Assign rank 1 to the smallest absolute difference; average ranks for ties.
3. Restore signs to the ranks → signed ranks.
4. Sum the positive ranks – denote this as $W_+$.
   • Under $H_0$, $W_+$ should be close to half of total rank sum.
5. Decision rule depends on sample size:

Sample size	Method	Test statistic	Critical value source
$n \geq 30$	Large‑sample normal approximation	$z = \dfrac{W_+ - \frac{n(n+1)}{4}}{\sqrt{\frac{n(n+1)(2n+1)}{24}}}$	Standard normal $z_{\alpha/2}$
$n < 30$	Small‑sample exact table	$W_+$	Wilcoxon signed‑rank table (e.g., for $n=10$, $\alpha=0.05$, critical = 8)

Exam tip: The large‑sample formula uses the mean $\mu = n(n+1)/4$ and variance $\sigma^2 = n(n+1)(2n+1)/24$. These come from the null distribution of signed ranks.

Worked Example (Marketing Campaign)

A company tracks website visits of 10 customers before and after a campaign.
Only the first four pairs are shown below (full data not given).

| Customer | Before | After | Difference $d$ | $|d|$ | Rank of $|d|$ | Signed rank | |----------|--------|-------|----------------|-------|----------------|-------------| | 1 | 50 | 55 | +5 | 5 | 4 | +4 | | 2 | 60 | 65 | +5 | 5 | 4 | +4 | | 3 | 45 | 48 | +3 | 3 | 2 | +2 | | 4 | 80 | 75 | –5 | 5 | 4 | –4 | | … | … | … | … | … | … | … |

Note: ties in absolute difference (e.g., three values of 5) receive average rank $(4+5+6)/3 = 5$ in a full ranking; this partial table uses a simplified rank for illustration.

Complete the ranking for all 10 customers, then compute $W_+$ = sum of positive signed ranks.

Result (per lecturer): For $n=10$ at $\alpha=0.05$ (two‑tailed), the critical value from the Wilcoxon table is 8. The computed $W_+$ exceeds 8, so we reject $H_0$ – there is a statistically significant difference in website visits before vs. after the campaign.

Interpretation: The marketing campaign changed website traffic. To determine whether it increased visits, conduct a one‑tailed test (right‑tail: $H_1$: median difference $>0$).

When to Choose This Test

flowchart LR
A[Two related samples?] -->|Yes| B{Normality plausible?}
B -->|No| C[Paired t‑test]
B -->|Yes, but n is small <30| D[Paired Wilcoxon signed‑rank test]
B -->|Yes, n large| E[Paired t‑test acceptable]
C --> F[Safe]
D --> F

Key Takeaways

• Paired Wilcoxon signed‑rank test compares median difference in matched/paired data.
• Difference $d_i$ is computed; absolute values are ranked; signs restored; positive ranks summed → $W_+$.
• For $n<30$, use exact Wilcoxon table; for $n\geq30$, use normal approximation $z$.
• No normality assumption required – robust for small or skewed samples.
• The test tells you whether a difference exists; direction requires a one‑tailed version.
• Do not confuse with Wilcoxon rank‑sum test (for independent samples).

Summary of Module 1: Non-Parametric Methods

Non-parametric methods provide robust alternatives to parametric tests when underlying assumptions (e.g., normality, homoscedasticity) are violated. This module covered three core techniques: chi-square test of independence, goodness-of-fit test, and Wilcoxon signed-rank test. Each is widely applied in business for categorical data, distribution checks, and paired comparisons.

1. Chi-Square Test of Independence

Determines whether two categorical variables are associated. Intuition: If the variables are independent, the observed frequency of each combination should be close to what we’d expect by chance.

Test statistic:

$\chi^2 = \sum \frac{(O_{ij} - E_{ij})^2}{E_{ij}}$

where $O_{ij}$ = observed count in cell $(i,j)$, $E_{ij}$ = expected count under independence $= \frac{\text{row total} \times \text{column total}}{\text{grand total}}$.

Process:

1. Build a contingency table of observed frequencies.
2. Compute expected frequencies assuming no relationship.
3. Calculate $\chi^2$ and compare to a critical value (degrees of freedom = $(r-1)(c-1)$).

Example: A business wants to know if product type purchased is related to customer geographic region. A chi-square test on the contingency table of product × region can reveal regional preferences without any distributional assumptions.

Applications:

• Customer demographics, product categories, payment methods, satisfaction ratings.
• Can also be used as a non-parametric alternative to the correlation coefficient for continuous data (by discretizing into categories).

Key takeaways

• Only requires categorical data; no normality assumption.
• Tests independence between two variables.
• Calculated from observed vs. expected frequencies in a contingency table.
• Widely used in retail, marketing, healthcare, manufacturing.

2. Chi-Square Goodness-of-Fit Test

Assesses whether observed data follow a specific theoretical distribution (e.g., normal, Poisson). Intuition: Does the actual frequency distribution match the expected pattern?

Test statistic:

$\chi^2 = \sum \frac{(O_i - E_i)^2}{E_i}$

where $O_i$ = observed count in category $i$, $E_i$ = expected count from the theoretical distribution.

Example:

• Retail sales forecasting: A company assumes customer spending follows a normal distribution. The test compares observed sales bins against normal expectations. If significant deviation is found, the forecasting model must be adjusted (better inventory, less waste/stockout).
• Quality control: Defects in manufacturing are expected to follow a Poisson distribution (constant average rate). The test verifies alignment; a significant result indicates a process shift.

Other applications:

• Finance: Check if stock returns follow a normal distribution (common in risk modeling).
• Marketing: Verify whether customer purchase patterns match expected distributions to fine-tune promotions.

Key takeaways

• Tests whether data match a specific distribution (normal, Poisson, etc.).
• Critical for validating assumptions behind forecasting, quality control, and risk models.
• Significant deviation prompts model adjustment or process investigation.
• Uses same $\chi^2$ formula as test of independence, but with one-way categories.

3. Wilcoxon Signed-Rank Test

A non-parametric alternative to the paired t-test (or one-sample t-test) when normality is not satisfied. It operates on ranks of the differences, making it robust to outliers and skewed data.

When to use:

• One sample vs. a hypothetical median.
• Two related (paired) samples: before/after, matched pairs.

Idea: Rank the absolute differences, then sum the ranks for positive and negative differences. The test statistic compares the smaller sum against a critical value.

Example: A company launches a new product feature and measures customer satisfaction for the same group before and after. Even if the satisfaction ratings are not normally distributed, the Wilcoxon test can reliably tell if the median difference is significant.

Business scenarios:

• Marketing campaign effectiveness – small sample, volatile sales figures.
• Employee productivity after a training program.
• Financial returns before/after an interest rate change – returns are often skewed with extreme outliers.

Advantages over paired t-test:

• No normality assumption required.
• Focuses on median difference rather than mean.
• Less influenced by extreme values.

Key takeaways

• Ranks differences between paired observations; does not require normality.
• Ideal for small samples, skewed data, or outliers.
• Use for one-sample or paired comparisons (before/after).
• Commonly applied in HR, marketing, and finance.

---

Why Non-Parametric Methods Matter

When assumptions like normality or homoscedasticity fail, these three tools offer reliable, assumption-light alternatives:

Method	Data Type	Question Answered	Parametric Alternative
Chi-square independence	Categorical (two variables)	Are they related?	– (no direct param.)
Chi-square goodness-of-fit	Categorical (one variable vs. distribution)	Does the data fit a distribution?	–
Wilcoxon signed-rank	Paired continuous/ordinal	Is the median different?	Paired t-test

Key takeaway for business analytics: Non-parametric methods handle real-world data (non‑normal, categorical, small samples) without sacrificing rigor. They enable confident decision‑making when parametric assumptions are untenable.