Term 3 · Module 1 of 4

Analysis of Variance ANOVA Goodness of Fit Covariance and Correlation

Business Statistics for Entrepreneurs III

Practise this module in Claude, ChatGPT or Gemini

Set it up free
Mind map

Covariance and Correlation

Covariance: direction of joint movement

Covariance describes whether paired values of two quantitative variables tend to lie on the same side of their respective averages. A positive value means above-average values of one tend to accompany above-average values of the other (and below with below); a negative value means one tends to be above average when the other is below. It describes linear co-movement, not cause and effect.

For paired sample observations (xi,yi)(x_i,y_i), with sample means xˉ\bar{x} and yˉ\bar{y}:

sxy=∑i=1n(xi−xˉ)(yi−yˉ)n−1s_{xy}=\frac{\sum_{i=1}^{n}(x_i-\bar{x})(y_i-\bar{y})}{n-1}

The n−1n-1 denominator reflects the degrees-of-freedom adjustment used for sample variance. Each product is positive in the upper-right and lower-left quadrants around (xˉ,yˉ)(\bar{x},\bar{y}), and negative in the other two. If x=yx=y, covariance reduces to sample variance.

In a sample of 55 Canara Bank customers, monthly credit-card spend had mean ₹3,852, median ₹3,864 and standard deviation ₹1,454. Spend and household income had sample covariance 16,109; spend and household size had covariance 1,610. The sign indicates positive association. The magnitude depends on units: expressing income in rupees instead of lakhs rescales covariance without changing the underlying relationship.

Pearson correlation: unit-free linear association

Pearson’s sample correlation coefficient standardizes covariance by both sample standard deviations:

rxy=sxysxsyr_{xy}=\frac{s_{xy}}{s_xs_y}

It is unit-free and lies between −1-1 and 11. A value of 11 means all points lie on a positively sloped straight line; −1-1 means a negatively sloped line. Values nearer either endpoint mean tighter linear alignment; a value near zero means weak linear association, not necessarily no relationship.

For spend and income, sx=13.74s_x=13.74 lakhs, sy=1,454s_y=1{,}454, and sxy=16,109s_{xy}=16{,}109:

r=16,109(13.74)(1,454)≈0.806≈0.81r=\frac{16{,}109}{(13.74)(1{,}454)}\approx0.806\approx0.81

For spend and household size, sx=1.76s_x=1.76 and sxy=1,610s_{xy}=1{,}610:

r=1,610(1.76)(1,454)≈0.629≈0.63r=\frac{1{,}610}{(1.76)(1{,}454)}\approx0.629\approx0.63

Both are positive; the sample shows stronger linear association between spend and income. Excel COVARIANCE.S and CORREL return sample covariance and correlation; reported results are 16,109 and 0.806 for spend-income, and 1,610 and 0.629 for spend-household size.

Pearson correlation only detects linear association. For x=(3,2,1,−1,−2,−3)x=(3,2,1,-1,-2,-3) and y=1/x2=(1/9,1/4,1,1,1/4,1/9)y=1/x^2=(1/9,1/4,1,1,1/4,1/9), symmetry makes sample covariance and correlation zero, even though yy is exactly determined by xx through a nonlinear relationship. A scatterplot is essential to spot curvature. Utility spending can be higher at very hot and very cold temperatures than at moderate temperatures, a U-shaped relation with near-zero linear correlation.

Explainable correlation has a plausible relationship; spurious correlation does not have a supported causal explanation. A third, latent variable may affect both measured variables: summer temperature can coincide with higher ice-cream sales and crime (more homes left empty during vacations); poor village health can lead both to more doctors and more deaths. A reported correlation of 0.99 between Maine divorce rates and per-capita margarine consumption is presented as spurious, without an identified hidden cause. The hemline theory attributed shorter skirts to economic prosperity and longer skirts in downturns, explaining the proposed association through ability to afford stockings; such patterns require scrutiny. For a nominal or ordinal variable, other measures such as Spearman rank correlation may be appropriate. Population claims about correlation require inference; the related significance test uses a tt distribution, though its procedure is outside this treatment.

Key takeaways

  • Covariance’s sign gives direction; its magnitude changes with measurement units.
  • Pearson rr standardizes covariance and measures only linear association.
  • Zero correlation does not rule out nonlinear dependence.
  • Association, even when strong, is not proof of causation; inspect plausible confounders and plots.

One-Way Analysis of Variance (ANOVA)

Why compare several means with ANOVA?

Analysis of variance (ANOVA) tests whether means of two or more populations can be treated as equal using samples from each. Applications include policy outcomes across states, salaries across educational backgrounds, products or technologies, and disease risk across groups. The groups are called treatments or factor levels; the measured outcome is the response. In one-way ANOVA, one factor defines the groups and one quantitative response is measured.

For KK population means:

H0:μ1=μ2=⋯=μKH_0:\mu_1=\mu_2=\cdots=\mu_K HA:not all μj are equalH_A:\text{not all }\mu_j\text{ are equal}

The alternative means at least one differs; it does not claim every mean differs. ANOVA is called analysis of variance because it tests mean differences by decomposing and comparing variability. A given separation between sample means is compelling when observations within groups are tightly clustered; the same separation is weak evidence when within-group spread is large. ANOVA compares between-treatment variation with within-treatment (error) variation.

Sums of squares and the F test

Let group jj contain njn_j observations, total N=∑jnjN=\sum_j n_j, group mean xˉj\bar{x}_j, and grand mean xˉ..\bar{x}_{..}. Partition total variability:

SST=∑j=1K∑i=1nj(xij−xˉ..)2SS_T=\sum_{j=1}^{K}\sum_{i=1}^{n_j}(x_{ij}-\bar{x}_{..})^2 SSTr=∑j=1Knj(xˉj−xˉ..)2SS_{Tr}=\sum_{j=1}^{K}n_j(\bar{x}_j-\bar{x}_{..})^2 SSE=∑j=1K∑i=1nj(xij−xˉj)2SS_E=\sum_{j=1}^{K}\sum_{i=1}^{n_j}(x_{ij}-\bar{x}_j)^2 SST=SSTr+SSESS_T=SS_{Tr}+SS_E

Treatment degrees of freedom are K−1K-1; error degrees of freedom are N−KN-K. Divide each sum by its degrees of freedom:

MSTr=SSTrK−1,MSE=SSEN−K,F=MSTrMSEMS_{Tr}=\frac{SS_{Tr}}{K-1},\qquad MS_E=\frac{SS_E}{N-K},\qquad F=\frac{MS_{Tr}}{MS_E}

Under H0H_0, both mean squares estimate the common population variance, so FF tends to be near 1. A large FF indicates treatment means vary more than expected from within-group noise. Its reference distribution has (K−1,N−K)(K-1,N-K) degrees of freedom; reject for a sufficiently small upper-tail pp-value (at most α\alpha), or when FF exceeds the upper-tail critical value. Failure to reject is not proof that means are equal.

Worked example: investors in Palnadu

Fifteen investors were grouped by age: young (n1=4n_1=4), middle-aged (n2=5n_2=5), and senior (n3=6n_3=6). Sample means in thousands were 60, 57.2 and 54.8. These sample differences alone do not settle whether population means differ: within-group variability matters too. From the data, SSTr=64.36SS_{Tr}=64.36, MSTr=32.18MS_{Tr}=32.18, MSE=5.64MS_E=5.64, hence:

F=32.185.64=5.71F=\frac{32.18}{5.64}=5.71

The reported pp-value is 0.02, so at 0.05 reject equality: at least one population mean differs. The source’s raw table is not reproduced in its transcript text, so the sums of squares cannot be recalculated observation by observation; reported results are retained.

Same group sizes, different evidence

For another sample of 15 investors with group sizes 4, 5 and 6, means 41, 42, 41.67 and standard deviations 24.08, 21.12, 18.61, reported values are SST=5257.6SS_T=5257.6, SSTr=2.27SS_{Tr}=2.27, SSE=5255SS_E=5255, MSTr=1.13MS_{Tr}=1.13, MSE=438MS_E=438. Thus F≈1.13/438≈0.003F\approx1.13/438\approx0.003 and reported p=0.997p=0.997: do not reject. Similar averages amid high spread offer little evidence of different population means.

For a third sample with same sizes, means 60, 37.2 and 20, standard deviations 2.31, 2.28 and 2.31. Reported SST=3345.6SS_T=3345.6, SSTr=3287SS_{Tr}=3287, SSE=58.8SS_E=58.8, MSTr=1643MS_{Tr}=1643, MSE=4.9MS_E=4.9, so F≈335F\approx335 and pp is near zero: reject. Means are far apart relative to within-group spread. Boxplots help visualize, but inference comes from the test.

ANOVA table and computation

Source Sum of squares Degrees of freedom Mean square Test statistic
Treatments SSTrSS_{Tr} K−1K-1 SSTr/(K−1)SS_{Tr}/(K-1) MSTr/MSEMS_{Tr}/MS_E
Error SSESS_E N−KN-K SSE/(N−K)SS_E/(N-K) —
Total SSTSS_T N−1N-1 — —

For hand calculation, find the grand mean, square each observation’s deviation from it to get SSTSS_T, calculate group means and their weighted deviations for SSTrSS_{Tr}, then calculate within-group deviations for SSESS_E (or subtract SSTrSS_{Tr} from SSTSS_T). Verify the partition before calculating degrees of freedom, mean squares and FF. Excel’s Analysis ToolPak offers “ANOVA: Single Factor”; select group columns, mark labels if included, set α\alpha (0.05 in examples), and choose an output location. Software speeds arithmetic; understanding the table supports interpretation.

Post-hoc pairwise comparisons

A significant omnibus test says at least one mean differs, but not which. Post-hoc analysis investigates pairs. In the demonstrated simple Excel approach, run one-way ANOVA on each pair. For Palnadu investors: young versus middle-aged had SST=48.22SS_T=48.22, SSTr=17.42SS_{Tr}=17.42, SSE=30.8SS_E=30.8, MSE=4.4MS_E=4.4, F=3.96F=3.96, p=0.09p=0.09 (do not reject); middle-aged versus senior had SST=72.9SS_T=72.9, SSTr=15.3SS_{Tr}=15.3, SSE=57.6SS_E=57.6, p=0.16p=0.16 (do not reject); young versus senior had MSTr=64.1MS_{Tr}=64.1, MSE=5.86MS_E=5.86, p=0.01p=0.01 (reject). These comparisons suggest young and senior means differ. Repeated pairwise tests can be tedious and have limitations; advanced methods are beyond scope.

Further ANOVA examples

Audit-team composition

Twenty audit-duration observations were split among local-only (n=7n=7), corporate-only (n=6n=6) and mixed (n=7n=7) teams. Grand mean 21.05 days; group means 25.3, 20.8, 17. Data give SST=324.95SS_T=324.95, SSTr≈240.6SS_{Tr}\approx240.6, SSE=84.26SS_E=84.26 (rounding differences), with degrees of freedom 2 and 17:

MSTr=240.6/2=120.3,MSE=84.3/17≈4.9,F=120.3/4.9≈24.3MS_{Tr}=240.6/2=120.3,\quad MS_E=84.3/17\approx4.9,\quad F=120.3/4.9\approx24.3

At α=0.05\alpha=0.05, critical F2,17=3.59F_{2,17}=3.59. Since 24.3>3.5924.3>3.59, reject: average audit completion times are not all equal. This does not identify which pair differs; three pairwise comparisons would be needed.

Assembly methods

For 34 transmission assemblies, methods A–D had sample sizes 8, 11, 9, 6; grand mean 107.2 and means 117, 100, 102.6, 114.2. From observations, SST=11,588SS_T=11{,}588 and SSTr=1,800.4SS_{Tr}=1{,}800.4 (rounding), so SSE=9,787.7SS_E=9{,}787.7. Degrees of freedom are 3 and 30:

MSTr=1,800.4/3=600.1,MSE=9,787.7/30=326.3MS_{Tr}=1{,}800.4/3=600.1,\quad MS_E=9{,}787.7/30=326.3 F=600.1/326.3=1.84F=600.1/326.3=1.84

Critical F3,30=2.92F_{3,30}=2.92 at 0.05. Since 1.84<2.921.84<2.92, do not reject. The data do not show statistically significant differences among average assembly times.

Biryani prices by restaurant location

Thirty restaurants were classified as shopping (n=13n=13), office (n=9n=9), or station (n=8n=8). Grand mean ₹410; group means ₹390.7, ₹393.3 and ₹460. Reported SST=172,600SS_T=172{,}600, while SSTrSS_{Tr} was inconsistently reported as 27,307 or 27,372; use 27,307 here, giving SSE=145,293SS_E=145{,}293. The within-group calculation was approximately 50,892 + 36,800 + 57,600 = 145,292, a rounding difference.

MSTr=27,307/2=13,653.5,MSE=145,292/27≈5,381MS_{Tr}=27{,}307/2=13{,}653.5,\quad MS_E=145{,}292/27\approx5{,}381 F≈13,653/5,381=2.54F\approx13{,}653/5{,}381=2.54

Critical F2,27=3.35F_{2,27}=3.35 at 0.05, so do not reject. Though the station-group sample mean is higher, its sample is small and spread substantial; comparing means by eye alone would mislead.

Key takeaways

  • ANOVA tests the omnibus claim that all group population means are equal.
  • The FF ratio compares between-group variation to within-group noise.
  • A significant result identifies some difference, not which groups differ; follow-up comparisons answer that question.
  • Sample mean separation must be read alongside sample size and spread.
  • Check SST=SSTr+SSESS_T=SS_{Tr}+SS_E and use the correct degrees of freedom.

Chi-Squared Tests and Goodness of Fit

The chi-square goodness-of-fit idea

A goodness-of-fit (GOF) test asks whether observed category counts are plausible under a specified probability distribution. It compares counts, not individual numerical distances. The null specifies the claimed probabilities; the alternative says the population distribution differs. The statistic grows when observed counts depart from expected counts:

χ2=∑j=1K(Oj−Ej)2Ej\chi^2=\sum_{j=1}^{K}\frac{(O_j-E_j)^2}{E_j}

Here OjO_j is observed count and Ej=npjE_j=np_j is expected under H0H_0. Under the course procedure, with KK categories, the reference distribution has K−1K-1 degrees of freedom. The test is right-tailed: large departures produce large χ2\chi^2. Reject if p≤αp\le\alpha or statistic exceeds the upper-tail critical value. Examples require expected counts at least 5 per category; combine or redesign categories if needed.

Discrete multinomial example: television audiences

A proposed historical audience distribution for four Telugu channels is p=(0.29,0.28,0.23,0.20)p=(0.29,0.28,0.23,0.20) (the final probability is the remainder to one). The null says the current population follows these proportions; the alternative says at least one differs. In a sample of 500, expected counts are:

E=(500)(0.29,0.28,0.23,0.20)=(145,140,115,100)E=(500)(0.29,0.28,0.23,0.20)=(145,140,115,100)

For sample counts in the worked example, four contributions sum to χ2=48.29\chi^2=48.29. With K−1=3K-1=3 degrees of freedom, this is far into the right tail at α=0.05\alpha=0.05, so reject: data do not support historical viewership proportions. This does not reveal the true replacement distribution; another proposed distribution can be tested. Under a second proposal, reported statistic is 5.52 with 3 degrees of freedom, below critical value 7.815, so do not reject that proposal. Transcript text omits all observed channel counts and the second proposal’s probabilities, so those inputs cannot be reconstructed.

Binomial GOF: six surgeries per day

A hospital example tests whether daily surgery counts have the distribution implied by six independent trials with probability p=0.5p=0.5 (success is a surgery on a particular type). Under H0H_0, XX is binomial with n=6,p=0.5n=6,p=0.5:

P(X=x)=(6x)(0.5)x(0.5)6−x=(6x)/64P(X=x)=\binom{6}{x}(0.5)^x(0.5)^{6-x}=\binom{6}{x}/64

Probabilities for counts 0 through 6 are (1,6,15,20,15,6,1)/64(1,6,15,20,15,6,1)/64. Multiply each by the number of observed days to obtain expected category counts; compare with observed frequencies using ∑(O−E)2/E\sum (O-E)^2/E. Reported statistic is 7.21 for seven categories, hence 6 degrees of freedom. At 0.05 critical value is 12.59; since 7.21<12.597.21<12.59, do not reject the binomial model with p=0.5p=0.5. Raw daily frequency table is not present in transcript text, so individual contributions cannot be recreated.

Continuous distributions: make probability bins first

For continuous data, convert the proposed distribution into bins, preferably equal-probability bins, so expected counts are adequate. Boundaries come from null-model quantiles; first and last bins include tails. Count sample observations per bin, set Ej=npjE_j=np_j, and apply the same statistic.

Normal demand: Khaja’s store

The claim is hourly demand is normal with mean 70 and standard deviation 20. For n=25n=25, make five equal-probability bins, each expected count 5. Standard-normal quintile cutoffs are approximately z=−0.84,−0.25,0.25,0.84z=-0.84,-0.25,0.25,0.84; transform by x=70+20zx=70+20z to boundaries 53.2, 65, 75, 86.8. Intervals are (−∞,53.2](-\infty,53.2], (53.2,65](53.2,65], (65,75](65,75], (75,86.8](75,86.8], (86.8,∞)(86.8,\infty). Observed counts are 4, 1, 3, 4, 5, giving contributions:

(4−5)25=0.2,(1−5)25=3.2,(3−5)25=0.8,(4−5)25=0.2,(5−5)25=0\frac{(4-5)^2}{5}=0.2,\quad\frac{(1-5)^2}{5}=3.2,\quad\frac{(3-5)^2}{5}=0.8,\quad\frac{(4-5)^2}{5}=0.2,\quad\frac{(5-5)^2}{5}=0

χ2=0.2+3.2+0.8+0.2+0=4.4\chi^2=0.2+3.2+0.8+0.2+0=4.4

With 5 bins, df=4df=4 and the 0.05 critical value is 9.48. Since 4.4<9.484.4<9.48, do not reject; the sample could have come from the stated normal distribution.

Normal demand: Venkatesh’s store

A second normal GOF example tests mean 70 and standard deviation 20, using sample size 25 and five equal-probability intervals. Reported observed counts lead to χ2=6.4\chi^2=6.4. With df=4df=4, critical value 9.48 at 0.05; since 6.4<9.486.4<9.48 and p>0.05p>0.05, do not reject the stated normal model. The sample does not provide enough evidence that demand follows a different distribution.

Exponential call duration: Madhavi’s call center

The null claims durations follow an exponential distribution with mean 6 minutes; sample size is 50. The CDF is:

F(x)=1−e−λx=1−e−x/6,λ=1/6F(x)=1-e^{-\lambda x}=1-e^{-x/6},\qquad \lambda=1/6

Use deciles so each of 10 bins has expected count 50(0.1)=550(0.1)=5. The cutoff at cumulative probability qq is x=−6ln⁡(1−q)x=-6\ln(1-q). Reported cutoffs for q=0.1,…,0.9q=0.1,\ldots,0.9 are 0.632, 1.34, 2.14, 3.06, 4.16, 5.50, 7.22, 9.66 and 13.82 minutes. Bins are [0,0.632][0,0.632], successive intervals between cutoffs, and (13.82,∞)(13.82,\infty). Observed counts are 4, 1, 3, 4, 5, 5, 8, 5, 7, 8; expected count in each bin is 5.

(4−5)25=0.2,  (1−5)25=3.2,  (3−5)25=0.8,  (4−5)25=0.2,  0,  0,  (8−5)25=1.8,  0,  (7−5)25=0.8,  (8−5)25=1.8\frac{(4-5)^2}{5}=0.2,\;\frac{(1-5)^2}{5}=3.2,\;\frac{(3-5)^2}{5}=0.8,\;\frac{(4-5)^2}{5}=0.2,\;0,\;0,\;\frac{(8-5)^2}{5}=1.8,\;0,\;\frac{(7-5)^2}{5}=0.8,\;\frac{(8-5)^2}{5}=1.8

χ2=0.2+3.2+0.8+0.2+0+0+1.8+0+0.8+1.8=8.8\chi^2=0.2+3.2+0.8+0.2+0+0+1.8+0+0.8+1.8=8.8

With 10 bins, the course uses df=9df=9; critical χ9,0.052=16.92\chi^2_{9,0.05}=16.92. Since 8.8<16.928.8<16.92, do not reject the exponential mean-6 model. One caption line mistakenly says mean 0.6; the CDF, λ=1/6\lambda=1/6, cutoffs and stated claim establish mean 6 minutes.

A practical GOF checklist

  1. State H0H_0 as the proposed population distribution and parameter values; HAH_A says it does not follow that specification.
  2. For discrete outcomes, list categories and null probabilities. For continuous outcomes, define intervals using null-model quantiles.
  3. Check expected frequency is at least 5 in each category; choose sensible equal-probability bins where possible.
  4. Count observations and calculate Ej=npjE_j=np_j.
  5. Compute each contribution (Oj−Ej)2/Ej(O_j-E_j)^2/E_j and add them.
  6. Use right-tail chi-square and course degrees of freedom K−1K-1; compare pp with α\alpha or statistic with critical value.
  7. State the contextual conclusion. Not rejecting means compatible with the proposed distribution; it does not prove it true.

Key takeaways

  • GOF evaluates a specified probability distribution through observed and expected category frequencies.
  • A large chi-square statistic means observed counts are collectively far from expectation.
  • Continuous distributions require bins; equal-probability bins make expected counts straightforward.
  • Non-rejection is lack of evidence against the proposed model, not proof of truth.
  • “The model is rejected” does not identify the true model.
Revise before your examAll 1 exam tip from this module in one list
  1. Correlation is not causation.** A high rr alone does not establish that changing one variable changes the other.

Test yourself on this module

Questions from these notes, one at a time, with feedback. Your weak topics are saved.

Set it up free Takes about a minute. Free plans work.