Covariance and Correlation
Covariance: direction of joint movement
Covariance describes whether paired values of two quantitative variables tend to lie on the same side of their respective averages. A positive value means above-average values of one tend to accompany above-average values of the other (and below with below); a negative value means one tends to be above average when the other is below. It describes linear co-movement, not cause and effect.
For paired sample observations , with sample means and :
The denominator reflects the degrees-of-freedom adjustment used for sample variance. Each product is positive in the upper-right and lower-left quadrants around , and negative in the other two. If , covariance reduces to sample variance.
In a sample of 55 Canara Bank customers, monthly credit-card spend had mean ₹3,852, median ₹3,864 and standard deviation ₹1,454. Spend and household income had sample covariance 16,109; spend and household size had covariance 1,610. The sign indicates positive association. The magnitude depends on units: expressing income in rupees instead of lakhs rescales covariance without changing the underlying relationship.
Pearson correlation: unit-free linear association
Pearson’s sample correlation coefficient standardizes covariance by both sample standard deviations:
It is unit-free and lies between and . A value of means all points lie on a positively sloped straight line; means a negatively sloped line. Values nearer either endpoint mean tighter linear alignment; a value near zero means weak linear association, not necessarily no relationship.
For spend and income, lakhs, , and :
For spend and household size, and :
Both are positive; the sample shows stronger linear association between spend and income. Excel COVARIANCE.S and CORREL return sample covariance and correlation; reported results are 16,109 and 0.806 for spend-income, and 1,610 and 0.629 for spend-household size.
Pearson correlation only detects linear association. For and , symmetry makes sample covariance and correlation zero, even though is exactly determined by through a nonlinear relationship. A scatterplot is essential to spot curvature. Utility spending can be higher at very hot and very cold temperatures than at moderate temperatures, a U-shaped relation with near-zero linear correlation.
Explainable correlation has a plausible relationship; spurious correlation does not have a supported causal explanation. A third, latent variable may affect both measured variables: summer temperature can coincide with higher ice-cream sales and crime (more homes left empty during vacations); poor village health can lead both to more doctors and more deaths. A reported correlation of 0.99 between Maine divorce rates and per-capita margarine consumption is presented as spurious, without an identified hidden cause. The hemline theory attributed shorter skirts to economic prosperity and longer skirts in downturns, explaining the proposed association through ability to afford stockings; such patterns require scrutiny. For a nominal or ordinal variable, other measures such as Spearman rank correlation may be appropriate. Population claims about correlation require inference; the related significance test uses a distribution, though its procedure is outside this treatment.
Key takeaways
- Covariance’s sign gives direction; its magnitude changes with measurement units.
- Pearson standardizes covariance and measures only linear association.
- Zero correlation does not rule out nonlinear dependence.
- Association, even when strong, is not proof of causation; inspect plausible confounders and plots.
One-Way Analysis of Variance (ANOVA)
Why compare several means with ANOVA?
Analysis of variance (ANOVA) tests whether means of two or more populations can be treated as equal using samples from each. Applications include policy outcomes across states, salaries across educational backgrounds, products or technologies, and disease risk across groups. The groups are called treatments or factor levels; the measured outcome is the response. In one-way ANOVA, one factor defines the groups and one quantitative response is measured.
For population means:
The alternative means at least one differs; it does not claim every mean differs. ANOVA is called analysis of variance because it tests mean differences by decomposing and comparing variability. A given separation between sample means is compelling when observations within groups are tightly clustered; the same separation is weak evidence when within-group spread is large. ANOVA compares between-treatment variation with within-treatment (error) variation.
Sums of squares and the F test
Let group contain observations, total , group mean , and grand mean . Partition total variability:
Treatment degrees of freedom are ; error degrees of freedom are . Divide each sum by its degrees of freedom:
Under , both mean squares estimate the common population variance, so tends to be near 1. A large indicates treatment means vary more than expected from within-group noise. Its reference distribution has degrees of freedom; reject for a sufficiently small upper-tail -value (at most ), or when exceeds the upper-tail critical value. Failure to reject is not proof that means are equal.
Worked example: investors in Palnadu
Fifteen investors were grouped by age: young (), middle-aged (), and senior (). Sample means in thousands were 60, 57.2 and 54.8. These sample differences alone do not settle whether population means differ: within-group variability matters too. From the data, , , , hence:
The reported -value is 0.02, so at 0.05 reject equality: at least one population mean differs. The source’s raw table is not reproduced in its transcript text, so the sums of squares cannot be recalculated observation by observation; reported results are retained.
Same group sizes, different evidence
For another sample of 15 investors with group sizes 4, 5 and 6, means 41, 42, 41.67 and standard deviations 24.08, 21.12, 18.61, reported values are , , , , . Thus and reported : do not reject. Similar averages amid high spread offer little evidence of different population means.
For a third sample with same sizes, means 60, 37.2 and 20, standard deviations 2.31, 2.28 and 2.31. Reported , , , , , so and is near zero: reject. Means are far apart relative to within-group spread. Boxplots help visualize, but inference comes from the test.
ANOVA table and computation
| Source | Sum of squares | Degrees of freedom | Mean square | Test statistic |
|---|---|---|---|---|
| Treatments | ||||
| Error | — | |||
| Total | — | — |
For hand calculation, find the grand mean, square each observation’s deviation from it to get , calculate group means and their weighted deviations for , then calculate within-group deviations for (or subtract from ). Verify the partition before calculating degrees of freedom, mean squares and . Excel’s Analysis ToolPak offers “ANOVA: Single Factor”; select group columns, mark labels if included, set (0.05 in examples), and choose an output location. Software speeds arithmetic; understanding the table supports interpretation.
Post-hoc pairwise comparisons
A significant omnibus test says at least one mean differs, but not which. Post-hoc analysis investigates pairs. In the demonstrated simple Excel approach, run one-way ANOVA on each pair. For Palnadu investors: young versus middle-aged had , , , , , (do not reject); middle-aged versus senior had , , , (do not reject); young versus senior had , , (reject). These comparisons suggest young and senior means differ. Repeated pairwise tests can be tedious and have limitations; advanced methods are beyond scope.
Further ANOVA examples
Audit-team composition
Twenty audit-duration observations were split among local-only (), corporate-only () and mixed () teams. Grand mean 21.05 days; group means 25.3, 20.8, 17. Data give , , (rounding differences), with degrees of freedom 2 and 17:
At , critical . Since , reject: average audit completion times are not all equal. This does not identify which pair differs; three pairwise comparisons would be needed.
Assembly methods
For 34 transmission assemblies, methods A–D had sample sizes 8, 11, 9, 6; grand mean 107.2 and means 117, 100, 102.6, 114.2. From observations, and (rounding), so . Degrees of freedom are 3 and 30:
Critical at 0.05. Since , do not reject. The data do not show statistically significant differences among average assembly times.
Biryani prices by restaurant location
Thirty restaurants were classified as shopping (), office (), or station (). Grand mean ₹410; group means ₹390.7, ₹393.3 and ₹460. Reported , while was inconsistently reported as 27,307 or 27,372; use 27,307 here, giving . The within-group calculation was approximately 50,892 + 36,800 + 57,600 = 145,292, a rounding difference.
Critical at 0.05, so do not reject. Though the station-group sample mean is higher, its sample is small and spread substantial; comparing means by eye alone would mislead.
Key takeaways
- ANOVA tests the omnibus claim that all group population means are equal.
- The ratio compares between-group variation to within-group noise.
- A significant result identifies some difference, not which groups differ; follow-up comparisons answer that question.
- Sample mean separation must be read alongside sample size and spread.
- Check and use the correct degrees of freedom.
Chi-Squared Tests and Goodness of Fit
The chi-square goodness-of-fit idea
A goodness-of-fit (GOF) test asks whether observed category counts are plausible under a specified probability distribution. It compares counts, not individual numerical distances. The null specifies the claimed probabilities; the alternative says the population distribution differs. The statistic grows when observed counts depart from expected counts:
Here is observed count and is expected under . Under the course procedure, with categories, the reference distribution has degrees of freedom. The test is right-tailed: large departures produce large . Reject if or statistic exceeds the upper-tail critical value. Examples require expected counts at least 5 per category; combine or redesign categories if needed.
Discrete multinomial example: television audiences
A proposed historical audience distribution for four Telugu channels is (the final probability is the remainder to one). The null says the current population follows these proportions; the alternative says at least one differs. In a sample of 500, expected counts are:
For sample counts in the worked example, four contributions sum to . With degrees of freedom, this is far into the right tail at , so reject: data do not support historical viewership proportions. This does not reveal the true replacement distribution; another proposed distribution can be tested. Under a second proposal, reported statistic is 5.52 with 3 degrees of freedom, below critical value 7.815, so do not reject that proposal. Transcript text omits all observed channel counts and the second proposal’s probabilities, so those inputs cannot be reconstructed.
Binomial GOF: six surgeries per day
A hospital example tests whether daily surgery counts have the distribution implied by six independent trials with probability (success is a surgery on a particular type). Under , is binomial with :
Probabilities for counts 0 through 6 are . Multiply each by the number of observed days to obtain expected category counts; compare with observed frequencies using . Reported statistic is 7.21 for seven categories, hence 6 degrees of freedom. At 0.05 critical value is 12.59; since , do not reject the binomial model with . Raw daily frequency table is not present in transcript text, so individual contributions cannot be recreated.
Continuous distributions: make probability bins first
For continuous data, convert the proposed distribution into bins, preferably equal-probability bins, so expected counts are adequate. Boundaries come from null-model quantiles; first and last bins include tails. Count sample observations per bin, set , and apply the same statistic.
Normal demand: Khaja’s store
The claim is hourly demand is normal with mean 70 and standard deviation 20. For , make five equal-probability bins, each expected count 5. Standard-normal quintile cutoffs are approximately ; transform by to boundaries 53.2, 65, 75, 86.8. Intervals are , , , , . Observed counts are 4, 1, 3, 4, 5, giving contributions:
With 5 bins, and the 0.05 critical value is 9.48. Since , do not reject; the sample could have come from the stated normal distribution.
Normal demand: Venkatesh’s store
A second normal GOF example tests mean 70 and standard deviation 20, using sample size 25 and five equal-probability intervals. Reported observed counts lead to . With , critical value 9.48 at 0.05; since and , do not reject the stated normal model. The sample does not provide enough evidence that demand follows a different distribution.
Exponential call duration: Madhavi’s call center
The null claims durations follow an exponential distribution with mean 6 minutes; sample size is 50. The CDF is:
Use deciles so each of 10 bins has expected count . The cutoff at cumulative probability is . Reported cutoffs for are 0.632, 1.34, 2.14, 3.06, 4.16, 5.50, 7.22, 9.66 and 13.82 minutes. Bins are , successive intervals between cutoffs, and . Observed counts are 4, 1, 3, 4, 5, 5, 8, 5, 7, 8; expected count in each bin is 5.
With 10 bins, the course uses ; critical . Since , do not reject the exponential mean-6 model. One caption line mistakenly says mean 0.6; the CDF, , cutoffs and stated claim establish mean 6 minutes.
A practical GOF checklist
- State as the proposed population distribution and parameter values; says it does not follow that specification.
- For discrete outcomes, list categories and null probabilities. For continuous outcomes, define intervals using null-model quantiles.
- Check expected frequency is at least 5 in each category; choose sensible equal-probability bins where possible.
- Count observations and calculate .
- Compute each contribution and add them.
- Use right-tail chi-square and course degrees of freedom ; compare with or statistic with critical value.
- State the contextual conclusion. Not rejecting means compatible with the proposed distribution; it does not prove it true.
Key takeaways
- GOF evaluates a specified probability distribution through observed and expected category frequencies.
- A large chi-square statistic means observed counts are collectively far from expectation.
- Continuous distributions require bins; equal-probability bins make expected counts straightforward.
- Non-rejection is lack of evidence against the proposed model, not proof of truth.
- “The model is rejected” does not identify the true model.
Revise before your examAll 1 exam tip from this module in one list
- Correlation is not causation.** A high alone does not establish that changing one variable changes the other.
Test yourself on this module
Questions from these notes, one at a time, with feedback. Your weak topics are saved.