Compute score contributions from the data, not from the score

[kgdunn]

Summary

• score_contributions() on PCA, PLS, MBPCA and MBPLS computed (t_end - t_start) @ P, an expression that never receives the observation's data. With one component it reduces exactly to -t_1 * p_1, so it returned the loading vector rescaled by a constant and gave every observation the same ranking of variables. It is now the term-by-term breakdown of the sum that forms the score, c_ik = x_ik * R_ka, which sums back to t_ia.
• Added group_contributions() for the cluster and two-period forms from the same paper, which is what the old t_start / t_end pair was reaching for, and scaling= for the paper's two presentation scalings.
• Breaking: score_contributions() now takes X and returns one row per observation, matching t2_contributions() and spe_contributions() beside it. The old signature cannot be supported alongside the new one, since a score vector does not carry the information the calculation needs, so old-form calls raise a TypeError naming the replacement.

The defect

A contribution is meant to answer "which variables put this observation where it is?". The old expression could not answer it, because it is a function of the scores and loadings only. Measured on real data:

	process-improve (before)	correct
Distinct variable rankings across the 50 food-texture observations	1	34
Top-blamed variable differs from the correct answer	36 of 50 (food texture)
Top-blamed variable differs from the correct answer	47 of 54 (LDPE)

One ranking for fifty observations is the loading ranking, reproduced fifty times.

That is precisely the failure the diagnostic exists to prevent. Miller, Swanson and Heckler, who introduced contribution plots, give a batch whose loadings point at the silver-concentration variables while its contributions point at the salt and silver pressure variables; the pressures were the real fault, confirmed by univariate charts and traced to a replaced valve. Their Figure 2 shows the two bar charts side by side and they share no visual resemblance. A loading describes the whole data set; a contribution describes one observation, and a variable with a large loading contributes nothing when that observation sits at its mean.

Worth noting: docs/user_guide/pca.rst already stated the correct formula, x_{i,k} p_{k,a}, in prose immediately above code calling the old back-projection. The documentation and the implementation had drifted apart.

What the calculation should be

The defining property is that the per-variable terms add up to the score being decomposed:

c_ik = x_ik * R_ka          sum_k c_ik = t_ia

with R the score-generating matrix: loadings_ for PCA, direct_weights_ for PLS (so that T = XR). That generalisation from PCA loadings to any latent-variable model's weights follows Westerhuis, Gurden and Smilde (2000). For the multiblock models the super score at component a is formed from the block data deflated through the earlier components, so the decomposition starts from that same deflated data; summing over every block and variable returns the super score.

t2_contributions() and spe_contributions() were already correct and are unchanged. t2_contributions multiplies by X_values and telescopes to T², which is exactly the pattern score_contributions was missing one function away.

Migration

Before	After
model.score_contributions(model.scores_.iloc[i])	model.score_contributions(X).iloc[i]
model.score_contributions(t_a, t_b)	model.group_contributions(X, group=[a], reference=[b])
model.score_contributions(t, weighted=True)	model.t2_contributions(X) (already correct)

Test plan

• New tests assert the defining property, that contributions sum to the score, for every component of PCA, PLS, MBPCA and MBPLS, and on the real LDPE blocks. The previous tests asserted only that the output equalled dt @ P.T, which re-derived the implementation and so held whatever it computed; that is why this survived.
• A test asserts the ranking varies across observations, which is the specific defect above.
• A test asserts a variable sitting at its mean contributes zero regardless of its loading.
• Tests cover the migration errors, both scalings, and the selector forms of group_contributions (labels, positions, boolean mask).
• uv run pytest tests/test_multivariate.py tests/test_multiblock_reference.py tests/test_multivariate_tsr.py tests/test_multivariate_robustness.py -> 274 passed, 1 skipped.
• uv run ruff check . and uv run mypy src/process_improve both pass.
• Both case-study notebooks execute and print the identity holding: the flagged food-texture sample sums to -1.1107 against a score of -1.1107, the flagged tablet to -53.4725 against -53.4725.
• uv run sphinx-build docs docs/_build/html -b html -W reaches build finished with one warning, an autodoc import failure on process_improve.batch.plotting caused by seaborn not being installed in this sandbox. It is unrelated to this change (it also breaks tests/batch/test_plotting.py here) and does not occur in CI, which installs --all-extras.

Pre-existing failures unrelated to this change, confirmed identical on a stashed tree: 130 failures across the experiments / design-generation suites.

Checklist

• Version bumped in pyproject.toml (MINOR: 1.60.0 -> 1.61.0; the method changes shape and the old convention now raises). CITATION.cff version and date-released synced.
• Tests added or updated where relevant
• ruff check . passes
• CHANGELOG.md updated

References

Miller, P., Swanson, R.E. and Heckler, C.E. (1994). "Contribution plots: a missing link in multivariate quality control." Applied Mathematics and Computer Science, 8(4), 775-792.

Westerhuis, J.A., Gurden, S.P. and Smilde, A.K. (2000). "Generalized contribution plots in multivariate statistical process monitoring." Chemometrics and Intelligent Laboratory Systems, 51(1), 95-114.

---

Generated by Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[kgdunn]

Pushed af70dac for the Codecov patch coverage. Writing the missing tests turned up a real defect rather than only a coverage gap, and it carries a small semantic decision worth your eye.

group_contributions() selected rows inconsistently. _select_rows fell back to positional selection whenever a label was missing, which made the meaning depend on the data. On the LDPE blocks, indexed 1..54:

model.group_contributions(x_blocks, group=[0..9], reference=[10..19])
#                                          ^ positions        ^ labels

0 is not a label, so the group resolved positionally; every entry of 10..19 is a label, so the reference resolved by label. The two halves of one comparison were resolved by different rules and the answer was silently off by one row. My own test caught it: it expected the shift in average super score and got -0.2859 against -0.4018.

Selection is now by index label or boolean mask only. Callers wanting positions pass X.index[64:74], which the error message says explicitly. I preferred refusing over guessing here: a rule that quietly changes with the index is worse than one that fails loudly, particularly on a diagnostic people act on. Happy to reinstate a positional path behind an explicit argument if you would rather have the convenience.

Coverage of the new code after this commit:

• _scale_block_contributions fully exercised, including both of the paper's scalings and the joint-over-blocks behaviour that distinguishes them from scaling each block on its own.
• _diagnostics.py at 97% under the contribution tests alone; the remaining misses sit in unrelated functions.
• The MBPCA and MBPLS guards were each tested on one class only. Both are now covered, along with blocks supplied as plain ndarrays.

278 passed, 1 skipped across the four multivariate suites; ruff check and mypy both clean.

---

Generated by Claude Code

[kgdunn]

Added tests/test_multivariate_contribution_plots.py in 5af109e: 24 tests written against the paper rather than against the code, each citing the equation, page or figure it encodes.

Source recorded in the module docstring and linked from every test group:
https://literature.learnche.org/item/78/contribution-plots-a-missing-link-in-multivariate-quality-control

Paper	What is pinned
Eq. 1 & 2, p.4	T² from x S⁻¹ x' equals T² from the scores when no dimension is dropped; t2_contributions sums to T²
Eq. 3, p.6-7	Contributions sum to the score, every component, for PCA, PLS and the multiblock super score; each term is literally x_ij · p_jd
p.7-8, Fig. 2	A large loading contributes nothing when the observation sits at its mean, so the loading ranking and the contribution ranking disagree
Eq. 4, p.10 & p.12	Q contributions are the squares of the k residual elements: non-negative, summing to Q. Also per block
p.14	The group average form; and its limiting case, where averaging over every batch gives zero because the data are centred
p.18, Fig. 9	The +0.1 / -0.1 level-shift weights, asserted against the formula exactly as the paper prints it, and against the group/reference spelling
p.18	A drift weighted by the first-order orthogonal polynomial
p.20, Method 1	Divide by the largest absolute contribution in the data set; exactly one variable-batch pair attains ±1
p.21, Method 2	Row absolute values sum to 1, and equal the proportion exactly when every contribution shares a sign
p.20	Both scalings "leave the pattern of bar heights unchanged": constant positive ratio within a row, ranking untouched

These are not vacuous. The old tests asserted contrib == dt @ P.T, which re-derives the implementation and so holds whatever it computes; that is why the defect survived. To check the new ones do not have the same property, I temporarily reintroduced the back-projection:

9 failed, 12 passed

including every load-bearing test. Restored, and the suite is green again.

Two things came out of writing them.

The paper's general form is wider than the API was. Page 14 says the data may be replaced by "a suitably chosen average, or other linear combination of the data", and page 18 gives a case group/reference cannot express: a run of batches drifting rather than stepping, weighted by the first-order orthogonal polynomial. group_contributions() now takes weights= for that, with group/reference as sugar over it. This is a small addition beyond the bug fix, made because the paper is the specification for this PR; say the word and I will pull it back out.

The last Codecov gaps were the multiblock spe_contributions input guards, which is the multiblock form of equation 4, so they are covered in the same file.

303 passed, 1 skipped across the five multivariate suites; ruff check and mypy clean.

---

Generated by Claude Code