Skip to content

EmbeddingCollapseMetric: effective-rank score has a 1/N floor when N <= D #8996

Description

@rubenG1009

Describe the bug

_effective_rank_score (monai/metrics/embedding_collapse.py:298) mean-centers the embedding matrix, which caps its attainable rank at min(N-1, D), but normalizes the effective rank by min(N, D).

Since the score is 1 - eff_rank/max_rank, this imposes a hard floor of 1/N whenever N <= D - so the metric structurally cannot report "no collapse" in the regime it was designed for (high-dimensional embeddings, modest sample counts).

_per_class_rank is affected more, since it partitions N by class: the smaller per-class N makes the floor larger.

To Reproduce

Isotropic embeddings - no collapse is present, so the correct score is ~0:

N D score now expected
2 768 0.5000 0.0
4 768 0.2503 ~0.0004
8 768 0.1262 ~0.0014
64 768 0.0261 ~0.0106
1024 768 0.1234 0.1234 (N > D: unaffected)

Two maximally-spread points score exactly 0.50 ("50% collapsed") when they span the only subspace two samples can span:

import torch
from monai.metrics.embedding_collapse import _effective_rank_score

emb = torch.zeros(2, 768)
emb[0, 0], emb[1, 0] = 1.0, -1.0
print(float(_effective_rank_score(emb)))  # 0.5, expected 0.0

The observed floor matches 1 - (N-1)/N = 1/N exactly.

Expected behavior

Normalize by min(N-1, D), the attainable maximum after centering. This is a no-op for N > D, where both expressions equal D.

Environment

Introduced in #8815 (merged 2026-04-21), shipped in 1.6.0.

I have a fix with tests ready and will open a PR shortly.

One design question: for N = 2 the centered matrix is always rank-1, so the score is a constant regardless of the data. Is returning 0.0 right there, or would None be preferable so it is excluded from aggregate(), as _class_collapse does for a single class?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions