pairwise_superiority

mcda.pairwise_superiority(scores, polarity, rope=0.0, method_names=None, alpha=0.05)

Compare every method pair across datasets with a probability of superiority.

Reads a tool-by-dataset matrix on one metric (or a composite). Each pair is compared on the datasets where both are observed. Two methods are equivalent on a dataset when their scores differ by no more than rope; otherwise the higher-scoring method outperforms the other, with the direction of “higher” set by polarity. The probability of superiority of A over B is the fraction of shared datasets on which A outperforms B, and a sign test on the decisive datasets says whether the difference is more than chance.

Parameters

Name	Type	Description	Default
scores		Array-like of shape (n_methods, n_datasets) in native units. Missing cells are NaN and dropped per pair.	required
polarity	str	"higher_is_better" or "lower_is_better", the direction in which a higher score means a method outperforms.	required
rope	float	The region of practical equivalence in native units. A difference within it is treated as equivalent. Pass the metric’s comparability.noise_floor to count an outperformance only past the smallest interpretable difference. Default 0.	0.0
method_names	Sequence[str] | None	Optional length n_methods labels carried in the report.	None
alpha	float	Significance level for equivalent_pairs. Default 0.05.	0.05

Returns

Type	Description
PairwiseSuperiorityReport

Raises

Type	Description
ValueError	If scores is not 2D, has fewer than two methods or two datasets, polarity is not monotone, rope is negative, or method_names has the wrong length.

Examples

>>> import numpy as np
>>> from beam.mcda import pairwise_superiority
>>> scores = np.array([[0.9, 0.8, 0.7], [0.5, 0.6, 0.4], [0.2, 0.1, 0.3]])
>>> report = pairwise_superiority(scores, "higher_is_better")
>>> int(report.order[0])  # method 0 outperforms the others on every dataset
0
>>> float(report.probability_superior[0, 1])
1.0