bayesian_sign_comparison
mcda.bayesian_sign_comparison(report, prior_strength=1.0, prior_placement='rope', decision_threshold=0.95, n_samples=50000, seed=42)
Posterior probability that one method is practically better than another.
Reads a PairwiseSuperiorityReport and applies the Bayesian sign test of Benavoli et al. (2017) to each pair’s outperformance counts. The proportions of future datasets in the three regions (A practically better, equivalent, B practically better) follow a Dirichlet posterior whose parameters are the observed counts plus prior_strength pseudo-observations placed by prior_placement. The probability that a region holds the largest share is estimated by drawing n_samples from the posterior; the posterior mean share of each region is the closed-form Dirichlet mean.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
report |
PairwiseSuperiorityReport | A PairwiseSuperiorityReport from beam.mcda.pairwise_superiority. Its ROPE (set when the report was built, often the metric’s noise floor) is the equivalence band used here, so the equivalence count already reflects it. |
required |
prior_strength |
float | Number of prior pseudo-observations. Default 1, a single weak prior dataset. Must be non-negative. | 1.0 |
prior_placement |
str | Where the prior mass sits: "rope" (default, all on the equivalence region, the weak prior that the two are equivalent, matching the baycomp default), "uniform" (split across the three regions), or "neutral" (split across the two directional regions, none on the equivalence region). |
'rope' |
decision_threshold |
float | The posterior probability a region must reach for a decisive per_pair decision label. Default 0.95. |
0.95 |
n_samples |
int | Monte Carlo draws from each posterior. Default 50000. | 50000 |
seed |
int | Seed for the draws, so two runs reproduce. Default 42. | 42 |
Returns
| Type | Description |
|---|---|
| BayesianSignReport |
Raises
| Type | Description |
|---|---|
| ValueError | If prior_strength is negative, prior_placement is unknown, or decision_threshold is not in (0, 1]. |
Examples
>>> import numpy as np
>>> from beam.mcda import pairwise_superiority, bayesian_sign_comparison
>>> high = [0.9, 0.8, 0.7, 0.85, 0.75, 0.95, 0.8, 0.9]
>>> low = [0.2, 0.1, 0.3, 0.15, 0.25, 0.05, 0.2, 0.1]
>>> sup = pairwise_superiority(np.array([high, low]), "higher_is_better")
>>> bayes = bayesian_sign_comparison(sup, seed=0)
>>> bayes.per_pair[0].decision # method 0 better on every dataset
'a_better'
>>> float(bayes.probability_better[0, 1]) > 0.95
True