aggregation_agreement
mcda.aggregation_agreement
How much the recommendation depends on the choice of aggregation method.
beam offers five aggregations (SAW, TOPSIS, VIKOR, PROMETHEE II, COMET), each resting on different assumptions about how per-metric scores combine into one composite. The headline ranking uses one of them. This module checks whether another aggregation would order the tools the same way. It re-ranks the same normalized matrix under each aggregation, holding the weighting fixed, and reports how closely the resulting orderings agree.
The agreement is measured with the Kendall tau-b rank-correlation coefficient, which handles the tied ranks that competition ranking produces. A high mean pairwise tau means the recommendation is stable under the aggregation choice; a low one means the choice of method is itself a degree of freedom the report should disclose. The consensus ranking averages the per-method ranks, and a flag marks whether every aggregation puts the same tool first.
This sits alongside the other choice-sensitivity diagnostics: leave-one-metric-out and leave-one-dataset-out vary the inputs, SMAA varies the weights, and this varies the aggregation rule.
Classes
| Name | Description |
|---|---|
| AggregationAgreementReport | Outcome of re-ranking one matrix under several aggregations. |
AggregationAgreementReport
mcda.aggregation_agreement.AggregationAgreementReport(self, methods, ranks_by_method, tau_matrix, mean_pairwise_tau, consensus_ranks, consensus_order, top_tool, top_is_unanimous, rank_low, rank_high, tool_names=None)
Outcome of re-ranking one matrix under several aggregations.
Fields: methods: the aggregation names that ran, in order. An aggregation that raises on the input (for example a degenerate matrix) is dropped. ranks_by_method: per-method 1-based ranks (1 is best), keyed by method name, each of length n_tools. tau_matrix: (n_methods, n_methods) Kendall tau-b between every pair of methods’ rankings, indexed by the order in methods. The diagonal is 1. An entry is nan when one ranking is constant, so tau-b is undefined. mean_pairwise_tau: mean of the off-diagonal tau values, ignoring any nan. A scalar summary of how much the orderings agree; 1 is exact agreement across every pair. consensus_ranks: (n_tools,) ranks from the mean rank across methods, 1 is best. Ties share the lowest rank. consensus_order: tool indices sorted from best to worst by the consensus. top_tool: the consensus rank-1 tool index. top_is_unanimous: True when every method ranks top_tool first. rank_low, rank_high: (n_tools,) best and worst rank each tool takes across the methods, the rank span behind the funky-heatmap consensus panel. tool_names: optional labels carried for reporting.
Functions
| Name | Description |
|---|---|
| aggregation_agreement | Re-rank one matrix under several aggregations and measure their agreement. |
aggregation_agreement
mcda.aggregation_agreement.aggregation_agreement(scores, polarity, weights='equal', methods=None, normalization=None, bounds=None, baselines=None, targets=None, missing='error', tool_names=None)
Re-rank one matrix under several aggregations and measure their agreement.
Normalization and weighting happen before aggregation and do not depend on the aggregation rule, so the weight vector is identical across the methods and the only thing that varies is how the normalized scores combine into a composite. For each method in methods the function runs the full pipeline through beam.mcda.run, collects the per-tool ranks, and compares every pair of rankings with the Kendall tau-b coefficient.
A method that raises on the input is dropped from the report rather than failing the whole analysis, matching how the funky-heatmap consensus panel already treats a method that cannot run. At least two methods must succeed.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
scores |
Array-like of shape (n_tools, n_metrics). |
required | |
polarity |
Sequence[str] | Length n_metrics sequence of "higher_is_better" or "lower_is_better". Use beam.cards.polarities_for to source this from the registry. |
required |
weights |
Forwarded to run. A named scheme or an explicit array. Held fixed across the methods. |
'equal' |
|
methods |
Sequence[str] | None | Aggregation names to compare. Default is the five beam aggregations (SAW, TOPSIS, VIKOR, PROMETHEE II, COMET). | None |
normalization |
Optional per-metric normalization context forwarded to every run. Pass the values from beam.mcda.registry_context so the comparison rests on the same normalized matrix as the headline ranking. |
None |
|
bounds |
Optional per-metric normalization context forwarded to every run. Pass the values from beam.mcda.registry_context so the comparison rests on the same normalized matrix as the headline ranking. |
None |
|
baselines |
Optional per-metric normalization context forwarded to every run. Pass the values from beam.mcda.registry_context so the comparison rests on the same normalized matrix as the headline ranking. |
None |
|
targets |
Optional per-metric normalization context forwarded to every run. Pass the values from beam.mcda.registry_context so the comparison rests on the same normalized matrix as the headline ranking. |
None |
|
missing |
str | Missing-data policy forwarded to every run. Default "error". |
'error' |
tool_names |
Sequence[str] | None | Optional length n_tools labels carried in the report. |
None |
Returns
| Type | Description |
|---|---|
| AggregationAgreementReport |
Raises
| Type | Description |
|---|---|
| ValueError | If fewer than two of the requested methods produce a ranking. |
Examples
>>> import numpy as np
>>> from beam.mcda import aggregation_agreement
>>> scores = np.array([[0.9, 30.0], [0.7, 50.0], [0.5, 40.0]])
>>> report = aggregation_agreement(
... scores, ["higher_is_better", "lower_is_better"]
... )
>>> report.top_is_unanimous
True