critical_difference
mcda.critical_difference(scores, higher_is_better=True, alpha=0.05, tool_names=None)
Run the Friedman test and Nemenyi post-hoc on a tool by dataset matrix.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
scores |
np.ndarray | 2D array of shape (n_tools, n_datasets) for one metric or for an MCDA composite. Each column holds the scores of every tool on one dataset. |
required |
higher_is_better |
bool | Whether a larger score is better. When False the ranking is reversed before ranks are taken, so the reported average ranks always put the better tool nearer 1. The Friedman statistic itself does not depend on the direction. |
True |
alpha |
float | Significance level for the Nemenyi critical difference. | 0.05 |
tool_names |
Sequence[str] | None | Optional labels, length n_tools, kept in the report. |
None |
Returns
| Type | Description |
|---|---|
| CriticalDifferenceReport |
Notes
Friedman needs at least three tools and two datasets, and a complete tool by dataset table: every tool ranked on every dataset. A missing cell raises an error, since the Friedman ranks per dataset are only defined over a complete column. The missing-data generalization is the Skillings-Mack (1981) test in beam.mcda.skillings_mack; alternatively, restrict the diagram to the block of tools and datasets where all of them ran.