critical_difference
mcda.critical_difference(scores, higher_is_better=True, alpha=0.05, tool_names=None)
Run the Friedman test and Nemenyi post-hoc on a tool by dataset matrix.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
scores |
np.ndarray | 2D array of shape (n_tools, n_datasets) for one metric or for an MCDA composite. Each column holds the scores of every tool on one dataset. |
required |
higher_is_better |
bool | Whether a larger score is better. When False the ranking is reversed before ranks are taken, so the reported average ranks always put the better tool nearer 1. The Friedman statistic itself does not depend on the direction. |
True |
alpha |
float | Significance level for the Nemenyi critical difference. | 0.05 |
tool_names |
Sequence[str] | None | Optional labels, length n_tools, carried in the report. |
None |
Returns
| Type | Description |
|---|---|
| CriticalDifferenceReport |
Notes
Friedman needs at least three tools and two datasets, and a complete tool by dataset table: every tool ranked on every dataset. A missing cell is refused rather than dropped or filled, since the Friedman ranks per dataset are only defined over a complete column. The missing-data generalization is the Skillings-Mack (1981) test, which is not implemented here; restrict the diagram to the block of tools and datasets where all of them ran.