specification_curve
mcda.specification_curve
List every ranking the analyst’s choices produce, and report how stable it is.
rank_sensitivity runs the full factorial of the weighting scheme, the aggregation rule and (for a tensor) the dataset, and splits the rank variance into a share per factor. That answers which choice moves the ranking. It does not list the rankings themselves.
specification_curve does. It reads the per-combination ranks that rank_sensitivity already computed and turns them into one record per combination: the factor levels that define it, the full tool ordering it produces, and the tool it ranks first. From those records it reports how stable the top method is: the fraction of combinations that rank the same tool first, the fraction that produce the single most common ordering, and how many distinct tools rank first in at least one combination.
This is the specification-curve form used in meta-research (Simonsohn, Simmons and Nelson 2020; Steegen, Tuerlinckx, Gelman and Vanpaemel 2016): report the ranking under every combination of choices rather than one, so the reader can see how much it varies.
Classes
| Name | Description |
|---|---|
| Specification | One combination of analyst choices and the ranking it produces. |
| SpecificationCurveReport | Every ranking the factorial produces, with stability summaries. |
Specification
mcda.specification_curve.Specification(self, index, weighting, aggregation, dataset, ranks, ordering, top_tool)
One combination of analyst choices and the ranking it produces.
Attributes
| Name | Type | Description |
|---|---|---|
| index | int | Position of this combination in the factorial, in row-major order over the weighting, aggregation and (for a tensor) dataset axes. |
| weighting | str | The weighting scheme of this combination. |
| aggregation | str | The aggregation rule of this combination. |
| dataset | str | None | The dataset of this combination for a tensor input, or None for a matrix. |
| ranks | tuple[int, …] | The competition rank of every tool under this combination, indexed by tool. Rank 1 is best. Ties share the lower rank. |
| ordering | tuple[int, …] | Tool indices sorted from best to worst rank, ties broken by tool index. |
| top_tool | int | The tool index ranked first. On a tie the lowest tool index is taken. |
SpecificationCurveReport
mcda.specification_curve.SpecificationCurveReport(self, factors, weightings, methods, dataset_names, tool_names, n_specifications, specifications, curve_order, most_frequent_top_tool, most_frequent_top_fraction, distinct_top_tools, n_distinct_top_tools, modal_order, modal_order_fraction)
Every ranking the factorial produces, with stability summaries.
Attributes
| Name | Type | Description |
|---|---|---|
| factors | tuple[str, …] | The factor names that vary, in axis order: ("weighting", "aggregation") for a matrix, with "dataset" appended for a tensor. |
| weightings, methods | The weighting schemes and aggregations that formed the factorial. | |
| dataset_names | tuple[str, …] | None | The datasets that formed the third factor, or None for a matrix. |
| tool_names | tuple[str, …] | None | Tool labels in index order, or None when the input carried none. |
| n_specifications | int | The number of combinations in the factorial. |
| specifications | tuple[Specification, …] | One Specification per combination, in factorial order. |
| curve_order | tuple[int, …] | Indices into specifications sorted for plotting: by the rank that the method ranking first most often takes, then by combination index. A specification curve reads left to right along this order. |
| most_frequent_top_tool | int | The tool index that ranks first in the most combinations. |
| most_frequent_top_fraction | float | The fraction of combinations that rank most_frequent_top_tool first. |
| distinct_top_tools | tuple[int, …] | The tool indices that rank first in at least one combination, sorted. |
| n_distinct_top_tools | int | The length of distinct_top_tools: how many tools rank first in at least one combination. |
| modal_order | tuple[int, …] | The single most common full tool ordering across the combinations. |
| modal_order_fraction | float | The fraction of combinations that produce modal_order. |
Functions
| Name | Description |
|---|---|
| specification_curve | Build the specification curve from a rank_sensitivity report. |
specification_curve
mcda.specification_curve.specification_curve(report)
Build the specification curve from a rank_sensitivity report.
Reads the per-combination ranks in report.ranks and produces one Specification per combination, then the stability summaries. Does no new ranking: it post-processes the factorial rank_sensitivity already ran, so it works the same for a matrix and a tensor input.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
report |
RankSensitivityReport | A RankSensitivityReport from beam.mcda.rank_sensitivity. |
required |
Returns
| Type | Description |
|---|---|
| SpecificationCurveReport |
Examples
>>> import numpy as np
>>> from beam.mcda import rank_sensitivity, specification_curve
>>> scores = np.array([[0.9, 30.0], [0.7, 50.0], [0.5, 40.0], [0.3, 70.0]])
>>> rs = rank_sensitivity(scores, ["higher_is_better", "lower_is_better"])
>>> sc = specification_curve(rs)
>>> sc.n_specifications == len(rs.weightings) * len(rs.methods)
True
>>> 0.0 <= sc.most_frequent_top_fraction <= 1.0
True