card_data_consistency

mcda.card_data_consistency(scores, polarity, bounds, baselines=None, targets=None, noise_floors=None, metric_ids=None, range_tol=0.0)

Audit each metric card’s declared numeric claims against the raw scores.

Reads the native-unit score matrix against the values the cards declare, before any normalization, and reports where they disagree. Pass the card values from a resolved context: context.bounds, context.baselines, context.targets and context.noise_floors of a beam.mcda.registry_context line up with context.polarity and the matrix columns.

The checks per metric:

  • malformed_range (violation): the declared lower bound exceeds the upper.
  • out_of_range (violation): an observed score falls below the lower bound or above the upper bound by more than range_tol.
  • baseline_out_of_range (violation): a declared chance baseline lies outside the declared range. A target_value metric has no chance level, so its baseline, if any, is not checked.
  • target_out_of_range (violation): a declared target lies outside the declared range.
  • nonpositive_noise_floor (violation): a declared noise floor is zero or negative, which is not a width in native units.
  • noise_floor_exceeds_spread (note): a positive noise floor is at least as large as the whole observed spread, so the metric separates no pair of tools on this data.
  • degenerate (note): the metric is constant across the observed tools, so it carries no ranking signal here.
  • no_observations (note): every cell of the metric is NaN.

Parameters

Name Type Description Default
scores Array-like of shape (n_tools, n_metrics) in native units. Missing cells are NaN and excluded from the per-metric statistics. required
polarity Sequence[str] Length n_metrics polarity strings. Only target_value is read specially, to skip the baseline check; the range, target and noise-floor checks do not depend on direction. required
bounds Sequence[tuple[float | None, float | None]] Length n_metrics (lower, upper) pairs from the cards, either side None where the card declares no bound. required
baselines Sequence[float | None] | None Optional length n_metrics chance scores, None where the card declares none. Defaults to all None. None
targets Sequence[float | None] | None Optional length n_metrics target values, None where the card declares none. Defaults to all None. None
noise_floors Sequence[float | None] | None Optional length n_metrics noise floors in native units, None where the card declares none. Defaults to all None. None
metric_ids Sequence[str] | None Optional length n_metrics labels carried into the findings. None
range_tol float Non-negative absolute tolerance on the range edges, to absorb float round-off at an exact boundary. Default 0.0. 0.0

Returns

Type Description
CardDataConsistencyReport

Raises

Type Description
ValueError If scores is not 2D, a per-metric sequence has the wrong length, or range_tol is negative.

Examples

>>> import numpy as np
>>> from beam.mcda import card_data_consistency
>>> scores = np.array([[0.4, 5.0], [0.9, 12.0]])  # second metric as percent
>>> report = card_data_consistency(
...     scores,
...     ["higher_is_better", "higher_is_better"],
...     [(0.0, 1.0), (0.0, 1.0)],
...     metric_ids=["ari", "accuracy"],
... )
>>> report.ok
False
>>> [f.code for f in report.violations]
['out_of_range']