leave_one_dataset_out

mcda.leave_one_dataset_out(tensor, polarity, reduction_rules, dataset_names=None, metric_ids=None, weights='equal', method='saw', normalization=None, bounds=None, baselines=None, targets=None, missing='error', on_zero_coverage='error')

Pool all datasets and rank, then re-rank with each dataset left out.

The headline ranking pools a tool by dataset by metric tensor across the datasets (one reduction rule per metric) into a tool by metric matrix and ranks it. This analysis asks how much that ranking depends on any single dataset: for each dataset d, drop d, pool the remaining datasets the same way, re-rank, and compare to the base ranking. The result is a per-tool rank stability (the fraction of leave-one-out runs in which the tool keeps its base rank) and the largest rank shift caused by removing any single dataset.

The reduction is nan-aware. A dataset whose removal would leave some tool with no observation for a metric cannot be pooled, so that omission is skipped and excluded from the stability denominator; the evaluated dataset indices are reported in the result.

The normalization context applies per metric and the metric axis is unchanged when a dataset is dropped, so normalization, bounds, baselines and targets are forwarded to every run unchanged.

Parameters

Name	Type	Description	Default
tensor		Array-like of shape (n_tools, n_datasets, n_metrics).	required
polarity	Sequence[str]	Length n_metrics sequence of polarity strings.	required
reduction_rules	Sequence[str]	Length n_metrics sequence of cross-dataset reduction rule names, one per metric. Source these from each card’s recommended_aggregation_across_datasets.	required
dataset_names	Sequence[str] | None	Optional length n_datasets labels, carried in the report.	None
metric_ids	Sequence[str] | None	Optional length n_metrics labels used in reduction error messages.	None
weights		Forwarded to run.	'equal'
method		Forwarded to run.	'equal'
normalization		Optional per-metric normalization context forwarded to every run. Pass the values from beam.mcda.registry_context so the leave-one-out runs normalize the same way as the headline ranking.	None
bounds		Optional per-metric normalization context forwarded to every run. Pass the values from beam.mcda.registry_context so the leave-one-out runs normalize the same way as the headline ranking.	None
baselines		Optional per-metric normalization context forwarded to every run. Pass the values from beam.mcda.registry_context so the leave-one-out runs normalize the same way as the headline ranking.	None
targets		Optional per-metric normalization context forwarded to every run. Pass the values from beam.mcda.registry_context so the leave-one-out runs normalize the same way as the headline ranking.	None

Returns

Type	Description
DatasetSensitivityReport