source_variance_decomposition

heterogeneity.source_variance_decomposition(methods, datasets, benchmarks, scores)

Decompose benchmark-score variance into method, benchmark, and interaction.

Parameters

Name	Type	Description	Default
methods	Sequence[str]	Four parallel sequences, one entry per observation: the method label, the dataset label, the benchmark label, and the score on one metric. Rows with a NaN score are dropped. Pass scores from one metric (or one already-pooled composite) per call; do not mix metrics or polarities.	required
datasets	Sequence[str]	Four parallel sequences, one entry per observation: the method label, the dataset label, the benchmark label, and the score on one metric. Rows with a NaN score are dropped. Pass scores from one metric (or one already-pooled composite) per call; do not mix metrics or polarities.	required
benchmarks	Sequence[str]	Four parallel sequences, one entry per observation: the method label, the dataset label, the benchmark label, and the score on one metric. Rows with a NaN score are dropped. Pass scores from one metric (or one already-pooled composite) per call; do not mix metrics or polarities.	required
scores	Sequence[str]	Four parallel sequences, one entry per observation: the method label, the dataset label, the benchmark label, and the score on one metric. Rows with a NaN score are dropped. Pass scores from one metric (or one already-pooled composite) per call; do not mix metrics or polarities.	required

Returns

Type	Description
SourceVarianceReport

Raises

Type	Description
ValueError	If the sequences differ in length, or fewer than two methods or two benchmarks remain after dropping NaN scores.
RNotAvailableError	If the R toolchain with lme4 is not available.
RExecutionError	If the R subprocess fails.