Uncertainty Quantification API

Confidence interval estimation and coverage probability analysis.

UncertaintyQuantifier

class lrdbenchmark.analysis.uncertainty.UncertaintyQuantifier(n_block_bootstrap: int = 64, block_size: int | None = None, n_wavelet_bootstrap: int = 64, wavelet: str = 'db4', max_wavelet_level: int | None = None, n_parametric: int = 48, confidence_level: float = 0.95, random_state: int | None = None, max_failures: int = 16)[source]

Compute confidence intervals for H estimators using reusable resampling strategies.

__init__(n_block_bootstrap: int = 64, block_size: int | None = None, n_wavelet_bootstrap: int = 64, wavelet: str = 'db4', max_wavelet_level: int | None = None, n_parametric: int = 48, confidence_level: float = 0.95, random_state: int | None = None, max_failures: int = 16) → None[source]

compute_intervals(estimator: Any, data: ndarray, base_result: Dict[str, Any], true_value: float | None = None, data_model_name: str | None = None, data_model_params: Dict[str, Any] | None = None, data_model_registry: Dict[str, Any] | None = None) → Dict[str, Any][source]: Compute uncertainty summaries for an estimator applied to data.

_studentized_bootstrap_interval(estimator_cls: Type[Any], estimator_params: Dict[str, Any], data: ndarray, rng: Generator, base_estimate: float) → IntervalSummary[source]

Studentized (bias-corrected) bootstrap interval.

Uses t-distribution critical values and bias correction for improved coverage probability in small samples.

Supported Methods

Block Bootstrap: Moving-block bootstrap for dependent time series
Wavelet Bootstrap: Wavelet-domain resampling preserving scale-wise energy
Parametric Monte Carlo: Simulation from known data model
Studentized Bootstrap: Bias-corrected intervals with t-distribution CIs

CoverageAnalyzer

Monte Carlo estimation of confidence interval coverage probabilities.

class lrdbenchmark.analysis.uncertainty.CoverageAnalyzer(n_trials: int = 100, confidence_level: float = 0.95, tolerance: float = 0.05, random_state: int | None = None)[source]

Monte Carlo analyzer for confidence interval coverage probabilities.

Assesses whether UQ methods produce calibrated confidence intervals by measuring how often the true value falls within the reported CI.

Example

>>> analyzer = CoverageAnalyzer(n_trials=200)
>>> results = analyzer.analyze_estimator_coverage(
...     estimator_cls=DFAEstimator,
...     data_model_cls=FBMModel,
...     true_H=0.7,
...     length=1000
... )
>>> print(f"Coverage: {results['block_bootstrap'].empirical_coverage:.2%}")

__init__(n_trials: int = 100, confidence_level: float = 0.95, tolerance: float = 0.05, random_state: int | None = None)[source]

Initialize coverage analyzer.

Parameters:

n_trials (int) – Number of Monte Carlo trials
confidence_level (float) – Nominal confidence level
tolerance (float) – Acceptable deviation from nominal coverage (for calibration check)
random_state (int, optional) – Random seed

analyze_estimator_coverage(estimator_cls: Type, data_model_cls: Type, true_H: float, length: int, estimator_params: Dict[str, Any] | None = None, data_model_params: Dict[str, Any] | None = None, uq_methods: List[str] | None = None, n_bootstrap: int = 64) → Dict[str, CoverageResult][source]

Analyze coverage probability for an estimator-model combination.

Parameters:

estimator_cls (Type) – Estimator class (e.g., DFAEstimator)
data_model_cls (Type) – Data generation model class (e.g., FBMModel)
true_H (float) – True Hurst parameter
length (int) – Length of generated time series
estimator_params (dict, optional) – Parameters for estimator
data_model_params (dict, optional) – Additional parameters for data model
uq_methods (list of str, optional) – UQ methods to test: ‘block_bootstrap’, ‘percentile’, ‘studentized’
n_bootstrap (int) – Number of bootstrap samples per trial

Returns:

Coverage results for each UQ method

Return type:

dict

_compute_ci(estimator_cls: Type, estimator_params: Dict[str, Any], data: ndarray, method: str, n_bootstrap: int, seed: int) → tuple | None[source]: Compute confidence interval using specified method.

generate_coverage_report(results: Dict[str, CoverageResult]) → Dict[str, Any][source]

Generate a summary report from coverage results.

Returns:: Summary with calibration status and recommendations
Return type:: dict

CoverageResult

class lrdbenchmark.analysis.uncertainty.CoverageResult(method: str, nominal_level: float, empirical_coverage: float, n_trials: int, n_covered: int, coverage_error: float, standard_error: float, calibrated: bool)[source]

Results from coverage probability analysis.

method: str

nominal_level: float

empirical_coverage: float

n_trials: int

n_covered: int

coverage_error: float

standard_error: float

calibrated: bool

to_dict() → Dict[str, Any][source]

__init__(method: str, nominal_level: float, empirical_coverage: float, n_trials: int, n_covered: int, coverage_error: float, standard_error: float, calibrated: bool) → None

Example Usage

from lrdbenchmark.analysis.uncertainty import UncertaintyQuantifier, CoverageAnalyzer

# Compute confidence intervals
uq = UncertaintyQuantifier(confidence_level=0.95)
intervals = uq.compute_intervals(
    estimator=dfa_estimator,
    data=signal,
    base_result=estimation_result,
    true_value=0.7
)

print(f"Block bootstrap CI: {intervals['block_bootstrap']['confidence_interval']}")
print(f"Studentized CI: {intervals['studentized_bootstrap']['confidence_interval']}")

# Analyze coverage probability
analyzer = CoverageAnalyzer(n_trials=200)
coverage = analyzer.analyze_estimator_coverage(
    estimator_cls=DFAEstimator,
    data_model_cls=FBMModel,
    true_H=0.7,
    length=1000
)

for method, result in coverage.items():
    print(f"{method}: {result.empirical_coverage:.1%} coverage")

Utility Functions

lrdbenchmark.analysis.uncertainty.run_comprehensive_coverage_analysis(estimator_cls: Type, data_model_cls: Type, H_values: List[float], length: int = 1000, n_trials: int = 100, seed: int = 42) → Dict[str, Any][source]

Run coverage analysis across multiple H values.

Parameters:

estimator_cls (Type) – Estimator class
data_model_cls (Type) – Data model class
H_values (list of float) – H values to test
length (int) – Series length
n_trials (int) – Trials per H value
seed (int) – Random seed

Returns:

Comprehensive coverage results

Return type:

dict