Lecture 13 Notes: Systematics, Calibration, and Error Budgets

ASTR 476: Observational Cosmology · 2026-09-27

Preparation: algebra, introductory mechanics, and the previous module's stated prerequisites. This is a 60-minute instructional module; use the slide timings as a guide and pause at the active check.

Terms Developed in Context

These terms organize the chain from recorded signal to calibrated observable, likelihood, and bounded cosmological claim.

Statistical uncertainty

Variation expected from finite noisy data under the adopted sampling model. In this module it is used to answer How do we know what is limiting the measurement?

Observable/model connection: The term links The package makes distinct records for live-verified Pantheon+ source data, published summary values not re-fetched from the paper, and theoretical or synthetic illustrations to Calibration uncertainty and the quantitative relation \(C_total = C_stat + C_cal + C_model + ...\). State which part is measured, calibrated, derived, or model-dependent.

Limit: A checksum proves file identity, not scientific validity; provenance, versioning, units, and independent checks are also required. That boundary must remain attached when the term is used in a cosmological claim.

Calibration uncertainty

Error in the mapping from instrument response to a physical observable. In this module it is used to answer How do we know what is limiting the measurement?

Observable/model connection: The term links The package makes distinct records for live-verified Pantheon+ source data, published summary values not re-fetched from the paper, and theoretical or synthetic illustrations to Model systematic and the quantitative relation \(C_total = C_stat + C_cal + C_model + ...\). State which part is measured, calibrated, derived, or model-dependent.

Limit: A checksum proves file identity, not scientific validity; provenance, versioning, units, and independent checks are also required. That boundary must remain attached when the term is used in a cosmological claim.

Model systematic

Bias or uncertainty caused by an incomplete or misspecified physical or population model. In this module it is used to answer How do we know what is limiting the measurement?

Observable/model connection: The term links The package makes distinct records for live-verified Pantheon+ source data, published summary values not re-fetched from the paper, and theoretical or synthetic illustrations to Error budget and the quantitative relation \(C_total = C_stat + C_cal + C_model + ...\). State which part is measured, calibrated, derived, or model-dependent.

Limit: A checksum proves file identity, not scientific validity; provenance, versioning, units, and independent checks are also required. That boundary must remain attached when the term is used in a cosmological claim.

Error budget

A structured accounting of uncertainty sources, correlations, and propagation to the final claim. In this module it is used to answer How do we know what is limiting the measurement?

Observable/model connection: The term links The package makes distinct records for live-verified Pantheon+ source data, published summary values not re-fetched from the paper, and theoretical or synthetic illustrations to Blinding and the quantitative relation \(C_total = C_stat + C_cal + C_model + ...\). State which part is measured, calibrated, derived, or model-dependent.

Limit: A checksum proves file identity, not scientific validity; provenance, versioning, units, and independent checks are also required. That boundary must remain attached when the term is used in a cosmological claim.

Blinding

A procedure that hides or offsets sensitive results to reduce analyst-driven confirmation bias. In this module it is used to answer How do we know what is limiting the measurement?

Observable/model connection: The term links The package makes distinct records for live-verified Pantheon+ source data, published summary values not re-fetched from the paper, and theoretical or synthetic illustrations to Statistical uncertainty and the quantitative relation \(C_total = C_stat + C_cal + C_model + ...\). State which part is measured, calibrated, derived, or model-dependent.

Limit: A checksum proves file identity, not scientific validity; provenance, versioning, units, and independent checks are also required. That boundary must remain attached when the term is used in a cosmological claim.

Physical and observational argument

A useful error budget names where uncertainty originates and how it propagates: detector response, photometric calibration, standardization, redshift, selection, population variation and cosmological modeling. Independent noise often averages down; a shared calibration mode does not. Covariance is a mathematical representation of these dependencies, while nuisance parameters can expose physically interpretable uncertainty. Provenance controls another failure mode: unknown units, reordered rows, stale releases or undocumented transformations. A digest confirms which bytes were analyzed, but only tests, source verification and scientific review establish whether those bytes and assumptions are appropriate.

Precision cosmology depends on calibration chains and nuisance models as much as sample size. An error budget names sources, correlations, mitigation, and residual validation rather than collapsing every uncertainty into one bar.

The package makes distinct records for live-verified Pantheon+ source data, published summary values not re-fetched from the paper, and theoretical or synthetic illustrations.

Derivation and interpretation

\[C_total = C_stat + C_cal + C_model + ...\]

Begin by naming observables, coordinate convention, and model assumptions; each algebraic transformation must preserve units and the distinction between measured and inferred quantities. Test a limiting case, then perturb one input and state which observable changes.

A checksum proves file identity, not scientific validity; provenance, versioning, units, and independent checks are also required.

Worked numerical anchor

The selected covariance block preserves source order; file hashes verify identity but do not validate a scientific interpretation.

Reproduce the value with the course computation module, retain full precision in code, and round only for communication. Report units and all conditional assumptions with the result.

Evidence, systematics, and limits

A checksum proves file identity, not scientific validity; provenance, versioning, units, and independent checks are also required.

The package makes distinct records for live-verified Pantheon+ source data, published summary values not re-fetched from the paper, and theoretical or synthetic illustrations.

Separate the directly recorded observable, its calibration, the derived intermediate quantity, and the model-dependent parameter. An uncertainty bar is not a complete likelihood when shared covariance or selection is omitted.

Study and retrieval practice

  1. Build a categorized error budget with statistical and shared systematic terms.
  2. Explain calibration propagation through a multi-stage distance measurement.
  3. Audit provenance, version, units, and transformations for a derived result.

Checkpoint: Record source URL, retrieval date, source digest, transform, code version, and caveat for every data product.

Answer in words before calculating; then identify one observation or null test that could falsify your interpretation.

Systematics, Calibration, and Error BudgetsThe package makes distinct records for live-verified Pantheon+ source data, published summary values not re-fetched from the paper, and theoretical or synthetic illustrations.A credible error budget tracks origin and correlationphotometry0.68 (schematic weight)calibration0.52 (schematic weight)population0.39 (schematic weight)selection0.30 (schematic weight)model0.24 (schematic weight)relative contribution (illustrative only)Schematic categories, not measured error fractions
The package makes distinct records for live-verified Pantheon+ source data, published summary values not re-fetched from the paper, and theoretical or synthetic illustrations.

Lecture 13 notes · Sources and caveats in reference-log.md.