ASTR 474 Lab 07: Reproducible Computational Research Audit

Purpose and learning objectives

L13-L14 · full workflow and independent review. This investigation develops a numerical method and tests its output against a declared benchmark, invariant, uncertainty model, or independent implementation.

Run a specified scientific workflow from the pinned environment.
Measure an explicit numerical or statistical diagnostic with units and provenance.
Separate numerical error from physical model discrepancy and interpret a limitation.

Preparation, setup, and inputs

Software: CPython 3.14.6; dependencies in requirements.txt. From repository root run .venv/Scripts/python.exe materials/ASTR474/src/astr474_computations.py. Then use the corresponding CSV/JSON under materials/ASTR474/data/generated/. Required data: All generated ASTR474 data and source artifacts.

Apparatus: computer with local CPU, terminal, text editor, and browser; no telescope, GPU, network account, or paid service is required. Preserve the generated input files and record file hash/version.

Model scope: The workflow includes level-2 published JPL vector values and clearly identified level-3 synthetic controlled experiments.

Procedure

Create a clean environment from the pinned requirements and record exact runtime versions.
Run the shared computation module and compare every generated artifact with its source inputs and seed.
Choose one numerical experiment; reproduce its key result with a second method or an analytic check.
Write a claim-evidence table with physical model, numerical error, stochastic uncertainty, parameter uncertainty, and model discrepancy as separate rows.
Package the code, traceable source records, generated outputs, provenance, command, checksums, and a limitations statement for a peer to rerun.

Measurement record

Quantity	Value or field	Unit/convention	Source/method
Input configuration	Record solver, version, seed, timestep/grid/tolerance	As defined by the experiment	Command line and source file
Primary diagnostic	Record every resolution/sample/fit row, not just the preferred result	State units and normalization	Generated CSV/JSON
Independent reference	The final validation dossier retains four distinct measures: three-body energy drift 5.33e-13, shock L1 error 0.00521, transmission standard error 0.00244, and fitted residual RMS 3.0897 m/s.	As stated in the generated example	Analytic/JPL/invariant cross-check
Uncertainty/sensitivity	Repeat with changed resolution, seed, parameter, or method	Report interval or relative change	Student analysis

Analysis and uncertainty

Peer review checks uncertainty categories and whether the claimed precision is supported; reruns are not independent evidence of correctness.

Plot the diagnostic against the controlled parameter; label axes, units, normalization, and reference. Explain any non-monotonic point rather than smoothing or dropping it. If propagating independent errors, state why independence is plausible; preserve correlated/common-mode terms separately.

Deliverables and assessment

Runnable source or notebook export with command, package versions, and all input/output paths.
One scientific figure plus a machine-readable table of measurements and convergence/sampling/residual diagnostics.
A 700-1000 word technical report describing model, method, validation, uncertainty categories, and one failed assumption.
A short claim-evidence table separating result, numerical check, and domain limitation.

Rubric: 30% correct method and setup; 25% validation and numerical diagnostics; 20% uncertainty/provenance; 15% physical interpretation; 10% clarity and reproducibility. Revisions and resubmissions are encouraged under the syllabus mastery policy.

Data and source provenance

The workflow includes level-2 published JPL vector values and clearly identified level-3 synthetic controlled experiments. Provenance levels, request records, and source verification limits are documented in reference-log.md; generated files and seeds are documented in data/README.md.

References: