Lecture 01: Computational Experiments and Floating-Point Arithmetic — Lecture Notes

Terms Developed in Context

The terms below connect the mathematical model, implementation choices, computed diagnostics, and limits of the scientific claim.

Floating-point number

A finite binary representation of a real number with bounded precision and exponent range. In this lecture it is used to address When does a numerically returned digit carry physical information?

Model connection: Read the term against Machine epsilon and the governing relation \(\mathrm{fl}(a\circ b)=(a\circ b)(1+\delta),\quad |\delta|\lesssim \epsilon_{\rm mach}\). The relevant computational evidence is a controlled output, residual, invariant, or refinement sequence rather than visual smoothness alone.

Interpretive limit: Machine precision is not the same as scientific accuracy. A stable calculation can still solve a misspecified physical model. This distinction separates numerical behavior from physical accuracy and model adequacy.

Machine epsilon

The scale of the spacing between one and the next larger representable number. In this lecture it is used to address When does a numerically returned digit carry physical information?

Model connection: Read the term against Roundoff error and the governing relation \(\mathrm{fl}(a\circ b)=(a\circ b)(1+\delta),\quad |\delta|\lesssim \epsilon_{\rm mach}\). The relevant computational evidence is a controlled output, residual, invariant, or refinement sequence rather than visual smoothness alone.

Interpretive limit: Machine precision is not the same as scientific accuracy. A stable calculation can still solve a misspecified physical model. This distinction separates numerical behavior from physical accuracy and model adequacy.

Roundoff error

The discrepancy introduced when an exact arithmetic result is rounded to the available representation. In this lecture it is used to address When does a numerically returned digit carry physical information?

Model connection: Read the term against Cancellation and the governing relation \(\mathrm{fl}(a\circ b)=(a\circ b)(1+\delta),\quad |\delta|\lesssim \epsilon_{\rm mach}\). The relevant computational evidence is a controlled output, residual, invariant, or refinement sequence rather than visual smoothness alone.

Interpretive limit: Machine precision is not the same as scientific accuracy. A stable calculation can still solve a misspecified physical model. This distinction separates numerical behavior from physical accuracy and model adequacy.

Cancellation

Loss of significant information when nearly equal finite-precision quantities are subtracted. In this lecture it is used to address When does a numerically returned digit carry physical information?

Model connection: Read the term against Conditioning and the governing relation \(\mathrm{fl}(a\circ b)=(a\circ b)(1+\delta),\quad |\delta|\lesssim \epsilon_{\rm mach}\). The relevant computational evidence is a controlled output, residual, invariant, or refinement sequence rather than visual smoothness alone.

Interpretive limit: Machine precision is not the same as scientific accuracy. A stable calculation can still solve a misspecified physical model. This distinction separates numerical behavior from physical accuracy and model adequacy.

Conditioning

Sensitivity of the exact problem solution to small perturbations in its inputs. In this lecture it is used to address When does a numerically returned digit carry physical information?

Model connection: Read the term against Floating-point number and the governing relation \(\mathrm{fl}(a\circ b)=(a\circ b)(1+\delta),\quad |\delta|\lesssim \epsilon_{\rm mach}\). The relevant computational evidence is a controlled output, residual, invariant, or refinement sequence rather than visual smoothness alone.

Interpretive limit: Machine precision is not the same as scientific accuracy. A stable calculation can still solve a misspecified physical model. This distinction separates numerical behavior from physical accuracy and model adequacy.

Problem and physical meaning

A computation is an experiment with a model, finite precision, data, and a falsifiable check. IEEE-754 arithmetic rounds every operation; cancellation and conditioning can make a small input perturbation large relative to the desired answer.

When does a numerically returned digit carry physical information? Begin from the scientific observable or state variable, and name what is directly specified versus inferred. The model should be simple enough to validate, but not quietly presented as the entire astrophysical system.

Model and derivation

The governing expression for this lecture is:

Define the symbols and the independent variable before numerical evaluation. Dimensional analysis checks that both sides have matching units; if the model is dimensionless, state the adopted scale factors and the transformation back to physical units.

A computation is an experiment with a model, finite precision, data, and a falsifiable check. IEEE-754 arithmetic rounds every operation; cancellation and conditioning can make a small input perturbation large relative to the desired answer.

Numerical implementation

Define dimensional inputs; nondimensionalize where useful; record dtype and versions; establish analytic limits; perturb inputs and precision; report only stable significant digits.

Keep physical quantities and numerical arrays conceptually separate. Convert units at well-defined boundaries, preserve raw inputs, and record dtype, solver, tolerances/timestep, grid, and software versions. A numerical method is part of the model evidence, not an invisible implementation detail.

Worked, programmatically computed example

NumPy reports spacing 2.22045e-16 at 1.0; the half-spacing increment evaluates to 0 in the specified expression order.

These values are generated in src/astr474_computations.py, then shared with lecture/lab/assignment renderers. Re-run the generator after changing an input; do not hand-edit a displayed result without recomputing dependent artifacts.

At least one independent check is required: an analytic solution, a second numerical method, a conservation law, or a controlled convergence sequence. When these disagree, diagnose units, frame, discretization, source terms, and model scope before selecting a result.

Error budget and limitations

Machine precision is not the same as scientific accuracy. A stable calculation can still solve a misspecified physical model.

Classify each uncertainty before combining terms: local/global solver error, timestep or spatial discretization, Monte Carlo sampling, parameter uncertainty, measurement uncertainty, and model discrepancy are not interchangeable. Conservation can test internal consistency but does not prove the model is physically complete.

Connection to evidence and research practice

Lab 01 measures tolerance convergence and cancellation; PS01 asks students to design an independent numerical check.

For real parameters, the source record distinguishes the fetched product and epoch from derived values. For synthetic experiments, the initial conditions, generator, seed, and exact file are recorded. A simulation is not an observation, even when its parameters resemble a real object.

Misconceptions to avoid

• More displayed digits do not imply more physical accuracy.
• A successful solver exit or smooth plot does not establish convergence.
• Agreement between two solvers with the same wrong model is not independent physical validation.
• A formal covariance or Monte Carlo interval is conditional on the adopted model and noise process.

Study and review questions

1. When does a numerically returned digit carry physical information?
2. Which numerical parameter would you vary first, what observable would you track, and how would you know the sequence reached an asymptotic regime?
3. Design one independent check and state what failure would change your scientific interpretation.

Preparation for the next lecture: explain how this method's assumptions change when the problem becomes more coupled, stochastic, or data-driven.

Lecture-specific derivation and assumptions

For a normalized floating-point number, write fl(x)=x(1+delta), with |delta| bounded approximately by unit roundoff u=2^-53 for binary64 elementary operations away from underflow/overflow. The spacing at 1 is 2^-52, so adding half that spacing lands on a tie; round-to-nearest, ties-to-even returns 1. Subtracting 1 then returns zero. If the increment is formed first, it remains representable. The subtraction has a large condition number because the exact result is tiny compared with the operands: algebraic equivalence does not imply numerical equivalence.

Algorithm trace and computed-output interpretation

Recreate both expression orders in float64 and print `np.spacing(1.0)`. Confirm that the first result is zero and the reordered result is half a spacing. Repeat with values near 2^k, where spacing scales as 2^(k-52), then perturb the input scale. The diagnostic addresses representation and conditioning only; it does not show that either expression encodes a scientifically accurate model.

Computed result: NumPy reports spacing 2.22045e-16 at 1.0; the half-spacing increment evaluates to 0 in the specified expression order.

The computed spacing is the smallest adjacent representable increment at unity, not a universal absolute accuracy. A result can be stable to floating-point perturbations and still be biased by rounded input data, omitted physics, or poor conditioning. Preserve extra precision only if the source information and downstream operations can use it.

Two-meeting teaching arc

Each weekly module is taught in two 75-minute meetings. The sequence below includes a formative check in each meeting and is intended to use the lecture's worked computation, not merely present it.

1. Meeting 1 (75 min): 10 min model-versus-computation framing; 20 min binary representation and rounding; 25 min cancellation derivation; 15 min paired expression-order audit; 5 min exit check on significant digits.
2. Meeting 2 (75 min): 10 min recap; 20 min conditioning and sensitivity; 25 min perturbation experiment across scales; 15 min dimensional/type checks; 5 min predict the ODE error-control question for Lecture 02.

Targeted study questions

1. At what scale does the half-spacing perturbation become representable, and why?
2. Which reformulation reduces cancellation without changing the mathematical model?
3. How would you distinguish roundoff sensitivity from uncertainty in a measured input?