Lecture 11: Likelihoods, Residuals, and Parameter Estimation
What evidence distinguishes a fitted parameter from a visually plausible curve?
14-week course · 3 credits · Week 11
Motivation
What evidence distinguishes a fitted parameter from a visually plausible curve?
Least-squares fitting minimizes a chosen residual objective, not a universal truth criterion. Heteroscedastic observations require uncertainty-scaled residuals; structured residuals reveal model inadequacy even when an optimizer reports success.
Today’s result must be interpretable as a physical statement with declared data, algorithm, uncertainty, and a check that could fail.
Learning objectives
After this lecture, you can
What evidence distinguishes a fitted parameter from a visually plausible curve?
Define a physically interpretable model
The inverse Jacobian curvature is a local approximation, not a robust interval when parameters are degenerate, boundaries bind, noise is non-Gaussian, or the model is wrong.
Core language
Terms for This Lecture
Likelihood
Definition: A probability model for observed data viewed as a function of model parameters.
Why it matters here: It helps answer What evidence distinguishes a fitted parameter from a visually plausible curve? within the numerical model described here.
Relationships: Connect it to Residual, the governing expression \(\chi^2(\theta)=\sum_i\left[\frac{y_i-m(t_i;\theta)}{\sigma_i}\right]^2\), and the computed diagnostic developed later in the lecture.
Boundary or common confusion: The inverse Jacobian curvature is a local approximation, not a robust interval when parameters are degenerate, boundaries bind, noise is non-Gaussian, or the model is wrong.
Residual
Definition: The observed value minus its model prediction under a stated sign convention.
Why it matters here: It helps answer What evidence distinguishes a fitted parameter from a visually plausible curve? within the numerical model described here.
Relationships: Connect it to Weighted least squares, the governing expression \(\chi^2(\theta)=\sum_i\left[\frac{y_i-m(t_i;\theta)}{\sigma_i}\right]^2\), and the computed diagnostic developed later in the lecture.
Boundary or common confusion: The inverse Jacobian curvature is a local approximation, not a robust interval when parameters are degenerate, boundaries bind, noise is non-Gaussian, or the model is wrong.
Weighted least squares
Definition: Parameter estimation that scales residuals by their uncertainties or covariance.
Why it matters here: It helps answer What evidence distinguishes a fitted parameter from a visually plausible curve? within the numerical model described here.
Relationships: Connect it to Identifiability, the governing expression \(\chi^2(\theta)=\sum_i\left[\frac{y_i-m(t_i;\theta)}{\sigma_i}\right]^2\), and the computed diagnostic developed later in the lecture.
Boundary or common confusion: The inverse Jacobian curvature is a local approximation, not a robust interval when parameters are degenerate, boundaries bind, noise is non-Gaussian, or the model is wrong.
Identifiability
Definition: The degree to which distinct parameter values produce distinguishable predictions.
Why it matters here: It helps answer What evidence distinguishes a fitted parameter from a visually plausible curve? within the numerical model described here.
Relationships: Connect it to Jacobian, the governing expression \(\chi^2(\theta)=\sum_i\left[\frac{y_i-m(t_i;\theta)}{\sigma_i}\right]^2\), and the computed diagnostic developed later in the lecture.
Boundary or common confusion: The inverse Jacobian curvature is a local approximation, not a robust interval when parameters are degenerate, boundaries bind, noise is non-Gaussian, or the model is wrong.
Jacobian
Definition: The matrix of model derivatives with respect to parameters used in optimization and local uncertainty estimates.
Why it matters here: It helps answer What evidence distinguishes a fitted parameter from a visually plausible curve? within the numerical model described here.
Relationships: Connect it to Likelihood, the governing expression \(\chi^2(\theta)=\sum_i\left[\frac{y_i-m(t_i;\theta)}{\sigma_i}\right]^2\), and the computed diagnostic developed later in the lecture.
Boundary or common confusion: The inverse Jacobian curvature is a local approximation, not a robust interval when parameters are degenerate, boundaries bind, noise is non-Gaussian, or the model is wrong.
Physical and mathematical model
State what is being evolved
Least-squares fitting minimizes a chosen residual objective, not a universal truth criterion. Heteroscedastic observations require uncertainty-scaled residuals; structured residuals reveal model inadequacy even when an optimizer reports success.
Define the state, coordinates, units, initial/boundary conditions, and domain before interpreting solver output.
Numerical method
Algorithm choice is a modeling decision
Define a physically interpretable model; scale parameters; optimize weighted residuals with SciPy; inspect residual versus time and phase; estimate covariance from the Jacobian only under local linearity and stated noise assumptions.
Write the model and its assumptions.
Choose a solver and identify its stability, accuracy, and conservation properties.
Record configuration, tolerances or timestep, and output sampling.
Compare against an analytic limit, independent method, or controlled refinement.
Visual reasoning
Radial-velocity observations, fitted model, and residuals
All points and residuals are seeded synthetic data; the curve uses the fitted parameters.
Computed example
One result, with a check
The seeded fit returns K=20.8286 m/s, phase=0.69746 rad, gamma=3.1325 m/s, residual RMS=3.0897 m/s, and chi-square=67.882 for 61 degrees of freedom.
Inputs and method are generated by src/astr474_computations.py; the machine-readable values are in data/generated/computed_results.json.
The inverse Jacobian curvature is a local approximation, not a robust interval when parameters are degenerate, boundaries bind, noise is non-Gaussian, or the model is wrong.
Verification
What would falsify this result?
Repeat with a smaller timestep, tighter tolerance, or finer spatial grid.
Track conservation or residual structure in addition to visual agreement.
Check units, coordinate frame, boundary conditions, and invariants independently.
Change a random seed when the method is stochastic; report the resulting sampling uncertainty.
Do not report a smoother curve as proof of convergence.
Failure modes
Numerical agreement has a domain
The inverse Jacobian curvature is a local approximation, not a robust interval when parameters are degenerate, boundaries bind, noise is non-Gaussian, or the model is wrong.
Solver error, discretization error, stochastic error, parameter uncertainty, and model discrepancy are different quantities. Diagnose and report them separately.
Active reasoning
Make a prediction before rerunning
What evidence distinguishes a fitted parameter from a visually plausible curve?
State a measurable outcome, a competing explanation, and the smallest computation that distinguishes them. Include an expected direction or scaling, then test it.
Workflow
From parameters to auditable output
Define a physically interpretable model; scale parameters; optimize weighted residuals with SciPy; inspect residual versus time and phase; estimate covariance from the Jacobian only under local linearity and stated noise assumptions.
Record the exact command, package versions, data path, random seed if applicable, and output checksum. Keep raw inputs unchanged and generated files separate.
Transfer
Lab and problem-set connection
Lab 06 fits a seeded synthetic radial-velocity series and requires parameter uncertainty and residual diagnostics.
Submit plots with labeled axes/units, a concise uncertainty argument, and enough code or intermediate values for a peer to reproduce the claim.
Synthesis
Three durable checks
Does the implemented equation match the intended physical model?
Does the numerical result converge under a controlled method change?
Does the interpretation stay inside the validity and provenance limits?
The inverse Jacobian curvature is a local approximation, not a robust interval when parameters are degenerate, boundaries bind, noise is non-Gaussian, or the model is wrong.