Lecture 11: Likelihoods, Residuals, and Parameter Estimation — Lecture Notes

Terms Developed in Context

The terms below connect the mathematical model, implementation choices, computed diagnostics, and limits of the scientific claim.

Likelihood

A probability model for observed data viewed as a function of model parameters. In this lecture it is used to address What evidence distinguishes a fitted parameter from a visually plausible curve?

Model connection: Read the term against Residual and the governing relation \(\chi^2(\theta)=\sum_i\left[\frac{y_i-m(t_i;\theta)}{\sigma_i}\right]^2\). The relevant computational evidence is a controlled output, residual, invariant, or refinement sequence rather than visual smoothness alone.

Interpretive limit: The inverse Jacobian curvature is a local approximation, not a robust interval when parameters are degenerate, boundaries bind, noise is non-Gaussian, or the model is wrong. This distinction separates numerical behavior from physical accuracy and model adequacy.

Residual

The observed value minus its model prediction under a stated sign convention. In this lecture it is used to address What evidence distinguishes a fitted parameter from a visually plausible curve?

Model connection: Read the term against Weighted least squares and the governing relation \(\chi^2(\theta)=\sum_i\left[\frac{y_i-m(t_i;\theta)}{\sigma_i}\right]^2\). The relevant computational evidence is a controlled output, residual, invariant, or refinement sequence rather than visual smoothness alone.

Interpretive limit: The inverse Jacobian curvature is a local approximation, not a robust interval when parameters are degenerate, boundaries bind, noise is non-Gaussian, or the model is wrong. This distinction separates numerical behavior from physical accuracy and model adequacy.

Weighted least squares

Parameter estimation that scales residuals by their uncertainties or covariance. In this lecture it is used to address What evidence distinguishes a fitted parameter from a visually plausible curve?

Model connection: Read the term against Identifiability and the governing relation \(\chi^2(\theta)=\sum_i\left[\frac{y_i-m(t_i;\theta)}{\sigma_i}\right]^2\). The relevant computational evidence is a controlled output, residual, invariant, or refinement sequence rather than visual smoothness alone.

Interpretive limit: The inverse Jacobian curvature is a local approximation, not a robust interval when parameters are degenerate, boundaries bind, noise is non-Gaussian, or the model is wrong. This distinction separates numerical behavior from physical accuracy and model adequacy.

Identifiability

The degree to which distinct parameter values produce distinguishable predictions. In this lecture it is used to address What evidence distinguishes a fitted parameter from a visually plausible curve?

Model connection: Read the term against Jacobian and the governing relation \(\chi^2(\theta)=\sum_i\left[\frac{y_i-m(t_i;\theta)}{\sigma_i}\right]^2\). The relevant computational evidence is a controlled output, residual, invariant, or refinement sequence rather than visual smoothness alone.

Interpretive limit: The inverse Jacobian curvature is a local approximation, not a robust interval when parameters are degenerate, boundaries bind, noise is non-Gaussian, or the model is wrong. This distinction separates numerical behavior from physical accuracy and model adequacy.

Jacobian

The matrix of model derivatives with respect to parameters used in optimization and local uncertainty estimates. In this lecture it is used to address What evidence distinguishes a fitted parameter from a visually plausible curve?

Model connection: Read the term against Likelihood and the governing relation \(\chi^2(\theta)=\sum_i\left[\frac{y_i-m(t_i;\theta)}{\sigma_i}\right]^2\). The relevant computational evidence is a controlled output, residual, invariant, or refinement sequence rather than visual smoothness alone.

Interpretive limit: The inverse Jacobian curvature is a local approximation, not a robust interval when parameters are degenerate, boundaries bind, noise is non-Gaussian, or the model is wrong. This distinction separates numerical behavior from physical accuracy and model adequacy.

Problem and physical meaning

Least-squares fitting minimizes a chosen residual objective, not a universal truth criterion. Heteroscedastic observations require uncertainty-scaled residuals; structured residuals reveal model inadequacy even when an optimizer reports success.

What evidence distinguishes a fitted parameter from a visually plausible curve? Begin from the scientific observable or state variable, and name what is directly specified versus inferred. The model should be simple enough to validate, but not quietly presented as the entire astrophysical system.

Model and derivation

The governing expression for this lecture is:

Define the symbols and the independent variable before numerical evaluation. Dimensional analysis checks that both sides have matching units; if the model is dimensionless, state the adopted scale factors and the transformation back to physical units.

Least-squares fitting minimizes a chosen residual objective, not a universal truth criterion. Heteroscedastic observations require uncertainty-scaled residuals; structured residuals reveal model inadequacy even when an optimizer reports success.

Numerical implementation

Define a physically interpretable model; scale parameters; optimize weighted residuals with SciPy; inspect residual versus time and phase; estimate covariance from the Jacobian only under local linearity and stated noise assumptions.

Keep physical quantities and numerical arrays conceptually separate. Convert units at well-defined boundaries, preserve raw inputs, and record dtype, solver, tolerances/timestep, grid, and software versions. A numerical method is part of the model evidence, not an invisible implementation detail.

Worked, programmatically computed example

The seeded fit returns K=20.8286 m/s, phase=0.69746 rad, gamma=3.1325 m/s, residual RMS=3.0897 m/s, and chi-square=67.882 for 61 degrees of freedom.

These values are generated in src/astr474_computations.py, then shared with lecture/lab/assignment renderers. Re-run the generator after changing an input; do not hand-edit a displayed result without recomputing dependent artifacts.

At least one independent check is required: an analytic solution, a second numerical method, a conservation law, or a controlled convergence sequence. When these disagree, diagnose units, frame, discretization, source terms, and model scope before selecting a result.

Error budget and limitations

The inverse Jacobian curvature is a local approximation, not a robust interval when parameters are degenerate, boundaries bind, noise is non-Gaussian, or the model is wrong.

Classify each uncertainty before combining terms: local/global solver error, timestep or spatial discretization, Monte Carlo sampling, parameter uncertainty, measurement uncertainty, and model discrepancy are not interchangeable. Conservation can test internal consistency but does not prove the model is physically complete.

Connection to evidence and research practice

Lab 06 fits a seeded synthetic radial-velocity series and requires parameter uncertainty and residual diagnostics.

For real parameters, the source record distinguishes the fetched product and epoch from derived values. For synthetic experiments, the initial conditions, generator, seed, and exact file are recorded. A simulation is not an observation, even when its parameters resemble a real object.

Misconceptions to avoid

• More displayed digits do not imply more physical accuracy.
• A successful solver exit or smooth plot does not establish convergence.
• Agreement between two solvers with the same wrong model is not independent physical validation.
• A formal covariance or Monte Carlo interval is conditional on the adopted model and noise process.

Study and review questions

1. What evidence distinguishes a fitted parameter from a visually plausible curve?
2. Which numerical parameter would you vary first, what observable would you track, and how would you know the sequence reached an asymptotic regime?
3. Design one independent check and state what failure would change your scientific interpretation.

Preparation for the next lecture: explain how this method's assumptions change when the problem becomes more coupled, stochastic, or data-driven.

Lecture-specific derivation and assumptions

With fixed angular frequency omega, write v(t)=gamma+K sin(omega*t+phi)=gamma+a sin(omega*t)+b cos(omega*t), where a=K cos(phi), b=K sin(phi). The design matrix is linear in (gamma,a,b); weighted least squares solves (X^T W X)c=X^T W y. Recover K=sqrt(a^2+b^2) and phi=atan2(b,a). This independent linear parameterization is a useful cross-check on nonlinear optimization and avoids the amplitude/phase curvature during the fit.

Algorithm trace and computed-output interpretation

Fit the 64 seeded observations with sigma=3 m/s and fixed P=11.2 d. Record K, phase, offset, local standard errors, weighted chi-square, 61 degrees of freedom, residual RMS, and maximum residual. Plot residual versus time and orbital phase. The supplied nonlinear solver's covariance rescales local curvature by reduced chi-square; an alternative treating sigma as known uses (J^T W J)^-1 without that scale factor, and must state its assumption.

Computed result: The seeded fit returns K=20.8286 m/s, phase=0.69746 rad, gamma=3.1325 m/s, residual RMS=3.0897 m/s, and chi-square=67.882 for 61 degrees of freedom.

A low residual RMS does not validate the period or sinusoidal model; phase coverage and residual structure matter. Standard errors are local and conditional on fixed period, Gaussian independent errors, and the model. A trend, additional periodicity, or phase-dependent scatter would motivate a richer model rather than merely a different optimizer.

Two-meeting teaching arc

Each weekly module is taught in two 75-minute meetings. The sequence below includes a formative check in each meeting and is intended to use the lecture's worked computation, not merely present it.

1. Meeting 1 (75 min): 10 min inference setup; 20 min likelihood and weighted residuals; 25 min sinusoid linearization; 15 min identifiability/phase coverage; 5 min design-matrix check.
2. Meeting 2 (75 min): 10 min recap; 20 min run nonlinear and linear fits; 25 min interpret covariance and residuals; 15 min model criticism; 5 min state a falsifiable extension.

Targeted study questions

1. Derive the conversion between (a,b) and (K,phi), including the phase quadrant.
2. What changes in the covariance if the period is also fitted?
3. Which residual pattern would indicate a second companion rather than white noise?