tensor/optim/doc.go

// Copyright (c) 2026 Petr Balvín <opensource@petrbalvin.org> (https://petrbalvin.org)
// SPDX-License-Identifier: MIT

// Package optim fits, minimises and solves: nonlinear least squares,
// local and global minimisation, constrained minimisation, linear and
// quadratic programming, and root finding for one equation or a system
// of them.
//
// # What it is
//
// Every entry point consumes and returns the library's Array. The
// caller's objective, residual, gradient or constraint function
// receives the candidate point as a rank-1 array of its own and returns
// a value and an error; the answer comes back as a fresh array the
// caller owns, and nothing the caller passed in is modified.
//
// Callbacks run one evaluation at a time on the goroutine that entered
// the solver. The one opt-out is ParallelJacobian on LMOptions and
// RootSystemOptions: setting it consents to the residual callback being
// read from several goroutines at once while a finite-difference
// Jacobian sweeps its columns, and the numbers come out identical
// either way.
//
// The methods, by entry point:
//
//   - LevenbergMarquardt fits a model to data by damped Gauss-Newton on
//     the residual vector, with a central-difference or an analytic
//     Jacobian.
//   - Minimise is the derivative-free Nelder-Mead simplex;
//     MinimiseLBFGS is limited-memory BFGS with an Armijo backtracking
//     line search and optional box walls.
//   - MinimiseConstrained adds the linear rows l ≤ A·x ≤ u to an
//     objective through an augmented Lagrangian; the same machinery
//     carries the functional rows of MinimiseNonlinearConstrained.
//   - MinimiseLinear and MinimiseLinearRows solve a linear program by
//     the two-phase revised simplex under Bland's rule; MinimiseQP
//     solves the strictly convex quadratic program by a primal
//     active-set method and returns the row multipliers.
//   - MinimiseDifferentialEvolution, MinimiseCMAES and
//     MinimiseSimulatedAnnealing search a landscape with several
//     basins without derivatives.
//   - FindRoot, FindRootBrent and FindRootNewton solve one equation in
//     one unknown; FindRootSystem solves as many equations as unknowns.
//
// # Tolerances
//
// The tolerances are absolute in the units of the quantity they
// measure, the caller's own: a gradient coordinate, a row violation,
// an objective spread, a reduced cost. An objective or a constraint
// whose natural scale sits many orders of magnitude away from one
// should be rescaled to O(1) before it is handed to a solver, because
// a solution that is converged in a small unit is reported as
// converged at the point the solver started from.
//
// # Budgets and honesty
//
// No entry point returns a point it did not earn. A local solver that
// spends its iteration budget without meeting its tolerance is refused
// with an error naming the figure it reached and the tolerance it fell
// short of; a line search that stalls, a damping that collapses and a
// search direction that vanishes are refused the same way. Setting
// AllowBudgetExit reports the best point reached instead, which is the
// documented escape hatch and never the default. The constraint
// entries use it for their inner solves on purpose: the outer loop
// judges those points by the rows' feasibility, so an inexact inner
// solve still carries the iteration forward. Differential evolution is
// the exception that needs no flag, because its generation budget
// tunes the search rather than deciding convergence. MinimiseCMAES
// runs once: the restart schemes are the caller's loop, and the
// result of one run is reported as such.
//
// # What it does not do
//
// The package is real-valued: a complex starting point, cost, bound,
// constraint matrix or callback payload is refused with an error. It
// also leaves the modelling to the caller. MinimiseLinear requires the
// standard form A·x = b with x ≥ 0 exactly, and MinimiseLinearRows is
// the wrapper that converts the house two-sided rows into it.
// MinimiseDifferentialEvolution clamps to its box and requires one,
// while MinimiseCMAES has no bounds at all and expects a caller who
// needs them to reparametrise. A local minimum is a local minimum:
// Minimise and MinimiseLBFGS answer for the basin they started in, and
// multistart or a global searcher is what finds the others.
package optim