Nonlinear Least-Squares Fitting

# Title: Nonlinear Least-Squares Fitting

## 1 The GSL::MultiFit::FdfSolver class

`GSL::MultiFit::FdfSolver.new(T, n, p)`

  This creates an instance of the `GSL::MultiFit::FdfSolver` class of type T for n observations and p parameters. The type T is given by a Ruby constant,

  - GSL::MultiFit::LMSDER
  - GSL::MultiFit::LMDER

  For example, the following code creates an instance of a Levenberg-Marquardt solver for 100 data points and 3 parameters,

      solver = MultiFit::FdfSolver.new(MultiFit::LMDER, 100, 3)

`GSL::MultiFit::FdfSolver#set(f, x)`
  This method initializes, or reinitializes, an existing solver self to use the function f and the initial guess x. The function f is an instance of the `GSL::MultiFit::Function_fdf` class (see below). The initial guess of the parameters x is given by a GSL::Vector object.
`GSL::MultiFit::FdfSolver#name`
  This returns the name of the solver self as a String.
`GSL::MultiFit::FdfSolver#x`
`GSL::MultiFit::FdfSolver#dx`
`GSL::MultiFit::FdfSolver#f`
`GSL::MultiFit::FdfSolver#J`
`GSL::MultiFit::FdfSolver#jacobian`
`GSL::MultiFit::FdfSolver#jac`
  Access to the members (see `gsl_multifit_nlin.h`)

## 2 Providing the function to solve: GSL::MultiFit::Function_fdf class

`GSL::MultiFit::Function_fdf.new()`
`GSL::MultiFit::Function_fdf.new(f, df, p)`
`GSL::MultiFit::Function_fdf.new(f, df, fdf, p)`
  Constructor for the `Function_fdf` class, to a function with p parameters, The first two or three arguments are Ruby Proc objects to evaluate the function to minimize and its derivative (Jacobian).
`GSL::MultiFit::Function_fdf#set_procs(f, df, p)`
`GSL::MultiFit::Function_fdf#set_procs(f, df, fdf, p)`
  This initialize of reinitialize the function self with p parameters by two or three Proc objects f, df and fdf.
`GSL::MultiFit::Function_fdf#set_data(t, y)`
`GSL::MultiFit::Function_fdf#set_data(t, y, sigma)`
  This sets the data t, y, sigma of length n, to the function self.

## 3 Iteration

`GSL::MultiFit::FdfSolver#iterate`
  THis performs a single iteration of the solver self. If the iteration encounters an unexpected problem then an error code will be returned. The solver maintains a current estimate of the best-fit parameters at all times. This information can be accessed with the method `position`.
`GSL::MultiFit::FdfSolver#position`
  This returns the current position (i.e. best-fit parameters) of the solver self, as a `GSL::Vector` object.
`GSL::MultiFit.covar(J, epsrel)`
`GSL::MultiFit.covar(J, epsrel, covar)`

  This method uses the Jacobian matrix J to compute the covariance matrix of the best-fit parameters. If an existing matrix covar is given, it is overwritten, and if not, this method returns a new matrix. The parameter epsrel is used to remove linear-dependent columns when J is rank deficient.

  The covariance matrix is given by,

      covar = (J^T J)^{-1}

  and is computed by QR decomposition of J with column-pivoting. Any columns of R which satisfy

      |R_{kk}| <= epsrel |R_{11}|

  are considered linearly-dependent and are excluded from the covariance matrix (the corresponding rows and columns of the covariance matrix are set to zero).

### 3.1 Search Stopping Parameters

A minimization procedure should stop when one of the following conditions is true:

- A minimum has been found to within the user-specified precision.
- A user-specified maximum number of iterations has been reached.
- An error has occurred.

The handling of these conditions is under user control. The method below allows the user to test the current estimate of the best-fit parameters.

`GSL::MultiFit::FdfSolver#test_delta(epsabs, epsrel)`

  This method tests for the convergence of the sequence by comparing the last step with the absolute error epsabs and relative error (epsrel to the current position. The test returns `GSL::SUCCESS` if the following condition is achieved,

      |dx_i| < epsabs + epsrel |x_i|

  for each component of x and returns `GSL::CONTINUE` otherwise.

`GSL::MultiFit::FdfSolver#test_gradient(g, epsabs)`
`GSL::MultiFit::FdfSolver#test_gradient(epsabs)`

  This function tests the residual gradient g against the absolute error bound epsabs. If g is not given, it is calculated internally. Mathematically, the gradient should be exactly zero at the minimum. The test returns `GSL::SUCCESS` if the following condition is achieved,

      \sum_i |g_i| < epsabs

  and returns `GSL::CONTINUE` otherwise. This criterion is suitable for situations where the precise location of the minimum, x, is unimportant provided a value can be found where the gradient is small enough.

`GSL::MultiFit::FdfSolver#gradient`
  This method returns the gradient g of \Phi(x) = (1/2) ||F(x)||^2 from the Jacobian matrix and the function values, using the formula g = J^T f.
`GSL::MultiFit.test_delta(dx, x, epsabs, epsrel)`
`GSL::MultiFit.test_gradient(g, epsabs)`
`GSL::MultiFit.gradient(jac, f, g)`
`GSL::MultiFit.covar(jac, epsrel)`
`GSL::MultiFit.covar(jac, epsrel, covar)`
  Singleton methods of the `GSL::MultiFit` module.

## 4 Example

The following example program fits a weighted exponential model with background to experimental data, Y = A exp(-lambda t) + b. The first part of the program sets up the functions `procf` and `procdf` to calculate the model and its Jacobian. The appropriate fitting function is given by,

    f_i = ((A exp(-lambda t_i) + b) - y_i)/sigma_i

where we have chosen t_i = i. The Jacobian matrix `jac` is the derivative of these functions with respect to the three parameters (A, lambda, b). It is given by,

    J_{ij} = d f_i / d x_j

where x_0 = A, x_1 = lambda and x_2 = b.

    require("gsl")
    include GSL::MultiFit

    # x: Vector, list of the parameters to determine
    # t, y, sigma: Vectors, observational data
    # f: Vector, function to minimize
    procf = Proc.new { |x, t, y, sigma, f|
      a = x[0]
      lambda = x[1]
      b = x[2]
      n = t.size
      for i in 0...n do
        yi = a*Math::exp(-lambda*t[i]) + b
        f[i] = (yi - y[i])/sigma[i]
      end
    }

    # jac: Matrix, Jacobian
    procdf = Proc.new { |x, t, y, sigma, jac|
      a = x[0]
      lambda = x[1]
      n = t.size
      for i in 0...n do
        ti = t[i]
        si = sigma[i]
        ei = Math::exp(-lambda*ti)
        jac.set(i, 0, ei/si)
        jac.set(i, 1, -ti*a*ei/si)
        jac.set(i, 2, 1.0/si)
      end
    }

    f = MultiFit::Function_fdf.new(procf, procdf, 2)

    # Create data
    r = Rng.new()
    t = Vector.new(n)
    y = Vector.new(n)
    sigma = Vector.new(n)
    for i in 0...n do
      t[i] = i
      y[i] = 1.0 + 5*Math::exp(-0.1*t[i]) + r.gaussian(0.1)
      sigma[i] = 0.1
    end

    f.set_data(t, y, sigma)
    x = Vector.new(1.0, 0.0, 0.0)    # initial guess

    solver = FdfSolver.new(FdfSolver::LMSDER, n, np)

    solver.set(f, x)

    iter = 0
    solver.print_state(iter)
    begin
      iter += 1
      status = solver.iterate
      solver.print_state(iter)
      status = solver.test_delta(1e-4, 1e-4)
    end while status == GSL::CONTINUE and iter < 500

    printf("A      = %.5f +/- %.5f\n", position[0], Math::sqrt(covar.get(0, 0)))
    printf("lambda = %.5f +/- %.5f\n", position[1], Math::sqrt(covar.get(1, 1)))
    printf("b      = %.5f +/- %.5f\n", position[2], Math::sqrt(covar.get(2, 2)))

prev next

Reference index top