# Created by Octave 3.2.3, Tue Jan 05 12:02:55 2010 EST <mandrake@n3.mandriva.com>
# name: cache
# type: cell
# rows: 3
# columns: 37
# name: <cell-element>
# type: string
# elements: 1
# length: 16
LinearRegression
# name: <cell-element>
# type: string
# elements: 1
# length: 720
 general linear regression

 [p,y_var,r,p_var]=LinearRegression(F,y)
 [p,y_var,r,p_var]=LinearRegression(F,y,weight)
 
 determine the parameters p_j  (j=1,2,...,m) such that the function
 f(x) = sum_(i=1,...,m) p_j*f_j(x) fits as good as possible to the 
 given values y_i = f(x_i)
 
 parameters
 F  n*m matrix with the values of the basis functions at the support points 
    in column j give the values of f_j at the points x_i  (i=1,2,...,n)
 y  n column vector of given values
 weight  n column vector of given weights
 
 return values
 p     m vector with the estimated values of the parameters
 y_var estimated variance of the error
 r     weighted norm of residual
 p_var estimated variance of the parameters p_j

# name: <cell-element>
# type: string
# elements: 1
# length: 27
 general linear regression


# name: <cell-element>
# type: string
# elements: 1
# length: 6
adsmax
# name: <cell-element>
# type: string
# elements: 1
# length: 1581
ADSMAX  Alternating directions method for direct search optimization.
        [x, fmax, nf] = ADSMAX(FUN, x0, STOPIT, SAVIT, P) attempts to
        maximize the function FUN, using the starting vector x0.
        The alternating directions direct search method is used.
        Output arguments:
               x    = vector yielding largest function value found,
               fmax = function value at x,
               nf   = number of function evaluations.
        The iteration is terminated when either
               - the relative increase in function value between successive
                 iterations is <= STOPIT(1) (default 1e-3),
               - STOPIT(2) function evaluations have been performed
                 (default inf, i.e., no limit), or
               - a function value equals or exceeds STOPIT(3)
                 (default inf, i.e., no test on function values).
        Progress of the iteration is not shown if STOPIT(5) = 0 (default 1).
        If a non-empty fourth parameter string SAVIT is present, then
        `SAVE SAVIT x fmax nf' is executed after each inner iteration.
        By default, the search directions are the co-ordinate directions.
        The columns of a fifth parameter matrix P specify alternative search
        directions (P = EYE is the default).
        NB: x0 can be a matrix.  In the output argument, in SAVIT saves,
            and in function calls, x has the same shape as x0.
        ADSMAX(fun, x0, STOPIT, SAVIT, P, P1, P2,...) allows additional
        arguments to be passed to fun, via feval(fun,x,P1,P2,...).

# name: <cell-element>
# type: string
# elements: 1
# length: 69
ADSMAX  Alternating directions method for direct search optimization.

# name: <cell-element>
# type: string
# elements: 1
# length: 7
battery
# name: <cell-element>
# type: string
# elements: 1
# length: 123
 battery.m: repeatedly call bfgs using a battery of 
 start values, to attempt to find global min
 of a nonconvex function

# name: <cell-element>
# type: string
# elements: 1
# length: 9
 battery.

# name: <cell-element>
# type: string
# elements: 1
# length: 7
bfgsmin
# name: <cell-element>
# type: string
# elements: 1
# length: 1747
 bfgsmin: bfgs or limited memory bfgs minimization of function

 Usage: [x, obj_value, convergence, iters] = bfgsmin(f, args, control)

 The function must be of the form
 [value, return_2,..., return_m] = f(arg_1, arg_2,..., arg_n)
 By default, minimization is w.r.t. arg_1, but it can be done
 w.r.t. any argument that is a vector. Numeric derivatives are
 used unless analytic derivatives are supplied. See bfgsmin_example.m
 for methods.

 Arguments:
 * f: name of function to minimize (string)
 * args: a cell array that holds all arguments of the function
 	The argument with respect to which minimization is done
 	MUST be a vector
 * control: an optional cell array of 1-8 elements. If a cell
   array shorter than 8 elements is provided, the trailing elements
   are provided with default values.
 	* elem 1: maximum iterations  (positive integer, or -1 or Inf for unlimited (default))
 	* elem 2: verbosity
 		0 = no screen output (default)
 		1 = only final results
 		2 = summary every iteration
 		3 = detailed information
 	* elem 3: convergence criterion
 		1 = strict (function, gradient and param change) (default)
 		0 = weak - only function convergence required
 	* elem 4: arg in f_args with respect to which minimization is done (default is first)
 	* elem 5: (optional) Memory limit for lbfgs. If it's a positive integer
 		then lbfgs will be use. Otherwise ordinary bfgs is used
 	* elem 6: function change tolerance, default 1e-12
 	* elem 7: parameter change tolerance, default 1e-6
 	* elem 8: gradient tolerance, default 1e-5

 Returns:
 * x: the minimizer
 * obj_value: the value of f() at x
 * convergence: 1 if normal conv, other values if not
 * iters: number of iterations performed

 Example: see bfgsmin_example.m

# name: <cell-element>
# type: string
# elements: 1
# length: 63
 bfgsmin: bfgs or limited memory bfgs minimization of function


# name: <cell-element>
# type: string
# elements: 1
# length: 15
bfgsmin_example
# name: <cell-element>
# type: string
# elements: 1
# length: 16
 initial values

# name: <cell-element>
# type: string
# elements: 1
# length: 16
 initial values


# name: <cell-element>
# type: string
# elements: 1
# length: 5
cdiff
# name: <cell-element>
# type: string
# elements: 1
# length: 1378
 c = cdiff (func,wrt,N,dfunc,stack,dx) - Code for num. differentiation
   = "function df = dfunc (var1,..,dvar,..,varN) .. endfunction
 
 Returns a string of octave code that defines a function 'dfunc' that
 returns the derivative of 'func' with respect to it's 'wrt'th
 argument.

 The derivatives are obtained by symmetric finite difference.

 dfunc()'s return value is in the same format as that of  ndiff()

 func  : string : name of the function to differentiate

 wrt   : int    : position, in argument list, of the differentiation
                  variable.                                Default:1

 N     : int    : total number of arguments taken by 'func'. 
                  If N=inf, dfunc will take variable argument list.
                                                         Default:wrt

 dfunc : string : Name of the octave function that returns the
                   derivatives.                   Default:['d',func]

 stack : string : Indicates whether 'func' accepts vertically
                  (stack="rstack") or horizontally (stack="cstack")
                  arguments. Any other string indicates that 'func'
                  does not allow stacking.                Default:''

 dx    : real   : Step used in the symmetric difference scheme.
                                                  Default:10*sqrt(eps)

 See also : ndiff, eval, todisk


# name: <cell-element>
# type: string
# elements: 1
# length: 54
 c = cdiff (func,wrt,N,dfunc,stack,dx) - Code for num.

# name: <cell-element>
# type: string
# elements: 1
# length: 16
cg_min_example_2
# name: <cell-element>
# type: string
# elements: 1
# length: 57
 [x,v,niter] = conjgrad_min("testfunc","dtestf",xinit) ;

# name: <cell-element>
# type: string
# elements: 1
# length: 57
 [x,v,niter] = conjgrad_min("testfunc","dtestf",xinit) ;


# name: <cell-element>
# type: string
# elements: 1
# length: 16
cg_min_example_3
# name: <cell-element>
# type: string
# elements: 1
# length: 77
 [xlev,vlev,nlev] = feval (optim_func, "ff", "dff", xinit, "extra", extra) ;

# name: <cell-element>
# type: string
# elements: 1
# length: 77
 [xlev,vlev,nlev] = feval (optim_func, "ff", "dff", xinit, "extra", extra) ;


# name: <cell-element>
# type: string
# elements: 1
# length: 6
d2_min
# name: <cell-element>
# type: string
# elements: 1
# length: 2782
 [x,v,nev,h,args] = d2_min(f,d2f,args,ctl,code) - Newton-like minimization

 Minimize f(x) using 1st and 2nd derivatives. Any function w/ second
 derivatives can be minimized, as in Newton. f(x) decreases at each
 iteration, as in Levenberg-Marquardt. This function is inspired from the
 Levenberg-Marquardt algorithm found in the book "Numerical Recipes".

 ARGUMENTS :
 f    : string : Cost function's name

 d2f  : string : Name of function returning the cost (1x1), its
                 differential (1xN) and its second differential or it's
                 pseudo-inverse (NxN) (see ctl(5) below) :

                 [v,dv,d2v] = d2f (x).

 args : list   : f and d2f's arguments. By default, minimize the 1st
     or matrix : argument.

 ctl  : vector : Control arguments (see below)
      or struct

 code : string : code will be evaluated after each outer loop that
                 produced some (any) improvement. Variables visible from
                 "code" include "x", the best parameter found, "v" the
                 best value and "args", the list of all arguments. All can
                 be modified. This option can be used to re-parameterize 
                 the argument space during optimization

 CONTROL VARIABLE ctl : (optional). May be a struct or a vector of length
 ---------------------- 5 or less where NaNs are ignored. Default values
                        are written <value>.
 FIELD  VECTOR
 NAME    POS

 ftol, f N/A    : Stop search when value doesn't improve, as tested by

                   f > Deltaf/max(|f(x)|,1)

             where Deltaf is the decrease in f observed in the last
             iteration.                                     <10*sqrt(eps)>

 utol, u N/A    : Stop search when updates are small, as tested by

                   u > max { dx(i)/max(|x(i)|,1) | i in 1..N }

             where  dx is the change in the x that occured in the last
             iteration.                                              <NaN>

 dtol, d N/A    : Stop search when derivative is small, as tested by
 
                   d > norm (dv)                                     <eps>

 crit, c ctl(1) : Set one stopping criterion, 'ftol' (c=1), 'utol' (c=2)
                  or 'dtol' (c=3) to the value of by the 'tol' option. <1>

 tol, t  ctl(2) : Threshold in termination test chosen by 'crit'  <10*eps>

 narg, n ctl(3) : Position of the minimized argument in args           <1>
 maxev,m ctl(4) : Maximum number of function evaluations             <inf>
 maxout,m       : Maximum number of outer loops                      <inf>
 id2f, i ctl(5) : 0 if d2f returns the 2nd derivatives, 1 if           <0>
                  it returns its pseudo-inverse.

 verbose, v N/A : Be more or less verbose (quiet=0)                    <0>

# name: <cell-element>
# type: string
# elements: 1
# length: 75
 [x,v,nev,h,args] = d2_min(f,d2f,args,ctl,code) - Newton-like minimization


# name: <cell-element>
# type: string
# elements: 1
# length: 5
deriv
# name: <cell-element>
# type: string
# elements: 1
# length: 21
 deriv(f,x0[,h,O,N])

# name: <cell-element>
# type: string
# elements: 1
# length: 21
 deriv(f,x0[,h,O,N])


# name: <cell-element>
# type: string
# elements: 1
# length: 4
dfdp
# name: <cell-element>
# type: string
# elements: 1
# length: 719
 numerical partial derivatives (Jacobian) df/dp for use with leasqr
 --------INPUT VARIABLES---------
 x=vec or matrix of indep var(used as arg to func) x=[x0 x1 ....]
 f=func(x,p) vector initialsed by user before each call to dfdp
 p= vec of current parameter values
 dp= fractional increment of p for numerical derivatives
      dp(j)>0 central differences calculated
      dp(j)<0 one sided differences calculated
      dp(j)=0 sets corresponding partials to zero; i.e. holds p(j) fixed
 func=string naming the function (.m) file
      e.g. to calc Jacobian for function expsum prt=dfdp(x,f,p,dp,'expsum')
----------OUTPUT VARIABLES-------
 prt= Jacobian Matrix prt(i,j)=df(i)/dp(j)
================================

# name: <cell-element>
# type: string
# elements: 1
# length: 80
 numerical partial derivatives (Jacobian) df/dp for use with leasqr
 --------INP

# name: <cell-element>
# type: string
# elements: 1
# length: 7
expdemo
# name: <cell-element>
# type: string
# elements: 1
# length: 32
 An example of expfit in action

# name: <cell-element>
# type: string
# elements: 1
# length: 32
 An example of expfit in action


# name: <cell-element>
# type: string
# elements: 1
# length: 6
expfit
# name: <cell-element>
# type: string
# elements: 1
# length: 2344
 USAGE  [alpha,c,rms] = expfit( deg, x1, h, y )

 Prony's method for non-linear exponential fitting

 Fit function:   \sum_1^{deg} c(i)*exp(alpha(i)*x)

 Elements of data vector y must correspond to
 equidistant x-values starting at x1 with stepsize h

 The method is fully compatible with complex linear
 coefficients c, complex nonlinear coefficients alpha
 and complex input arguments y, x1, non-zero h .
 Fit-order deg  must be a real positive integer.

 Returns linear coefficients c, nonlinear coefficients
 alpha and root mean square error rms. This method is
 known to be more stable than 'brute-force' non-linear
 least squares fitting.

 Example
    x0 = 0; step = 0.05; xend = 5; x = x0:step:xend;
    y = 2*exp(1.3*x)-0.5*exp(2*x);
    error = (rand(1,length(y))-0.5)*1e-4;
    [alpha,c,rms] = expfit(2,x0,step,y+error)

  alpha =
    2.0000
    1.3000
  c =
    -0.50000
     2.00000
  rms = 0.00028461

 The fit is very sensitive to the number of data points.
 It doesn't perform very well for small data sets.
 Theoretically, you need at least 2*deg data points, but
 if there are errors on the data, you certainly need more.

 Be aware that this is a very (very,very) ill-posed problem.
 By the way, this algorithm relies heavily on computing the
 roots of a polynomial. I used 'roots.m', if there is
 something better please use that code.

 Copyright (C) 2000 Gert Van den Eynde
 SCK-CEN (Nuclear Energy Research Centre)
 Boeretang 200
 2400 Mol
 Belgium
 na.gvandeneynde@na-net.ornl.gov

 This code is under the GNU Public License (GPL) version 2 or later.
 I hope that it is useful, but it is WITHOUT ANY WARRANTY, without
 even the implied warranty of MERCHANTABILITY or FITNESS FOR A
 PARTICULAR PURPOSE.
 __________________________________________________________________
 Modified for full compatibility with complex fit-functions by
 Rolf Fabian <fabian@tu-cottbus.de>                2002-Sep-23
 Brandenburg University of Technology Cottbus
 Dep. of Air Chemistry and Pollution Control

 Demo for a complex fit-function:
 deg= 2; N= 20; x1= -(1+i), x= linspace(x1,1+i/2,N).';
 h = x(2) - x(1)
 y= (2+i)*exp( (-1-2i)*x ) + (-1+3i)*exp( (2+3i)*x );
 A= 5e-2; y+= A*(randn(N,1)+randn(N,1)*i); % add complex noise
 [alpha,c,rms]= expfit( deg, x1, h, y )
 __________________________________________________________________

# name: <cell-element>
# type: string
# elements: 1
# length: 48
 USAGE  [alpha,c,rms] = expfit( deg, x1, h, y )


# name: <cell-element>
# type: string
# elements: 1
# length: 7
fminunc
# name: <cell-element>
# type: string
# elements: 1
# length: 1407
 [x,v,flag,out,df,d2f] = fminunc (f,x,opt,...) - M*tlab-like optimization

 Imitation of m*tlab's fminunc(). The optional 'opt' argument is a struct,
 e.g. produced by 'optimset()'.

 Supported options
 -----------------
 Diagnostics, [off|on] : Be verbose
 Display    , [off|iter|notify|final]
                       : Be verbose unless value is "off"
 GradObj    , [off|on] : Function's 2nd return value is derivatives
 Hessian    , [off|on] : Function's 2nd and 3rd return value are
                         derivatives and Hessian.
 TolFun     , scalar   : Termination criterion (see 'ftol' in minimize())
 TolX       , scalar   : Termination criterion (see 'utol' in minimize())
 MaxFunEvals, int      : Max. number of function evaluations
 MaxIter    , int      : Max. number of algorithm iterations

 These non-m*tlab are provided to facilitate porting code to octave:
 -----------------------
 "MinEquiv" , [off|on] : Don't minimize 'fun', but instead return the
                         option passed to minimize().

 "Backend"  , [off|on] : Don't minimize 'fun', but instead return
                         [backend, opt], the name of the backend
                         optimization function that is used and the
                         optional arguments that will be passed to it. See
                         the 'backend' option of minimize().

 This function is a front-end to minimize().

# name: <cell-element>
# type: string
# elements: 1
# length: 43
 [x,v,flag,out,df,d2f] = fminunc (f,x,opt,.

# name: <cell-element>
# type: string
# elements: 1
# length: 6
leasqr
# name: <cell-element>
# type: string
# elements: 1
# length: 2910
function [f,p,kvg,iter,corp,covp,covr,stdresid,Z,r2]=
                   leasqr(x,y,pin,F,{stol,niter,wt,dp,dFdp,options})

 Levenberg-Marquardt nonlinear regression of f(x,p) to y(x).

 Version 3.beta
 Optional parameters are in braces {}.
 x = column vector or matrix of independent variables, 1 row per
   observation: x = [x0 x1....xm].
 y = column vector of observed values, same number of rows as x.
 wt = column vector (dim=length(x)) of statistical weights.  These
   should be set to be proportional to (sqrt of var(y))^-1; (That is,
   the covariance matrix of the data is assumed to be proportional to
   diagonal with diagonal equal to (wt.^2)^-1.  The constant of
   proportionality will be estimated.); default = ones(length(y),1).
 pin = column vec of initial parameters to be adjusted by leasqr.
 dp = fractional increment of p for numerical partial derivatives;
   default = .001*ones(size(pin))
   dp(j) > 0 means central differences on j-th parameter p(j).
   dp(j) < 0 means one-sided differences on j-th parameter p(j).
   dp(j) = 0 holds p(j) fixed i.e. leasqr wont change initial guess: pin(j)
 F = name of function in quotes; the function shall be of the form y=f(x,p),
   with y, x, p of the form y, x, pin as described above.
 dFdp = name of partial derivative function in quotes; default is "dfdp", a
   slow but general partial derivatives function; the function shall be
   of the form prt=dfdp(x,f,p,dp,F) (see dfdp.m).
 stol = scalar tolerance on fractional improvement in scalar sum of
   squares = sum((wt.*(y-f))^2); default stol = .0001;
 niter = scalar maximum number of iterations; default = 20;
 options = matrix of n rows (same number of rows as pin) containing
   column 1: desired fractional precision in parameter estimates.
     Iterations are terminated if change in parameter vector (chg) on two
     consecutive iterations is less than their corresponding elements
     in options(:,1).  [ie. all(abs(chg*current parm est) < options(:,1))
      on two consecutive iterations.], default = zeros().
   column 2: maximum fractional step change in parameter vector.
     Fractional change in elements of parameter vector is constrained to be 
     at most options(:,2) between sucessive iterations.
     [ie. abs(chg(i))=abs(min([chg(i) options(i,2)*current param estimate])).],
     default = Inf*ones().

          OUTPUT VARIABLES
 f = column vector of values computed: f = F(x,p).
 p = column vector trial or final parameters. i.e, the solution.
 kvg = scalar: = 1 if convergence, = 0 otherwise.
 iter = scalar number of iterations used.
 corp = correlation matrix for parameters.
 covp = covariance matrix of the parameters.
 covr = diag(covariance matrix of the residuals).
 stdresid = standardized residuals.
 Z = matrix that defines confidence region (see comments in the source).
 r2 = coefficient of multiple determination.

 All Zero guesses not acceptable

# name: <cell-element>
# type: string
# elements: 1
# length: 80
function [f,p,kvg,iter,corp,covp,covr,stdresid,Z,r2]=
                   leasqr(

# name: <cell-element>
# type: string
# elements: 1
# length: 10
leasqrdemo
# name: <cell-element>
# type: string
# elements: 1
# length: 131
 leasqrdemo

 An example showing how to use non-linear least squares to fit 
 simulated data to the function:

      y = a e^{-bx}

# name: <cell-element>
# type: string
# elements: 1
# length: 12
 leasqrdemo


# name: <cell-element>
# type: string
# elements: 1
# length: 8
line_min
# name: <cell-element>
# type: string
# elements: 1
# length: 538
 [a,fx,nev] = line_min (f, dx, args, narg) - Minimize f() along dx

 INPUT ----------
 f    : string  : Name of minimized function
 dx   : matrix  : Direction along which f() is minimized
 args : list    : List of argument of f
 narg : integer : Position of minimized variable in args.  Default=1

 OUTPUT ---------
 a    : scalar  : Value for which f(x+a*dx) is a minimum (*)
 fx   : scalar  : Value of f(x+a*dx) at minimum (*)
 nev  : integer : Number of function evaluations

 (*) The notation f(x+a*dx) assumes that args == list (x).

# name: <cell-element>
# type: string
# elements: 1
# length: 67
 [a,fx,nev] = line_min (f, dx, args, narg) - Minimize f() along dx


# name: <cell-element>
# type: string
# elements: 1
# length: 6
mdsmax
# name: <cell-element>
# type: string
# elements: 1
# length: 1527
MDSMAX  Multidirectional search method for direct search optimization.
        [x, fmax, nf] = MDSMAX(FUN, x0, STOPIT, SAVIT) attempts to
        maximize the function FUN, using the starting vector x0.
        The method of multidirectional search is used.
        Output arguments:
               x    = vector yielding largest function value found,
               fmax = function value at x,
               nf   = number of function evaluations.
        The iteration is terminated when either
               - the relative size of the simplex is <= STOPIT(1)
                 (default 1e-3),
               - STOPIT(2) function evaluations have been performed
                 (default inf, i.e., no limit), or
               - a function value equals or exceeds STOPIT(3)
                 (default inf, i.e., no test on function values).
        The form of the initial simplex is determined by STOPIT(4):
          STOPIT(4) = 0: regular simplex (sides of equal length, the default),
          STOPIT(4) = 1: right-angled simplex.
        Progress of the iteration is not shown if STOPIT(5) = 0 (default 1).
        If a non-empty fourth parameter string SAVIT is present, then
        `SAVE SAVIT x fmax nf' is executed after each inner iteration.
        NB: x0 can be a matrix.  In the output argument, in SAVIT saves,
            and in function calls, x has the same shape as x0.
        MDSMAX(fun, x0, STOPIT, SAVIT, P1, P2,...) allows additional
        arguments to be passed to fun, via feval(fun,x,P1,P2,...).

# name: <cell-element>
# type: string
# elements: 1
# length: 70
MDSMAX  Multidirectional search method for direct search optimization.

# name: <cell-element>
# type: string
# elements: 1
# length: 8
minimize
# name: <cell-element>
# type: string
# elements: 1
# length: 4104
 [x,v,nev,...] = minimize (f,args,...) - Minimize f

 ARGUMENTS
 f    : string  : Name of function. Must return a real value
 args : list or : List of arguments to f (by default, minimize the first)
        matrix  : f's only argument

 RETURNED VALUES
 x   : matrix  : Local minimum of f. Let's suppose x is M-by-N.
 v   : real    : Value of f in x0
 nev : integer : Number of function evaluations 
     or 1 x 2  : Number of function and derivative evaluations (if
                 derivatives are used)
 

 Extra arguments are either a succession of option-value pairs or a single
 list or struct of option-value pairs (for unary options, the value in the
 struct is ignored).
 
 OPTIONS : DERIVATIVES   Derivatives may be used if one of these options
 ---------------------   uesd. Otherwise, the Nelder-Mean (see
                         nelder_mead_min) method is used.
 
 'd2f', d2f     : Name of a function that returns the value of f, of its
                  1st and 2nd derivatives : [fx,dfx,d2fx] = feval (d2f, x)
                  where fx is a real number, dfx is 1x(M*N) and d2fx is
                  (M*N)x(M*N). A Newton-like method (d2_min) will be used.

 'hess'         : Use [fx,dfx,d2fx] = leval (f, args) to compute 1st and
                  2nd derivatives, and use a Newton-like method (d2_min).

 'd2i', d2i     : Name of a function that returns the value of f, of its
                  1st and pseudo-inverse of second derivatives : 
                  [fx,dfx,id2fx] = feval (d2i, x) where fx is a real
                  number, dfx is 1x(M*N) and d2ix is (M*N)x(M*N).
                  A Newton-like method will be used (see d2_min).

 'ihess'        : Use [fx,dfx,id2fx] = leval (f, args) to compute 1st
                  derivative and the pseudo-inverse of 2nd derivatives,
                  and use a Newton-like method (d2_min).

            NOTE : df, d2f or d2i take the same arguments as f.
 
 'order', n     : Use derivatives of order n. If the n'th order derivative
                  is not specified by 'df', 'd2f' or 'd2i', it will be
                  computed numerically. Currently, only order 1 works.
 
 'ndiff'        : Use a variable metric method (bfgs) using numerical
                  differentiation.

 OPTIONS : STOPPING CRITERIA  Default is to use 'tol'
 ---------------------------
 'ftol', ftol   : Stop search when value doesn't improve, as tested by

              ftol > Deltaf/max(|f(x)|,1)

                 where Deltaf is the decrease in f observed in the last
                 iteration.                                 Default=10*eps

 'utol', utol   : Stop search when updates are small, as tested by

              tol > max { dx(i)/max(|x(i)|,1) | i in 1..N }

                 where  dx is the change in the x that occured in the last
                 iteration.

 'dtol',dtol    : Stop search when derivatives are small, as tested by

              dtol > max { df(i)*max(|x(i)|,1)/max(v,1) | i in 1..N }

                 where x is the current minimum, v is func(x) and df is
                 the derivative of f in x. This option is ignored if
                 derivatives are not used in optimization.

 MISC. OPTIONS
 -------------
 'maxev', m     : Maximum number of function evaluations             <inf>

 'narg' , narg  : Position of the minimized argument in args           <1>
 'isz'  , step  : Initial step size (only for 0 and 1st order method)  <1>
                  Should correspond to expected distance to minimum
 'verbose'      : Display messages during execution

 'backend'      : Instead of performing the minimization itself, return
                  [backend, control], the name and control argument of the
                  backend used by minimize(). Minimimzation can then be
                  obtained without the overhead of minimize by calling, if
                  a 0 or 1st order method is used :

              [x,v,nev] = feval (backend, args, control)
                   
                  or, if a 2nd order method is used :

              [x,v,nev] = feval (backend, control.d2f, args, control)


# name: <cell-element>
# type: string
# elements: 1
# length: 11
 [x,v,nev,.

# name: <cell-element>
# type: string
# elements: 1
# length: 15
nelder_mead_min
# name: <cell-element>
# type: string
# elements: 1
# length: 2753
 [x0,v,nev] = nelder_mead_min (f,args,ctl) - Nelder-Mead minimization

 Minimize 'f' using the Nelder-Mead algorithm. This function is inspired
 from the that found in the book "Numerical Recipes".

 ARGUMENTS
 ---------
 f     : string : Name of function. Must return a real value
 args  : list   : Arguments passed to f.
      or matrix : f's only argument
 ctl   : vector : (Optional) Control variables, described below
      or struct

 RETURNED VALUES
 ---------------
 x0  : matrix   : Local minimum of f
 v   : real     : Value of f in x0
 nev : number   : Number of function evaluations
 
 CONTROL VARIABLE : (optional) may be named arguments (i.e. "name",value
 ------------------ pairs), a struct, or a vector of length <= 6, where
                    NaN's are ignored. Default values are written <value>.
  OPT.   VECTOR
  NAME    POS
 ftol,f  N/A    : Stopping criterion : stop search when values at simplex
                  vertices are all alike, as tested by 

                   f > (max_i (f_i) - min_i (f_i)) /max(max(|f_i|),1)

                  where f_i are the values of f at the vertices.  <10*eps>

 rtol,r  N/A    : Stop search when biggest radius of simplex, using
                  infinity-norm, is small, as tested by :

              ctl(2) > Radius                                     <10*eps>

 vtol,v  N/A    : Stop search when volume of simplex is small, tested by
            
              ctl(2) > Vol

 crit,c ctl(1)  : Set one stopping criterion, 'ftol' (c=1), 'rtol' (c=2)
                  or 'vtol' (c=3) to the value of the 'tol' option.    <1>

 tol, t ctl(2)  : Threshold in termination test chosen by 'crit'  <10*eps>

 narg  ctl(3)  : Position of the minimized argument in args            <1>
 maxev ctl(4)  : Maximum number of function evaluations. This number <inf>
                 may be slightly exceeded.
 isz   ctl(5)  : Size of initial simplex, which is :                   <1>

                { x + e_i | i in 0..N } 
 
                Where x == nth (args, narg) is the initial value 
                 e_0    == zeros (size (x)), 
                 e_i(j) == 0 if j != i and e_i(i) == ctl(5)
                 e_i    has same size as x

                Set ctl(5) to the distance you expect between the starting
                point and the minimum.

 rst   ctl(6)   : When a minimum is found the algorithm restarts next to
                  it until the minimum does not improve anymore. ctl(6) is
                  the maximum number of restarts. Set ctl(6) to zero if
                  you know the function is well-behaved or if you don't
                  mind not getting a true minimum.                     <0>

 verbose, v     Be more or less verbose (quiet=0)                      <0>

# name: <cell-element>
# type: string
# elements: 1
# length: 70
 [x0,v,nev] = nelder_mead_min (f,args,ctl) - Nelder-Mead minimization


# name: <cell-element>
# type: string
# elements: 1
# length: 6
nmsmax
# name: <cell-element>
# type: string
# elements: 1
# length: 1705
NMSMAX  Nelder-Mead simplex method for direct search optimization.
        [x, fmax, nf] = NMSMAX(FUN, x0, STOPIT, SAVIT) attempts to
        maximize the function FUN, using the starting vector x0.
        The Nelder-Mead direct search method is used.
        Output arguments:
               x    = vector yielding largest function value found,
               fmax = function value at x,
               nf   = number of function evaluations.
        The iteration is terminated when either
               - the relative size of the simplex is <= STOPIT(1)
                 (default 1e-3),
               - STOPIT(2) function evaluations have been performed
                 (default inf, i.e., no limit), or
               - a function value equals or exceeds STOPIT(3)
                 (default inf, i.e., no test on function values).
        The form of the initial simplex is determined by STOPIT(4):
           STOPIT(4) = 0: regular simplex (sides of equal length, the default)
           STOPIT(4) = 1: right-angled simplex.
        Progress of the iteration is not shown if STOPIT(5) = 0 (default 1).
           STOPIT(6) indicates the direction (ie. minimization or 
                   maximization.) Default is 1, maximization.
                   set STOPIT(6)=-1 for minimization
        If a non-empty fourth parameter string SAVIT is present, then
        `SAVE SAVIT x fmax nf' is executed after each inner iteration.
        NB: x0 can be a matrix.  In the output argument, in SAVIT saves,
            and in function calls, x has the same shape as x0.
        NMSMAX(fun, x0, STOPIT, SAVIT, P1, P2,...) allows additional
        arguments to be passed to fun, via feval(fun,x,P1,P2,...).

# name: <cell-element>
# type: string
# elements: 1
# length: 66
NMSMAX  Nelder-Mead simplex method for direct search optimization.

# name: <cell-element>
# type: string
# elements: 1
# length: 8
optimset
# name: <cell-element>
# type: string
# elements: 1
# length: 1021
 opt = optimset (...)         - manipulate m*tlab-style options structure
 
 This function returns a m*tlab-style options structure that can be used
 with the fminunc() function.

 INPUT : Input consist in one or more structs followed by option-value
 pairs. The option that can be passed are those of m*tlab's 'optimset'.
 Whether fminunc() accepts them is another question (see fminunc()).
 
 Two extra options are supported which indicate how to use directly octave
 optimization tools (such as minimize() and other backends):

 "MinEquiv", [on|off] : Tell 'fminunc()' not to minimize 'fun', but
                        instead return the option passed to minimize().

 "Backend", [on|off] : Tell 'fminunc()' not to minimize 'fun', but
                       instead return the [backend, opt], the name of the
                       backend optimization function that is used and the
                       optional arguments that will be passed to it. See
                       the 'backend' option of minimize().
 

# name: <cell-element>
# type: string
# elements: 1
# length: 18
 opt = optimset (.

# name: <cell-element>
# type: string
# elements: 1
# length: 9
poly_2_ex
# name: <cell-element>
# type: string
# elements: 1
# length: 353
  ex = poly_2_ex (l, f)       - Extremum of a 1-var deg-2 polynomial

 l  : 3 : Values of variable at which polynomial is known.
 f  : 3 : f(i) = Value of the degree-2 polynomial at l(i).
 
 ex : 1 : Value for which f reaches its extremum
 
 Assuming that f(i) = a*l(i)^2 + b*l(i) + c = P(l(i)) for some a, b, c,
 ex is the extremum of the polynome P.


# name: <cell-element>
# type: string
# elements: 1
# length: 69
  ex = poly_2_ex (l, f)       - Extremum of a 1-var deg-2 polynomial


# name: <cell-element>
# type: string
# elements: 1
# length: 8
polyconf
# name: <cell-element>
# type: string
# elements: 1
# length: 1439
 [y,dy] = polyconf(p,x,s)

   Produce prediction intervals for the fitted y. The vector p 
   and structure s are returned from polyfit or wpolyfit. The 
   x values are where you want to compute the prediction interval.

 polyconf(...,['ci'|'pi'])

   Produce a confidence interval (range of likely values for the
   mean at x) or a prediction interval (range of likely values 
   seen when measuring at x).  The prediction interval tells
   you the width of the distribution at x.  This should be the same
   regardless of the number of measurements you have for the value
   at x.  The confidence interval tells you how well you know the
   mean at x.  It should get smaller as you increase the number of
   measurements.  Error bars in the physical sciences usually show 
   a 1-alpha confidence value of erfc(1/sqrt(2)), representing
   one standandard deviation of uncertainty in the mean.

 polyconf(...,1-alpha)

   Control the width of the interval. If asking for the prediction
   interval 'pi', the default is .05 for the 95% prediction interval.
   If asking for the confidence interval 'ci', the default is
   erfc(1/sqrt(2)) for a one standard deviation confidence interval.

 Example:
  [p,s] = polyfit(x,y,1);
  xf = linspace(x(1),x(end),150);
  [yf,dyf] = polyconf(p,xf,s,'ci');
  plot(xf,yf,'g-;fit;',xf,yf+dyf,'g.;;',xf,yf-dyf,'g.;;',x,y,'xr;data;');
  plot(x,y-polyval(p,x),';residuals;',xf,dyf,'g-;;',xf,-dyf,'g-;;');

# name: <cell-element>
# type: string
# elements: 1
# length: 26
 [y,dy] = polyconf(p,x,s)


# name: <cell-element>
# type: string
# elements: 1
# length: 10
rosenbrock
# name: <cell-element>
# type: string
# elements: 1
# length: 196
 Rosenbrock function - used to create example obj. fns.

 Function value and gradient vector of the rosenbrock function
 The minimizer is at the vector (1,1,..,1),
 and the minimized value is 0.


# name: <cell-element>
# type: string
# elements: 1
# length: 50
 Rosenbrock function - used to create example obj.

# name: <cell-element>
# type: string
# elements: 1
# length: 13
samin_example
# name: <cell-element>
# type: string
# elements: 1
# length: 16
 dimensionality

# name: <cell-element>
# type: string
# elements: 1
# length: 16
 dimensionality


# name: <cell-element>
# type: string
# elements: 1
# length: 14
test_fminunc_1
# name: <cell-element>
# type: string
# elements: 1
# length: 92
 Plain run, just to make sure ######################################
 Minimum wrt 'x' is y0

# name: <cell-element>
# type: string
# elements: 1
# length: 80
 Plain run, just to make sure ######################################
 Minimum wr

# name: <cell-element>
# type: string
# elements: 1
# length: 10
test_min_1
# name: <cell-element>
# type: string
# elements: 1
# length: 63
 [x,v,niter] = feval (optim_func, "testfunc","dtestf", xinit);

# name: <cell-element>
# type: string
# elements: 1
# length: 63
 [x,v,niter] = feval (optim_func, "testfunc","dtestf", xinit);


# name: <cell-element>
# type: string
# elements: 1
# length: 10
test_min_2
# name: <cell-element>
# type: string
# elements: 1
# length: 60
 [xlev,vlev,nlev] = feval(optim_func, "ff", "dff", xinit) ;

# name: <cell-element>
# type: string
# elements: 1
# length: 60
 [xlev,vlev,nlev] = feval(optim_func, "ff", "dff", xinit) ;


# name: <cell-element>
# type: string
# elements: 1
# length: 10
test_min_3
# name: <cell-element>
# type: string
# elements: 1
# length: 166
 [xlev,vlev,nlev] = feval (optim_func, "ff", "dff", xinit, "extra", extra) ;
 [xlev,vlev,nlev] = feval \
     (optim_func, "ff", "dff", list (xinit, obsmat, obses));

# name: <cell-element>
# type: string
# elements: 1
# length: 80
 [xlev,vlev,nlev] = feval (optim_func, "ff", "dff", xinit, "extra", extra) ;
 [x

# name: <cell-element>
# type: string
# elements: 1
# length: 10
test_min_4
# name: <cell-element>
# type: string
# elements: 1
# length: 178
 Plain run, just to make sure ######################################
 Minimum wrt 'x' is y0
 [xlev,vlev,nlev] = feval (optim_func, "ff", "dff", list (x0,y0,1));
 ctl.df = "dff";

# name: <cell-element>
# type: string
# elements: 1
# length: 80
 Plain run, just to make sure ######################################
 Minimum wr

# name: <cell-element>
# type: string
# elements: 1
# length: 15
test_minimize_1
# name: <cell-element>
# type: string
# elements: 1
# length: 92
 Plain run, just to make sure ######################################
 Minimum wrt 'x' is y0

# name: <cell-element>
# type: string
# elements: 1
# length: 80
 Plain run, just to make sure ######################################
 Minimum wr

# name: <cell-element>
# type: string
# elements: 1
# length: 22
test_nelder_mead_min_1
# name: <cell-element>
# type: string
# elements: 1
# length: 29
 Use vanilla nelder_mead_min

# name: <cell-element>
# type: string
# elements: 1
# length: 29
 Use vanilla nelder_mead_min


# name: <cell-element>
# type: string
# elements: 1
# length: 22
test_nelder_mead_min_2
# name: <cell-element>
# type: string
# elements: 1
# length: 70

 Test using volume #################################################

# name: <cell-element>
# type: string
# elements: 1
# length: 70

 Test using volume #################################################


# name: <cell-element>
# type: string
# elements: 1
# length: 13
test_wpolyfit
# name: <cell-element>
# type: string
# elements: 1
# length: 34
          x         y          dy

# name: <cell-element>
# type: string
# elements: 1
# length: 34
          x         y          dy


# name: <cell-element>
# type: string
# elements: 1
# length: 12
wpolyfitdemo
# name: <cell-element>
# type: string
# elements: 1
# length: 362
 wpolyfitdemo(p)
 Generate some random data for the polynomial p, then fit that
 data.  If p ends with 0, then the fit will be constrained to
 go through the origin.  

 To force a variety of weights, poisson statistics are used to 
 estimate the variance on the individual points, but gaussian 
 statistics are used to generate new values within that variance.

# name: <cell-element>
# type: string
# elements: 1
# length: 80
 wpolyfitdemo(p)
 Generate some random data for the polynomial p, then fit that


# name: <cell-element>
# type: string
# elements: 1
# length: 6
wsolve
# name: <cell-element>
# type: string
# elements: 1
# length: 1736
 [x,s] = wsolve(A,y,dy)

 Solve a potentially over-determined system with uncertainty in
 the values. 

     A x = y +/- dy

 Use QR decomposition for increased accuracy.  Estimate the 
 uncertainty for the solution from the scatter in the data.

 The returned structure s contains

    normr = sqrt( A x - y ), weighted by dy
    R such that R'R = A'A
    df = n-p, n = rows of A, p = columns of A

 See polyconf for details on how to use s to compute dy.
 The covariance matrix is inv(R'*R).  If you know that the
 parameters are independent, then uncertainty is given by
 the diagonal of the covariance matrix, or 

    dx = sqrt(N*sumsq(inv(s.R'))')

 where N = normr^2/df, or N = 1 if df = 0.

 Example 1: weighted system

    A=[1,2,3;2,1,3;1,1,1]; xin=[1;2;3]; 
    dy=[0.2;0.01;0.1]; y=A*xin+randn(size(dy)).*dy;
    [x,s] = wsolve(A,y,dy);
    dx = sqrt(sumsq(inv(s.R'))');
    res = [xin, x, dx]

 Example 2: weighted overdetermined system  y = x1 + 2*x2 + 3*x3 + e

    A = fullfact([3,3,3]); xin=[1;2;3];
    y = A*xin; dy = rand(size(y))/50; y+=dy.*randn(size(y));
    [x,s] = wsolve(A,y,dy);
    dx = s.normr*sqrt(sumsq(inv(s.R'))'/s.df);
    res = [xin, x, dx]

 Note there is a counter-intuitive result that scaling the
 uncertainty in the data does not affect the uncertainty in
 the fit.  Indeed, if you perform a monte carlo simulation
 with x,y datasets selected from a normal distribution centered
 on y with width 10*dy instead of dy you will see that the
 variance in the parameters indeed increases by a factor of 100.
 However, if the error bars really do increase by a factor of 10
 you should expect a corresponding increase in the scatter of 
 the data, which will increase the variance computed by the fit.

# name: <cell-element>
# type: string
# elements: 1
# length: 24
 [x,s] = wsolve(A,y,dy)


