A point x is projected into the interior of [u,v] and x[i] is rounded to the nearest integer multiple of self.dx[i]

Return the projected and rounded version of x

Sort x in increasing order, remove multiple entries.

Warning: when you use this function, make sure x and w is row vector

Minimization of the quadratic polynomial p(x) = a*x^2+b*x over [xl,xu]

Output: x minimizer in [xl,xu]

Update the search bounds [self.u, self.v]

The search box [self.u, self.v] is updated if the user changed it. The vectors self.u and self.v are defined such that [self.u, self.v] is the smallest box containing [u', v'], all new input points and, in the case of a continuation call to Snobfit, also the box [u1, v1] from the previous iteration. [self.u, self.v] is considered to be the box to be explored and a box tree of [self.u, self.v] is generated;

Note: All suggested new evaluation points are in [u', v'].

Compute the safeguarded nearest neighbors.

If we have |X| < n + dn + 1 for the current set X of evaluation points,
or if all function values != NaN (if any) are identical, go to Step 11.
Otherwise, for each new point x a vector pointing to n + dn
safeguarded nearest neighbors is computed. The neighbor lists of
some old points are updated in the same way if necessary.

Note: we denote the X the set of points for which the objective
      function has already been evaluated at some stage of SNOBFIT.

Calculate the fictitious values f and df for nan points

For any new input point x with function value NaN (which means that the function value could not be determined at that point) and for all old points marked infeasible whose nearest neighbors have changed, we define fictitious values f and df as specified in Section 3.

Later we test an objective function with nan values

Create local surrogate models.

Local surrogate models are created by linear least squares fit. Local fits (as described in Section 3) around all new points and all old points with changed nearest neighbors are computed and the potential points y of Class 2 and 3 and their estimated function values fy are determined as in Section 4.

The current best point xbest and the current best function value fbest
in [self.u1, self.v1] are determined. If the objective function
has not been evaluated in [self.u1, self.v1] yet, (i.e., n1 = 0),
it mean that we doesn't generate the recommended evalution points of
Class 1. And go to Step 8.

A local quadratic fit around self.xbest is computed as described
in Section 3 and the point z of Class 1 is generated as described
in Section 4. If such a point z was generated, let z be contained in
the subbox [\underline{x}^j, ar{x}^j ] (in the case that z belongs
to more than one box, a box with minimal smallness is selected).
if
             min_i( (ar{x}^j_i - \underline{x}^j_i )/(v_i-u_i) ) >
      0.05 * max_i( (ar{x}^j_i - \underline{x}^j_i )/(v_i-u_i) ),

the point z is put into the list of recommended evaluation points,
and otherwise (if the smallest side length of the box
[ ar{x}_j , \underline{x}_j ] relative to [self.u, self.v] is too
small compared to the largest one) we set self.J4 = {j}, i.e.,
a point of Class 4 is to be generated in this box later,
which seems to be more appropriate for such a long, narrow box.
This gives n1 recommended evaluation points (n1 = 0 or 1) of Class 1.

Use random number to assign new evaluation points for each Class.

To generate the remaining self.globloc := nreq - n1 recommended
evaluation points, let glob1 := floor(self.p*self.globloc)
                       glob2 := ceil*(self.p*self.globloc).

Then a random number generator sets self.glob = glob1 with
probability self.globloc*self.p - glob1 and self.glob=glob2 otherwise.
Then self.globloc - self.glob points of Classes 2 and 3 together
and self.glob points of Class 4 are to be generated.

Generate the new points for Class 2,3,4

If X contains any local points, first at most

loc := self.globloc - self.glob

points generated from the local points y are chosen in the order of
ascending model function values fy to yield points of Class 2.

If the desired number of loc points has not been reached yet,
afterwards points y pertaining to nonlocal x which belong to X are
taken (again in the order of ascending fy).
For each potential point of Class 2 or 3 generated in this way,
a subbox [ \underline{x}^j , ar{x}^j ] of the box tree with
minimal smallness is determined with
y belong to  [ \underline{x}^j , ar{x}^j ]
if
             min_i( (ar{x}^j_i - \underline{x}^j_i )/(v_i-u_i) ) >
      0.05 * max_i( (ar{x}^j_i - \underline{x}^j_i )/(v_i-u_i) ),

the point y is not put into the list of recommended evaluation points
but instead we set self.J4 = self.J4 and {j}, i.e., the box is marked
for the generation of a recommended evaluation point of Class 4.

Note: we denote the X the set of points for which the objective
      function has already been evaluated at some stage of SNOBFIT.

Let Smin and Smax denote the minimal and maximal smallness, respectively, among the boxes in current box tree, and let

M := floor(1/3(Smax-Smin)).

For each smallness S = Smin + m, m = 0, . . . ,M, the boxes are sorted according to ascending function values f(x) (i.e., self.f( self.x) ). First a point of Class 4 is generated from the box with S = Smin with smallest f(x). If self.J4 != Null, then the points of Class 4 belonging to the subboxes with indices in self.J4 are generated. Subsequently, a point of Class 4 is generated in the box with smallest f(x) at each nonempty smallness level Smin+m, m = 1, ... ,M, and then the smallness levels from Smin to Smin+M are gone through again etc. until we either have nreq recommended evaluation points or there are no eligible boxes for generating points of Class 4 any more. We assign to the points of Class 4 the model function values obtained from the local models pertaining to the points in their boxes.

Merit function of the soft optimality theorem

Merit function of the soft optimality theorem

Input:
------
f      objective function value
F      vector containing the values of the constraint functions
       (m-vector)
F1     m-vector of lower bounds of the constraints
F2     m-vector of upper bounds of the constraints
f0     scalar parameter in the merit function
Delta  scalar, positive parameter in the merit function
sigma  positive m-vector, where sigma(i) is the permitted violation
       of constraint i

One call of snobfit

Output:
-------
request nreq x (n+3)-matrix
   request[j,0:n] is the jth newly generated point,
   request[j,n+1] is its estimated function value and
   request[j,n+2] is its uncertainty of estimated function value
   request[j,n+3] indicates for which reason the point
                  = 1 best prediction
                  = 2 putative local minimizer
                  = 3 alternative good point
                  = 4 explore empty region
                  = 5 to fill up the required number of function values
                      if too little points of other classes are found
xbest  current best point
fbest  current best function value (i.e. function value at xbest)

Main Loop

The optimization is stopped at one of the follow conditions: 1: if approximately ncall fucntion values have been exceeded.

2: if at least minfval function values were obtained and

the best function value wasn't improved in the last nstop calls to SNOBFIT

3: stop when within tolerance of the known global minimum. E.g.,

for least squares problems, the global minimum is 0 for a perfect match. Various test functions also have known global minima.

Main Loop

The optimization is stopped at one of the follow conditions: 1: if approximately ncall fucntion values have been exceeded.

2: if at least minfval function values were obtained and

the best function value wasn't improved in the last nstop calls to SNOBFIT

3: stop when within tolerance of the known global minimum. E.g.,

for least squares problems, the global minimum is 0 for a perfect match. Various test functions also have known global minima.