optiml.opti.unconstrained.line_search.line_search module
- class optiml.opti.unconstrained.line_search.line_search.LineSearch(f, max_f_eval=1000, m1=0.01, a_start=1, tau=0.9, min_a=1e-16)[source]
Bases:
object- Parameters:
f – the objective function.
max_f_eval – (integer scalar, optional, default value 1000): the maximum number of function evaluations (hence, iterations will be not more than max_f_eval because at each iteration at least a function evaluation is performed, possibly more due to the line search).
m1 – (real scalar, optional, default value 0.01): first parameter of the Armijo-Wolfe-type line search (sufficient decrease). Has to be in (0,1).
a_start – (real scalar, optional, default value 1): starting value of alpha in the line search (> 0).
tau – (real scalar, optional, default value 0.9): scaling parameter for the line search. In the Armijo-Wolfe line search it is used in the first phase: if the derivative is not positive, then the step is divided by tau (which is < 1, hence it is increased). In the Backtracking line search, each time the step is multiplied by tau (hence it is decreased).
min_a – (real scalar, optional, default value 1e-16): if the algorithm determines a step size value <= min_a, this is taken as an indication that something has gone wrong (the gradient is not a direction of descent, so maybe the function is not differentiable) and computation is stopped. It is legal to take min_a = 0, thereby in fact skipping this test.
- class optiml.opti.unconstrained.line_search.line_search.BacktrackingLineSearch(f, max_f_eval=1000, m1=0.01, a_start=1, tau=0.9, min_a=1e-16)[source]
Bases:
LineSearchPerforms a Backtracking Line Search.
phi0 = phi(0), phi_p0 = phi’(0) < 0
a_start > 0 is the first value to be tested, which is decreased by multiplying it by tau < 1 until the Armijo condition with parameter m1 is satisfied.
- Returns:
the optimal step and the optimal f-value
- Parameters:
f – the objective function.
max_f_eval – (integer scalar, optional, default value 1000): the maximum number of function evaluations (hence, iterations will be not more than max_f_eval because at each iteration at least a function evaluation is performed, possibly more due to the line search).
m1 – (real scalar, optional, default value 0.01): first parameter of the Armijo-Wolfe-type line search (sufficient decrease). Has to be in (0,1).
a_start – (real scalar, optional, default value 1): starting value of alpha in the line search (> 0).
tau – (real scalar, optional, default value 0.9): scaling parameter for the line search. In the Armijo-Wolfe line search it is used in the first phase: if the derivative is not positive, then the step is divided by tau (which is < 1, hence it is increased). In the Backtracking line search, each time the step is multiplied by tau (hence it is decreased).
min_a – (real scalar, optional, default value 1e-16): if the algorithm determines a step size value <= min_a, this is taken as an indication that something has gone wrong (the gradient is not a direction of descent, so maybe the function is not differentiable) and computation is stopped. It is legal to take min_a = 0, thereby in fact skipping this test.
- class optiml.opti.unconstrained.line_search.line_search.ArmijoWolfeLineSearch(f, max_f_eval=1000, m1=0.01, m2=0.9, a_start=1, tau=0.9, sfgrd=0.01, min_a=1e-16)[source]
Bases:
LineSearchPerforms an Armijo-Wolfe Line Search.
phi0 = phi(0), phi_p0 = phi’(0) < 0
a_start > 0 is the first value to be tested: if phi’(as) < 0 then a_start is divided by tau < 1 (hence it is increased) until this does not happen any longer.
m1 and m2 are the standard Armijo-Wolfe parameters; note that the strong Wolfe condition is used.
- Returns:
the optimal step and the optimal f-value
- Parameters:
f – the objective function.
max_f_eval – (integer scalar, optional, default value 1000): the maximum number of function evaluations (hence, iterations will be not more than max_f_eval because at each iteration at least a function evaluation is performed, possibly more due to the line search).
m1 – (real scalar, optional, default value 0.01): first parameter of the Armijo-Wolfe-type line search (sufficient decrease). Has to be in (0,1).
m2 – (real scalar, optional, default value 0.9): typically the second parameter of the Armijo-Wolfe-type line search (strong curvature condition). It should to be in (0,1); if not, it is taken to mean that the simpler Backtracking line search should be used instead.
a_start – (real scalar, optional, default value 1): starting value of alpha in the line search (> 0).
tau – (real scalar, optional, default value 0.9): scaling parameter for the line search. In the Armijo-Wolfe line search it is used in the first phase: if the derivative is not positive, then the step is divided by tau (which is < 1, hence it is increased). In the Backtracking line search, each time the step is multiplied by tau (hence it is decreased).
sfgrd – (real scalar, optional, default value 0.01): safeguard parameter for the line search. To avoid numerical problems that can occur with the quadratic interpolation if the derivative at one endpoint is too large w.r.t. The one at the other (which leads to choosing a point extremely near to the other endpoint), a safeguarded version of interpolation is used whereby the new point is chosen in the interval [as * (1 + sfgrd) , am * (1 - sfgrd)], being [as , am] the current interval, whatever quadratic interpolation says. If you experience problems with the line search taking too many iterations to converge at “nasty” points, try to increase this.
min_a – (real scalar, optional, default value 1e-16): if the algorithm determines a step size value <= min_a, this is taken as an indication that something has gone wrong (the gradient is not a direction of descent, so maybe the function is not differentiable) and computation is stopped. It is legal to take min_a = 0, thereby in fact skipping this test.