[NLopt-discuss] SLSQP, positive directional derivative

[Joshua N Pritikin]

During optimization, I have a problem that gets to a point where the
gradient looks like this,

[0] gradient = t( matrix(c( # 51x1
-31780.195562, -38508.674735, -50973.208738, -55408.084812,
-66931.026056, -74286.656477, -80710.037658, -32059.100573,
-40421.260358, -47351.363022, -56331.397546, -67570.244335,
-71730.720066, -80617.938563, -30100.959330, -40235.309256,
-47047.853982, -56263.225828, -67009.403836, -75987.083372,
-84897.553874, -62848.356157, -210680.805240, -214470.061730,
-234536.487353, -241645.958717, -262177.928304, -278085.836093,
-286557.250051, -63655.124325, -214492.388535, -219036.315591,
-224526.862707, -240634.971741, -248375.386046, -263267.566427,
-289505.767031, -64899.664139, -213624.942908, -222084.527486,
-223847.520561, -236782.323103, -258931.376366, -276562.312245,
-304241.522192, -25035.332609, -25125.410641, -25003.755675,
-4302.824452, -4986.836691, -3574.837605), byrow=TRUE, nrow=1, ncol=51))

This point is nowhere near the minimum. Is it significant that all the
gradients are negative? Is that why SLSQP cannot determine a search
direction? When this occurs, could SLSQP use the gradient as the search
direction?

[Grey Gordon]

Is this an analytical gradient? Perhaps it is incorrect. For minimization, its clear from this gradient where the direction of steepest descent is. So if it can’t use this information to get closer to a local minimum, then I would think there is something wrong with the input or bizarre about the function.

Best,
Grey

Post by Joshua N Pritikin
During optimization, I have a problem that gets to a point where the
gradient looks like this,
[0] gradient = t( matrix(c( # 51x1
-31780.195562, -38508.674735, -50973.208738, -55408.084812,
-66931.026056, -74286.656477, -80710.037658, -32059.100573,
-40421.260358, -47351.363022, -56331.397546, -67570.244335,
-71730.720066, -80617.938563, -30100.959330, -40235.309256,
-47047.853982, -56263.225828, -67009.403836, -75987.083372,
-84897.553874, -62848.356157, -210680.805240, -214470.061730,
-234536.487353, -241645.958717, -262177.928304, -278085.836093,
-286557.250051, -63655.124325, -214492.388535, -219036.315591,
-224526.862707, -240634.971741, -248375.386046, -263267.566427,
-289505.767031, -64899.664139, -213624.942908, -222084.527486,
-223847.520561, -236782.323103, -258931.376366, -276562.312245,
-304241.522192, -25035.332609, -25125.410641, -25003.755675,
-4302.824452, -4986.836691, -3574.837605), byrow=TRUE, nrow=1, ncol=51))
This point is nowhere near the minimum. Is it significant that all the
gradients are negative? Is that why SLSQP cannot determine a search
direction? When this occurs, could SLSQP use the gradient as the search
direction?
--
Joshua N. Pritikin
Department of Psychology
University of Virginia
485 McCormick Rd, Gilmer Hall Room 102
Charlottesville, VA 22904
http://people.virginia.edu/~jnp3bc
_______________________________________________
NLopt-discuss mailing list
http://ab-initio.mit.edu/cgi-bin/mailman/listinfo/nlopt-discuss

[Joshua N Pritikin]

Post by Grey Gordon
Is this an analytical gradient?

No, finite differences

Post by Grey Gordon
Perhaps it is incorrect.

It is correct. If I run steepest descent for 3 iterations then the fit
rapidly improves.

Post by Grey Gordon
For minimization, its clear from this gradient where the direction of
steepest descent is.

For sure!

Post by Grey Gordon
So if it can’t use this information to get closer to a local minimum,
then I would think there is something wrong with the input or bizarre
about the function.

No, that isn't what I'm saying. This gradient works great even without
any Hessian approximation. The problem is that SLSQP reports "positive
directional derivative for linesearch" and gives up. It would be nice if
SLSQP could start optimization from here, even though the starting
values are not great.

Post by Grey Gordon

Post by Joshua N Pritikin
During optimization, I have a problem that gets to a point where the
gradient looks like this,
[0] gradient = t( matrix(c( # 51x1
-31780.195562, -38508.674735, -50973.208738, -55408.084812,
-66931.026056, -74286.656477, -80710.037658, -32059.100573,
-40421.260358, -47351.363022, -56331.397546, -67570.244335,
-71730.720066, -80617.938563, -30100.959330, -40235.309256,
-47047.853982, -56263.225828, -67009.403836, -75987.083372,
-84897.553874, -62848.356157, -210680.805240, -214470.061730,
-234536.487353, -241645.958717, -262177.928304, -278085.836093,
-286557.250051, -63655.124325, -214492.388535, -219036.315591,
-224526.862707, -240634.971741, -248375.386046, -263267.566427,
-289505.767031, -64899.664139, -213624.942908, -222084.527486,
-223847.520561, -236782.323103, -258931.376366, -276562.312245,
-304241.522192, -25035.332609, -25125.410641, -25003.755675,
-4302.824452, -4986.836691, -3574.837605), byrow=TRUE, nrow=1, ncol=51))
This point is nowhere near the minimum. Is it significant that all the
gradients are negative? Is that why SLSQP cannot determine a search
direction? When this occurs, could SLSQP use the gradient as the search
direction?