Derivative-free optimization

The ALGLIB numerical library offers a powerful suite of derivative-free solvers, capable of efficiently handling unconstrained and constrained problems with box, linear and nonlinear constraints. The suite includes nonlinear programming solvers, nonlinear least squares solvers and global optimization solvers.

ALGLIB is available in C++, C#, Java and other languages under free and commercial licenses.

Derivative-free optimization overview

Algorithms and problem types

The term 'derivative-free optimization' is usually applied to two distinct kinds of optimization problems: those where continuous derivatives are unavailable due to non-existence, or those where continuous derivatives are unavailable but do exist. Depending on the situation, the following methods can be used:

• Metaheuristic algorithms are used for problems with non-existent derivatives and/or for global optimization. Parallelized differential evolution (DE) is usually a good choice for both unconstrained and constrained problems, generally performing better than genetic algorithms (GA) or particle swarm optimization (PSO). This functionality is provided by the mindf subpackage.
• Numerical differentiation allows one to solve smooth derivative-free problems with any nonlinear programming algorithm. It is also easy to parallelize. The ALGLIB suite of smooth nonlinear optimizers provides built-in numerical differentiation functionality in the lsfit (curve fitting), minnlc (nonlinear programming) and minlm (nonlinear least squares) subpackages.
• Model-based methods approach smooth derivative-free optimization by maintaining and updating a smooth (usually quadratic) model of the target. The nls subpackage implements a model-based derivative-free nonlinear least squares solver.

Most of the solvers above support box, linear and nonlinear constraints. They also support parallelism, although to different extents.

For example, differential evolution and numerical differentiation are inherently parallelizable, whilst model-based solvers tend to show more limited parallelism potential. On the other hand, model-based algorithms usually require fewer target function evaluations than those based on numerical differentiation, as they more efficiently reuse available information.

Programming languages supported

ALGLIB supports many programming languages, including C++, C#, Java, Python, and others:

• C++ version. Highly optimized, portable (x86/x64/ARM, works on Windows, Linux, and POSIX systems), and is a self-contained library
• C# version. Unified C# API for two backends - a fully managed C# implementation and a high-performance native computational core written in C
• Java, Python and other languages. A wrapper for a high-performance native computational core written in C. Seamless integration with the target programming language

A distinctive feature of ALGLIB is that it provides exactly the same API in all programming languages. This is achieved with our exclusive technology of automatic code translation and wrapper generation.

Benchmarks

Metaheuristics

In order to evaluate performance and robustness of our differential evolution solver we compared it with several metaheuristic solvers from other well-known open source software packages. We used a small-dimensional subset of the CUTEst set of test problems. See the benchmark page for more information about the benchmark setup.

The chart above clearly shows that ALGLIB is the fastest solver amongst ones compared.