Nonlinear programming

ALGLIB, a free and commercial open-source numerical library, includes a rich suite of large-scale nonlinear programming algorithms. These algorithms have undergone extensive testing across a wide range of industrial optimization problems.

It is one of the fastest open-source solver suites (see the benchmark), featuring parallel algorithms for smooth, nonsmooth and stochastic problems. It is available in C++, C#, Java and several other programming languages, in both free and commercial editions.

Nonlinear programming suite overview

Design goals

ALGLIB NLP suite design goals include:

• Performance. The solvers incorporate many algorithmic improvements and low level optimizations to improve speed.
• Robustness. Reliable handling of degenerate problems.
• Convenience. A uniform API is provided for all combinations of solvers and languages, supporting both user-supplied gradients and numerical differentiation.
• Parallelism support. Numerical differentiation can be efficiently parallelized, greatly speeding up derivative-free optimization.
• Advanced debug features. ALGLIB includes an OptGuard integrity checker that helps to detect common problems like errors in the analytic Jacobian or the optimization problem being nonsmooth.

Algorithms

ALGLIB includes solvers for all kinds of constrained nonlinear optimization problems:

• Smooth problems, the most popular ones, can be solved by the state-of-the-art large-scale Sequential Quadratic Programming (SQP) solver provided by the minnlc subpackage. This solver includes many advanced features that accelerate convergence and improve robustness. Its sparse version scales to problems with thousands and hundreds of thousands of variables, utilizing every bit of sparsity in constraints. The dense version is optimized for medium-sized problems with dense constraints.
• Nonsmooth problems can be solved by the Adaptive Gradient Sampling (AGS) algorithm, which is provided by the minns subpackage. As nonsmooth optimization algorithms are much slower than smooth ones, this solver should not be used for smooth problems.
• Stochastic/noisy problems arise in industrial applications more often than one may expect: when a smooth target function is a result of some complex numerical simulation, it often behaves so erratically on a small scale that it effectively becomes stochastic. Such problems can be solved with metaheuristic algorithms from the mindf subpackage.

The solvers mentioned above support all types of constraint: box, linear and nonlinear ones. Smooth solvers support parallel numerical differentiation which greatly accelerates optimization without analytic derivatives. Metaheuristics are also inherently parallelizable.

Comparison with other open-source solvers

In order to evaluate the performance and robustness of our NLP solvers we compared them with several open-source nonlinear programming solvers using a subset of the CUTEst test problem set (see the smooth NLP and metaheuristic NLP benchmark pages for a more detailed discussion).

ALGLIB is consistently faster and more robust than other open-source solvers!

Programming languages supported

ALGLIB supports many programming languages, including C++, C#, Java, Python, and others:

• C++ version. Highly optimized, portable (x86/x64/ARM, works on Windows, Linux, and POSIX systems), and is a self-contained library
• C# version. Unified C# API for two backends - a fully managed C# implementation and a high-performance native computational core written in C
• Java, Python and other languages. A wrapper for a high-performance native computational core written in C. Seamless integration with the target programming language

A distinctive feature of ALGLIB is that it provides exactly the same API in all programming languages. This is achieved with our exclusive technology of automatic code translation and wrapper generation.

Performance

SIMD support

SIMD presence greatly increases speed of the linear algebra utilized by all ALGLIB optimizers. It provides a nearly uniform speed-up for problems of any size, from tens to tens of thousands of variables. SIMD support is available in both backends (C/C++ and NET), with roughly similar performance results.

Multithreading

ALGLIB can parallelize the following parts of the optimization process:

• Numerical differentiation by smooth optimizers.
• Batch evaluation requests issued by metaheuristic derivative-free optimizers.

If you are numerically differentiating an expensive target or one with many variables, or if you use a metaheuristic solver with its immense computational demands, then you can greatly benefit from the parallelism support.