26.09.2024  ALGLIB 4.03 is released

We are excited to announce the official release of ALGLIB 4.03!

This version marks a major breakthrough in quadratic optimization with ALGLIB, featuring GENIPM - a convex/nonconvex QP solver that supports quadratic and conic constraints.

Like most ALGLIB optimizers, it comes in two editions: DENSE-GENIPM and SPARSE-GENIPM. The former leverages dense linear algebra for optimal performance on small and dense problems, while the latter uses sparse algebra for scalability, especially on large-scale sparse problems.

Both editions of the solver (dense and sparse) support quadratic constraints, which can be either convex or nonconvex (with the nonconvex case returning a local solution). Additionally, this release includes early access support for second-order cone constraints. As of ALGLIB 4.03, this feature is stable but currently available only in DENSE-GENIPM. The upcoming ALGLIB 4.04 will extend full conic constraint support to both solver editions.

Other notable new features include:

1. A new sparse QP solver specifically for problems with linear equality constraints. Although it doesn't handle inequality constraints (nonlinear, linear or even box ones), this solver delivers superior performance on equality-only problems by solving them in a single Newton step.
2. Improved heuristics for the LP/QP presolver.
3. Support for ndarrays in ALGLIB for Python.

There are two backward-incompatible changes: we have discontinued support for the legacy QP-BLEIC solver and the SLP nonlinear programming solver. These solvers are now outclassed by modern interior-point and sequential quadratic programming methods, and maintaining them diverts effort from more impactful improvements.

As always, a full list of changes can be found in our issues tracker.

Finally, a look at our future plans: the next release, the third in 2024, will focus on implementing sparse conically constrained optimization, bringing parity between the DENSE and SPARSE editions of GENIPM. There is also a high probability that we will introduce additional cone types, such as the power cone and p-norm cone. Finally, it is planned to add at least one new nonlinear optimizer.

Thank you for using ALGLIB!
Sergei Bochkanov, ALGLIB LTD

30.05.2024  ALGLIB 4.02 is released

We are happy to announce that ALGLIB 4.02 has now been officially released. This version includes several significant updates:

1. Faster sparse SQP nonlinear programming solver. The solver is now fully-sparse, supporting sparse nonlinear constraints and scalable to hundreds of thousands of variables and constraints. It also has several internal improvements, like stabilized LM-SR1 update that behaves better than LBFGS on non-convex problems.
2. Differential evolution improvements: the ability to choose optimizer profiles (either performance-oriented or robustness-oriented), a DE/current-to-best/1 mutation strategy, controllable RNG seed, and new stopping criteria. The commercial edition also features SIMD kernels that accelerate mutation and crossover stages.
3. Nonmonotonic steps for LSFIT/MIN solvers. This addition significantly improves robustness when optimizing objective functions with small rounding-related/simulation-related noise or minor smoothness defects. It can also improve performance on highly nonlinear problems (up to a 1.5-2x improvement was observed), although the result heavily depends on the problem properties. For backward compatibility reasons, this feature is disabled by default.
4. Faster parallel sparse Cholesky factorization. The new parallel Cholesky code supports more fine-grained parallelism. It also has improvements that help to deal with mild cases of oversubscription (e.g. when you run as many threads as you have CPU cores, but someone else runs a heavy job in the background).

The new release fixes an alignment-related bug in the Linux version of the native computational core. This bug affected commercial and free ALGLIB 4.01 for CPython/Java/Delphi and commercial ALGLIB 4.01 for C# (the C++ version is not affected). We strongly advise anyone using these versions to upgrade to 4.02. More details are given in the issues tracker entry.

There are also many smaller additions. As always, a full list of changes can be found in our issues tracker.

Finally, we have upgraded our commercial licensing agreement to version 9, which adds clause 5.6 discussing compliance with sanctions and relevant national laws.

Looking to the future, we confirm our intention to make three releases per year. The next release (ALGLIB 4.03) is planned somewhere around August and will include an interior point solver for QCQP and SOCP. Either 4.03 or 4.04 (also planned for this year) will feature a general nonlinear interior point solver, which will be a good addition to our NLP suite.

Thank you for using ALGLIB!
Sergei Bochkanov, ALGLIB LTD

28.12.2023  ALGLIB 4.01 is released

We are pleased to announce that ALGLIB 4.01 has now been officially released. This version includes several significant updates:

1. Large-scale sparse SQP nonlinear programming solver. The new solver replaces the previous SQP implementation, offering scalability to thousands of variables and 1.5x-2x faster convergence. Additional features include two-sided nonlinear constraints and the ability to recover from target function breakdowns.
2. Constrained differential evolution solver. This state-of-the-art global derivative-free optimizer features an adaptive choice of training parameters using the award-winning EPSDE strategy by R.Mallipeddi and P.N.Suganthan.
3. Derivative-free box-constrained nonlinear least squares solver. The new NLS subpackage offers functionality similar to that of the MINLM subpackage, but in the derivative-free mode.
4. Parallel nonlinear optimization. Our nonlinear optimization suite (MINNLC, MINLM, LSFIT, NLS, MINDF optimizers) now supports parallel model evaluation and numerical differentiation. This feature is available only in the commercial edition and can greatly accelerate the solution of problems with expensive target functions. As of ALGLIB 4.01, this feature is available in C++ and C#, with plans to extend it to Java in the 4.02 release.
5. New linear solvers. This release features direct sparse linear least squares and a faster sparse nonsymmetric linear solver.

There are also many smaller additions and several bug fixes. As always, a full list of changes can be found in our issues tracker.

We wish you all the best in the upcoming year!
Sergei Bochkanov, ALGLIB LTD

01.12.2023  Upcoming pricing changes, special provisions for long-term users

Dear ALGLIB Users!

We would like to inform you of some upcoming changes to our pricing for new users, effective January 1, and about special provisions for long-term users.

In recent years, ALGLIB has significantly improved, with advancements in optimization, interpolation, and linear algebra suites, while maintaining our open-source nature. However, our pricing has remained constant since 2012, despite these enhancements and a cumulative inflation rate of over 34% since then.

We intend to stay in our current price segment but must implement a necessary price correction. Therefore, beginning Jan-01, we will increase prices by an average of 18% for new licenses, support plan renewals (for new users), and license upgrades.

Our aim is to increase prices for new users while minimizing the impact on our long-term users. Here are two measures we are implementing:

1. First, a 'grandfathering' policy: support plans active as of December 31 will see their next renewal price increased by merely 8% (the average annual inflation rate for 2022-2023).

This policy will be applied for, at least, several consecutive years: if you renew an active or recently expired support plan, the renewal price will follow the world's inflation rate, or remain the same (depending on our pricing decisions during that period). It is discussed in a more detail in our FAQ.

2. Second, an opportunity to secure the current renewal/upgrade price, even if your support plan expires long after December 31. It is possible by either procuring the renewal/upgrade before December 31, or asking for a quotation.

The former option is easy to implement - the price will be fixed even if payment occurs after December 31, and the new support plan will commence immediately after the expiration of the current one, so you will lose no money by renewing early. The latter option is especially suitable for users in large corporations with complex decision processes. It allows you to lock in the current price without any obligations from your side.

If you have any questions or need assistance with renewals, upgrades, or securing a quotation, please don't hesitate to reach out to us. Thank you for using ALGLIB!

Sergei Bochkanov, ALGLIB LTD

22.05.2023  ALGLIB 4 is released

We're pleased to announce that ALGLIB 4 has now been officially released. This version features significant updates:

1. ALGLIB for Java: We've expanded our library to include Java, offering both free and commercial editions. By utilizing our exclusive automatic code generation technology, we provide the same API as in other programming languages supported by ALGLIB.
2. Updates to ALGLIB Suite: The commercially licensed ALGLIB Suite now comes with C sources for the HPC core. This addition is relevant for non-C++ developers, like those working with ALGLIB for C# or Java. For those using single-language editions, upgrading to the Suite is simple - simply pay the difference in price.
3. New and Improved Functions: ALGLIB 4 introduces new features such as a multi-objective optimizer, more robust SQP code, 1D spline serialization and faster 1D spline fitting. We also want to thank Horst Greiner working at Hembach Photonik for contributing the code for second derivatives for a bicubic spline.

The decision to transition from version numbering 3.x to 4.x is attributed to two key reasons:

Firstly, the inclusion of ALGLIB for Java is a major addition, significant enough to warrant a major version increase.

Secondly, this release incorporates several changes that break backward compatibility. ALGLIB 3.x maintained numerous legacy API design decisions. However, recent advances in our automatic code and wrapper generation technology allow us to capture more information about program semantics, such as data constancy, modifications, in-place changes, reallocations, resizing, etc. This development allows us to create a more concise API, free from old inconsistencies.

Transitioning from the old to the new API is straightforward. If you are using a strongly typed language (such as C++ or C#), recompiling your program will generally be sufficient. If it compiles, it works. In some cases, you might need to remove unnecessary 'ref' modifiers or pay attention to const-correctness, but all fixes are straightforward.

For more information on the transition from 3.x to 4.0, including a list of affected functions, please refer to the separate article on this topic.

As always, a full list of changes can be found in our issues tracker.

19.12.2022  ALGLIB 3.20.0 is released

ALGLIB 3.20.0 comes with several new major features and with an important addition to our commercial product line - ALGLIB Suite.

ALGLIB Suite is a commercial license that includes all programming languages (4, as of the moment of writing) supported by ALGLIB. The product price is, however, only 1.5x higher than the corresponding license for just one of the languages. Just as with single-language ALGLIB versions, it is possible to buy DEV-1, DEV-3, COMPANY-WIDE and CORPORATE licenses for ALGLIB Suite. Users, who already own a single-language license, can upgrade it by paying the difference in price.

Speaking of the new features, ALGLIB 3.20.0 includes:

• biharmonic RBF construction is now accelerated with far field expansions (AKA fast multipoles). Model construction time reduced from O(N²) to roughly O(N·logN).
• sparse Cholesky decomposition now supports parallelism (commercial version). Parallel speedup achieved depends on the sparsity pattern: e.g. problems with long dependency chains (e.g. tridiagonal or band) are inherently sequential.
• several minor additions, including ability to use OpenSSL as an entropy source and 2D splines with missing cells.

A full list of changes can be found in our issues tracker.

13.06.2022  ALGLIB 3.19.0 is released

The most important feature in ALGLIB 3.19.0 is a new large-scale RBF solver that supports thin plate splines, biharmonic splines and multiquadrics. The solver supports both interpolation and fitting (smoothing) problems.

New RBF algorithm uses a domain decomposition method to solve linear systems. As a result, it has O(N) memory requirements and O(N²) running time, which is a major improvement over O(N²) memory and O(N³) time required by other open source implementations. It can be used on datasets with more than 100.000 points.

Another interesting improvement is a support for red zones in the C++ version of the library. A Red Zone is a fixed size area added before and after each dynamically allocated memory block. The Red Zone is filled by the special red zone control value during its allocation. When the dynamically allocated block is freed, its control value is checked. Any change means that someone (either ALGLIB or user code that works with ALGLIB-allocated arrays) performed an out-of-bounds write. Red Zones are essential for finding memory access errors that silently corrupt your data and/or crash your program.

A full list of changes can be found in our issues tracker.

19.05.2022  ALGLIB Project is now a UK company!

Dear ALGLIB users!

I am Sergey Bochkanov, the founder of the ALGLIB project, and I want to tell you that starting from 19th May, 2022, we are moving operations to ALGLIB LTD, a UK Limited Company.

The new company will be our core business unit. It independently licenses its intellectual property (i.e. it is not a reseller), uses banking accounts in the UK and USA for its transactions, operates under English law and pays taxes to HMRC.

Our former business unit will operate for some time in order to serve existing long-term support contracts. Because sole proprietorship is hard to separate from the person behind it, it exists no matter where I live.

However, it is advised to move your contracts to our new company - it can be performed online in just a minute from your account. This option is completely free and available to any customer, even one with an expired support contract.

UPDATE: The relocation from Russia was completed. We no longer have a physical presence in Russia.

Sergey Bochkanov, ALGLIB Project.

05.03.2022  Sergey Bochkanov on the project status in light of RUS/UKR situation

Dear ALGLIB users!

I am Sergey Bochkanov, the founder of ALGLIB project, and I want to address you about the status of the project (which is based in Russia) in light of the recent events.

I am shocked and saddened by what is happening now.

I have always believed in cooperation and openness, that is why I have decided to work on the open source product in the first place. ALGLIB is trusted by people from all countries and industries due to its open nature. I hope that you will continue to trust us.

Below I address the following points: (a) the future of the project, and (b) our contingency plan.

First, ALGLIB development continues. There is an upcoming release with new large-scale thin plate splines and improved detection of memory access errors. The next release will include parallel sparse factorizations, LP and QP solvers, and there is a good chance that both releases will be made in 2022.

We have resources to provide the same level of commercial support as before. Our plan for the next several months is to continue our operations from Russia, making everything possible to be able to work with our users worldwide.

We are a small IT company that uses the local branch of the Western bank for its transactions and has no ties to the military sector. As for now, we have not been hit by the sanctions. We also have a USA-based reseller (FastSpring) that processes plastic card payments, and several other contingency plans.

Should anything change in the situation, we will notify you as soon as possible.

27.10.2021  ALGLIB 3.18.0 is released, with NET 5 SIMD support!

The most important improvement in ALGLIB 3.18.0 is that ALGLIB became the first open source numerical library to support .NET 5 SIMD intrinsics!

Hardware intrinsics are well known to C/C++ programmers, but C# lacked them for tens of year. As a result, it was impossible to utilize modern SIMD instructions in C# and performance of the numerical C# code was way below that of the well-optimized C/C++ one. Many numerical libraries (including ALGLIB) solved the problem by providing C# interface to high-performance native code. However, having native code in your application has its own drawbacks.

Thankfully, .NET Core 3.0 finally introduced SIMD support into C# and it matured in .NET 5, and we are glad to announce that ALGLIB for C# now includes optional SIMD kernels for many performance-critical functions. Here 'optional' means that one may compile ALGLIB under .NET 5 with SIMD intrinsics enabled and enjoy unprecedented performance - or compile it without SIMD and enjoy unprecedented portability to any .NET version released since .NET Framework 2. This feature is present in both commercial and open source editions of ALGLIB.

In addition to that major improvement, ALGLIB 3.18.0 also includes following features:

• sparse GMRES solver for linear equations
• sparse Cholesky decomposition now comes with AVX2/FMA-capable C++/C# kernels
• faster AMD ordering (optimized handling of matrices with dense rows)
• interior point LP and QP solvers became much faster due to new Cholesky and extensive optimization of the solver code itself
• finally, the open source edition of ALGLIB for C++ also become faster due to introduction of AVX2/FMA-capable linear algebra kernels

We also fixed several minor bugs (see issues tracker for a complete list). Thanks to all for staying with ALGLIB!

29.12.2020  ALGLIB 3.17.0 is released

We are glad to announce that (a) 2020 hopefully will be over soon, and (b) we just published a new ALGLIB release!

As always, ALGLIB 3.17.0 comes with new major features, lots of minor additions and a few bugfixes. Major new functionality includes:

• sparse supernodal Cholesky decomposition (with expert functions and convenience wrappers) + corresponding sparse direct linear solver. Fill-in reducing permutation and indefinite factorizations included.
• large-scale sparse interior point (barrier) LP solver
• large-scale sparse interior point (barrier) semidefinite QP solver

Minor improvements include:

• random vectors/matrices with normal entries
• stronger OptGuard integrity checking for optimizers helps to detect more errors in user-provided functions/gradients
• stabilized Hessian updates for SQP solver help to handle problems with wildly changing curvature
• many improvements in sparse matrix functionality (faster transposition, quick tests for element existence, symmetric permutation of sparse symmetric matrix)

We also introduced a few compatibility-breaking changes to names/signatures of sparse direct solvers (see entry in our issues tracker for complete list). These changes were introduced in order to make API compatible with the rest of the library.

Finally, a few stability fixes:

• better handling of huge bounds (like x<1.0E10) by optimizers
• fixed tolerance thresholds in the infeasibility detection code (original values were too strict)
• better initial point selection in dense/sparse IPM method

Full list of changes can be found in the Change Log.

19.12.2019  ALGLIB 3.16.0 is released

New ALGLIB release introduces several major additions, a pack of minor improvements and one bugfix. The major new functionality is outlined below:

• interior point QP solver was implemented, with dense and sparse versions.
• new subroutine for fast fitting by penalized cubic spline with O(N*logN) running time
• new SQP nonlinear programming solver
• parallel variable importance estimation for random forests
• compressed binary storage format for large random forests (with 3.7x-5.7x memory usage reduction)
• finally, new trace infrastructure for ALGLIB solvers and algorithms which allows to get algorithm log in human-readable form

Minor improvements include:

• sparse nonsymmetric real solver
• sparsegemv() function for CRS and Skyline matrices
• bivariate normal CDF and PDF
• Lagrange multipliers being reported by QP solvers
• two-sided linear constraints being supported by QP solvers
• improved stability of SLP solver
• more precise choice of pivot element in LP solver

Finally, one bugfix and one backward-incompatible change:

• we fixed breaking change in C++ exception handling in user callbacks (previous ALGLIB releases replaced user-generated exceptions by ALGLIB own exceptions, now user exceptions are forwarded as is)
• we changed order of parameters of minqpsetlcmixed(), now it accepts sparse constraints before dense ones

Full list of changes can be found in the Change Log.

21.02.2019  ALGLIB 3.15.0 is released

New release of ALGLIB introduces several major additions, a pack of minor improvements and three bugfixes. The major new functionality is outlined below:

• revised dual simplex solver was implemented
• multilayer inverse distance weighting interpolation algorithm similar to HRBFs in quality, but better scalable to multi-megapoint datasets
• ALGLIB optimizers can now detect common errors in problem statements like nonsmoothness or discontinuity of the target/constraints. Such errors are reported via uniform OptGuard API supported by all nonlinear optimizers.
• k-nn classification/regression algorithm was implemented
• sparse LU factorization with complete (row/col) and partial (row) pivoting is added

We also implemented following minor additions:

• termination requests for MinNLC optimizer and constraint violation reports
• progress reports and termination requests for LSQR linear solver
• progress reports and termination requests for HRBF model builder
• unordered distance-based k-nn queries; overall performance of kd-trees was improved in this release
• dfprocess0() and dfclassify() convenience wrappers for random forest inference
• copy-with-transpose for CRS matrices, faster sparsetrsv()

Finally, three minor bugs were fixed, two of them in gradient checking functionality and one in logisticfit4() function. Full list of changes can be found in the Change Log.

16.06.2018  ALGLIB 3.14.0 is released

New release of ALGLIB introduces many new algorithms and one major backward incompatible change in multithreaded API. Following algorithms were added:

• improved random forests construction algorithm, which is from 2x to 10x faster than previous version and produces orders of magnitude smaller forests.
• sparse truncated PCA for large and sparse datasets
• bicubic least squares fitting algorithm which fits 2D splines to large-scale scattered point clouds (two algorithms are introduced: BlockLLS which can handle up to 1.000.000 points, and FastDDM, which is a bit harder to tune, but can deal with up to 100M points)
• a few minor improvements to 2D splines: ~2x faster evaluation/differentiation, serialization support
• new nonlinear programming solver is introduced: MinSLP.
• least squares circle/sphere fitting algorithm is implemented
• and a few minor changes: transpose support for sparse CRS matrices, faster sparsemv and sparsemtv functions

Finally, we want to tell you about changes in the API. Pre-3.14 versions of ALGLIB used smp_ prefix in the function name in order to indicate that you need parallel processing capabilities. Since version 3.14 we switched to more convenient interface which allows you to (a) specify global threading behavior by means of setglobalthreading() function/method, or (b) specify threading settings at function call level by appending alglib::parallel or alglib::serial to the list of parameters.

Former (alglib::parallel) means that computations are parallelized when feasible (ALGLIB automatically decides whether it makes sense to start worker threads or not), latter means that computations are always performed in serial manner. The default behavior is to perform all computations serially, unless explicitly told by user to parallelize them. When told to use parallel processing, ALGLIB tries to use all cores available (but you can change it with setnworkers call).

Old smp_ prefix is not supported in newer ALGLIB releases; you have to change your code by removing this prefix from function names and by adding alglib::parallel to such calls.

We also fixed several bugs, full list of changes can be found in the Change Log.

29.12.2017  ALGLIB 3.13.0 is released

On the last business day of 2017 we are happy to announce next release of ALGLIB: version 3.13.0! This release introduces many interesting updates: from new functions to precompiled multiplatform Linux binaries. Changes can be divided into the following categories: "infrastructural" changes, new "major" features, new "minor" features, and bug fixes.

Major infrastructural changes in this release are:

• optional exception-free error handling in ALGLIB for C++. Embedded system developers can compile ALGLIB in completely exception-free mode, with global flag variables used to report critical errors normally reported with exceptions.
• Multiplatform Linux support in native edition of ALGLIB for C#.
• We also added Linux support to ALGLIB for CPython and ALGLIB for Delphi.

Following major features were added in this release:

• Singular Spectrum Analysis (SSA) algorithm for time-series data. Our implementation is well optimized and includes trend extraction, prediction, prediction with averaging, fast incremental updates to the model.
• Direct solver for sparse linear systems stored in Skyline (SKS) storage format
• Performance improvements for QP-DENSE-AUL quadratic solver. Algorithm became orders of magnitude faster on problems with huge sets of inequality constraints.
• Up to 4x faster BLEIC and QP-BLEIC solvers. We significantly remastered internals of these solvers, with significant improvement of constraint handling code.
• Warm start support for subspace eigensolver. If you solve a sequence of related eigenproblems, you can reuse previously found basis as initial point for the new solver session.

Following minor features were added in this release:

• Easy creation of band matrices stored in sparse SKS format (sparsecreatesksband() function)
• New set of real Level 2 BLAS functions: GEMV, SYMV, TRSV.
• sparseset() function now supports SKS matrices
• minqp solver now supports automatic inferring of variable scales from the diagonal of the quadratic term
• ALGLIB for C++ now allows to "attach" double precision vector/matrix to externally stored data without copying with attach_to_ptr() function

We also fixed two bugs:

• fixed memory leak in the Intel MKL (to be exact, not a leak - just excessive allocation of temporaries which are not automatically freed; read an Intel article on this issue). This fix is relevant for C++ and C# versions of commercial ALGLIB.
• fixed potential memory leak in the native computational core

Full list of changes can be found in the Change Log.

24.08.2017  ALGLIB 3.12.0 is released, now with Delphi support!

The most important change in ALGLIB 3.12.0 is introduction of ALGLIB for Delphi/FreePascal. New ALGLIB product is a Delphi/FPC wrapper around precompiled computational core written in C. As other our products, it is available in two editions: free (full functionality, limited performance, non-commercial license) and commercial (HPC computational core with multithreading/SIMD support) versions.

This release is mostly centered around introduction of Delphi version. However, we also made several minor improvements in generic numeric functionality of our package:

• rbfsetpoints() function now checks for NAN/INF in its inputs
• k-means clustering algorithm and inverse iteration eigensolver now use deterministic seeds for their initializer, which makes results reproducible across different runs
• we also fixed minor error in QQP solver - incorrect automatic scaling of quadratic term

Full list of changes can be found in the Change Log.

11.05.2017  ALGLIB 3.11.0 is released

New release of ALGLIB includes following improvements:

• linearly constrained nonlinear least squares (MinLM and LSFit optimizers) - you can now perform nonlinear fits subject to linear constraints on parameters
• fitting minimum circumscribed, minimum zone and maximum inscribed N-spheres to data (in 2D - fitting circles, in 3D - fitting spheres)
• we improved robustness of MinNLC optimizer and added one more preconditioning mode - "exact robust"
• new optimizer is added to ALGLIB suite - box constrained MinBC optimizer, which is optimized for box-only constraints (such constraints allow us to use activation strategies which are impossible for general linear constraints)
• streaming serialization/unserialization is added to C# and C++ versions of ALGLIB
• subspace iteration direct/sparse/out-of-core eigensolver
• new fast truncated PCA (relying on subspace iteration)
• much faster hierarchical RBFs with parallelism support - many orders of magnitudes faster on some datasets, can handle more than 100.000 points
• linearly constrained QP solver
• better kd-trees - with box queries and thread-safe versions of query functions

One more major change: we merged VB.NET and IronPython wrappers with primary C# distribution they rely on - now all three languages are packed together.

We also made several stability fixess, details can be found in the Change Log.

P.S. Thanks to all our users for your attention, stay with us! :-)

19.08.2015  ALGLIB 3.10.0 is released

New release of ALGLIB includes following improvements:

• CSV import - you can now read 2D matrix from CSV file
• AGS (Adaptive Gradient Sampling) algorithm for nonsmooth nonconvex constrained optimization, which makes ALGLIB one of the few commercial packages supporting nonsmooth optimization.
• Ward's method for hierarchical clustering
• special lightweight linear solvers without condition number estimation and iterative refinement - orders of magnitude faster than their "feature-rich" counterparts
• parallel k-means clustering (parallelism is supported only in commercial edition of ALGLIB)

We also made several stability fixess, details can be found in the Change Log.

12.12.2014  ALGLIB 3.9.0 is released

New release of ALGLIB 3.9.0 brings a lot of significant improvements! Both commercial and non-commercial users have access to following functionality:

• significant improvements in sparse/dense linear algebra support: SKS sparse matrix storage format, linear algebra operations for SKS-based matrices, SKS Cholesky factorizer, a lot of additional sparse matrix functions.
• improvements in solvers and optimizers: new box-constrained QP solver - QuickQP, nonlinearly constrained (!) Augmented Lagrangian optimizer, improved BLEIC optimizer, polynomial solver and many other minor improvements.
• additional interpolation/fitting functions: 4/5-parameter logistic fitting, RDP (Ramer-Douglas-Peucker) algorithm
• faster LDA (linear discriminant analysis) algorithm

Commercial users (C++ and C#) may benefit from 100% MKL-accelerated dense linear algebra. Now all important dense linear algebra functions are accelerated by built-in (or provided by user) installation of Intel MKL.

IMPORTANT: we should note that we made changes in the default solver used by MinQP subpackage. Previously it used Cholesky-based solver, now it uses faster BLEIC-based QP solver with default stopping conditions. Both solvers converge to good solutions, but because they are different solvers, slightly different solutions can be returned. If you use MinQP subpackage in its default configuration, we recommend you to set QP solver explicitly with carefully tuned stopping conditions.

We also made several stability and bug fixes, details can be found in the Change Log.

13.08.2014  ALGLIB Project publishes technical report on StrongNet

Our technical report proposes two innovations in the design of neural networks: (a) strong neurons – highly nonlinear/nonsmooth neurons with multiple outputs and (b) mostly unsupervised architecture – backpropagation-free design with all layers except for the last one being trained in a completely unsupervised setting.

The new neural design (called StrongNet) was tested on a well-known MNIST benchmark. This demonstrated that StrongNet is able to capture structural information about images in the "mostly unsupervised" setting. Three-layer StrongNet has a test set error on MNIST as low as 1.1%. It outperforms 11 out of 14 two/three-layer networks cited on the MNIST homepage (as of Aug 2014).

read full report (PDF)

12.03.2014  New article - "ALGLIB+MKL: first results"

In November (2013) we presented new product - ALGLIB 3.8.1 (Commercial Edition), the first ALGLIB version which was shipped with Intel MKL - high-performance numerical library by Intel. It was first release which featured C# interface to native computational core, thus both C++ and C# users were able to use MKL as part of ALGLIB.

MKL-capable ALGLIB is a significant step in ALGLIB development, result of technical partnership with Intel Numerics Russia. Now, after 4 months with new ALGLIB, we decided to publish an article which discusses benefits MKL brings to ALGLIB.

read "ALGLIB+MKL: first results"

25.11.2013  ALGLIB 3.8.2 is released

This release was made in less than 3 weeks since previous one, and includes only one improvement in ALGLIB for C++ – better, smoother integration with Intel MKL. Only C++ version of Commercial Edition was changed, and only this change was made, so if you use Free Edition (or anything other than C++), you may ignore this release.

Experimental support for Intel MKL was introduced in ALGLIB 3.8.1, which was released in the beginning of November. Both C++ and C# versions of ALGLIB started to use MKL. ALGLIB for C# was favorably received by customers, but C++ users reported about two inconveniences. Both of them due to the fact that MKL-capable version of ALGLIB for C++ was delivered as precompiled static library (binary addition to the main source code distribution).

First, several users reported about linking problems with combined ALGLIB+MKL static library – distribution of precompiled C++ libraries has a lot of tricky issues. Second, some of our users already own license for MKL, and they wanted to use MKL and ALGLIB together in one application – but they wanted ALGLIB to use their own installation of MKL.

As result, in new ALGLIB for C++ we changed distribution model. ALGLIB itself is distributed as source-only, and again you should compile it yourself – with your own compiler, own options, and perfect compatibility with your build environment. If you want to use Intel MKL, you have two options here...

First, you may link with special lightweight Intel MKL distribution, which is shipped with ALGLIB for C++ as DLL file. In this case you do not have to buy separate license for MKL, but this MKL distribution can be used only as part of the ALGLIB. Second option is to link ALGLIB with your own MKL installation (you should have license for Intel MKL). In this case you may use Intel MKL both directly – and as part of ALGLIB.

Both options involve no precompiled static libraries. In the first case the only library you should use is import library, which is portable across different compilers. In the second case you should just add one more C file (ALGLIB-MKL interface) to your application, so again there should be no compatibility issues.

07.11.2013  ALGLIB 3.8.1 is released

ALGLIB 3.8.1 is released! This new release of ALGLIB features two major improvements in Commercial Edition: 1) C# interface to highly optimized native computational core, 2) experimental integration with Intel MKL.

1. Prior to 3.8.1, ALGLIB for C# was 100% NET application. Being pure NET assembly, it was portable and easy to integrate into existing NET projects. However, on some important computational problems (linear algebra, large scale data mining) C# performance was 2x-4x times slower than that of optimized native code.

Starting from 3.8.1, ALGLIB for C# includes both pure NET computational core – and its native companion, highly optimized native core with C# interface. Both computational cores provide same functionality, and even more – they provide 100% identical C# interface. You can easily plug HPC core into existing applications which use previous versions of ALGLIB. As for ALGLIB 3.8.1, HPC core can be used only under Windows systems. However, we plan to introduce support for LSB-compliant Linux systems in the next release. One more limitation – HPC core is not included in ALGLIB for VB.NET, although in next ALGLIB release VB users will have same level of performance as C# programmers.

2. Second important improvement is integration with Intel MKL, which can be accessed from C++ and C#. Intel MKL is used internally to accelerate linear algebra algorithms. We obtained developer's license from Intel which allows us to ship precompiled ALGLIB binaries linked with Intel MKL.

C++ version of ALGLIB now includes precompiled static library alglibxx_hpc.lib (ALGLIB core), mkl4alglibxx.lib (MKL imports) and mkl4alglibxx.dll (MKL core). When you link your application with these precompiled static/dynamic libraries, ALGLIB will use MKL internally to provide same interface – but with 2x-4x (depending on hardware being used) increase in linear algebra performance. As for C#, this version of ALGLIB includes native computational core (as we told before). It is already linked with Intel MKL, so you don't need additional steps in order to use MKL from C#.

We want to make several notes here. First, due to licensing restrictions, you can not use Intel MKL directly. You can use ALGLIB functionality which is internally accelerated by MKL's functions, but you can not access these functions directly. We do not provide you MKL header files because we do not have right to do so. Second, because MKL support is in the experimental stage, for now we ship only Windows libraries. Linux binaries are planned to be included in the next release of ALGLIB.

Finally, during implementation of this subproject we received several important suggestions and ideas from developers of local Intel's office, and we would like to thank them (especially Gennady Fedorov and Sergey Maidanov) for their help.

3. These improvements can be used only by commercial users who own license for new multi-threaded branch of ALGLIB. If you bought your license before release of 3.8.0 and didn't upgraded your license yet, it is good time to consider an upgrade. Upgrade is done for free, the only thing we need from you is a formal confirmation that you agree with new licensing terms.

In addition to improvements mentioned above, we made several minor fixes in both Commercial and Free editions of ALGLIB and fixed one bug in the MinBLEIC optimizer (of course, bug fixes were included in all editions – commercial and free).

07.08.2013  ALGLIB 3.8.0 is released

ALGLIB 3.8.0 was released today! This release features a lot of performance-related improvements, new features and bug fixes. Both editions of ALGLIB (Free and Commercial one) share following improvements:

• ranking functionality (descriptive statistics) - function which replaces data by their ranks
• new QP solver (QP-BLEIC) which is capable of solving sparse and non-convex QP problems subject to bound/linear constraints
• improved FFT (higher performance, but still single-threaded)
• better basecase C# code for matrix-matrix multiplication, which achieves performance close to that of generic C code
• a lot of minor improvements (uphill steps in BLEIC optimizer, better weight initialization for MLP's, Akima spline for less than 5 points)

ALGLIB Commercial Edition features following performance-related improvements (our former customers should upgrade their license in order to access these improvements):

• parallel ranking (descriptive statistics)
• parallel correlation/covariation functions
• parallel evaluation of distance matrix for hierarchical clustering
• parallel training of neural ensembles
• parallel cross-validation for neural networks
• parallel training of neural network
• parallel linear algebra functions (real/complex TRSM, GEMM, SYRK)

Both editions of ALGLIB share following bug fixes:

• fixed - minor bugs in implementaton of internal C++ structures (shared pool, smart pointer, proxy array/matrix)
• fixed - possible division by zero in estimator for errors-in-parameters (issue 524)
• fixed - minor error in calculation of covariance matrix for least squares fit (issue 506)
• fixed - minor bug in Student's t-test (issue 518)
• fixed - minor bug in the sign test (two sided alternative, issue 504)
• fixed - stack overflow in kd-tree construction routine (issue 521)

02.08.2013  ALGLIB Project announces changes in product line!

ALGLIB Project is going to release ALGLIB 3.8.0 in several days and we are glad to announce changes in our product line!

Prior to ALGLIB 3.8.0 we had single product - ALGLIB Standard Edition - which was distributed under two licenses, open source and commercial. Starting from 3.8.0, we have product line of two related products - ALGLIB Free Edition and ALGLIB Commercial Edition. Both editions provide same functionality, i.e. they both can solve same set of mathematical problems. However, they differ in their performance.

Free Edition is a single-threaded library with no extensive low-level optimization being applied. It is descendant of pre-3.8.0 ALGLIB Standard Edition, which had no multithreading support. Commercial Edition provides multithreaded acceleration for many important functions, and includes better optimized computational core. Functions whose performance is different in FE and CE are explicitly labelled as such in ALGLIB Reference Manual.

ALGLIB Free Edition is still licensed under GPL. ALGLIB Commercial Edition, obviously, is distributed under commercial licensing terms. We had to change commercial licensing terms for this new edition, new licensing terms can be downloaded as PDF from our website. Most important changes are:
    1) different terms for distribution of ALGLIB CE source code,
    2) special provisions for future inclusion of third-party binary components (Intel MKL - planned for 3.8.2).

IMPORTANT! Special note for commercial users who bought pre-3.8.0 ALGLIB and have active support-and-maintenance agreement. ALGLIB CE is a new product, related to, but different from ALGLIB SE, which you paid for. We offer you free upgrade from Standard Edition to new multithreaded Commercial Edition, but in order to do that you have to accept new licensing terms. New performance-related code is not compatible with former ALGLIB license, so accepting new license is the only way to get new ALGLIB CE.

Upgrading your license is easy - just write to updates@alglib.net, tell us your agreement number and confirm that you are ready to upgrade to new license. We can not perform such upgrade without your consent, because it will be done by entering into new agreement.

If you decide not to upgrade, you still will be able to receive new versions of ALGLIB (within your support period), but it will be single-threaded ALGLIB without new performance-related improvements. However, it will have full set of functions - the only difference from new ALGLIB CE will be its single-threaded performance and lack of planned SSE/AVX/IntelMKL extensions.

18.01.2013  ALGLIB 3.7.0 is released

ALGLIB 3.7.0 was released today, with new algorithms and fixes:

• BLEIC optimizer was significantly refactored. First, it uses new three-stage active set algorithm suggested by Elvira Illarionova, which combines gradient projection with iterations of equality constrained L-BFGS. Second, because we do not have nested inner/outer iterations, more transparent stopping criteria can be specified for algorithm. Last, it uses new constraint activation/deactivation strategy which can correctly handle degenerate constraints.
• ALGLIB support for neural networks was significantly improved. We have new training interface which greatly simplifies training of multiple networks with same settings/datasets. It is now possible to specify training set by sparse matrix.
• Better support for clustering - new version of ALGLIB includes hierarchical cluster analysis algorithm provided by clustering subpackage. It includes several distance metrics (Euclidean, 1-norm, infinity-norm, Pearson and Spearman correlation-based metrics, cosine distance) and several linkage types (single linkage, complete linkage, average linkage). K-means clustering functionality (which existed long before new algorithm) was merged with new clustering unit.
• Sparse linear solvers (CG and LSQR) now support automatic diagonal precondtitioner.
• Linear/nonlinear least squares solvers (lsfit subpackage) now report errors in the coefficients.
• Sparse matrix functionality now includes new functions for conversion between hash-table and CRS representations, we've also improved performance of SparseRewriteExisting function.
• Backward-incompatible changes in MinBLEIC subpackage: three separate functions which specify different kinds of stopping criteria - MinBLEICSetInnerCond(), MinBLEICSetOuterCond() and MinBLEICSetMaxIts() - were removed and replaced by one function - MinBLEICSetCond().
• we've made several minor fixes (see Change Log).

Commercial version of ALGLIB 3.7.0 will be delivered to customers with active support/maintenance agreement within few days. Good luck with ALGLIB!

04.07.2012  ALGLIB 3.6.0 is released

ALGLIB 3.6.0 was released today, with new algorithms and fixes:

• Quadratic optimizer now supports arbitrary combination of boundary and linear equality/inequality constraints. New version of the optimizer uses combination of the augmented Lagrangian and active set methods.
• Spline1D unit now supports monotone cubic spline interpolation
• Support for vector-valued bilinear and bicubic splines
• Support for scalar and vector-valued trilinear (3D) splines
• Better support for sparse matrices: efficient enumeration of non-zero elements with SparseEnumerate(), faster SparseGet() for matrices stored in CRS format.
• Optmization and nonlinear fitting algorithms (LSFit, MinLM, MinBLEIC, MinLBFGS, MinCG subpackages) can verify correctness of the user-supplied gradient (the most common source of errors in numerical programs).
• we've made several minor fixes (see Change Log).

Commercial version of ALGLIB 3.6.0 will be delivered to customers with active support/maintenance agreement within few days. Good luck with ALGLIB!

22.05.2012  New articles for ALGLIB User Guide

We've updated ALGLIB User Guide - online book about ALGLIB and numerical analysis algorithms in general. Update includes three articles on interpolation and data analysis algorithms:

• article about new RBF algorithms, which describes robust and efficient RBF algorithms with O(N·logN) performance which were implemented in ALGLIB 3.5.0. Article contains detailed description of algorithms, discussion of their strong and weak sides, performance/robustness comparison with SciPy.
• introductory course on RBF interpolation, which discusses basic ideas of the RBF interpolation, gives several charts, examples and advices.
• article on basic concepts of neural networks, which discusses generalization, overfitting, early stopping and weight decay, with examples and charts.

We continue our work on bringing you more information - and developing new algorithms. Good luck with ALGLIB!

27.03.2012  ALGLIB 3.5.0 is released

New version of ALGLIB is released - ALGLIB 3.5.0, with many improvements and new algorithms:

• Major breakthrough in a scattered 2D/3D interpolation - two RBF interpolation/fitting algorithms with N*log(N) working time. First algorithm uses compact basis functions with ACBF preconditioning, requires no tuning, and can efficiently solve problems with approximately uniformly distributed points. Second algorithm uses non-compact multiscale/multilayer RBF functions, requires only minor amount of tuning (initial radius and layers count), and can solve problems with arbitrary distribution of points, with controllable amount of smoothing being applied to the data. Both algorithms were successfully used on problems with up to 100.000 points and even more (even 1.000.000 points is not a problem, although you will need large amounts of RAM).
• ALGLIB now supports portable serialization of neural network ensembles. "Portable" means that serialized structures can be moved between different machines and even versions of ALGLIB for different languages. Development of this addition was funded by IDx LLC - a company which used ALGLIB classification features for analysis of medical data.
• New feature - support for sparse matrices. Sparse matrix structure can store matrices using one of the two formats: Hash-Table representation (allows easy initialization) and CRS ("frozen" representation, which can't be modified, but can be used in a linear algebra algorithms).
• Two sparse linear solvers - linear CG and LSQR (can solve M*N problems in the least squares sense).
• New statistical functions - separate functions for mean/variance/stddev.
• New RNG functions - generation of random numbers from distribution given by finite sample.
• New data analysis methods - SMA, EMA, LRMA filters.
• Several fixes and tweaks: better detection of the feasible point in the BLEIC algorithm, better version of MinQP solver, several bug fixes (see Change Log).

Commercial version of the new release will be delivered to customers with active support/maintenance agreement within few days. Good luck with ALGLIB!

03.01.2012  Commercial licensing policy changed - more rights for our customers

ALGLIB Project announces introduction of new version of the ALGLIB Commercial License Agreement. New agreement gives same freedoms as previous one, but additionally it allows our customers to:

• distribute their applications in both Binary and Source Code forms (if they want to)
• distribute ALGLIB source code with their applications (if they want to)
• permanently transfer ALGLIB License Agreement to another legal entity without written consent of the ALGLIB Project

These options can be used for no additional fee by any customer, from large company to one-developer startup. We believe that such license covers most use cases. However, if you feel that for some reason our commercial license does not suit your needs, feel free to contact us and describe your situation. We are always ready to discuss new licensing options.

04.10.2011  We've moved into a new office!

ALGLIB Project - a company which supports ALGLIB - has moved to the new location. Our new office is located at the 12th floor of the "Capital-Nizhny" office building. Thus, our mailing address has changed to 603006, Russian Federation, Nizhny Novgorod, Maxim Gorky street, 117, room 1210. Our legal address remains unchanged.

17.06.2011  ALGLIB 3.4.0 is released

New version of ALGLIB is released - ALGLIB 3.4.0. This release includes many improvements:

• ALGLIB now supports portable serialization of decision forests, KD-trees and neural networks. "Portable" means that serialized structures can be moved between different machines and even versions of ALGLIB for different languages. Development of this addition was funded by IDx LLC - a company which used ALGLIB classification features for analysis of medical data.
• Optimization algorithms now can find extrema of functions with singularities (as long as correct values are returned near the singular point, and +INF is returned exactly at the singularity). Special post-processing is applied to the function value in order to prevent overflow.
• All optimizers (BLEIC, CG, L-BFGS, LM) now support numerical differentiation. Only fixed step size is supported, but it can be different for different variables. Numerical differentiation allows fast prototyping when solving small and moderately dimensioned problems.
• ALGLIB for C++ now supports SSE2 under GCC and Sun Studio (previous version supported MSVC only).
• We've added a lot of new examples and significantly improved our User Guide on numerical optimization.
• New data analysis algorithm was added - Markov Chains for Population/proportional Data (MCPD) solver. This solver can be used for portfolio analysis, when you want to restore hidden transition matrix using aggregated portfolio data.

We've also fixed several bugs:

• Minor bug in the integration code (bug #399). It was triggered when user tried to integrate function with high numerical noise (algorithm was unable to stop because of noise).
• Major bug in the Python version of ALGLIB. Python wrapper was unable to find ALGLIB shared library under some systems. This bug was found and resolved by Tom Dimiduk at Department of Physics of Harvard University.

28.04.2011  Commercial licensing/pricing policy changed

ALGLIB Project announces following changes in the commercial licensing policy. First, we've introduced new version of the commercial license agreement. It gives same freedoms as previous one, but additionally it:

• gives warranties related to the intellectual property issues and protection from third party claims
• includes support and maintenance plan
• extends no-questions-asked money back guarantee from 30 days to 90 days

We've also introduced set of new licensing options targeted at large businesses and government entities:

• source code redistribution agreement
• ability to move jurisdiction to Customer's state
• group licensing (one license for several connected companies)

Finally, we've changed our pricing model. New pricing model features differentiated price, which depends on the number of developers working with ALGLIB. Thus, it has some traits of per-developer licensing - but it is not per-developer licensing because there is no need to pay additional fees every time you add one mode developer.

There is unlimited company-wide license and there is a set of discounts based on the number of developers actually using ALGLIB. For example, there are discounts for companies with 1, 3, 5 or 8 developers. Additional discounts are provided to companies from developing countries or companies which need only minor fraction of ALGLIB functionality. These discounts are applied at the moment of purchase and are not subject to recalculation/revocation in the future (except for the situation when you understand that you need more developers right after the purchase). We've discussed details of our new licensing/pricing model in our FAQ. Speaking short, we give discount for small companies, but we do not charge them when they begin to grow.

We hope that our new licensing model will make ALGLIB accessible both to small and large companies. In case you feel that your company needs special handling (i.e. some special kind of discount), feel free to contact us.

06.03.2011  ALGLIB 3.3.0 is released

New release of ALGLIB - now in five programming environments (C++, C#, CPython, IronPython, VB.NET). This release contains following improvements:

• Diagonal preconditioner for optimization algorithms (L-BFGS, CG, BLEIC)
• Variable scaling for optimization algorithms (L-BFGS, CG, BLEIC, Lev-Mar), which allows to tune convergence criteria and improve convergence speed for badly scaled problems.
• QP-solver for quadratic programming problems with dense symmetric positive definite matrices and boundary constraints
• Levenberg-Marquardt optimizer (and lsfit unit) now support optimization/fitting with boundary constraints
• BLEIC optimizer (optimization with bound and linear equality constraints) now uses faster and more robust algorithm

29.12.2010  ALGLIB for IronPython is released

We are ready to announce last release of the Release Month - ALGLIB for IronPython!

Similar to its CPython counterpart, ALGLIB for IronPython is a wrapper - but wrapper around computational core written in C#. ALGLIB for IronPython - is a 100% NET library, which includes only safe code and which is compatible with both 32-bit and 64-bit architectures. And, as far as we know, ALGLIB is a first major numerical analysis library, which is fully compatible with IronPython.

This release is the last one for 2010, and our Release Month is over. Good luck to all ALGLIB users in the upcoming 2011!

25.12.2010  One more release - ALGLIB for CPython

Release Month continues with one more branch of ALGLIB! ALGLIB for CPython is an automatically generated wrapper for computational core written in C++. This branch of ALGLIB combines performance of C++ with flexibility of Python.

One of the most interesting features of new ALGLIB is 100% compatibility with both branches of Python - 2.x and 3.x, which allows to use package with any version of interpreter since 2.5. It makes ALGLIB one of the few numerical packages which can be used with Python 3.

In order to achieve such portability, ALGLIB for CPython uses ctypes to access computational core, which is implemented as external shared library. As result, it can be used with CPython only - other implementations of Python don't support ctypes fully enough. However, Release Month is not over yet :) in a several days we will introduce one more version of ALGLIB, for one more programming language from (spoiler!) Python family.

23.12.2010  ALGLIB for VB.NET is released

New branch of ALGLIB is released - ALGLIB for VB.NET! From now - in open source and commercial editions.

ALGLIB for VB.NET is an automatically generate wrapper for ALGLIB for C#. All ALGLIB functionality can be accessed through one VB module - xalglib.vb, which automatically calls appropriate C# functions and converts data structures between VB.NET and C#. And you don't have to study C# in order to use ALGLIB and to setup C# compiler - C# computational core is distributed in the precompiled form (although you still have sources).

20.12.2010  ALGLIB 3.2.0 is released

We announce new version of ALGLIB - ALGLIB 3.2.0. This release includes many improvements and new algorithms:

• new algorithm - nonlinear optimizer with linear equality/inequality constraints
• improvements in the nonlinear CG: a) user may "suggest" initial step length (algorithm will try to determine step length using suggested value as starting point) b) improved automatic detection of the stopping conditions which are too stringent.
• ALGLIB for C++ now has limited SSE2 support (turned on when compiling with AE_INTEL defined at the global level). As for now, SSE2 support is still experimental and can be used only when compiling under MSVC. However, in the ALGLIB 3.3 (scheduled for January-February) it will be accessible to GCC/Sun Studio users too.
• ALGLIB for C# is distributed with precompiled NET 2.0 assembly

31.10.2010  ALGLIB 3.1.0 is released

We are happy to announce new release - ALGLIB 3.1.0. This release includes following improvements:

• new algorithm for curve fitting by penalized regression spline
• new algorithm for calculation of covariance/correlation matrices, which efficiently works even with large matrices which do not fit into CPU cache
• improved nonlinear least squares solver (optimization without analytic gradient)
• improved Levenberg-Marquardt optimizer (optimization without analytic Jacobian)
• new functions for conversion between barycentric representation of polynomial and power basis

We also significantly improved ALGLIB documentation, especially 'Optimization' è 'Interpolation' sections of ALGLIB User Guide. Many new examples were added, cross-links between ALGLIB User Guide and ALGLIB Reference Manual were implemented.

30.09.2010  Final release of ALGLIB 3.0.0 for C++ and C#

Final release of ALGLIB 3.0.0 is ready - both C++ and C# versions. This release contains some minor tweaks for better portability as well as some new functionality. We've fixed some minor bugs in ALGLIB 3.0.0.RC1 (and discovered some bugs in GCC and Sun Studio during our testing).

C# version features significant changes in the interface provided by the library. As with C++ version, 104 .cs files were merged into 12 packages. Function names were changed from alglib.classname.functionname to alglib.functionname. Inconvenient reverse communication interface used by optimizers and fitting functions was replaced by more convenient version which accepts delegates.

There is also new functionality in the ALGLIB 3.0 in comparison with Release Candidate 1: better step selection for CG-based algorithms, grid conversion and differentiation using cubic splines, internal sorting algorithm improvements.

01.09.2010  New release - ALGLIB 3.rc1 for C++

Totally new version of ALGLIB is released - ALGLIB 3.0.0.rc1. This new version contains several major changes which makes it very different from previous releases.

First, it contains many backward incompatible changes. Names/parameters of some functions were changed. Obsolete constructions which were there because of backward compatibility only are now gone. See Change Log for more information. Sounds complex? Not really - if you can compile your code with new ALGLIB, everything will work.

Second, there was major restructurization of code which was made after discussion with ALGLIB users. ALGLIB 2.6.0 contained 104 units, and our users find it inconvenient to have 104 files in their project. Now, in ALGLIB 3.x all units are merged into 12 packages with simple dependency structure. So you have to change names of headers/units you use.

Third, project idea was changed a bit. Before 3.x core concept was automatic translation from pseudocode to each of target languages. FreePascal ALGLIB was 100% FreePascal, VBA ALGLIB was 100% VBA. It was very beatiful idea, but the more we wanted from ALGLIB, the harder it was to implement it in all languages. So from 3.0 we've changed our focus from automatic translation to automatic interface generation. Automatic translation is still used, but we translate to several "rich" languages only: C, C++, C#, multiple precision C++. Other languages will get interface to precompiled C version.

Such large-scale changes need some time, so now we are only able to introduce C++ version. During September we will introduce C# (which is almost ready) and Python versions (former will be implemented in pure C#, while latter will be interface to precompiled C version). Python interface was announced long before this release. It was planned to be introduced in June, but because of switch to ALGLIB 3.x with its backward incompatible changeds we've decided to make it available after first 3.x release.

There is also new functionality in the ALGLIB 3.0: nonlinear solver, efficient restart functions for optimizers, improved error handling in many functions. Inconvenient reverse communication interface used by optimizers and fitting functions was replaced by more convenient version which accepts function pointers (however, you still can use reverse communication interface).

01.06.2010  ALGLIB 2.6.0 is released

New ALGLIB release features:

1. two bugfixes (see. Change Log).

2. improved spline interpolation algorithms: added support for Catmull-Rom spline; added support for periodic boundary conditions.

3. new algorithms for parametric spline interpolation in 2D and 3D are introduced. Two types of curves are supported: with non-periodic and periodic boundary conditions, i.e. open curves and loops.

Now several words about the future. New release of ALGLIB is almost the same as ALGLIB 2.5.0 - there were no new algorithms except for improved spline interpolation. The reason for this is active work under new extension of ALGLIB package - interface between ALGLIB and Python. Two interfaces are planned to be released - one for double precision ALGLIB, another for multiple precision ALGLIB. At least one of them should be ready in June, with another following several weeks later.

12.04.2010  ALGLIB 2.5.0 is released

New ALGLIB release features:

1. two bugfixes (see Change Log).

2. new optimization algorithms: nonlinear conjugate gradient method and active set algorithm for bound constrained optimization.

3. API of L-BFGS and Levenberg-Marquardt algorithms was slightly changed: see #0000326 and #0000327 for more info. These changes are backward incompatible. However, it should be easy to modify existing programs to work with new interface.

10.03.2010  ALGLIB 2.4.0 is released

1. New ALGLIB release contains improved versions of several linear algebra algorithms: QR decomposition, matrix inversion, condition number estimation.

2. New algorithms implemented: complex QR and LQ decompositions, condition number estimation for triangular matrices, exact and approximate nearest neighbor search using kd-trees, new multidimensional scattered data interpolation/fitting algorithm with O(N·logN) complexity (modified Shepard's method with fast k-NN queries).

3. Minor fixes in one of the units (see Change Log).

30.01.2010  ALGLIB 2.3.0 is released

1. New ALGLIB release features improved linear algebra subroutines (mostly in C++ version). Performance of C++ implementation of Level 1 BLAS was significantly improved. A number of cache oblivious Level 2 and Level 3 BLAS algorithms was implemented. Triangular factorization algorithms (LU decomposition and Cholesky decomposition) were rewritten in new cache-oblivious manner. Now they are relying on Level 3 ALGLIB BLAS and their performance was significantly improved. New condition number estimation algorithm is much more faster than the previous version (partly due to reduction of unnecessary matrix copying).

2. New algorithms were added: a set of linear solvers for real/complex/SPD/HPD matrices.

3. Minor fixes (see Change Log for more info).

Plans for the near future: improve performance of orthogonal factorization algorithms (QR/QL/LQ/RQ, bi/tridiagonal reduction) and rewrite them in cache-oblivious manner; further improve C++ version performance using SSE intrinsics.

18.01.2010  ALGLIB 2.2.1 is released

ALGLIB 2.2.1 is a small bugfix-only release. This release fixes incorrect assertion in minlm unit.

December, 2009. ALGLIB 2.2.0.

1. One of the most important changes is new script-based compilation/testing system. Now ALGLIB may be compiled using Bash-script build (or batch file with similar name - for Windows users). One more script - check - may be used to test ALGLIB (or individual units).

2. ALGLIB was tested under wide variety of platforms and compilers, package portability was significantly improved. As of December, 2009, ALGLIB was tested under x86 and x86_64, under Windows and Linux, with Microsoft compilers, GCC, Mono and FreePascal. More wide testing is expected - both with new compilers and new hardware/software platforms.

3. Now ALGLIB supports Pascal in two fashions - proprietary Delphi compiler and open-source FreePascal compiler. ALGLIB for FreePascal is a different set of source codes, which targets slightly different language standard (FPC default mode). Although there are minor issues with current FPC release, FreePascal looks like an interesting compiler to support because of its open sourceness and cross-platform capabilities.

4. A lot of sources introduced: new constrained/unconstrained linear least squares fitting codes, non-linear least squares fitting sources, rational/polynomial fitting in barycentric form, fitting by splines, high quality random number generator, random matrix generator. See http://bugs.alglib.net/changelog_page.php for complete list of changes.

October, 2009. ALGLIB 2.1.2.

ALGLIB 2.1.2 is another bugfix-only release. This release fixes compilation error under original (Linux) GCC and incorrect handling of near-underflow vectors by reflections unit.

September, 2009. ALGLIB 2.1.1.

ALGLIB 2.1.1 - is a minor update to ALGLIB 2.1.0, bugfix-only release. The only change is a bug in the neural networks package being fixed. Major improvements will wait till 2.2.0 release, which is expected to be released in November, 2009.

September, 2009. ALGLIB 2.1.0.

The most important in the version 2.1.0 is a change of license. The decision to change the license was not easy. The project was distributed under BSD for about two years. But over time it became clear that GPL is more suitable for further project development. So starting from version 2.1.0, ALGLIB is distributed under GPL 2+ (GPL version 2 or later). License change, of course, affects only 2.1.0 and following versions. Users of previous versions may continue to use ALGLIB and distribute derivative code under less strict, BSD-compatible terms.

About other changes:

• Minor fixes in neural networks package (workaround for errors in standard libraries of several compilers)
• Added real/complex FFT implementation with O(N·log(N)) complexity for any N (composite/prime)
• Added real/complex fast convolution source
• Added real/complex fast cross-correlation source

Backward incompatible changes:

• Improved adaptive 1D integrator (improved interface, ability to integrate functions with singularities). Unit status changed from "obsolete" to "stable".
• Improved Gauss and Gauss-Kronrod quadrature generators (more stable algorithms, improved interface). Unit status changed from "obsolete" to "stable".
• C# sources are now in alglib namespace, so C# users should add "using alglib;" for their programs to work with new version.
• All ALGLIB/C# classes are now public.

See bug tracker for more information on changes, roadmap - for more information on progress with next release.

July, 2009. ALGLIB 2.0.1.

ALGLIB 2.0.1 is a small bug fix release (see bug tracker for more information). The most important changes: improved performance of rational interpolation algorithm; improved compatibility with tools/compilers which checks for uninitialized variables. The next release is scheduled for August/September; new integration sources will be introduced as well as FFT/FHT/FST/FCT sources and fast convolution/correlation sources.

April, 2009. Site updates.

For a long time - almost a year - the project web-site has not been updated. Such a long break does not mean that the project development stopped. For most of the time we were actively developing new source codes. Now it's my pleasure to announce the following news to the ALGLIB users:

1. A publicly available bug tracker that uses Mantis open-source software has been opened at bugs.alglib.net. At the moment the database is all but empty as earlier we registered bugs in a local software that doesn't allow for data to be transferred into Mantis.
2. The 'Data Analysis' section has been opened. The topics of the section are: regression, classification and other ways of statistical data analysis. A range of algorithms has been implemented: linear and logit regression, random forests of decision trees, neural networks and ensembles of neural networks, linear discriminant analysis, principal component analysis method and k-means++ clustering algorithm.
3. The Optimization section is redesigned. A new version of L-BFGS algorithm and an improved version of the Levenberg-Marquardt algorithm have been issued (the improved version dramatically exceeds the performance of the original algorithm). The new versions of algorithms use reverse communication to calculate the function/gradient values.
4. A number of bugs have been eliminated and some minor improvements have been made. Their description can be found in the bug tracker.
5. Several modules that are complicated to use (optimization, classification and regression) are now accompanied by examples. The online documentation contains references to the demonstration source codes in different languages. At the moment only the most complicated algorithms have examples. However, as the site develops it is planned to add them to all modules except those evident in usage.

Welcome the new ALGLIB 2.0!

Many ALGLIB users told us about their need to download all source codes in one zip archive. Also many users told us that absence of version numbers confuses them and prevents them from tracking changes in the projects. In fact, the project has been developing for a long time in a non-traditional way, as a collection of modules, where the users only chosen the ones they needed. This approach has its advantages, for instance, it allows for creation of compact programs. However there are drawbacks too: the changes in multitude of modules are very hard to track.

This is why the source code access mechanism was changed. Now users can download the entire project in a single zip archive, while still having the possibility to download modules one by one. Individual modules still don't have version numbers. However now the entire release has a number that changes with each alteration. The current version has the number 2.0.

Please note that the version numbers do not begin with 1 as the project has undergone a serious development since 1999, only a part of which was seen to English-speaking users (the project has been originally developed as a Russian language site). Version 2.0 contains all changes made to ALGLIB during the years.

A thought about the future...

Here I will set the priority tasks for the nearest future:

• Finding some time :)
• Implementation of new algorithms (the field of interest includes complex linear algebra and iterative linear algebra algorithms, statistics and data analysis; more details about the plans at http://bugs.alglib.net/roadmap_page.php).
• Testing ALGLIB in a 64-bit environment.
• Adding new languages (plain C, possibly Java and Python).
• Keeping on improving documentation. In particular it is planned to create the offline documentation version (at the moment the documentation is bound to the online environment too tightly).
• Benchmarking ALGLIB's performance against other programming packages (on several tasks, including linear algebra, data analysis and optimization) and publish the results in the form of an article.
• Improving the testing system (including usage of more compilers and versions as well as organizing 24/7 testing of complicated algorithms for more complete testing).

03 June, 2008. Last changes:

• IMPORTANT: AP library for C++ is updated (new version: 1.2).
• IMPORTANT: C++ sources can use lightweight BLAS-like library with SSE2 support. AP library 1.2 is required to use ABLAS!
• BUGFIX: minor fix in BuildPolynomialLeastSquares: correct handling of some degenerate cases
• BUGFIX: minor fix in several unit tests (incorrect error reports on some compilers

14 April, 2008. Last changes:

• IMPORTANT: finished translation of the special functions.
• IMPORTANT: Linear algebra subroutines now take 0-based arrays as parameters. Old 1-based subroutines are left for backward compatibility.
• Fixed problem in incomplete gamma function algorithm, which causes compilation error on some compilers.
• Improved exponential/normal random numbers generation.
• Improved memory management in C++ - real arrays now aligned on 16-byte boundary (basis for future SSE2 optimizations).
• Improved performance of BLAS operations in C++/Delphi due to use of pointers.

09 August, 2007. Last changes:

• ALGLIB is relicensed under BSD
• More sources: "Statistics" (first source codes to be uploaded both in Russian and in English) and "Interpolation"

Future plans:

• Finish translation of the so-long-awaited "Special functions" within a month :)

31 March, 2007. Last changes:

• C# and MPFR sources have been added to the site
• New Hermitian eigenvalue algorithms
• Translation of the "Special functions" has begun

Future plans:

• Translate "Special functions" from Russian to English
• Relicense ALGLIB under some non-restrictive open source license

03 June, 2006. Welcome to the ALGLIB.NET website!

The main news for today is opening of the English-language version of the site, which you are visiting now. Nearly seven years ago, when this site only appeared, hardly anyone could imagine what this website will turn into, a number of years later. For this time, the website revealed only its best sides in the Russian segment of the global network. Now it is time to keep moving ahead. Due to the translation of this information into English, this site has become available to a larger amount of people than before. I hope that you will find all necessary information on this site and will come back for more!

Sergey Bochkanov, Author of ALGLIB.