Tips for ABINIT users and developers - OBSOLETE!

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
<WRAP important>**IMPORTANT** \\ The information provided in this page is valid for Abinit 8.5 and future versions only.</WRAP>

====== Configuring linear algebra ======

===== Underlying concepts =====

Linear algebra is the most critical dependency to consider, since Abinit could not be built nor run without it. It is also the most complex one, due to:
  * the different components to select and their various interactions with external factors, e.g. using ScaLAPACK only makes sense if your environment supports MPI;
  * the way each component is provided, e.g. one component per library or all components in one library;
  * the tremendous diversity of the available implementations, with several of them present on the same computer at once.

It logically corresponds to the most complex core block of the build system. On one side, there are //__vendors__//, who develop specific components for specific architectures. On another side, there are //__architectures__//, which include various kinds of optimizations and offer various degrees of parallelism. And on a third side, there are //__components__//, defined by the kind of mathematical operations they provide and the architectures they aim at.

We have designed our implementation to cover the highest possible number of cases, while preserving a relative simplicity in the configuration of their parameters. For this, we have grouped the 3 above concepts under //flavors// that users select depending on their situations and preferences.

<WRAP tip> This document is best understood when consulted peacefully. Please take your time to read it and feel free to come back as many times as you wish.</WRAP>

===== Linear algebra flavors =====

For sake of clarity, we provide here 3 different perspectives on the linear algebra flavors supported by Abinit and their relationships with the other concepts. The build system lets you configure linear algebra from each of these 3 perspectives, mainly by setting its //with_linalg_flavor// option in different ways (explained in detail below). You can choose between:
  * a vendor-first approach, where the criterion is to use as many components as possible from the same vendor;
  * an architecture-first approach, where the components are selected depending on which elements of the computer architecture are enabled (e.g. OpenMP, MPI, GPU, ...);
  * a component-first approach, where you specify explicitly each linear algebra component (e.g. BLAS, LAPACK, ScaLAPACK, ...) you want to use.

The following table lists the available flavors supported by Abinit and their relationships with the other relevant concepts:

^ Flavor   ^ Vendor ^ Software project ^ Architectures ^ Components ^
| acml     | [[http://developer.amd.com/|AMD]] | [[http://developer.amd.com/tools-and-sdks/archive/amd-core-math-library-acml/|ACML]] | serial | blas, lapack |
| atlas    | [[http://www.csc.lsu.edu/~whaley/#Proj|LSU]] | [[http://math-atlas.sourceforge.net/|ATLAS]] | serial | blas |
| elpa2014 | [[http://www.mpcdf.mpg.de/|MPCDF]] | [[http://elpa.mpcdf.mpg.de/|ELPA 2014]] | mpi | elpa |
| elpa2015 | [[http://www.mpcdf.mpg.de/|MPCDF]] | [[http://elpa.mpcdf.mpg.de/|ELPA 2015]] | mpi | elpa |
| elpa2016 | [[http://www.mpcdf.mpg.de/|MPCDF]] | [[http://elpa.mpcdf.mpg.de/|ELPA 2016]] | mpi | elpa |
| elpa2017 | [[http://www.mpcdf.mpg.de/|MPCDF]] | [[http://elpa.mpcdf.mpg.de/|ELPA 2017]] | mpi | elpa |
| essl     | [[http://www.ibm.com/|IBM]] | [[https://www-03.ibm.com/systems/power/software/essl/|ESSL]] | serial, openmp, mpi | blas, lapack, scalapack |
| magma10  | [[http://www.icl.cs.utk.edu/|UTK]] | [[http://icl.cs.utk.edu/magma/|MAGMA 1.0]] | gpu | magma |
| magma15  | [[http://www.icl.cs.utk.edu/|UTK]] | [[http://icl.cs.utk.edu/magma/|MAGMA 1.5]] | gpu | magma |
| magma25  | [[http://www.icl.cs.utk.edu/|UTK]] | [[http://icl.cs.utk.edu/magma/|MAGMA 2.5]] | gpu | magma |
| mkl      | [[http://developer.intel.com/|Intel]] | [[https://software.intel.com/en-us/intel-mkl|MKL]] | serial, openmp, mpi | blas, lapack, scalapack |
| netlib   | [[http://netlib.org/|Netlib]] | [[http://netlib.org/liblist.html|Netlib Repository]] | serial, mpi | blas, lapack, lapacke, scalapack |
| openblas | [[https://github.com/xianyi|Zhang Xianyi]] | [[http://www.openblas.net/|OpenBLAS]] | serial | blas |
| plasma1  | [[http://www.icl.cs.utk.edu/|UTK]] | [[http://icl.cs.utk.edu/plasma/|PLASMA]] | openmp | plasma |

Please refer to the instructions of each vendor listed above by clicking on the corresponding link to obtain information on how to install their linear algebra implementation.

===== Automatic detection =====

By default, the build system assumes that you want the fastest possible linear algebra operations. When no linear algebra option is specified, it thus looks for each component as soon as its prerequisites are met. The only required components are BLAS and LAPACK, since Abinit relies on them in each and every situation. Unless forced by the user, the other components are enabled only if their prerequisites are met, their detection succeeds, and are proved working.

In the absence of any option, the build system will use an internal heuristic to look for linear algebra libraries and enable/disable components depending on the following criteria:
  - If a compiler vendor has a preferred linear algebra implementation, it will be tried first.
  - If a build framework such as [[https://hpcugent.github.io/easybuild/|EasyBuild]] is present, it will be used to set the libraries.
  - If MPI is enabled, ScaLAPACK and ELPA will be looked for.
  - If everything else fails, the build system will look for a standard Netlib implementation available system-wide.

In all cases, the build system will try to mitigate side-effects on other components, e.g. FFT, but there is no guarantee that it will succeed by itself. This selection and detection process is known to work in the most common cases. If you have a customized linear algebra configuration, you will likely want to use some options to direct the build system to the right libraries and give it instructions to minimize side-effects.

===== Linear algebra options =====

To deal more easily with the various linear algebra implementations that can be used concurrently on the same computer, Abinit provides the //with_linalg_flavor// option. It will tune the detection tests made by the build system so that the probability of finding the correct libraries and setting-up the environment is maximized.

<WRAP tip>Please note that, when using //with_linalg_flavor//, the //LD_LIBRARY_PATH// and //PATH// environment variables must be properly set for the detection to succeed, in particular if you do not specify any other linear algebra option.</WRAP>

You can tune the detection mechanism through the //with_linalg_flavor// option in 3 ways:
  * by just specifying a plus-separated list of flavors, in which case the build system will figure out how to assign which flavor to which component and complain about inconsistencies;
  * architecture-wise, by specifying a comma-separated list of //arch:value// statements, where //arch// can be //serial//, //openmp//, //mpi//, or //gpu//, and //value// is a plus-separated list of flavors;
  * component-wise, by specifying a comma-separated list of //component:value// statements, where //component// is one of the keywords from the first column of the above table, and //value// is a corresponding flavor -- only one is permitted in this case.

Please note that you can only use one of these 3 methods at a time. Mixes are explicitly forbidden. Omitting an architectural item or a linear algebra component with the 2 latter methods will be equivalent to disabling this element.

The other options provide hints to the build system and determine which kinds of tests will be performed:
  * //with_linalg_fcflags// defines which Fortran compiler flags should be applied at compile time; this is useful for compilers providing native linear algebra support;
  * //with_linalg_ldflags// defines which flags should be passed to the linker when building programs; this is useful for compilers providing native linear algebra support;
  * //with_linalg_incs// defines which include flags should be applied at compile time; this option is only useful for LAPACKe;
  * //with_linalg_libs// defines which libraries should be linked with the programs performing linear algebra operations.

===== Creating your own blend of flavors =====

__Example__:
<code>with_linalg_flavor="openblas+netlib"</code>
will cause the build system to look for OpenBLAS for the //blas// component and use Netlib for the other ones, i.e. //lapack// for the serial case and //scalapack// if MPI is enabled in the current build. Unspecified components will be detected using the default behavior, e.g. the build system will look for all available implementations of //elpa// if MPI is enabled but do nothing if MPI is disabled.

===== Architecture-first method =====

__Example__:
<code>with_linalg_flavor="serial:atlas+netlib,mpi:netlib,gpu:magma15"</code>
will cause the build system to look for Atlas for //blas//, Netlib for //lapack// and //scalapack//, and MAGMA 1.5 for //magma//. However, it will never look for //elpa//, since it has not been explicitly stated, and will stop with an error if one of the specified elements is not present or not working. When using this method, you must specify the //serial// block. If one of the //blas// or //lapack// components is omitted, then //netlib// is assumed.

The following table links the serial and multicore architectures supported by Abinit with the other relevant concepts:

^ Architecture ^ Flavors ^ Components ^
| serial       | acml, atlas, essl, mkl, netlib, openblas | blas, lapack    |
| openmp       | essl, plasma1                            | plasma          |
| mpi          | elpa2014, elpa2015, essl, mkl, netlib    | elpa, scalapack |
| gpu          | magma10, magma15                         | magma           |

===== Component-first method =====

__Example__:
<code>with_linalg_flavor="blas:atlas,lapack:netlib,elpa:elpa2015"</code>
will cause the build system to look for the //blas//, //lapack// and //elpa// components, but not for //scalapack//, even if MPI is enabled. It will also fail if any component is missing or not working. When using this method, you must specify the //blas// and //lapack// components.

The following table summarizes the linear algebra components supported by Abinit and their relationships with the other relevant concepts:

^ Component ^ Type      ^ Flavors                                       ^ Remarks                   ^
| blas      | Mandatory | acml, asl, essl, atlas, mkl, netlib, openblas | Required in any case      |
| lapack    | Mandatory | acml, asl, essl, mkl, netlib                  | Requires BLAS             |
| lapacke   | Optional  | netlib                                        | Requires LAPACK           |
| scalapack | Optional  | mkl, netlib                                   | Requires MPI and LAPACK   |
| elpa      | Optional  | elpa2014, elpa2015                            | Requires MPI and LAPACK   |
| plasma    | Optional  | plasma1                                       | Requires OpenMP (caveats) |
| magma     | Optional  | magma10, magma15                              | Requires MPI and GPU      |