Tips for ABINIT users and developers

User Tools

Site Tools

Trace:
build:fallbacks

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
build:fallbacks [2020/08/13 10:46] – [Objectives] Xavier Gonzebuild:fallbacks [2020/08/19 12:55] (current) – [Building the fallbacks with Abinit] Jean-Michel Beuken
Line 24: Line 24:
  
 Here are the most relevant links to get useful information about the project and its status: Here are the most relevant links to get useful information about the project and its status:
-/*  * https://gitlab.abinit.org/buildbot/abinit-fallbacks */ +  * [[https://gitlab.abinit.org/buildbot/abinit-fallbacks|sources]]
-  * [[https://gitlab.abinit.org/buildbot/abinit-fallbacks/-/tree/release-9.2|sources]]+
   * [[https://gitlab.abinit.org/buildbot/abinit-fallbacks/issues|issues]]   * [[https://gitlab.abinit.org/buildbot/abinit-fallbacks/issues|issues]]
   * [[https://gitlab.abinit.org/buildbot/abinit-fallbacks/milestones|milestones]]   * [[https://gitlab.abinit.org/buildbot/abinit-fallbacks/milestones|milestones]]
- 
-For historical reasons, a secondary repository is still kept on Launchpad, where the contents of the home repository are periodically mirrored: 
-  * https://launchpad.net/abinit-fallbacks 
- 
 ===== Minimum requirements ===== ===== Minimum requirements =====
  
Line 38: Line 33:
   * the [[http://www.gnu.org/software/wget/|Wget]] and/or [[http://curl.haxx.se/|cURL]] utilities.   * the [[http://www.gnu.org/software/wget/|Wget]] and/or [[http://curl.haxx.se/|cURL]] utilities.
 If your network configuration does not allow for the downloading of remote packages, you will have to place the source tarballs in the //~/.abinit/tarballs%%/%%// directory manually. If your network configuration does not allow for the downloading of remote packages, you will have to place the source tarballs in the //~/.abinit/tarballs%%/%%// directory manually.
- 
-Since the recommended way of getting the Abinit Fallbacks is by cloning its repository from the Abinit Website, we highly recommend you to install the GNU Autotools on your system first: 
-  * [[http://www.gnu.org/software/autoconf/|Autoconf]]; 
-  * [[http://savannah.gnu.org/projects/autoconf-archive/|Autoconf Archive]]; 
-  * [[http://www.gnu.org/software/automake/|Automake]]; 
-  * [[http://www.gnu.org/software/libtool/|Libtool]]. 
  
 To build the fallbacks, you will also need a working development environment, including in particular: To build the fallbacks, you will also need a working development environment, including in particular:
Line 58: Line 47:
 Since the fallbacks evolve at a very different pace from Abinit and have very different objectives, they are available as a separate package. Typically, in one year, there will be 3 minor releases of Abinit, while 2 or 3 of the fallbacks at most will be upgraded asynchronously, mainly during active development phases. In parallel, the build system that wraps their installation will be refactored, upgraded, and tested. Since the fallbacks evolve at a very different pace from Abinit and have very different objectives, they are available as a separate package. Typically, in one year, there will be 3 minor releases of Abinit, while 2 or 3 of the fallbacks at most will be upgraded asynchronously, mainly during active development phases. In parallel, the build system that wraps their installation will be refactored, upgraded, and tested.
  
-The independent fallbacks can be downloaded either from the Abinit Website or Launchpad, at your preference. You can either download the source tarballs released with Abinit or clone one of the Git repositories directly.+The independent fallbacks can be downloaded either from the [[https://www.abinit.org/fallbacks|Abinit Website]]. You can either download the source tarballs released with Abinit or clone one of the Git repositories directly.
  
 +==== Requirements ====
 +
 +Since the recommended way of getting the Abinit Fallbacks is by cloning its repository from the Abinit Website, we highly recommend you to install the GNU Autotools on your system first:
 +  * [[http://www.gnu.org/software/autoconf/|Autoconf]];
 +  * [[http://savannah.gnu.org/projects/autoconf-archive/|Autoconf Archive]];
 +  * [[http://www.gnu.org/software/automake/|Automake]];
 +  * [[http://www.gnu.org/software/libtool/|Libtool]].
 ==== Getting the source code from the Abinit Website ==== ==== Getting the source code from the Abinit Website ====
  
-The official Abinit Fallbacks project page is [[https://gitlab.abinit.org/buildbot/abinit-fallbacks/-/tree/release-9.2|here]]+The official Abinit Fallbacks project page is [[https://gitlab.abinit.org/buildbot/abinit-fallbacks|here]]
  
 If you are an active Abinit Developer, we highly recommend you to [[developers:git:access_config|configure your access]] before anything else. We will suppose this is the case from now on. If you are an active Abinit Developer, we highly recommend you to [[developers:git:access_config|configure your access]] before anything else. We will suppose this is the case from now on.
  
 To get the latest version, just type the following: To get the latest version, just type the following:
-<code bash>git clone abinit-forge:buildbot/abinit-fallbacks</code> +<code bash>git clone abinit-forge:buildbot/abinit-fallbacks.git</code>
- +
-==== Getting the source code from Launchpad ==== +
- +
-The Launchpad mirror project page is there: [[https://launchpad.net/abinit-fallbacks]] +
- +
-It is provided for historical reasons and convenience, but all our activities related to the fallbacks take place on the Abinit Website. +
- +
-The first time you download the fallbacks from Launchpad, we highly recommend you to set your environment as described in the [[developers:git:access_config|Access configuration]] page. In the following, we will suppose you did so.+
  
-To get the latest version of the fallbacks, type: 
-<code bash>git clone lp:abinit-fallbacks</code> 
  
 ==== Downloading a source tarball ==== ==== Downloading a source tarball ====
Line 84: Line 70:
 When a new major or minor version of Abinit is about to be released, we usually package a snapshot of the fallbacks and distribute it as a source tarball. This file is called //abinit-fallbacks-X.Y.Z.tar.gz//, where X and Y correspond to the highest X.Y Abinit version for which compatibility has been tested, and Z is a patch level starting from 0. Please note that Z is used exclusively by the fallbacks and has nothing to do with Abinit. For instance, the tarball //abinit-falbacks-8.4.2.tar.gz// contains fallbacks compatible with all 8.4.* versions of Abinit. When a new major or minor version of Abinit is about to be released, we usually package a snapshot of the fallbacks and distribute it as a source tarball. This file is called //abinit-fallbacks-X.Y.Z.tar.gz//, where X and Y correspond to the highest X.Y Abinit version for which compatibility has been tested, and Z is a patch level starting from 0. Please note that Z is used exclusively by the fallbacks and has nothing to do with Abinit. For instance, the tarball //abinit-falbacks-8.4.2.tar.gz// contains fallbacks compatible with all 8.4.* versions of Abinit.
  
-Source tarballs generated so far can be downloaded from Launchpad (https://launchpad.net/abinit-fallbacks/+download), but this location might change in the future.+/* Source tarballs generated so far can be downloaded from Launchpad (https://launchpad.net/abinit-fallbacks/+download), but this location might change in the future. 
 +*/
  
 <WRAP info>Since Abinit dependencies evolve slowly, a fallbacks bundle with version X.Y.Z is often compatible with the X.(Y-1).* and X.(Y+1).* versions of Abinit.</WRAP> <WRAP info>Since Abinit dependencies evolve slowly, a fallbacks bundle with version X.Y.Z is often compatible with the X.(Y-1).* and X.(Y+1).* versions of Abinit.</WRAP>
Line 114: Line 101:
 ===== Building the fallbacks with Abinit ===== ===== Building the fallbacks with Abinit =====
  
-For convenience, a release of the fallbacks comes with the source tarball of Abinit. It is possible to build it along with Abinit, although much more efficient to build it and install it separately, since you do not have to rebuild the fallbacks each time you wish to make a new build of Abinit. Please note that the version of the fallbacks you may find within the Abinit source tree will always be outdated.+For convenience, a release of the fallbacks comes with the source tarball of Abinit.\\ 
 +It is possible to build it along with Abinit, although much more efficient to build it and install it separately, since you do not have to rebuild the fallbacks each time you wish to make a new build of Abinit.\\  
 +/*Please note that the version of the fallbacks you may find within the Abinit source tree will perhaps be outdated.*/
  
-Building the fallbacks within Abinit is mainly of interest to developers who want to experiment with compile flags and explore issues related to compilers and portability. It can also help - to a reduced extent only - users who are stuck within very rigid configurations and restricted build environments. Here is the way to proceed:+Building the fallbacks within Abinit is mainly of interest to developers who want to experiment with compile flags and explore issues related to compilers and portability. It can also help - to a reduced extent only - users who are stuck within very rigid configurations and restricted build environments.\\ 
 +Here is the way to proceed:
 <code bash> <code bash>
 tar xvzf abinit-x.y.z.tar.gz tar xvzf abinit-x.y.z.tar.gz
Line 122: Line 112:
 mkdir tmp-build mkdir tmp-build
 cd tmp-build cd tmp-build
-../configure \ + 
-  --prefix="/abinit/install/path" ... options ... \ +# Create a "config.ac9" file with all needed options 
-# Abinit drops a build script into the transient/ subdirectory and configure +../configure --with-config-file="config.ac9" 
-# fails because the fallbacks are missing and cannot be tested. +</code> 
-cd transient + 
-# Edit the build-abinit-fallbacks.sh script and set its install prefix +What happens in this case is that we first configure Abinit, so that it generates all the necessary information to build the fallbacks for missing external dependencies. The configure script of Abinit always gives priority to properly-installed external dependencies and will only select the minimal necessary fallbacks, in order to save time, trigger less bugs, and aim at maximum reliability.\\ 
-# as instructed therein+At this stage, the configure script has to fail, because it cannot anticipate the successful build of the fallbacks. 
 + 
 +<code bash> 
 +cd fallbacks
 ./build-abinit-fallbacks.sh ./build-abinit-fallbacks.sh
-make + 
-make install+# At the end of the process, the script provides the options with prefixes to fallbacks. 
 +# e.g. 
 +#   with_hdf5=/WORKSPACE_PREFIX/abinit-x.y.z/tmp-build/fallbacks/install_fb/vendor/version/hdf5/1.10.6 
 +# 
 +# You can link these fallbacks with Abinit by copying the options to your ac9 file. 
 cd .. cd ..
-../configure --prefix=/abinit/install/path \ +../configure --with-config-file="config.ac9" 
-  --with-fallbacks-prefix="/path/to/fallbacks/prefix" \ +
-  ... options ... +
 make -j8   # For example, if you have at least 8 cores available make -j8   # For example, if you have at least 8 cores available
 make check make check
 make install make install
 </code> </code>
 +/* ../configure --prefix=/abinit/install/path \
 +*  --with-fallbacks-prefix="/path/to/fallbacks/prefix" \
 +*  ... options ... 
 +*/
  
-What happens in this case is that we first configure Abinit, so that it generates all the necessary information to build the fallbacks for missing external dependencies. The configure script of Abinit always gives priority to properly-installed external dependencies and will only select the minimal necessary fallbacks, in order to save time, trigger less bugs, and aim at maximum reliability. At this stage, the configure script has to fail, because it cannot anticipate the successful build of the fallbacks.+Once the information is available to the configure script of the fallbacks, we can build them within the Abinit source tree and install them outside.\\ 
 +Next, Abinit uses them during its own build and we can install it normally after checking that everything is fine.\\ 
 +In further builds with the same compiler, the same fallbacks can be reused, i.e. no need to rebuild them.
  
-Once the information is available to the configure script of the fallbacks, we can build them within the Abinit source tree and install them outside. Next, Abinit uses them during its own build and we can install it normally after checking that everything is fine. In further builds with the same compiler, the same fallbacks can be reused, i.e. no need to rebuild them.+By default, the fallbacks are installed in "/WORKSPACE_PREFIX/abinit-x.y.z/tmp-build/fallbacks/install_fb".\\ 
 +The variable //fallbacks_prefix// can be changed in the //fallbacks/build-abinit-fallbacks.sh// script if you want to change the installation folder.\\ 
 +Please note that this path must be absolute, i.e. start with a slash. \\
  
-One line must be changed in the //transient/build-abinit-fallbacks.sh// script for it to work. This script can indeed not guess where you want to install the fallbacks. This is why you have to replace the **PREFIX** keyword by the full installation path of your choice. Please note that this path must be absolute, i.e. start with a slash. The line to change is located at the top of the script: 
 <code bash> <code bash>
 +#!/bin/bash
 +
 +# Init
 +fallbacks_prefix="@abinit_builddir@/fallbacks/install_fb/@abi_fc_vendor@/@abi_fc_version@"
 +
 +...
 +</code>
 +
 +/* -------------------------------------------------------------------------------------------------
 +One line must be changed in the //transient/build-abinit-fallbacks.sh// script for it to work. This script can indeed not guess where you want to install the fallbacks. This is why you have to replace the **PREFIX** keyword by the full installation path of your choice. Please note that this path must be absolute, i.e. start with a slash. The line to change is located at the top of the script:
 +
 #!/bin/sh #!/bin/sh
  
Line 162: Line 177:
  
 ... ...
-</code>+*/
  
 ===== Using installed fallbacks ===== ===== Using installed fallbacks =====
Line 196: Line 211:
 .    .   
 </code> </code>
-where files are ordered by compiler vendor, then compiler version, and finally build variant. Each variant corresponds to a set of build parameters and is summarized by a keyword:+where files are ordered by compiler vendor (intel, gnu, nag,...), then compiler version, and finally build variant. Each variant corresponds to a set of build parameters and is summarized by a keyword:
   * //gpu// for a GPU-aware configuration;   * //gpu// for a GPU-aware configuration;
-  * //mpi// for a MPI-aware configuration;+  * //openmpi// or //mpich// for a MPI-aware configuration;
   * //omp// for an OpenMP-aware configuration;   * //omp// for an OpenMP-aware configuration;
   * //sdebug// for a serial configuration with enhanced debug flags;   * //sdebug// for a serial configuration with enhanced debug flags;
Line 217: Line 232:
 ==== The configuration of a package fails ==== ==== The configuration of a package fails ====
  
-Errors may appear at configure time when you build the fallbacks. In order to help you diagnose the problems, most packages provide a //config.log// file. You will find it in //~fallbacks/sources/package-version/config.log//, where you replace //package// by the name of the package and //version// by the corresponding version. Open this file and look for the last occurence of the "failed" keyword, this is usually the place where you will get the most useful information. Looking around the corresponding line will give you useful hints about what made the configure script abort. With a little bit of practice, you will understand and fix most errors relatively quickly.+Errors may appear at configure time when you build the fallbacks. In order to help you diagnose the problems, most packages provide a //config.log// file. You will find it in //~fallbacks/sources/package-version/tmp-build/config.log//, where you replace //package// by the name of the package and //version// by the corresponding version. Open this file and look for the last occurence of the "failed" keyword, this is usually the place where you will get the most useful information. Looking around the corresponding line will give you useful hints about what made the configure script abort. With a little bit of practice, you will understand and fix most errors relatively quickly.
  
 ==== "make" fails ==== ==== "make" fails ====
build/fallbacks.1597308418.txt.gz · Last modified: 2020/08/13 10:46 by Xavier Gonze