14859
Comment: Add --disable-zlib option
|
6532
Replace "DOWNWARD_CPLEX_ROOT" by "cplex_DIR".
|
Deletions are marked like this. | Additions are marked like this. |
Line 1: | Line 1: |
Back to [[Doc/Heuristic]]. | Back to [[ObtainingAndRunningFastDownward]]. |
Line 5: | Line 5: |
Some configurations of the search component of Fast Downward, such as optimal cost partitioning for landmark heuristics, require a linear programming (LP) solver and will complain if the planner has not been built with support for such a solver. Setting up LP support requires three steps, explained below: | Some configurations of the search component of Fast Downward, such as optimal cost partitioning for landmark heuristics, require a linear programming (LP) solver and will complain if the planner has not been built with support for such a solver. Running an LP configuration requires three steps, explained below: |
Line 8: | Line 8: |
1. Installing the Open Solver Interface. | |
Line 10: | Line 9: |
1. Choosing a solver with command-line arguments. | |
Line 13: | Line 13: |
Fast Downward uses a generic interface (see next step) for accessing LP solvers and hence can be used together with different LP solvers. Currently, three LP solvers are supported: CPLEX, Gurobi, and COIN-LP. You can install one, two or all three solvers without causing conflicts. The solver used by the planner is selected by command-line arguments, not at compile time. We recommend using CPLEX, which in our experiments has led to better performance than COIN-LP. We have no experience with Gurobi, but others recommend it over CPLEX. (If you have data comparing these two within Fast Downward, we would be very interested in hearing about it.) | Fast Downward uses a generic interface for accessing LP solvers and hence can be used together with different LP solvers. Currently, CPLEX and !SoPlex are supported. You can install one or both solvers without causing conflicts. Installation varies by solver and operating system. |
Line 15: | Line 15: |
=== Installing one or more LP solvers: CPLEX === | === Installing CPLEX on Linux/macOS === |
Line 17: | Line 17: |
IBM offers a [[https://developer.ibm.com/academic/|free academic license]] that includes access to CPLEX. Once you are registered, you can download the 64 bit binary from the member area of the [[https://ibm.onthehub.com|Academic Initiative Website]]. Execute the binary and follow the guided installation. If you want to install in a global location, you have to execute the installer as {{{root}}}. |
IBM offers a [[http://ibm.com/academic|free academic license]] that includes access to CPLEX. Once you are registered, you find the software under Technology -> Data Science. Choose the right version and switch to HTTP download unless you have the IBM download manager installed. If you have problems using their website with Firefox, try Chrome instead. Execute the downloaded binary and follow the guided installation. If you want to install in a global location, you have to execute the installer as {{{root}}}. |
Line 20: | Line 20: |
After the installation, set the following environment variable (adapt the path if you did not install in the default location): | After the installation, set the following environment variable. The installer is for ILOG Studio, which contains more than just CPLEX, so the variable points to the subdirectory {{{/cplex}}} of the installation. Adapt the path if you installed another version or did not install in the default location: |
Line 22: | Line 24: |
export DOWNWARD_CPLEX_ROOT64=/opt/ibm/ILOG/CPLEX_Studio1263/cplex | export cplex_DIR=/opt/ibm/ILOG/CPLEX_Studio2211/cplex |
Line 25: | Line 27: |
If you need 32 bit builds, you also have to install the 32 bit version of CPLEX and use the environment variable {{{DOWNWARD_CPLEX_ROOT32}}} to identify its location. However, CPLEX will no longer support 32-bit builds starting with version 12.7 and previous 32-bit versions of CPLEX are not compatible with recent OS versions. If you get segmentation faults with 32-bit CPLEX builds, try to build one of the sample programs that come with CPLEX to see if it is installed correctly. | If you don't want to permanently modify your environment, you can also set these variables directly when calling CMake. The variable needs to be set when building Fast Downward's search component (Step 2.). |
Line 27: | Line 29: |
If you don't want to permanently modify your environment, you can also set these variables directly when calling CMake, or in {{{src/build.py}}}. The variables need to be set for when installing the Open Solver Interface (Step 2.) and when building Fast Downward's search component (Step 3.). | === Installing CPLEX on Windows === |
Line 29: | Line 31: |
For instructions for Windows, see below. | Follow the Linux instructions to acquire a license and access the Windows-version of the CPLEX installer. Please install CPLEX into a directory without spaces. For a silent installation, please consult: https://www.ibm.com/support/knowledgecenter/SSSA5P_12.9.0/ilog.odms.studio.help/Optimization_Studio/topics/td_silent_install.html |
Line 31: | Line 34: |
=== Installing one or more LP solvers: Gurobi === | /!\ '''Important Note:''' Setting up environment variables might require using / instead of the more Windows-common \ to work correctly. |
Line 33: | Line 36: |
We have no experience with installing Gurobi. On Linux, Gurobi is only available for 64bit, which means that it cannot be used with our default (32bit) build. | === Installing SoPlex on Linux/macOS === |
Line 35: | Line 38: |
After the installation, set the following environment variables: | !SoPlex is available under the Apache License from [[https://github.com/scipopt/soplex|Github]]. To be compatible with C++-20, we require a version of !SoPlex more recent than 6.0.3. At the time of this writing, 6.0.3 is the latest release, so we have to build from the unreleased version at the tip of the main branch. You can install !SoPlex as follows (adapt the path if you install another version or want to use another location): |
Line 37: | Line 42: |
export DOWNWARD_GUROBI_ROOT=/path/to/gurobi | git clone https://github.com/scipopt/soplex.git sudo apt install libgmp3-dev # The library is optional but important for SoPlex's performance export soplex_DIR=/opt/soplex-6.0.3x export CXXFLAGS="$CXXFLAGS -Wno-use-after-free" # Ignore compiler warnings about use-after-free cmake -S soplex -B build cmake --build build cmake --install build --prefix $soplex_DIR rm -rf soplex build |
Line 40: | Line 53: |
If you don't want to permanently modify your environment, you can also set these variables directly when calling CMake, or in {{{src/build.py}}}. The variables need to be set for when installing the Open Solver Interface (Step 2.) and when building Fast Downward's search component (Step 3.). | After installation, permanently set the environment variable {{{soplex_DIR}}} to the value you used in installation. |
Line 42: | Line 55: |
=== Installing one or more LP solvers: COIN-LP === | /!\ '''Note:''' Once a version including Salomé's fix is released, we can update this and can recommend the [[https://soplex.zib.de/index.php#download|SoPlex homepage]] for downloads instead. |
Line 44: | Line 57: |
COIN-LP is no longer bundled with OSI starting from version 0.104.0 and needs to be installed separately. Installation should be possible with | === Installing SoPlex on the grid in Basel === To build !SoPlex on the grid, you should load a module with the GMP library and a compatible compiler module. The following setup should work: |
Line 46: | Line 61: |
sudo apt-get install clp | module purge module load GCC/11.3.0.lua module load CMake/3.23.1-GCCcore-11.3.0.lua module load Python/3.10.4-GCCcore-11.3.0.lua module load GMP/6.2.1-GCCcore-11.3.0 |
Line 48: | Line 67: |
but we have no experience with this. We recommend against using it in serious experiments unless you have established that it offers comparable performance to CPLEX in your setting. | |
Line 50: | Line 68: |
== Step 2. Installing the Open Solver Interface == The Open Solver Interface (OSI) provides a common interface to different LP solvers. OSI must be compiled '''after''' installing the LP solver(s) in Step 1. If you install a solver later, repeat the installation steps of OSI. We assume in the following that 64-bit CPLEX is installed and the environment variable {{{DOWNWARD_CPLEX_ROOT64}}} is set up correctly. These instructions apply to COIN/OSI 0.107.8, newer versions from http://www.coin-or.org/download/source/Osi/ should work, too. If you experience problems, please [[HomePage#contact|let us know]]. === Installing the Open Solver Interface on Linux === Run the following commands. If you have chosen not to install CPLEX, '''omit''' the options related to this solver from the call to {{{./configure}}} (options for solvers that are not installed can lead to very cryptic error messages). If you installed other solvers like CLP or Gurobi, also add the corresponding options for them. You may have to include the path to shared libraries in {{{LDFLAGS}}}, i.e. {{{LDFLAGS="-L$DOWNWARD_CPLEX_ROOT64/lib -L/usr/lib/gcc/i686-linux-gnu/4.8"}}}. If there are other problems, see the [[#Troubleshooting|troubleshooting section]]. |
Because the library is loaded from a module, it is not in a default directory, so change the CMake call to {{{ cmake -S soplex -B build -DGMP_DIR="$EBROOTGMP" }}} |
Line 61: | Line 74: |
{{{#!highlight bash wget http://www.coin-or.org/download/source/Osi/Osi-0.107.8.tgz tar xvzf Osi-0.107.8.tgz cd Osi-0.107.8 |
== Step 2. Building Fast Downward with LP support == |
Line 66: | Line 76: |
sudo ./configure CC="gcc" CFLAGS="-m64 -pthread -Wno-long-long" \ CXX="g++" CXXFLAGS="-m64 -pthread -Wno-long-long" \ LDFLAGS="-L$DOWNWARD_CPLEX_ROOT64/lib/x86-64_linux/static_pic" \ --without-lapack --enable-static=yes \ --prefix="/opt/coin64" \ --disable-zlib \ --with-cplex-incdir=$DOWNWARD_CPLEX_ROOT64/include/ilcplex --with-cplex-lib="-lcplex -lm" sudo make sudo make install cd .. sudo rm -rf Osi-0.107.8 rm Osi-0.107.8.tgz }}} After installation, set the environment variable {{{DOWNWARD_COIN_ROOT64}}} to the prefix you used in the call to {{{./configure}}}. For example: {{{ export DOWNWARD_COIN_ROOT64=/opt/coin64 }}} If you want to compile for 32-bit, change {{{-m64}}} to {{{-m32}}} in the configuration step above and use the CPLEX paths to the 32-bit version. You also have to change the setting for {{{--prefix}}}, otherwise the 32-bit version will overwrite the 64-bit version. You can then use the environment variables {{{DOWNWARD_COIN_ROOT32}}} to identify the version. If you don't want to permanently modify your environment, you can also set these variables directly when calling CMake, or in {{{src/build.py}}}. The variables need to be set for when installing the Open Solver Interface (Step 2.) and when building Fast Downward's search component (Step 3.). === Installing the Open Solver Interface on Mac OS X === Follow the Linux instructions above with the following changes: * Replace {{{-m32}}} with {{{-arch i386}}}. * Building {{{CoinUtils}}} can create both static and dynamic libraries and there have been reported problems with both. * To use static libraries, delete the dynamic libraries from {{{$(DOWNWARD_COIN_ROOT)/lib/}}} after building {{{CoinUtils}}}. * '''or''' * To use dynamic libraries, replace {{{--enable-static=yes}}} with {{{--enable-static=no}}} and change the Fast Downward CMakefile to look for dynamic (*.dylib) instead of static (*.a) libraries of libOsiCpx, libOsiGrb, and libOsiClp. Installation of version 0.103.0 has been reported to fail on Mac OS X due to a bug in the {{{CoinUtils}}} configure script. (See [[http://issues.fast-downward.org/issue295|issue295]].) If you run into this problem, try downloading the [[attachment:coinutils-configure.patched|patched file]] attached to this page, make it executable, and replace the file {{{Osi-0.103.0/CoinUtils/configure}}} with it: {{{#! highlight bash cp Dowloads/coinutils-configure.patched Osi-0.103.0/CoinUtils/configure chmod +x Osi-0.103.0/CoinUtils/configure }}} === Installing the Open Solver Interface on Windows === We managed to install OSI with support for CPLEX on Windows using {{{msys2}}} using the following steps: 1. Install CPLEX into a directory without spaces. We assume this is {{{C:\cplex}}}. Usually you will install ILOG Studio, which contains more than just CPLEX, so the paths below use {{{C:\cplex\cplex}}} to point to the CPLEX part of it. 1. Install [[http://sourceforge.net/projects/msys2/|msys2]] (this is only necessary to compile Osi and not required to compile the planner) 1. Within msys2 use pacman to install base-devel: {{{pacman -S base-devel}}} 1. Download Osi (we use 0.103.0, newer versions should work but are untested) and unpack to {{{C:\msys64\home\<username>\Osi-0.103.0}}} 1. Start the "VS2013 x86 Native Tools Command Prompt", log in to msys from there, and change to the Osi directory: {{{#!highlight bash C:\msys64\usr\bin\bash --login -i cd Osi-0.103.0 }}} 1. Compile debug version of OSI: {{{#!highlight bash mkdir /c/osi mkdir /c/osi/mtd LD='link' STRIP=':' AR='lib' RANLIB=':' NM='dumpbin -symbols' \ CC='cl -nologo' CFLAGS=' -MTd -Zi -FS' CXX='cl -nologo' CXXFLAGS=' -MTd -Zi -EHsc -FS' \ ./configure --host=x86_64-w64-mingw32 \ --build=mingw32 \ --prefix=/c/osi/mtd \ --with-cplex-lib='-L/c/cplex/cplex/lib/x86_windows_vs2013/stat_mtd/cplex1262.lib' \ --with-cplex-incdir='/c/cplex/cplex/include/ilcplex' \ --disable-cplex-libcheck make make install }}} Afterwards, copy the debug symbol file {{{C:\msys64\home\<username>\Osi-0.103.0\vc120.pdb}}} to {{{C:\osi\mtd\lib\}}} to avoid compiler warnings when compiling a debug version. 1. Compile release version of OSI. Note the subtle differences to the command above: we use the directory {{{/c/osi/mt}}} instead of {{{/c/osi/mtd}}} in two places, the flag {{{-MT}}} instead of {{{-MTd}}} in two places, and the CPLEX library {{{stat_mta}}} instead of {{{stat_mtd}}}: {{{#!highlight bash mkdir /c/osi mkdir /c/osi/mt LD='link' STRIP=':' AR='lib' RANLIB=':' NM='dumpbin -symbols' \ CC='cl -nologo' CFLAGS=' -MT -Zi -FS' CXX='cl -nologo' CXXFLAGS=' -MT -Zi -EHsc -FS' \ ./configure --host=x86_64-w64-mingw32 \ --build=mingw32 \ --prefix=/c/osi/mt \ --with-cplex-lib='-L/c/cplex/cplex/lib/x86_windows_vs2013/stat_mta/cplex1262.lib' \ --with-cplex-incdir='/c/cplex/cplex/include/ilcplex' \ --disable-cplex-libcheck make make install }}} To compile a 64-bit version, start the "VS2013 x64 Native Tools Command Prompt" in step 5 instead of the x86 one. Then chose the 64-bit version of CLPEX in steps 6. and 7., and use a different prefix, e.g., {{{--prefix=/c/osi/mt64}}}. Everything else stays the same (in particular {{{--host=x86_64-w64-mingw32}}} and {{{--build=mingw32}}} do not change). Note that linking under Windows only works if the libraries were compiled with the same flags. For example, if OSI is compiled with the flag {{{-MTd}}} (dynamic debug) then the planner also has to use this flag for compilation. In our CMake build system some flags are automatically determined by the build type, e.g., if CMake is called with {{{CMAKE_BUILD_TYPE=DEBUG}}}, the flag {{{-MTd}}} is added by default. The CPLEX library also has to fit to everything. This is important when compiling Osi (in the --with-cplex-lib paramater, see above) and when compiling Fast Downward (handled by the CMake file). CPLEX has precompiled libraries for different versions of Visual Studio each as a static release and static debug version. This is documented in the file {{{C:/cplex/cplex/c_cpp.html}}}. To allow both debug and release builds of 32- and 64-bit version to link against the correct version of OSI, set the following environment variables: {{{#!highlight bash DOWNWARD_COIN_ROOT_RELEASE32=C:/osi/mt DOWNWARD_COIN_ROOT_DEBUG32=C:/osi/mtd DOWNWARD_COIN_ROOT_RELEASE64=C:/osi/mt64 DOWNWARD_COIN_ROOT_DEBUG64=C:/osi/mtd64 }}} == Step 3. Building Fast Downward with LP support == Once OSI is installed, you can build Fast Downward's search component with LP support by calling {{{./build.py}}}. Remove your previous build first: |
Once LP solvers are installed and the environment variables {{{cplex_DIR}}} and/or {{{soplex_DIR}}} are set up correctly, you can build Fast Downward's search component with LP support by calling {{{./build.py}}}. Remove your previous build first: |
Line 175: | Line 81: |
Fast Downward automatically includes an LP Solver in the build, if it is needed, and the solver and the necessary OSI adapter are detected on the system. If you want to explicitly build without the LP solver that is installed on your system, disable the LP solver plugin (with {{{-DPLUGIN_LP_SOLVER_ENABLED=FALSE}}}) and all plug-ins that require it, such as {{{POTENTIAL_HEURISTICS}}} (see ObtainingAndRunningFastDownward#Manual_Builds). | Fast Downward automatically includes an LP Solver in the build if it is needed and the solver is detected on the system. If you want to explicitly build without LP solvers that are installed on your system, use {{{./build.py release_no_lp}}}, or a [[ObtainingAndRunningFastDownward#Manual_Builds|manual build]] with the option {{{-DUSE_LP=NO}}}. == Step 3. Choosing a solver with command-line arguments == Features that use an LP solver have a command-line option `lpsolver` to switch between different solver types. See [[http://issues.fast-downward.org/issue752|issue752]] and [[http://issues.fast-downward.org/issue1076|issue1076]] for a discussion of the relative performance of CPLEX and !SoPlex. Note that !SoPlex is not a MIP solver, so using it for configurations that require integer variables will result in an error. Please use CPLEX for such cases. |
Line 181: | Line 95: |
If the configure step of COIN/OSI fails with the error {{{Cannot find symbol(s) CPXgetstat with CPX}}}, check the path to the CPLEX lib directory. Depending on your CPLEX installation, the libraries might be stored in a different place, e.g., in {{{$DOWNWARD_CPLEX_ROOT/lib/x86-64_sles10_4.1/static_pic}}}. Check that the directory exists and contains the file {{{libcplex.a}}}. If the configure step fails for another reason, have a look at their [[https://projects.coin-or.org/BuildTools/wiki/user-troubleshooting|troubleshooting page]] or their page of [[https://projects.coin-or.org/BuildTools/wiki/current-issues|current issues]]. |
|
Line 185: | Line 97: |
If Fast Downward gives you compilation errors mentioning the `isnan` function, you are likely affected by an OSI/compiler incompatibility (see [[http://issues.fast-downward.org/issue681]]). In this case try the following: | If you compiled Fast Downward on Windows (especially on !GitHub Actions) and cannot execute the binary in a new command line, then it might be unable to find a dynamically linked library. Use {{{dumpbin /dependents PATH\TO\DOWNWARD\BINARY}}} to list all required libraries and ensure that they can be found in your {{{PATH}}} variable. |
Line 187: | Line 99: |
1. Add `export DOWNWARD_USE_ISNAN_WORKAROUND=1` to your `~/.bashrc` file (or other shell configuration file). 1. Make sure the changed shell configuration takes effect by logging out and then logging in again. 1. In the main directory of the repository, delete all cmake build artifacts with `rm -r builds`. 1. Rebuild. If installing a 32-bit version of OSI on maia leads to an error about the wrong file format of {{{libstdc++.so}}}, then removing 64-bit specific paths from {{{LD_LIBRARY_PATH}}} might help. For example with g++-4.9.2 on maia, removing {{{/scicore/soft/apps/GCC/4.9.2/lib64}}} helps. If after troubleshooting you can get the LP package to work, please do let us know of your problem and its solution so that we can improve these instructions. If you still cannot get it to work, we may be able to provide some help, but note that the LP solvers and OSI library are external packages not developed by us. |
If after troubleshooting you can get the LP package to work, please do let us know of your problem and its solution so that we can improve these instructions. If you still cannot get it to work, we may be able to provide some help, but note that the LP solvers are external packages not developed by us. |
Back to ObtainingAndRunningFastDownward.
LP solver support
Some configurations of the search component of Fast Downward, such as optimal cost partitioning for landmark heuristics, require a linear programming (LP) solver and will complain if the planner has not been built with support for such a solver. Running an LP configuration requires three steps, explained below:
- Installing one or more LP solvers.
- Building Fast Downward with LP support.
- Choosing a solver with command-line arguments.
Step 1. Installing one or more LP solvers
Fast Downward uses a generic interface for accessing LP solvers and hence can be used together with different LP solvers. Currently, CPLEX and SoPlex are supported. You can install one or both solvers without causing conflicts. Installation varies by solver and operating system.
Installing CPLEX on Linux/macOS
IBM offers a free academic license that includes access to CPLEX. Once you are registered, you find the software under Technology -> Data Science. Choose the right version and switch to HTTP download unless you have the IBM download manager installed. If you have problems using their website with Firefox, try Chrome instead. Execute the downloaded binary and follow the guided installation. If you want to install in a global location, you have to execute the installer as root.
After the installation, set the following environment variable. The installer is for ILOG Studio, which contains more than just CPLEX, so the variable points to the subdirectory /cplex of the installation. Adapt the path if you installed another version or did not install in the default location:
export cplex_DIR=/opt/ibm/ILOG/CPLEX_Studio2211/cplex
If you don't want to permanently modify your environment, you can also set these variables directly when calling CMake. The variable needs to be set when building Fast Downward's search component (Step 2.).
Installing CPLEX on Windows
Follow the Linux instructions to acquire a license and access the Windows-version of the CPLEX installer. Please install CPLEX into a directory without spaces. For a silent installation, please consult: https://www.ibm.com/support/knowledgecenter/SSSA5P_12.9.0/ilog.odms.studio.help/Optimization_Studio/topics/td_silent_install.html
Important Note: Setting up environment variables might require using / instead of the more Windows-common \ to work correctly.
Installing SoPlex on Linux/macOS
SoPlex is available under the Apache License from Github. To be compatible with C++-20, we require a version of SoPlex more recent than 6.0.3. At the time of this writing, 6.0.3 is the latest release, so we have to build from the unreleased version at the tip of the main branch.
You can install SoPlex as follows (adapt the path if you install another version or want to use another location):
git clone https://github.com/scipopt/soplex.git sudo apt install libgmp3-dev # The library is optional but important for SoPlex's performance export soplex_DIR=/opt/soplex-6.0.3x export CXXFLAGS="$CXXFLAGS -Wno-use-after-free" # Ignore compiler warnings about use-after-free cmake -S soplex -B build cmake --build build cmake --install build --prefix $soplex_DIR rm -rf soplex build
After installation, permanently set the environment variable soplex_DIR to the value you used in installation.
Note: Once a version including Salomé's fix is released, we can update this and can recommend the SoPlex homepage for downloads instead.
Installing SoPlex on the grid in Basel
To build SoPlex on the grid, you should load a module with the GMP library and a compatible compiler module. The following setup should work:
module purge module load GCC/11.3.0.lua module load CMake/3.23.1-GCCcore-11.3.0.lua module load Python/3.10.4-GCCcore-11.3.0.lua module load GMP/6.2.1-GCCcore-11.3.0
Because the library is loaded from a module, it is not in a default directory, so change the CMake call to
cmake -S soplex -B build -DGMP_DIR="$EBROOTGMP"
Step 2. Building Fast Downward with LP support
Once LP solvers are installed and the environment variables cplex_DIR and/or soplex_DIR are set up correctly, you can build Fast Downward's search component with LP support by calling ./build.py. Remove your previous build first:
rm -rf builds
Fast Downward automatically includes an LP Solver in the build if it is needed and the solver is detected on the system. If you want to explicitly build without LP solvers that are installed on your system, use ./build.py release_no_lp, or a manual build with the option -DUSE_LP=NO.
Step 3. Choosing a solver with command-line arguments
Features that use an LP solver have a command-line option lpsolver to switch between different solver types. See issue752 and issue1076 for a discussion of the relative performance of CPLEX and SoPlex.
Note that SoPlex is not a MIP solver, so using it for configurations that require integer variables will result in an error. Please use CPLEX for such cases.
Troubleshooting
The LP-related libraries have a number of dependencies which might not be installed on your system. If for some reason one of the above steps fails, we would appreciate if you could attempt to troubleshoot it yourself.
If you get warnings about unresolved references with CPLEX, visit their help pages.
If you compiled Fast Downward on Windows (especially on GitHub Actions) and cannot execute the binary in a new command line, then it might be unable to find a dynamically linked library. Use dumpbin /dependents PATH\TO\DOWNWARD\BINARY to list all required libraries and ensure that they can be found in your PATH variable.
If after troubleshooting you can get the LP package to work, please do let us know of your problem and its solution so that we can improve these instructions. If you still cannot get it to work, we may be able to provide some help, but note that the LP solvers are external packages not developed by us.