Differences between revisions 32 and 33
Revision 32 as of 2015-02-11 11:26:17
Size: 8657
Comment: changed wording in mac instructions
Revision 33 as of 2015-08-03 17:02:46
Size: 12023
Comment: build instructions for OSI/Cplex on Windows
Deletions are marked like this. Additions are marked like this.
Line 27: Line 27:

For instructions for Windows, see below.
Line 101: Line 103:
=== Installing the Open Solver Interface on Windows ===

We managed to install OSI with support for CPLEX on Windows using {{{msys2}}} using the following steps:

 1. Install CPLEX into a directory without spaces. We assume this is {{{C:\cplex}}}. Usually you will install ILOG Studio, which contains more than just CPLEX, so the paths below use {{{C:\cplex\cplex}}} to point to the CPLEX part of it.
 1. Install [[http://sourceforge.net/projects/msys2/|msys2]] (this is only necessary to compile Osi and not required to compile the planner)
 1. Within msys2 use pacman to install base-devel: {{{pacman -S base-devel}}}
 1. Download Osi (we use 0.103.0, newer versions should work but are untested) and unpack to {{{C:\msys64\home\<username>\Osi-0.103.0}}}
 1. Start the "VS2013 x86 Native Tools Command Prompt", log in to msys from there, and change to the Osi directory:
    {{{#!highlight bash
    C:\msys64\usr\bin\bash --login -i
    cd Osi-0.103.0
    }}}
 1. Compile debug version of OSI:
    {{{#!highlight bash
    mkdir /c/osi
    mkdir /c/osi/mdd
    LD='link' STRIP=':' AR='lib' RANLIB=':' NM='dumpbin -symbols' \
    CC='cl -nologo' CFLAGS=' -MDd -Zi -FS' CXX='cl -nologo' CXXFLAGS=' -MDd -Zi -EHsc -FS' \
    ./configure --host=x86_64-w64-mingw32 \
                --build=mingw32 \
                --prefix=/c/osi/mdd \
                --with-cplex-lib='-L/c/cplex/cplex/lib/x86_windows_vs2013/stat_mdd/cplex1262.lib' \
                --with-cplex-incdir='/c/cplex/cplex/include/ilcplex' \
                --disable-cplex-libcheck
    make
    make install
    }}}
 1. Compile release version of OSI. Note the subtle differences to the command above: we use the directory {{{/c/osi/md}}} instead of {{{/c/osi/mdd}}} in two places, the flag {{{-MD}}} instead of {{{-MDd}}} in two places, and the CPLEX library {{{stat_mda}}} instead of {{{stat_mdd}}}:

    {{{#!highlight bash
    mkdir /c/osi
    mkdir /c/osi/md
    LD='link' STRIP=':' AR='lib' RANLIB=':' NM='dumpbin -symbols' \
    CC='cl -nologo' CFLAGS=' -MD -Zi -FS' CXX='cl -nologo' CXXFLAGS=' -MD -Zi -EHsc -FS' \
    ./configure --host=x86_64-w64-mingw32 \
                --build=mingw32 \
                --prefix=/c/osi/md \
                --with-cplex-lib='-L/c/cplex/cplex/lib/x86_windows_vs2013/stat_mda/cplex1262.lib' \
                --with-cplex-incdir='/c/cplex/cplex/include/ilcplex' \
                --disable-cplex-libcheck
    make
    make install
    }}}

Note that linking under Windows only works if the libraries were compiled with the same flags. For example, if OSI is compiled with the flag {{{-MDd}}} (dynamic debug) then the planner also has to use this flag for compilation.

In our upcoming cmake build system (see issue67) some flags are automatically determined by the build type, e.g., if cmake is called with {{{CMAKE_BUILD_TYPE=DEBUG}}}, the flag {{{-MDd}}} is added by default. The CPLEX library also has to fit to everything. This is important when compiling Osi (in the --with-cplex-lib paramater, see above) and when compiling Fast Downward (handled by the cmake file). CPLEX has precompiled libraries for different versions of Visual Studio each as a static release and static debug version. This is documented in the file {{{C:/cplex/cplex/c_cpp.html}}}

Back to Doc/Heuristic.

LP solver support

Some configurations of the search component of Fast Downward, such as optimal cost partitioning for landmark heuristics, require a linear programming (LP) solver and will complain if the planner has not been built with support for such a solver. Setting up LP support requires three steps, explained below:

  1. Installing one or more LP solvers.
  2. Installing the Open Solver Interface.
  3. Building Fast Downward with LP support.

Step 1. Installing one or more LP solvers

Fast Downward uses a generic interface (see next step) for accessing LP solvers and hence can be used together with different LP solvers. Currently, three LP solvers are supported: CPLEX, Gurobi, and COIN-LP. You can install one, two or all three solvers without causing conflicts. The solver used by the planner is selected by command-line arguments, not at compile time. We recommend using CPLEX, which in our experiments has led to better performance than COIN-LP. We have no experience with Gurobi, but others recommend it over CPLEX. (If you have data comparing these two within Fast Downward, we would be very interested in hearing about it.)

Installing one or more LP solvers: CPLEX

IBM offers a free academic license that includes access to CPLEX. Once you are registered, you can download a binary from the member area of the Academic Initiative Website. Execute the binary and follow the guided installation. If you want to install in a global location, you have to execute the installer as root.

After the installation, set the following environment variables (adapt the paths if you did not install in the default location):

export DOWNWARD_CPLEX_INCDIR=/opt/ibm/ILOG/CPLEX_Studio1251/cplex/include/ilcplex
export DOWNWARD_CPLEX_LIBDIR=/opt/ibm/ILOG/CPLEX_Studio1251/cplex/lib/x86_sles10_4.1/static_pic

If you don't want to permanently modify your environment, you can also set these variables directly in src/search/Makefile. The variables need to be set for when installing the Open Solver Interface (Step 2.) and when building Fast Downward's search component (Step 3.).

For instructions for Windows, see below.

Installing one or more LP solvers: Gurobi

We have no experience with installing Gurobi.

After the installation, set the following environment variables (adapt the paths if you did not install in the default location):

export DOWNWARD_GUROBI_INCDIR=/path/to/gurobi/include_dir
export DOWNWARD_GUROBI_LIBDIR=/path/to/gurobi/library_dir

If you don't want to permanently modify your environment, you can also set these variables directly in src/search/Makefile. The variables need to be set for when installing the Open Solver Interface (Step 2.) and when building Fast Downward's search component (Step 3.).

Installing one or more LP solvers: COIN-LP

COIN-LP is installed automatically by older version of the Open Solver Interface in the following step. It will be available without additional work in version 0.103.0 but needs to be installed separately for the current version. However, we recommend against using it in serious experiments unless you have established that it offers comparable performance to CPLEX in your setting.

Step 2. Installing the Open Solver Interface

The Open Solver Interface (OSI) provides a common interface to different LP solvers. OSI must be compiled after installing the LP solver(s) in Step 1. If you install a solver later, repeat the installation steps of OSI. We assume in the following that CPLEX and Gurobi are installed and the environment variables DOWNWARD_CPLEX_INCDIR, DOWNWARD_CPLEX_LIBDIR, DOWNWARD_GUROBI_INCDIR and DOWNWARD_GUROBI_LIBDIR are set up correctly.

These instructions apply to COIN/OSI 0.103.0, which was the last version that was bundled with Fast Downward. Newer versions from http://www.coin-or.org/download/source/Osi/ should work, too. If you experience problems, please let us know.

Installing the Open Solver Interface on Linux

Run the following commands. If you have chosen not to install CPLEX and/or Gurobi, omit the options related to these solvers from the call to ./configure (options for solvers that are not installed can lead to very cryptic error messages). If you install a current version of OSI that does not include CLP and you have installed CLP separately, also add the corresponding options for CLP. This is not necessary for versions that come with a bundled CLP. You may have to include the path to shared libraries in LDFLAGS, i.e. LDFLAGS="-L$DOWNWARD_CPLEX_LIBDIR -L/usr/lib/gcc/i686-linux-gnu/4.8".

   1 set -u
   2 OSI_VERSION=0.103.0
   3 wget http://www.coin-or.org/download/source/Osi/Osi-$OSI_VERSION.tgz
   4 tar xvzf Osi-$OSI_VERSION.tgz
   5 cd Osi-$OSI_VERSION
   6 
   7 sudo ./configure CC="gcc"  CFLAGS="-m32 -pthread -Wno-long-long" \
   8                  CXX="g++" CXXFLAGS="-m32 -pthread -Wno-long-long" \
   9                  LDFLAGS="-L$DOWNWARD_CPLEX_LIBDIR -L$DOWNWARD_GUROBI_LIBDIR" \
  10                  --without-lapack --enable-static=yes \
  11                  --prefix="path/to/coin" \
  12                  --with-cplex-incdir=$DOWNWARD_CPLEX_INCDIR --with-cplex-lib="-lcplex -lm" \
  13                  --with-gurobi-incdir=$DOWNWARD_GUROBI_INCDIR --with-gurobi-lib="-lgurobi"
  14 sudo make
  15 sudo make install
  16 cd ..
  17 rm -rf Osi-$OSI_VERSION
  18 rm Osi-$OSI_VERSION.tgz

After installation, set the environment variable DOWNWARD_COIN_ROOT to the prefix you used in the call to ./configure. For example:

export DOWNWARD_COIN_ROOT=/opt/coin

If you don't want to permanently modify your environment, you can also set this variables directly in src/search/Makefile. The variable need to be set when building Fast Downward's search component (Step 3.).

Installing the Open Solver Interface on Mac OS X

Follow the Linux instructions above with the following changes:

  • Replace -m32 with -arch i386.

  • Building CoinUtils can creates both static and dynamic libraries and there have been reported problems with both.

    • To use static libraries, delete the dynamic libraries from $(DOWNWARD_COIN_ROOT)/lib/ after building CoinUtils.

    • or

    • To use dynamic libraries, replace --enable-static=yes with --enable-static=no and change the Fast Downward Makefile to look for dynamic (*.dylib) instead of static (*.a) libraries of libOsiCpx, libOsiGrb, and libOsiClp.

Installation has been reported to fail on Mac OS X due to a bug in the CoinUtils configure script. (See issue295.) If you run into this problem, try downloading the patched file attached to this page, make it executable, and replace the file Osi-0.103.0/CoinUtils/configure with it:

    cp Dowloads/coinutils-configure.patched Osi-0.103.0/CoinUtils/configure
    chmod +x Osi-0.103.0/CoinUtils/configure

Installing the Open Solver Interface on Windows

We managed to install OSI with support for CPLEX on Windows using msys2 using the following steps:

  1. Install CPLEX into a directory without spaces. We assume this is C:\cplex. Usually you will install ILOG Studio, which contains more than just CPLEX, so the paths below use C:\cplex\cplex to point to the CPLEX part of it.

  2. Install msys2 (this is only necessary to compile Osi and not required to compile the planner)

  3. Within msys2 use pacman to install base-devel: pacman -S base-devel

  4. Download Osi (we use 0.103.0, newer versions should work but are untested) and unpack to C:\msys64\home\<username>\Osi-0.103.0

  5. Start the "VS2013 x86 Native Tools Command Prompt", log in to msys from there, and change to the Osi directory:
    •    1     C:\msys64\usr\bin\bash --login -i
         2     cd Osi-0.103.0
      
  6. Compile debug version of OSI:
    •    1     mkdir /c/osi
         2     mkdir /c/osi/mdd
         3     LD='link' STRIP=':' AR='lib' RANLIB=':' NM='dumpbin -symbols' \
         4     CC='cl -nologo' CFLAGS=' -MDd -Zi -FS' CXX='cl -nologo' CXXFLAGS=' -MDd -Zi -EHsc -FS' \
         5     ./configure --host=x86_64-w64-mingw32 \
         6                 --build=mingw32 \
         7                 --prefix=/c/osi/mdd \
         8                 --with-cplex-lib='-L/c/cplex/cplex/lib/x86_windows_vs2013/stat_mdd/cplex1262.lib' \
         9                 --with-cplex-incdir='/c/cplex/cplex/include/ilcplex' \
        10                 --disable-cplex-libcheck
        11     make
        12     make install
      
  7. Compile release version of OSI. Note the subtle differences to the command above: we use the directory /c/osi/md instead of /c/osi/mdd in two places, the flag -MD instead of -MDd in two places, and the CPLEX library stat_mda instead of stat_mdd:

    •    1     mkdir /c/osi
         2     mkdir /c/osi/md
         3     LD='link' STRIP=':' AR='lib' RANLIB=':' NM='dumpbin -symbols' \
         4     CC='cl -nologo' CFLAGS=' -MD -Zi -FS' CXX='cl -nologo' CXXFLAGS=' -MD -Zi -EHsc -FS' \
         5     ./configure --host=x86_64-w64-mingw32 \
         6                 --build=mingw32 \
         7                 --prefix=/c/osi/md \
         8                 --with-cplex-lib='-L/c/cplex/cplex/lib/x86_windows_vs2013/stat_mda/cplex1262.lib' \
         9                 --with-cplex-incdir='/c/cplex/cplex/include/ilcplex' \
        10                 --disable-cplex-libcheck
        11     make
        12     make install
      

Note that linking under Windows only works if the libraries were compiled with the same flags. For example, if OSI is compiled with the flag -MDd (dynamic debug) then the planner also has to use this flag for compilation.

In our upcoming cmake build system (see issue67) some flags are automatically determined by the build type, e.g., if cmake is called with CMAKE_BUILD_TYPE=DEBUG, the flag -MDd is added by default. The CPLEX library also has to fit to everything. This is important when compiling Osi (in the --with-cplex-lib paramater, see above) and when compiling Fast Downward (handled by the cmake file). CPLEX has precompiled libraries for different versions of Visual Studio each as a static release and static debug version. This is documented in the file C:/cplex/cplex/c_cpp.html

Step 3. Building Fast Downward with LP support

Once OSI is installed, you can build Fast Downward's search component with LP support by calling make with the option DOWNWARD_USE_LP=1. Remove your previous build first:

make distclean
make DOWNWARD_USE_LP=1

It is also possible to set DOWNWARD_USE_LP directly in src/search/Makefile or in your environment.

Troubleshooting

The LP-related libraries have a number of dependencies which might not be installed on your system. If for some reason one of the above steps fails, we would appreciate if you could attempt to troubleshoot it yourself.

If the configure step fails for COIN/OSI, have a look at their troubleshooting page or their page of current issues. If you get warnings about unresolved references with CPLEX, visit their help pages.

If after troubleshooting you can get the LP package to work, please do let us know of your problem and its solution so that we can improve these instructions. If you still cannot get it to work, we may be able to provide some help, but note that the LP solvers and OSI library are external packages not developed by us.