Skip to content

Compiling OpenFOAM 6

NOTE: This is a work in progress

Official Documentation♯

Requirements♯

  • gcc >= 4.8.5
    • default GCC on the system is 4.4.7
    • GCC provided by modulefiles is 4.8.1
    • GCC provided by the Software Collection devtoolset-3 is 4.9.2
  • but we will try using icc

module load intel/composerxe/2015.1.133

  • some issues using icc
    • use gcc-4.9 instead
  • cmake >= 3.3
  • boost >= 1.48
  • fftw >= 3.3.7 (opt)
  • Qt >= 5.9.3 (opt)

Edit Configs♯

Edit

/mnt/HA/opt/OpenFOAM6/OpenFOAM-6/etc/bashrc /mnt/HA/opt/OpenFOAM6/OpenFOAM-6/etc/cshrc

Should not have to modify anything in (yes, despite the names, these are really directories):

/mnt/HA/opt/OpenFOAM6/OpenFOAM-6/etc/config.sh/ /mnt/HA/opt/OpenFOAM6/OpenFOAM-6/etc/config.csh/

but it's worth taking a look.

Selecting Compiler♯

GCC 4.9 via Red Hat Software Collections♯

GCC 4.9 is provided by the devtoolset-3 software collection. To enable it, enable the collection in a new shell:

scl enable devtoolset-3 /bin/bash

Then, make sure the gcc/4.8.1 module is unloaded:

module unload gcc/4.8.1

Load cmake:

module load cmake/3.12.2

Rename Directories♯

When expanded, the downloaded source code tarballs do not produce the proper directory names. Rename them:

  • OpenFOAM-6
  • ThirdParty-6

i.e. remove "-version-6" from the directory names.

Configure♯

  • have to use cmake directly

Open MPI♯

Technically, this is a "third party" software. However, the version distributed with OpenFOAM 6 is old, and is not compatible with the hardware-specific libraries we have installed (MXM, FCA). While we already have a gcc48-compiled version installed on Proteus, we will compile an OpenFOAM6-private version.

Open MPI 1.8.1♯

Later releases in the Open MPI 1.8.x series are not compatible with the Mellanox drivers we have installed. 1.8.1 is known to work, so this is what we will use.

./configure CC=`which gcc` CXX=`which g++` FC=`which gfortran` --prefix=/mnt/HA/opt/OpenFOAM6 --with-sge \ --with-mxm=/opt/mellanox/mxm --with-fca=/opt/mellanox/fca

OpenBLAS♯

  • 0.3.3

Perl♯

  • 5.28.0

Python♯

FFTW♯

  • 3.3.8

Qt♯

Configure♯

./configure -prefix /mnt/HA/opt/OpenFOAM6  -opensource -release -rpath -no-evdev -qt-sqlite -qt-pcre -qt-zlib -nomake tests

If a make fails, just delete the entire directory, and untar the source tarball again. "make clean" and "make distclean" and "configure ... -recheck-all" don't seem to clear out some cached stuff.

NOTES:

  • XCB seems to be needed (so, can't pass "-no-xcb" to configure)

Third Party♯

The third party code should be in $FOAM_INST_DIR/ThirdParty-6 aka $WM_THIRD_PARTY_DIR

Download♯

Download all these packages, and expand the archives, in $WM_THIRD_PARTY_DIR:

CGAL depends on the other three packages. The makeCGAL script will build all of them in the correct order.

Order of Prereqs♯

  • Python
  • Open MPI
  • Qt
  • ParaView

ParaView♯

  • The makeParaView script downloads the source

./makeParaView -python -qmake `which qmake` -gcc -qt -qt-5.9.2

  • Or, download from:

https://github.com/Kitware/ParaView/releases

CGAL♯

  • https://www.cgal.org/
  • the makeCGAL script option for specifying boost version should take an underscore rather than hyphen, i.e.

./makeCGAL CGAL-4.1.0 boost_1_55_0 ...

MAY BE CHANGED

  • Edit the file ThirdParty-6/makeCGAL -- the option for setting the toolset to build Boost has changed for Boost 1.67.0; it used to be "intel", and should now be "intel-linux". Lines 152-156 should be:
    toolset="$WM_CC"
    if [ $WM_COMPILER == "Icc" ]; then
        toolset=intel-linux
    fi

And the line to bootstrap the Boost build needs to be changed: the toolset should be specified to bootstrap.sh:

        ./bootstrap.sh \
            --prefix=$BOOST_ARCH_PATH \
            --with-libraries=thread \
            --with-libraries=system \
            --with-toolset=$toolset \

To speed up the build, set:

    [juser@proteusi01 ThirdParty-6]$ export WM_NCOMPPROCS=16

Boost♯

Looks like Boost 1_67_0 may be too new.

GMP & MPFR♯

NOTE The makeCGAL script does not seem to build GMP and MPFR.

libquadmath -- for some reason, the gcc/4.8.1 module provided by Bright has a 32-bit libquadmath, which MPFR complains about when running the checks after building.

boost -- and if you unload gcc/4.8.1 you can't use the Proteus cmake/3.9.4 because that requires libraries from gcc/4.8.1

So,make sure to unload the gcc module to pick up the 64-bit library in /usr/lib64/libquadmath.so

export GMP_ARCH_PATH=${WM_THIRD_PARTY_DIR}/platforms/linux64Icc/gmp-6.1.2
export MPFR_ARCH_PATH=${WM_THIRD_PARTY_DIR}/platforms/linux64Icc/mpfr-4.0.1

Build and install GMP and MPFR using the ARCH_PATH directories above as the respective prefix directories.

For GMP:

[juser@proteusi01 gmp-6.1.2]$ ./configure --prefix=$GMP_ARCH_PATH

For MPFR:

[juser@proteusi01 mpfr-4.0.1]$ ./configure --prefix=$MPFR_ARCH_PATH --with-gmp=${GMP_ARCH_PATH}

Build CGAL♯

Then, build:

./makeCGAL CGAL-4.11.2 boost_1_67_0 gmp-6.1.2 mpfr-4.0.1 >& Make_CGAL.out &

See Also♯