Compiling OpenFOAM.org 10 and dev

COMPILE GUIDE IN PROGRESS

Modulefiles

Load

• cmake
• picotte-openmpi/intel/2020/4.1.4
• cgal/5.5.2

Edit OpenFOAM-dev/etc/bashrc

Set compiler line 65:

export WM_COMPILER=Icc

Edit OpenFOAM-dev/etc/config.sh/settings

Set to use Intel icc/icpc, line 51ff:

    x86_64)
        case "$WM_ARCH_OPTION" in
        32)
            ...
        64)
            WM_ARCH=linux64
            export WM_COMPILER_LIB_ARCH=64
            export WM_CC='icc'
            export WM_CXX='icpc'
            export WM_CFLAGS='-xHost -no-prec-div -m64 -fPIC'
            export WM_CXXFLAGS='-xHost -no-prec-div -m64 -fPIC -std=c+=11'

Environment after sourcing OpenFOAM setup bashrc

Environment setup /ifs/opt/src/OpenFOAM/OpenFOAM-dev/etc/bashrc sets these:

FOAM_APPBIN=/ifs/opt/src/OpenFOAM-org/OpenFOAM-dev/platforms/linux64IccDPInt32Opt/bin
FOAM_APP=/ifs/opt/src/OpenFOAM-org/OpenFOAM-dev/applications
FOAM_ETC=/ifs/opt/src/OpenFOAM-org/OpenFOAM-dev/etc
FOAM_EXT_LIBBIN=/ifs/opt/src/OpenFOAM-org/ThirdParty-dev/platforms/linux64IccDPInt32/lib
FOAM_INST_DIR=/ifs/opt/src/OpenFOAM-org
FOAM_JOB_DIR=/ifs/opt/src/OpenFOAM-org/jobControl
FOAM_LIBBIN=/ifs/opt/src/OpenFOAM-org/OpenFOAM-dev/platforms/linux64IccDPInt32Opt/lib
FOAM_MPI=openmpi-system
FOAM_RUN=/home/swbuild/OpenFOAM/swbuild-dev/run
FOAM_SETTINGS=
FOAM_SIGFPE=
FOAM_SITE_APPBIN=/ifs/opt/src/OpenFOAM-org/site/dev/platforms/linux64IccDPInt32Opt/bin
FOAM_SITE_LIBBIN=/ifs/opt/src/OpenFOAM-org/site/dev/platforms/linux64IccDPInt32Opt/lib
FOAM_SOLVERS=/ifs/opt/src/OpenFOAM-org/OpenFOAM-dev/applications/solvers
FOAM_SRC=/ifs/opt/src/OpenFOAM-org/OpenFOAM-dev/src
FOAM_TUTORIALS=/ifs/opt/src/OpenFOAM-org/OpenFOAM-dev/tutorials
FOAM_USER_APPBIN=/home/swbuild/OpenFOAM/swbuild-dev/platforms/linux64IccDPInt32Opt/bin
FOAM_USER_LIBBIN=/home/swbuild/OpenFOAM/swbuild-dev/platforms/linux64IccDPInt32Opt/lib
FOAM_UTILITIES=/ifs/opt/src/OpenFOAM-org/OpenFOAM-dev/applications/utilities
LD_LIBRARY_PATH=/ifs/opt/src/OpenFOAM-org/ThirdParty-dev/platforms/linux64Icc/gperftools-svn/lib:/ifs/opt/src/OpenFOAM-org/OpenFOAM-dev/platforms/linux64IccDPInt32Opt/lib/openmpi-system:/ifs/opt/src/OpenFOAM-org/ThirdParty-dev/platforms/linux64IccDPInt32/lib/openmpi-system:/ifs/opt/openmpi/intel/2020/4.1.4/lib:/home/swbuild/OpenFOAM/swbuild-dev/platforms/linux64IccDPInt32Opt/lib:/ifs/opt/src/OpenFOAM-org/site/dev/platforms/linux64IccDPInt32Opt/lib:/ifs/opt/src/OpenFOAM-org/OpenFOAM-dev/platforms/linux64IccDPInt32Opt/lib:/ifs/opt/src/OpenFOAM-org/ThirdParty-dev/platforms/linux64IccDPInt32/lib:/ifs/opt/src/OpenFOAM-org/OpenFOAM-dev/platforms/linux64IccDPInt32Opt/lib/dummy:/ifs/opt/intel/2020/compilers_and_libraries_2020.4.304/linux/compiler/lib/intel64_lin:/ifs/opt/intel/2020/compilers_and_libraries_2020.4.304/linux/ipp/lib/intel64:/ifs/opt/intel/2020/compilers_and_libraries_2020.4.304/linux/mkl/lib/intel64_lin:/ifs/opt/intel/2020/compilers_and_libraries_2020.4.304/linux/tbb/lib/intel64_lin/gcc4.8:/ifs/opt/intel/2020/debugger_2020/libipt/intel64/lib:/ifs/opt/intel/2020/compilers_and_libraries_2020.4.304/linux/daal/lib/intel64_lin:/ifs/opt/intel/2020/compilers_and_libraries_2020.4.304/linux/daal/../tbb/lib/intel64_lin/gcc4.8:/ifs/opt/ucx/1.14.0-rc6/lib:/cm/shared/apps/slurm/current/lib64/slurm:/cm/shared/apps/slurm/current/lib64:/cm/local/apps/gcc/9.2.0/lib:/cm/local/apps/gcc/9.2.0/lib64
OLDPWD=/ifs/opt/src/OpenFOAM-org/ThirdParty-dev/scotch_6.0.9
ParaView_DIR=/ifs/opt/src/OpenFOAM-org/ThirdParty-dev/platforms/linux64Icc/ParaView-5.10.1
PATH=/ifs/opt/src/OpenFOAM-org/ThirdParty-dev/platforms/linux64Icc/gperftools-svn/bin:/ifs/opt/src/OpenFOAM-org/ThirdParty-dev/platforms/linux64Icc/cmake-*/bin:/ifs/opt/src/OpenFOAM-org/OpenFOAM-dev/bin:/ifs/opt/src/OpenFOAM-org/OpenFOAM-dev/wmake:/home/swbuild/OpenFOAM/swbuild-dev/platforms/linux64IccDPInt32Opt/bin:/ifs/opt/src/OpenFOAM-org/site/dev/platforms/linux64IccDPInt32Opt/bin:/ifs/opt/src/OpenFOAM-org/OpenFOAM-dev/platforms/linux64IccDPInt32Opt/bin:/ifs/opt/openmpi/intel/2020/4.1.4/bin:/opt/mellanox/hpcx-v2.13-gcc-MLNX_OFED_LINUX-5-redhat8-cuda11-gdrcopy2-nccl2.12-x86_64/hcoll/bin:/opt/knem-1.1.3.90mlnx1/sbin:/opt/knem-1.1.3.90mlnx1/bin:/ifs/opt/intel/2020/compilers_and_libraries_2020.4.304/linux/bin/intel64:/ifs/opt/intel/2020/debugger_2020/gdb/intel64/bin:/ifs/opt/intel/2020/intelpython3/bin/libfabric:/ifs/opt/intel/2020/intelpython3/bin:/ifs/opt/intel/2020/intelpython2/bin:/ifs/opt/git/2.39.2/bin:/ifs/opt/cmake/3.25.3/bin:/ifs/opt/go/1.19/bin:/ifs/opt/vim/9.0.0902/bin:/cm/shared/apps/slurm/current/sbin:/cm/shared/apps/slurm/current/bin:/cm/local/apps/gcc/9.2.0/bin:/home/swbuild/.local/bin:/home/swbuild/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/dell/srvadmin/sbin:/home/swbuild/go/bin
PWD=/ifs/opt/src/OpenFOAM-org/ThirdParty-dev
WM_DIR=/ifs/opt/src/OpenFOAM-org/OpenFOAM-dev/wmake
WM_PROJECT_DIR=/ifs/opt/src/OpenFOAM-org/OpenFOAM-dev
WM_PROJECT_INST_DIR=/ifs/opt/src/OpenFOAM-org
WM_PROJECT=OpenFOAM
WM_PROJECT_USER_DIR=/home/swbuild/OpenFOAM/swbuild-dev
WM_THIRD_PARTY_DIR=/ifs/opt/src/OpenFOAM-org/ThirdParty-dev

ParaView_DIR=/ifs/opt/src/OpenFOAM-org/ThirdParty-dev/platforms/linux64Icc/ParaView-5.10.1
ParaView_GL=mesa
ParaView_MAJOR=5.10
ParaView_VERSION=5.10.1

Build Scotch/PT-Scotch

Edit Makefile.inc to use Intel compiler icc: ThirdParty-dev/scotch-6.0.9/src/Makefile.inc (which links to ThirdParty-dev/etc/wmakefiles/scotch/Makefile.inc.i686_pc_linux2.shliub-OpenFOAM).

EXE             =
LIB             = .so
OBJ             = .o

MAKE            = make
AR              = icc
ARFLAGS         = $(WM_CFLAGS) -shared -o
CAT             = cat
CCS             = icc
CCP             = mpicc
CCD             = mpicc
CFLAGS          = $(WM_CFLAGS) -O3 -DCOMMON_FILE_COMPRESS_GZ -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict=__restrict

ifeq ($(WM_LABEL_OPTION),Int64)
CFLAGS          += -DINTSIZE64
endif

CLIBFLAGS       = -shared
LDFLAGS         = -Xlinker --no-as-needed $(WM_LDFLAGS) -lz -lm -lrt
CP              = cp
LEX             = flex -Pscotchyy -olex.yy.c
LN              = ln
MKDIR           = mkdir
MV              = mv
RANLIB          = echo
YACC            = bison -pscotchyy -y -b y

cd ThirdParty-10
./Allwmake >& Make.all.out &

Build Paraview

Will download source tarball if not already there.

• Specify path to Qt5 qmake.
• Force use of GCC.

cd ThirdParty-dev
./makeParaView -qmake `which qmake-qt5` -gcc -verbose >& Make.paraview.out &

StreamingParticles build fails. However, going into the directory itself works OK.

cd ThirdParty-10/build/linux64Icc/ParaView-5.6.3/Plugins/StreamingParticles
make -j 32

Still no dice with GCC:

[ 90%] Built target pqCore
make[1]: Leaving directory '/ifs/opt/src/OpenFOAM/ThirdParty-10/build/linux64Icc/ParaView-5.6.3'
make: *** [Makefile:155: all] Error 2

No dice with OpenFOAM-dev which uses ParaView 5.10.1 because it requires a newer Qt:

CMake Warning at VTK/CMake/vtkModule.cmake:4365 (find_package):
  Could not find a configuration file for package "Qt5" that is compatible
  with requested version "5.12".

  The following configuration files were considered but not accepted:

    /usr/lib64/cmake/Qt5/Qt5Config.cmake, version: 5.11.1
    /lib64/cmake/Qt5/Qt5Config.cmake, version: 5.11.1

Call Stack (most recent call first):
  Qt/Core/CMakeLists.txt:168 (vtk_module_find_package)


CMake Error at VTK/CMake/vtkModule.cmake:4371 (message):
  Could not find the Qt5 external dependency.
Call Stack (most recent call first):
  Qt/Core/CMakeLists.txt:168 (vtk_module_find_package)

Build OpenFOAM

Use 32 make threads.

export WM_NCOMPPROCS=32

See Also

• Intel Developer articles - Recipe: Compile OpenFOAM* with Intel® Compiler for Intel® Xeon® Scalable Processors

References