Skip to content

Compiling CMAQ New

WORK IN PROGRESS

Official Installation Documentation♯

Modules to Load♯

  • gcc/7.4.0
  • intel/composerxe/2019u1
  • cmake
  • proteus-openmpi/intel/2019/3.1.4

Recipe♯

  • cd src/CMAQ-5.2.1
  • edit bldit_project.csh: set CMAQ_HOME = $HOME/CMAQ_Project
  • source bldit_project.csh -- creates $CMAQ_HOME
  • cd $CMAQ_HOME and edit config_cmaq.csh:
    • setenv myFC mpifort # (i.e. remove the extra "i")
    • setenv myFSTD "-O3 -xHost -fno-alias -mp1 -fp-model source"
    • setenv myCFLAGS "-O3 -xHost"

Other Packages♯

These need to be all built as static libraries, and then installed into their location in the CMAQ tree.

  1. Well, static libraries seem to have issues. Particularly, NetCDF-Fortran seems to have trouble linking with NetCDF-C
  2. And dynamic libraries also seem to have issues because CMAQ is expecting only static libraries. But the MPI libraries are all dynamic, and if HDF5+NetCDF were built to support parallel, they implicitly want the MPI libs.
  3. NetCDF depends on HDF5. So, HDF5 will be installed into the NetCDF directory.
  4. NetCDF needs a static libcurl - install libcurl privately.
  5. 3 conflicts with 1
  6. Can't use hdf5 module because it sets HDF5_ROOT

curl 7.64.1♯

HDF5 1.10.4♯

  • setenv PATH /home/dwc62/CMAQ_Project/lib/x86_64/intel/netcdf/bin:${PATH}
  • Depends on zlib and szip
    • which means these two libraries also have to be compiled statically and then installed in the NetCDF directory.
    • or may be able to use existing installation of these libs
  • issues with using cmake => use configure
  • setenv CC `which mpicc`
  • setenv FC `which mpifort`
  • ~~./configure --prefix=/home/dwc62/CMAQ_Project/lib/x86_64/intel/netcdf --enable-parallel --enable-fortran --disable-shared --enable-build-mode=production --with-zlib=/home/dwc62/CMAQ_Project/lib/x86_64/intel/netcdf --with-szlib=/home/dwc62/CMAQ_Project/lib/x86_64/intel/netcdf~~
  • Edit config.status to adjust link line:
    • S["build_alias"]=""
    • S["LIBS"]="-lrt /home/dwc62/CMAQ_Project/lib/x86_64/intel/netcdf/lib/libsz.a /home/dwc62/CMAQ_Project/lib/x86_64/intel/netcdf/lib/libz.a -ldl -lm "
  • module load zlib/cloudflare/intel/2019 szip/intel/2019
  • ./configure --prefix=/home/dwc62/CMAQ_Project/lib/x86_64/intel/netcdf --with-zlib=\(ZLIBHOME --with-szlib=\)SZIPHOME --enable-hl --enable-shared --enable-static --enable-build-mode=production --enable-parallel
  • Edit config.status to adjust:
    • S["AM_LDFLAGS"]=" -L/mnt/HA/opt_rh68/zlib/cloudflare/intel/2019/1.2.8/lib -L/mnt/HA/opt_rh68/szip/intel/2019/2.1.1/lib -Wl,-rpath,/mnt/HA/opt_rh68/zlib/cloudflare/intel/2019/1.2. 8/lib -Wl,-rpath,/mnt/HA/opt_rh68/szip/intel/2019/2.1.1/lib"
  • run config.status to regenerate makefiles

pNetCDF 1.11.2♯

  • DON'T INSTALL
  • Not part of NetCDF project; independently-developed code for using NetCDF-format files
  • NetCDF cmake build has option to use this; so build this first
  • See: https://parallel-netcdf.github.io
  • Might want libxml2 (else it picks up SGE's libxml2)
  • setenv TESTOUTDIR /lustre/scratch/dwc62/tmp
  • configure:

./configure --prefix=${PNETCDF_DIR} --enable-shared --enable-static --enable-largefile --enable-cxx --enable-fortran --enable-large-file-test --enable-subfiling

NetCDF 4.5.0♯

  • DON'T USE CMAKE
  • Using CMake - more reliable and able to set individual paths for libraries; but seems to be broken
  • Parallel
    • mpi-related stuff with cmake seems to be broken
    • mpi-related stuff with configure seems to be broken
    • build hdf5 serial and netcdf serial
    • in cmake, set CMAKE_C_FLAGS=-DHDF5_PARALLEL=1
      • BUT all parallel netcdf tests (except for a few trivial N=1 tests) fail
      • Disabling parallel => passes all tests
  • DO NOT use pnetcdf
  • CPPFLAGS=-I/mnt/HA/opt_rh68/zlib/cloudflare/intel/2019/1.2.8/include -I/mnt/HA/opt_rh68/szip/intel/2019/2.1.1/include -I/home/dwc62/CMAQ_Project/lib/x86_64/intel/netcdf/include
  • LDFLAGS=-L/mnt/HA/opt_rh68/zlib/cloudflare/intel/2019/1.2.8/lib -Wl,-rpath,/mnt/HA/opt_rh68/zlib/cloudflare/intel/2019/1.2.8/lib -L/mnt/HA/opt_rh68/szip/intel/2019/2.1.1/lib -Wl,-rpath,/mnt/HA/opt_rh68/szip/intel/2019/2.1.1/lib -L/home/dwc62/CMAQ_Project/lib/x86_64/intel/netcdf/lib
  • ~~LIBS=/mnt/HA/opt_rh68/zlib/cloudflare/intel/2019/1.2.8/lib/libz.a /mnt/HA/opt_rh68/szip/intel/2019/2.1.1/lib/libsz.a~~

  • ./configure --prefix=/home/dwc62/CMAQ_Project/lib/x86_64/intel/netcdf --enable-cdf5 --enable-examples --enable-shared --enable-static --enable-largefile --enable-diskless --with-temp-large=/local/scratch/508813.1.new.q --enable-mmap --enable-pnetcdf

  • Use configure

  • Set CPPFLAGS, LDFLAGS, LIBS
    • CPPFLAGS = -I/home/dwc62/CMAQ_Project/lib/x86_64/intel/netcdf/include -I/mnt/HA/opt_rh68/zlib/cloudflare/intel/2019/1.2.8/include -I/usr/local/include
    • LDFLAGS = -L/home/dwc62/CMAQ_Project/lib/x86_64/intel/netcdf/lib -Wl,-rpath,/home/dwc62/CMAQ_Project/lib/x86_64/intel/netcdf/lib -L/mnt/HA/opt_rh68/zlib/cloudflare/intel/2019/1.2.8/lib -Wl,-rpath,/mnt/HA/opt_rh68/zlib/cloudflare/intel/2019/1.2.8/lib -L/usr/local/lib -Wl,-rpath,/usr/local/lib -L/usr/local/lib64 -Wl,-rpath,/usr/local/lib64
  • set CC=mpicc
  • ./configure --prefix=/home/dwc62/CMAQ_Project/lib/x86_64/intel/netcdf --disable-shared --enable-static --enable-cdf5 --enable-largefile --enable-mmap --enable-parallel4 --disable-pnetcdf --with-temp-large=/lustre/scratch/dwc62/tmp

NetCDF Fortran 4.4.5♯

  • Expected by IOAPI
    • produces libnetcdff.a
  • Use cmake
  • Disable shared
  • Edit CMakeLists.txt
    • Comment out stanza starting with "# Determin C library setting for NC_HAS_CDF5"
  • No dice - libnetcdf has a bunch of references to stuff in libcurl (which is static, though there is a shared lib, too) and libhdf5 (which is also static + shared)

Configure?

  • FC = `which mpifort`
  • No dice - use cmake

IOAPI 3.2♯

  • foo
  • bar

See Also♯