Building Ascent¶
This page provides details on several ways to build Ascent from source.
For the shortest path from zero to Ascent, see Quick Start.
To build third party dependencies we recommend using build_ascent.sh, uberenv which leverages Spack, or Spack directly. We also provide a Docker example that leverages Spack.
Overview¶
Ascent uses CMake for its build system. Building Ascent creates two separate libraries:
libascent : a version for execution on a single node
libascent_mpi : a version for distributed-memory parallelism
The CMake variable( ENABLE_MPI ON | OFF ) controls the building the parallel version of Ascent and included proxy-apps.
The build dependencies vary according to which pipelines and proxy-applications are desired. For a minimal build with no parallel components, the following are required:
Conduit
C++ compilers
We recognize that building on HPC systems can be difficult, and we provide three build strategies.
A build_ascent.sh helper script
A spack based build
Manually compile dependencies using a CMake configuration file to keep compilers and libraries consistent
Most often, build_ascent or spack based build should be attempted first. These both automatically download and build all the third party dependencies and create a CMake configuration file for Ascent. Should you encounter build issues that are not addressed here, please ask questions using our github issue tracker.
Build Dependencies¶
Ascent requires Conduit and provides optional features that depend on third-party libraries:
Feature |
Required TPLS |
---|---|
VTK-h Rendering and Filtering Pipelines |
VTk-m (Serial, OpenMP, CUDA, Kokkos) |
MFEM High-Order to Low-Order Refinement for VTK-h Pipelines |
MFEM |
Devil Ray High-Order Ray Tracer Pipelines |
RAJA (Serial, OpenMP, CUDA, HIP), Umpire, MFEM |
General Expressions |
RAJA (Serial, OpenMP, CUDA, HIP), Umpire |
JIT Expressions |
OCCA, Umpire |
For a detailed account of features and what underpin them see Ascent Feature Map.
Getting Started¶
Clone the Ascent repo:
From Github
git clone --recursive https://github.com/Alpine-DAV/ascent.git
--recursive
is necessary because we are using a git submodule to pull in BLT (https://github.com/llnl/blt).
If you cloned without --recursive
, you can checkout this submodule using:
cd ascent
git submodule init
git submodule update
Configure a build:
config-build.sh
is a simple wrapper for the cmake call to configure ascent.
This creates a new out-of-source build directory build-debug
and a directory for the install install-debug
.
It optionally includes a host-config.cmake
file with detailed configuration options.
cd ascent
./config-build.sh
Build, test, and install Ascent:
cd build-debug
make -j 8
make test
make install
Build Options¶
Ascent’s build system supports the following CMake options:
Main CMake Options¶
Option |
Description |
Default |
---|---|---|
|
Controls if shared (ON) or static (OFF) libraries are built. |
(default = ON) |
|
Controls if Fortran components of Ascent are built. This includes the Fortran language bindings and Cloverleaf3D. |
(default = ON) |
|
Controls if the Ascent Python module and related tests are built. |
(default = OFF) |
|
Controls if MPI parallel versions of Ascent and proxy-apps are built. |
(default = ON) |
|
Controls if Serial (non-MPI) version of Ascent and proxy-apps are built. |
(default = ON) |
|
Controls if Ascent uses CUDA. |
(default = OFF) |
|
Controls if the proxy-apps and Ascent use with OpenMP. |
(default = OFF) |
|
Controls if Devil Ray is built. Requires RAJA + Umpire. (Devil Ray is now developed as part of Ascent) |
(default = OFF) |
|
Controls if AP Compositor is built. (AP Compositor is now developed as part of Ascent) |
(default = OFF) |
|
Controls if VTK-h is built.. Requires VTK-m. (VTK-h is now developed as part of Ascent) |
(default = OFF) |
|
Controls if Ascent examples are built. |
(default = ON) |
|
Controls if Ascent utilities are built. |
(default = ON) |
|
Controls if Ascent tests are built. |
(default = ON) |
|
Controls if data logging is built. |
(default = ON) |
|
Controls if the Ascent documentation is built (when sphinx is available). |
(default = ON) |
(Devil Ray Specific Options) |
||
|
Controls if Devil Ray Status support is built. |
(default = ON) |
|
Controls if Devil Ray is built with 64-bit floats |
(default = OFF, use 32-bit precision floats) |
CMake Options for Third-party Library Paths¶
Name |
Description |
---|---|
|
Path to a Conduit install (required) |
|
Path to a Caliper install (optional) |
|
Path to an Adiak install (optional) Caliper support requires Adiak. |
|
Path to a RAJA install (optional) |
|
Path to a Umpire install (optional) |
|
Path to an OCCA install (optional) |
|
Path to a VTK-m install (optional) |
|
Path to a Kokkos install (optional) |
|
Path to a ADIOS 2 install (optional) |
|
Path to a FIDES install (optional) |
|
Path to a BabelFlow install (optional) |
|
Path to a ParallelMergeTree install (optional) |
|
Path to a StreamStat install (optional) |
|
Path to a TopoFileParser install (optional) |
|
Path to a BLT install (default = |
Additional Build Notes¶
Python - The Ascent Python module builds for both Python 2 and Python 3. To select a specific Python, set the CMake variable PYTHON_EXECUTABLE to path of the desired python binary. The Ascent Python module requires the Conduit Python module.
MPI - We use CMake’s standard FindMPI logic. To select a specific MPI set the CMake variables
MPI_C_COMPILER
andMPI_CXX_COMPILER
, or the other FindMPI options for MPI include paths and MPI libraries. To run the mpi unit tests, you may also need change the CMake variablesMPIEXEC_EXECUTABLE
andMPIEXEC_NUMPROC_FLAG
, so you can use a different launcher, such as srun and set number of MPI tasks used.BLT - Ascent uses BLT (https://github.com/llnl/blt) as the foundation of its CMake-based build system. It is included as a submodule in Ascent’s git repo, and also available in our release tarballs. The
BLT_SOURCE_DIR
CMake option defaults tosrc/blt
, where we expect the blt submodule. The most compelling reason to override is to share a single instance of BLT across multiple projects.
Host Config Files¶
To handle build options, third party library paths, etc we rely on CMake’s initial-cache file mechanism.
cmake -C config_file.cmake
We call these initial-cache files host-config files, since we typically create a file for each platform or specific hosts if necessary.
The config-build.sh
script uses your machine’s hostname, the SYS_TYPE environment variable, and your platform name (via uname) to look for an existing host config file in the host-configs
directory at the root of the ascent repo. If found, it passes the host config file to CMake via the -C command line option.
cmake {other options} -C host-configs/{config_file}.cmake ../
You can find example files in the host-configs
directory.
These files use standard CMake commands. CMake set commands need to specify the root cache path as follows:
set(CMAKE_VARIABLE_NAME {VALUE} CACHE PATH "")
It is possible to create your own configure file, and an boilerplate example is provided in /host-configs/boilerplate.cmake
Warning
If compiling all of the dependencies yourself, it is important that you use the same compilers for all dependencies. For example, different MPI and Fortran compilers (e.g., Intel and GCC) are not compatible with one another.
Building Ascent and Third Party Dependencies¶
build_ascent¶
build_ascent.sh demonstrates how to build Ascent and its main dependencies:
#!/bin/bash
##############################################################################
# Demonstrates how to manually build Ascent and its dependencies, including:
#
# hdf5, conduit, vtk-m, mfem, raja, and umpire
#
# usage example:
# env enable_mpi=ON enable_openmp=ON ./build_ascent.sh
#
#
# Assumes:
# - cmake is in your path
# - selected compilers are in your path or set via env vars
# - [when enabled] MPI and Python (+numpy and mpi4py), are in your path
#
##############################################################################
set -eu -o pipefail
##############################################################################
# Build Options
##############################################################################
# shared options
enable_cuda="${enable_cuda:=OFF}"
enable_hip="${enable_hip:=OFF}"
enable_sycl="${enable_sycl:=OFF}"
enable_fortran="${enable_fortran:=OFF}"
enable_python="${enable_python:=OFF}"
enable_openmp="${enable_openmp:=OFF}"
enable_mpi="${enable_mpi:=OFF}"
enable_mpicc="${enable_mpicc:=OFF}"
enable_find_mpi="${enable_find_mpi:=ON}"
enable_tests="${enable_tests:=OFF}"
enable_verbose="${enable_verbose:=ON}"
build_jobs="${build_jobs:=6}"
build_config="${build_config:=Release}"
build_shared_libs="${build_shared_libs:=ON}"
# tpl controls
build_zlib="${build_zlib:=true}"
build_hdf5="${build_hdf5:=true}"
build_pyvenv="${build_pyvenv:=false}"
build_caliper="${build_caliper:=false}"
build_silo="${build_silo:=true}"
build_conduit="${build_conduit:=true}"
build_vtkm="${build_vtkm:=true}"
build_camp="${build_camp:=true}"
build_raja="${build_raja:=true}"
build_umpire="${build_umpire:=true}"
build_mfem="${build_mfem:=true}"
build_catalyst="${build_catalyst:=false}"
# ascent options
build_ascent="${build_ascent:=true}"
# see if we are building on windows
build_windows="${build_windows:=OFF}"
# see if we are building on macOS
build_macos="${build_macos:=OFF}"
if [[ "$enable_cuda" == "ON" ]]; then
echo "*** configuring with CUDA support"
CC="${CC:=gcc}"
CXX="${CXX:=g++}"
FTN="${FTN:=gfortran}"
CUDA_ARCH="${CUDA_ARCH:=80}"
CUDA_ARCH_VTKM="${CUDA_ARCH_VTKM:=ampere}"
fi
# NOTE: this script only builds kokkos when enable_hip=ON or enable_cycl=ON
if [[ "$enable_hip" == "ON" ]]; then
echo "*** configuring with HIP support"
CC="${CC:=/opt/rocm/llvm/bin/amdclang}"
CXX="${CXX:=/opt/rocm/llvm/bin/amdclang++}"
# FTN?
ROCM_ARCH="${ROCM_ARCH:=gfx90a}"
ROCM_PATH="${ROCM_PATH:=/opt/rocm/}"
build_kokkos="${build_kokkos:=true}"
elif [[ "$enable_sycl" == "ON" ]]; then
echo "*** configuring with SYCL support"
build_kokkos="${build_kokkos:=true}"
else
build_kokkos="${build_kokkos:=false}"
fi
case "$OSTYPE" in
win*) build_windows="ON";;
msys*) build_windows="ON";;
darwin*) build_macos="ON";;
*) ;;
esac
if [[ "$build_windows" == "ON" ]]; then
echo "*** configuring for windows"
fi
if [[ "$build_macos" == "ON" ]]; then
echo "*** configuring for macos"
fi
################
# path helpers
################
function ospath()
{
if [[ "$build_windows" == "ON" ]]; then
echo `cygpath -m $1`
else
echo $1
fi
}
function abs_path()
{
if [[ "$build_macos" == "ON" ]]; then
echo "$(cd $(dirname "$1");pwd)/$(basename "$1")"
else
echo `realpath $1`
fi
}
root_dir=$(pwd)
root_dir="${prefix:=${root_dir}}"
root_dir=$(ospath ${root_dir})
root_dir=$(abs_path ${root_dir})
script_dir=$(abs_path "$(dirname "${BASH_SOURCE[0]}")")
build_dir=$(ospath ${root_dir}/build)
source_dir=$(ospath ${root_dir}/source)
# root_dir is where we will build and install
# override with `prefix` env var
if [ ! -d ${root_dir} ]; then
mkdir -p ${root_dir}
fi
cd ${root_dir}
# install_dir is where we will install
# override with `prefix` env var
install_dir="${install_dir:=$root_dir/install}"
echo "*** prefix: ${root_dir}"
echo "*** build root: ${build_dir}"
echo "*** sources root: ${source_dir}"
echo "*** install root: ${install_dir}"
echo "*** script dir: ${script_dir}"
################
# tar options
################
tar_extra_args=""
if [[ "$build_windows" == "ON" ]]; then
tar_extra_args="--force-local"
fi
# make sure sources dir exists
if [ ! -d ${source_dir} ]; then
mkdir -p ${source_dir}
fi
################
# CMake Compiler Settings
################
cmake_compiler_settings=""
# capture compilers if they are provided via env vars
if [ ! -z ${CC+x} ]; then
cmake_compiler_settings="-DCMAKE_C_COMPILER:PATH=${CC}"
fi
if [ ! -z ${CXX+x} ]; then
cmake_compiler_settings="${cmake_compiler_settings} -DCMAKE_CXX_COMPILER:PATH=${CXX}"
fi
if [ ! -z ${FTN+x} ]; then
cmake_compiler_settings="${cmake_compiler_settings} -DCMAKE_Fortran_COMPILER:PATH=${FTN}"
fi
############################
# mpi related vars
############################
mpicc_exe="${mpicc_exe:=mpicc}"
mpicxx_exe="${mpicxx_exe:=mpic++}"
################
# print all build_ZZZ and enable_ZZZ options
################
echo "*** cmake_compiler_settings: ${cmake_compiler_settings}"
echo "*** build_ascent `enable` settings:"
set | grep enable_
echo "*** build_ascent `build` settings:"
set | grep build_
################
# Zlib
################
zlib_version=1.3.1
zlib_src_dir=$(ospath ${source_dir}/zlib-${zlib_version})
zlib_build_dir=$(ospath ${build_dir}/zlib-${zlib_version}/)
zlib_install_dir=$(ospath ${install_dir}/zlib-${zlib_version}/)
zlib_tarball=$(ospath ${source_dir}/zlib-${zlib_version}.tar.gz)
# build only if install doesn't exist
if [ ! -d ${zlib_install_dir} ]; then
if ${build_zlib}; then
if [ ! -d ${zlib_src_dir} ]; then
echo "**** Downloading ${zlib_tarball}"
curl -L https://github.com/madler/zlib/releases/download/v${zlib_version}/zlib-${zlib_version}.tar.gz -o ${zlib_tarball}
tar ${tar_extra_args} -xzf ${zlib_tarball} -C ${source_dir}
fi
echo "**** Configuring Zlib ${zlib_version}"
cmake -S ${zlib_src_dir} -B ${zlib_build_dir} ${cmake_compiler_settings} \
-DCMAKE_VERBOSE_MAKEFILE:BOOL=${enable_verbose} \
-DCMAKE_BUILD_TYPE=${build_config} \
-DCMAKE_INSTALL_PREFIX=${zlib_install_dir}
echo "**** Building Zlib ${zlib_version}"
cmake --build ${zlib_build_dir} --config ${build_config} -j${build_jobs}
echo "**** Installing Zlib ${zlib_version}"
cmake --install ${zlib_build_dir} --config ${build_config}
fi
else
echo "**** Skipping Zlib build, install found at: ${zlib_install_dir}"
fi # build_zlib
################
# HDF5
################
# release 1-2 GAH!
hdf5_version=1.14.1-2
hdf5_middle_version=1.14.1
hdf5_short_version=1.14
hdf5_src_dir=$(ospath ${source_dir}/hdf5-${hdf5_version})
hdf5_build_dir=$(ospath ${build_dir}/hdf5-${hdf5_version}/)
hdf5_install_dir=$(ospath ${install_dir}/hdf5-${hdf5_version}/)
hdf5_tarball=$(ospath ${source_dir}/hdf5-${hdf5_version}.tar.gz)
# build only if install doesn't exist
if [ ! -d ${hdf5_install_dir} ]; then
if ${build_hdf5}; then
if [ ! -d ${hdf5_src_dir} ]; then
echo "**** Downloading ${hdf5_tarball}"
curl -L https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-${hdf5_short_version}/hdf5-${hdf5_middle_version}/src/hdf5-${hdf5_version}.tar.gz -o ${hdf5_tarball}
tar ${tar_extra_args} -xzf ${hdf5_tarball} -C ${source_dir}
fi
#################
#
# hdf5 1.14.x CMake recipe for using zlib
#
# -DHDF5_ENABLE_Z_LIB_SUPPORT=ON
# Add zlib install dir to CMAKE_PREFIX_PATH
#
#################
echo "**** Configuring HDF5 ${hdf5_version}"
cmake -S ${hdf5_src_dir} -B ${hdf5_build_dir} ${cmake_compiler_settings} \
-DCMAKE_VERBOSE_MAKEFILE:BOOL=${enable_verbose} \
-DCMAKE_BUILD_TYPE=${build_config} \
-DHDF5_ENABLE_Z_LIB_SUPPORT=ON \
-DCMAKE_PREFIX_PATH=${zlib_install_dir} \
-DCMAKE_INSTALL_PREFIX=${hdf5_install_dir}
echo "**** Building HDF5 ${hdf5_version}"
cmake --build ${hdf5_build_dir} --config ${build_config} -j${build_jobs}
echo "**** Installing HDF5 ${hdf5_version}"
cmake --install ${hdf5_build_dir} --config ${build_config}
fi
else
echo "**** Skipping HDF5 build, install found at: ${hdf5_install_dir}"
fi # build_hdf5
################
# Silo
################
silo_version=4.11.1
silo_src_dir=$(ospath ${source_dir}/Silo-${silo_version})
silo_build_dir=$(ospath ${build_dir}/silo-${silo_version}/)
silo_install_dir=$(ospath ${install_dir}/silo-${silo_version}/)
silo_tarball=$(ospath ${source_dir}/silo-${silo_version}.tar.gz)
# build only if install doesn't exist
if [ ! -d ${silo_install_dir} ]; then
if ${build_silo}; then
if [ ! -d ${silo_src_dir} ]; then
echo "**** Downloading ${silo_tarball}"
curl -L https://github.com/LLNL/Silo/archive/refs/tags/${silo_version}.tar.gz -o ${silo_tarball}
# untar and avoid symlinks (which windows despises)
tar ${tar_extra_args} -xzf ${silo_tarball} -C ${source_dir} \
--exclude="Silo-${silo_version}/config-site/*" \
--exclude="Silo-${silo_version}/README.md"
# apply silo patches
cd ${silo_src_dir}
patch -p1 < ${script_dir}/2024_07_25_silo_4_11_cmake_fix.patch
# windows specifc patch
if [[ "$build_windows" == "ON" ]]; then
patch -p1 < ${script_dir}/2024_07_29_silo-pr389-win32-bugfix.patch
fi
cd ${root_dir}
fi
echo "**** Configuring Silo ${silo_version}"
cmake -S ${silo_src_dir} -B ${silo_build_dir} ${cmake_compiler_settings} \
-DCMAKE_VERBOSE_MAKEFILE:BOOL=${enable_verbose} \
-DCMAKE_BUILD_TYPE=${build_config} \
-DCMAKE_INSTALL_PREFIX=${silo_install_dir} \
-DSILO_ENABLE_SHARED=${build_shared_libs} \
-DCMAKE_C_FLAGS=-Doff64_t=off_t \
-DSILO_ENABLE_HDF5=ON \
-DSILO_ENABLE_TESTS=OFF \
-DSILO_BUILD_FOR_BSD_LICENSE=ON \
-DSILO_ENABLE_FORTRAN=OFF \
-DSILO_HDF5_DIR=${hdf5_install_dir}/cmake/ \
-DCMAKE_PREFIX_PATH=${zlib_install_dir}
echo "**** Building Silo ${silo_version}"
cmake --build ${silo_build_dir} --config ${build_config} -j${build_jobs}
echo "**** Installing Silo ${silo_version}"
cmake --install ${silo_build_dir} --config ${build_config}
fi
else
echo "**** Skipping Silo build, install found at: ${silo_install_dir}"
fi # build_silo
############################
# Python Virtual Env
############################
python_exe="${python_exe:=python3}"
venv_install_dir=$(ospath ${install_dir}/python-venv/)
venv_python_exe=$(ospath ${venv_install_dir}/bin/python3)
venv_sphinx_exe=$(ospath ${venv_install_dir}/bin/sphinx-build)
# build only if install doesn't exist
if [ ! -d ${venv_install_dir} ]; then
if ${build_pyvenv}; then
echo "**** Creating Python Virtual Env"
cd ${install_dir} && ${python_exe} -m venv python-venv
${venv_python_exe} -m pip install --upgrade pip
${venv_python_exe} -m pip install numpy sphinx sphinx_rtd_theme
if [[ "$enable_mpi" == "ON" ]]; then
${venv_python_exe} -m pip install mpi4py
fi
fi
else
echo "**** Skipping Python venv build, install found at: ${venv_install_dir}"
fi # build_pyvenv
if ${build_pyvenv}; then
venv_python_ver=`${venv_python_exe} -c "import sys;print('{0}.{1}'.format(sys.version_info.major, sys.version_info.minor))"`
venv_python_site_pkgs_dir=${venv_install_dir}/lib/python${venv_python_ver}/site-packages
fi
################
# Caliper
################
caliper_version=2.11.0
caliper_src_dir=$(ospath ${source_dir}/Caliper-${caliper_version})
caliper_build_dir=$(ospath ${build_dir}/caliper-${caliper_version}/)
caliper_install_dir=$(ospath ${install_dir}/caliper-${caliper_version}/)
caliper_tarball=$(ospath ${source_dir}/caliper-${caliper_version}-src-with-blt.tar.gz)
# build only if install doesn't exist
if [ ! -d ${caliper_install_dir} ]; then
if ${build_caliper}; then
if [ ! -d ${caliper_src_dir} ]; then
echo "**** Downloading ${caliper_tarball}"
curl -L https://github.com/LLNL/Caliper/archive/refs/tags/v${caliper_version}.tar.gz -o ${caliper_tarball}
tar ${tar_extra_args} -xzf ${caliper_tarball} -C ${source_dir}
# windows specifc patch
cd ${caliper_src_dir}
if [[ "$build_windows" == "ON" ]]; then
patch -p1 < ${script_dir}/2024_08_01_caliper-win-smaller-opts.patch
fi
cd ${root_dir}
fi
#
# Note: Caliper has optional Umpire support,
# if we want to support in the future, we will need to build umpire first
#
# -DWITH_CUPTI=ON -DWITH_NVTX=ON -DCUDA_TOOLKIT_ROOT_DIR={path} -DCUPTI_PREFIX={path}
# -DWITH_ROCTRACER=ON -DWITH_ROCTX=ON -DROCM_PREFIX={path}
caliper_windows_cmake_flags="-DCMAKE_CXX_STANDARD=17 -DCMAKE_WINDOWS_EXPORT_ALL_SYMBOLS=ON -DWITH_TOOLS=OFF"
caliper_extra_cmake_args=""
if [[ "$build_windows" == "ON" ]]; then
caliper_extra_cmake_args="${caliper_windows_cmake_flags}"
fi
# TODO enable_cuda
if [[ "$enable_hip" == "ON" ]]; then
caliper_extra_cmake_args="${caliper_extra_cmake_args} -DWITH_ROCTRACER=ON"
caliper_extra_cmake_args="${caliper_extra_cmake_args} -DWITH_ROCTX=ON"
caliper_extra_cmake_args="${caliper_extra_cmake_args} -DROCM_PREFIX:PATH=${ROCM_PATH}"
caliper_extra_cmake_args="${caliper_extra_cmake_args} -DROCM_ROOT_DIR:PATH=${ROCM_PATH}"
fi
if [[ "$enable_mpicc" == "ON" ]]; then
caliper_extra_cmake_args="${caliper_extra_cmake_args} -DMPI_C_COMPILER=${mpicc_exe}"
caliper_extra_cmake_args="${caliper_extra_cmake_args} -DMPI_CXX_COMPILER=${mpicxx_exe}"
fi
echo "**** Configuring Caliper ${caliper_version}"
cmake -S ${caliper_src_dir} -B ${caliper_build_dir} ${cmake_compiler_settings} \
-DCMAKE_VERBOSE_MAKEFILE:BOOL=${enable_verbose} \
-DCMAKE_BUILD_TYPE=${build_config} \
-DBUILD_SHARED_LIBS=${build_shared_libs} \
-DCMAKE_INSTALL_PREFIX=${caliper_install_dir} \
-DWITH_MPI=${enable_mpi} ${caliper_extra_cmake_args}
echo "**** Building Caliper ${caliper_version}"
cmake --build ${caliper_build_dir} --config ${build_config} -j${build_jobs}
echo "**** Installing Caliper ${caliper_version}"
cmake --install ${caliper_build_dir} --config ${build_config}
fi
else
echo "**** Skipping Caliper build, install found at: ${caliper_install_dir}"
fi # build_caliper
################
# Conduit
################
conduit_version=v0.9.2
conduit_src_dir=$(ospath ${source_dir}/conduit-${conduit_version})
conduit_build_dir=$(ospath ${build_dir}/conduit-${conduit_version}/)
conduit_install_dir=$(ospath ${install_dir}/conduit-${conduit_version}/)
conduit_tarball=$(ospath ${source_dir}/conduit-${conduit_version}-src-with-blt.tar.gz)
# build only if install doesn't exist
if [ ! -d ${conduit_install_dir} ]; then
if ${build_conduit}; then
if [ ! -d ${conduit_src_dir} ]; then
echo "**** Downloading ${conduit_tarball}"
curl -L https://github.com/LLNL/conduit/releases/download/${conduit_version}/conduit-${conduit_version}-src-with-blt.tar.gz -o ${conduit_tarball}
# untar and avoid symlinks (which windows despises)
tar ${tar_extra_args} -xzf ${conduit_tarball} -C ${source_dir} \
--exclude="conduit-${conduit_version}/src/tests/relay/data/silo/*"
# caliper vs adiak patch
if ${build_caliper}; then
cd ${conduit_src_dir}
echo ${conduit_src_dir}
patch -p 1 < ${script_dir}/2024_08_01_conduit-pr1311-detect-if-caliper-needs-adiak.patch
cd ${root_dir}
fi
fi
#
# extrat cmake args
#
conduit_extra_cmake_opts=-DENABLE_PYTHON=${enable_python}
if ${build_pyvenv}; then
conduit_extra_cmake_opts="${conduit_extra_cmake_opts} -DPYTHON_EXECUTABLE=${venv_python_exe}"
conduit_extra_cmake_opts="${conduit_extra_cmake_opts} -DSPHINX_EXECUTABLE=${venv_sphinx_exe}"
conduit_extra_cmake_opts="${conduit_extra_cmake_opts} -DPYTHON_MODULE_INSTALL_PREFIX=${venv_python_site_pkgs_dir}"
fi
if ${build_caliper}; then
conduit_extra_cmake_opts="${conduit_extra_cmake_opts} -DCALIPER_DIR=${caliper_install_dir}"
fi
if ${build_silo}; then
conduit_extra_cmake_opts="${conduit_extra_cmake_opts} -DSILO_DIR=${silo_install_dir}"
fi
if [[ "$enable_mpicc" == "ON" ]]; then
conduit_extra_cmake_opts="${conduit_extra_cmake_opts} -DMPI_C_COMPILER=${mpicc_exe}"
conduit_extra_cmake_opts="${conduit_extra_cmake_opts} -DMPI_CXX_COMPILER=${mpicxx_exe}"
fi
echo "**** Configuring Conduit ${conduit_version}"
cmake -S ${conduit_src_dir}/src -B ${conduit_build_dir} ${cmake_compiler_settings} \
-DCMAKE_VERBOSE_MAKEFILE:BOOL=${enable_verbose} \
-DCMAKE_BUILD_TYPE=${build_config} \
-DBUILD_SHARED_LIBS=${build_shared_libs} \
-DCMAKE_INSTALL_PREFIX=${conduit_install_dir} \
-DENABLE_FORTRAN=${enable_fortran} \
-DENABLE_MPI=${enable_mpi} \
-DENABLE_FIND_MPI=${enable_find_mpi} \
${conduit_extra_cmake_opts} \
-DENABLE_TESTS=OFF \
-DHDF5_DIR=${hdf5_install_dir} \
-DZLIB_DIR=${zlib_install_dir}
echo "**** Building Conduit ${conduit_version}"
cmake --build ${conduit_build_dir} --config ${build_config} -j${build_jobs}
echo "**** Installing Conduit ${conduit_version}"
cmake --install ${conduit_build_dir} --config ${build_config}
fi
else
echo "**** Skipping Conduit build, install found at: ${conduit_install_dir}"
fi # build_conduit
###############################
# Kokkos (only for hip or sycl)
###############################
kokkos_version=4.4.01
kokkos_src_dir=$(ospath ${source_dir}/kokkos-${kokkos_version})
kokkos_build_dir=$(ospath ${build_dir}/kokkos-${kokkos_version})
kokkos_install_dir=$(ospath ${install_dir}/kokkos-${kokkos_version}/)
kokkos_tarball=$(ospath ${source_dir}/kokkos-${kokkos_version}.tar.gz)
if [[ "$enable_hip" == "ON" ]] || [[ "$enable_sycl" == "ON" ]]; then
# build only if install doesn't exist
if [ ! -d ${kokkos_install_dir} ]; then
if ${build_kokkos}; then
if [ ! -d ${kokkos_src_dir} ]; then
echo "**** Downloading ${kokkos_tarball}"
curl -L https://github.com/kokkos/kokkos/archive/refs/tags/${kokkos_version}.tar.gz -o ${kokkos_tarball}
tar ${tar_extra_args} -xzf ${kokkos_tarball} -C ${source_dir}
fi
kokkos_extra_cmake_args=""
if [[ "$enable_hip" == "ON" ]]; then
kokkos_extra_cmake_args="-DKokkos_ENABLE_HIP=ON"
kokkos_extra_cmake_args="${kokkos_extra_cmake_args} -DKokkos_ENABLE_HIP_RELOCATABLE_DEVICE_CODE=OFF"
kokkos_extra_cmake_args="${kokkos_extra_cmake_args} -DCMAKE_CXX_COMPILER=${ROCM_PATH}/bin/hipcc"
kokkos_extra_cmake_args="${kokkos_extra_cmake_args} -DCMAKE_CXX_EXTENSIONS=OFF"
kokkos_extra_cmake_args="${kokkos_extra_cmake_args} -DCMAKE_CXX_STANDARD=17"
kokkos_extra_cmake_args="${kokkos_extra_cmake_args} -DKokkos_ENABLE_ROCTHRUST=OFF"
##
## build_ascent specific ROCM_ARCH Map for Kokkos options:
##
## TODO: Kokkos 4.5 has MI300A specific option, need to figure out how to
## map hat in when we update.
##
## gfx942 --> Kokkos_ARCH_AMD_GFX942 (MI300A, MI300X)
## (since Kokkos 4.2, since Kokkos 4.5 this should only be used for MI300X)
##
## gfx90a --> Kokkos_ARCH_AMD_GFX90A (MI200 series)
## (since Kokkos 4.2)
##
if [[ "$ROCM_ARCH" == "gfx942" ]]; then
kokkos_extra_cmake_args="${kokkos_extra_cmake_args} -DKokkos_ARCH_AMD_GFX942=ON"
fi
if [[ "$ROCM_ARCH" == "gfx90a" ]]; then
kokkos_extra_cmake_args="${kokkos_extra_cmake_args} -DKokkos_ARCH_AMD_GFX90A=ON"
fi
fi
if [[ "$enable_sycl" == "ON" ]]; then
kokkos_extra_cmake_args="-DCMAKE_CXX_FLAGS=-fPIC -fp-model=precise -Wno-unused-command-line-argument -Wno-deprecated-declarations -fsycl-device-code-split=per_kernel -fsycl-max-parallel-link-jobs=128"
kokkos_extra_cmake_args="${kokkos_extra_cmake_args} -DKokkos_ENABLE_SYCL=ON"
kokkos_extra_cmake_args="${kokkos_extra_cmake_args} -DKokkos_ARCH_INTEL_PVC=ON"
kokkos_extra_cmake_args="${kokkos_extra_cmake_args} -DCMAKE_CXX_EXTENSIONS=OFF"
kokkos_extra_cmake_args="${kokkos_extra_cmake_args} -DCMAKE_CXX_STANDARD=17"
fi
echo "**** Configuring Kokkos ${kokkos_version}"
cmake -S ${kokkos_src_dir} -B ${kokkos_build_dir} ${cmake_compiler_settings} \
-DCMAKE_VERBOSE_MAKEFILE:BOOL=${enable_verbose}\
-DCMAKE_BUILD_TYPE=${build_config} \
-DBUILD_SHARED_LIBS=${build_shared_libs} \
-DKokkos_ENABLE_SERIAL=ON \
-DCMAKE_INSTALL_PREFIX=${kokkos_install_dir} \
-DBUILD_TESTING=OFF ${kokkos_extra_cmake_args} \
-DCMAKE_INSTALL_PREFIX=${kokkos_install_dir}
echo "**** Building Kokkos ${kokkos_version}"
cmake --build ${kokkos_build_dir} --config ${build_config} -j${build_jobs}
echo "**** Installing VTK-m ${kokkos_version}"
cmake --install ${kokkos_build_dir} --config ${build_config}
fi
else
echo "**** Skipping Kokkos build, install found at: ${kokkos_install_dir}"
fi # build_kokkos
fi # if enable_hip || enable_sycl
################
# VTK-m
################
vtkm_version=v2.1.0
vtkm_src_dir=$(ospath ${source_dir}/vtk-m-${vtkm_version})
vtkm_build_dir=$(ospath ${build_dir}/vtk-m-${vtkm_version})
vtkm_install_dir=$(ospath ${install_dir}/vtk-m-${vtkm_version}/)
vtkm_tarball=$(ospath ${source_dir}/vtk-m-${vtkm_version}.tar.gz)
# build only if install doesn't exist
if [ ! -d ${vtkm_install_dir} ]; then
if ${build_vtkm}; then
if [ ! -d ${vtkm_src_dir} ]; then
echo "**** Downloading ${vtkm_tarball}"
curl -L https://gitlab.kitware.com/vtk/vtk-m/-/archive/${vtkm_version}/vtk-m-${vtkm_version}.tar.gz -o ${vtkm_tarball}
tar ${tar_extra_args} -xzf ${vtkm_tarball} -C ${source_dir}
# apply vtk-m patch
cd ${vtkm_src_dir}
patch -p1 < ${script_dir}/2023_12_06_vtkm-mr3160-rocthrust-fix.patch
patch -p1 < ${script_dir}/2024_05_03_vtkm-mr3215-ext-geom-fix.patch
patch -p1 < ${script_dir}/2024_07_02_vtkm-mr3246-raysubset_bugfix.patch
cd ${root_dir}
fi
vtkm_extra_cmake_args=""
if [[ "$enable_cuda" == "ON" ]]; then
vtkm_extra_cmake_args="${vtkm_extra_cmake_args} -DVTKm_ENABLE_CUDA=ON"
vtkm_extra_cmake_args="${vtkm_extra_cmake_args} -DCMAKE_CUDA_HOST_COMPILER=${CXX}"
vtkm_extra_cmake_args="${vtkm_extra_cmake_args} -DCMAKE_CUDA_ARCHITECTURES=${CUDA_ARCH}"
fi
if [[ "$enable_hip" == "ON" ]]; then
vtkm_extra_cmake_args="${vtkm_extra_cmake_args} -DVTKm_ENABLE_KOKKOS=ON"
vtkm_extra_cmake_args="${vtkm_extra_cmake_args} -DCMAKE_PREFIX_PATH=${kokkos_install_dir}"
vtkm_extra_cmake_args="${vtkm_extra_cmake_args} -DCMAKE_HIP_ARCHITECTURES=${ROCM_ARCH}"
vtkm_extra_cmake_args="${vtkm_extra_cmake_args} -DVTKm_ENABLE_KOKKOS_THRUST=OFF"
fi
if [[ "$enable_sycl" == "ON" ]]; then
vtkm_extra_cmake_args="${vtkm_extra_cmake_args} -DVTKm_ENABLE_KOKKOS=ON"
vtkm_extra_cmake_args="${vtkm_extra_cmake_args} -DCMAKE_PREFIX_PATH=${kokkos_install_dir}"
vtkm_extra_cmake_args="-DCMAKE_CXX_FLAGS=-fPIC -fp-model=precise -Wno-unused-command-line-argument -Wno-deprecated-declarations -fsycl-device-code-split=per_kernel -fsycl-max-parallel-link-jobs=128"
fi
if [[ "$enable_mpicc" == "ON" ]]; then
vtkm_extra_cmake_args="${vtkm_extra_cmake_args} -DMPI_C_COMPILER=${mpicc_exe}"
vtkm_extra_cmake_args="${vtkm_extra_cmake_args} -DMPI_CXX_COMPILER=${mpicxx_exe}"
fi
echo "**** Configuring VTK-m ${vtkm_version}"
cmake -S ${vtkm_src_dir} -B ${vtkm_build_dir} ${cmake_compiler_settings} \
-DCMAKE_VERBOSE_MAKEFILE:BOOL=${enable_verbose}\
-DCMAKE_BUILD_TYPE=${build_config} \
-DBUILD_SHARED_LIBS=${build_shared_libs} \
-DVTKm_USE_64BIT_IDS=OFF \
-DVTKm_USE_DOUBLE_PRECISION=ON \
-DVTKm_USE_DEFAULT_TYPES_FOR_ASCENT=ON \
-DVTKm_ENABLE_MPI=${enable_mpi} \
-DVTKm_ENABLE_OPENMP=${enable_openmp}\
-DVTKm_ENABLE_RENDERING=ON \
-DVTKm_ENABLE_TESTING=OFF\
-DBUILD_TESTING=OFF \
-DVTKm_ENABLE_BENCHMARKS=OFF ${vtkm_extra_cmake_args} \
-DCMAKE_INSTALL_PREFIX=${vtkm_install_dir}
echo "**** Building VTK-m ${vtkm_version}"
cmake --build ${vtkm_build_dir} --config ${build_config} -j${build_jobs}
echo "**** Installing VTK-m ${vtkm_version}"
cmake --install ${vtkm_build_dir} --config ${build_config}
fi
else
echo "**** Skipping VTK-m build, install found at: ${vtkm_install_dir}"
fi # build_vtkm
################
# Camp
################
camp_version=v2024.02.1
camp_src_dir=$(ospath ${source_dir}/camp-${camp_version})
camp_build_dir=$(ospath ${build_dir}/camp-${camp_version})
camp_install_dir=$(ospath ${install_dir}/camp-${camp_version}/)
camp_tarball=$(ospath ${source_dir}/camp-${camp_version}.tar.gz)
# build only if install doesn't exist
if [ ! -d ${camp_install_dir} ]; then
if ${build_camp}; then
if [ ! -d ${camp_src_dir} ]; then
echo "**** Downloading ${camp_tarball}"
curl -L https://github.com/LLNL/camp/releases/download/${camp_version}/camp-${camp_version}.tar.gz -o ${camp_tarball}
tar ${tar_extra_args} -xzf ${camp_tarball} -C ${source_dir}
fi
camp_extra_cmake_args=""
if [[ "$enable_cuda" == "ON" ]]; then
camp_extra_cmake_args="-DENABLE_CUDA=ON"
camp_extra_cmake_args="${camp_extra_cmake_args} -DCMAKE_CUDA_ARCHITECTURES=${CUDA_ARCH}"
fi
if [[ "$enable_hip" == "ON" ]]; then
camp_extra_cmake_args="-DENABLE_HIP=ON"
camp_extra_cmake_args="${camp_extra_cmake_args} -DCMAKE_HIP_ARCHITECTURES=${ROCM_ARCH}"
camp_extra_cmake_args="${camp_extra_cmake_args} -DROCM_PATH=${ROCM_PATH}"
fi
echo "**** Configuring Camp ${camp_version}"
cmake -S ${camp_src_dir} -B ${camp_build_dir} ${cmake_compiler_settings} \
-DCMAKE_VERBOSE_MAKEFILE:BOOL=${enable_verbose}\
-DCMAKE_BUILD_TYPE=${build_config} \
-DBUILD_SHARED_LIBS=${build_shared_libs} \
-DENABLE_TESTS=OFF \
-DENABLE_EXAMPLES=OFF ${camp_extra_cmake_args} \
-DCMAKE_INSTALL_PREFIX=${camp_install_dir}
echo "**** Building Camp ${camp_version}"
cmake --build ${camp_build_dir} --config ${build_config} -j${build_jobs}
echo "**** Installing Camp ${camp_version}"
cmake --install ${camp_build_dir} --config ${build_config}
fi
else
echo "**** Skipping Camp build, install found at: ${camp_install_dir}"
fi # build_camp
################
# RAJA
################
raja_version=v2024.02.1
raja_src_dir=$(ospath ${source_dir}/RAJA-${raja_version})
raja_build_dir=$(ospath ${build_dir}/raja-${raja_version})
raja_install_dir=$(ospath ${install_dir}/raja-${raja_version}/)
raja_tarball=$(ospath ${source_dir}/RAJA-${raja_version}.tar.gz)
raja_enable_vectorization="${raja_enable_vectorization:=ON}"
# build only if install doesn't exist
if [ ! -d ${raja_install_dir} ]; then
if ${build_raja}; then
if [ ! -d ${raja_src_dir} ]; then
echo "**** Downloading ${raja_tarball}"
curl -L https://github.com/LLNL/RAJA/releases/download/${raja_version}/RAJA-${raja_version}.tar.gz -o ${raja_tarball}
tar ${tar_extra_args} -xzf ${raja_tarball} -C ${source_dir}
fi
raja_extra_cmake_args=""
if [[ "$enable_cuda" == "ON" ]]; then
raja_extra_cmake_args="-DENABLE_CUDA=ON"
raja_extra_cmake_args="${raja_extra_cmake_args} -DCMAKE_CUDA_ARCHITECTURES=${CUDA_ARCH}"
fi
if [[ "$enable_hip" == "ON" ]]; then
raja_extra_cmake_args="-DENABLE_HIP=ON"
raja_extra_cmake_args="${raja_extra_cmake_args} -DCMAKE_HIP_ARCHITECTURES=${ROCM_ARCH}"
raja_extra_cmake_args="${raja_extra_cmake_args} -DROCM_PATH=${ROCM_PATH}"
fi
echo "**** Configuring RAJA ${raja_version}"
cmake -S ${raja_src_dir} -B ${raja_build_dir} ${cmake_compiler_settings} \
-DCMAKE_VERBOSE_MAKEFILE:BOOL=${enable_verbose}\
-DCMAKE_BUILD_TYPE=${build_config} \
-DBUILD_SHARED_LIBS=${build_shared_libs} \
-Dcamp_DIR=${camp_install_dir} \
-DENABLE_OPENMP=${enable_openmp} \
-DENABLE_TESTS=OFF \
-DRAJA_ENABLE_TESTS=OFF \
-DENABLE_EXAMPLES=OFF \
-DENABLE_EXERCISES=OFF ${raja_extra_cmake_args} \
-DCMAKE_INSTALL_PREFIX=${raja_install_dir} \
-DRAJA_ENABLE_VECTORIZATION=${raja_enable_vectorization}
echo "**** Building RAJA ${raja_version}"
cmake --build ${raja_build_dir} --config ${build_config} -j${build_jobs}
echo "**** Installing RAJA ${raja_version}"
cmake --install ${raja_build_dir} --config ${build_config}
fi
else
echo "**** Skipping RAJA build, install found at: ${raja_install_dir}"
fi # build_raja
################
# Umpire
################
umpire_version=2024.02.1
umpire_src_dir=$(ospath ${source_dir}/umpire-${umpire_version})
umpire_build_dir=$(ospath ${build_dir}/umpire-${umpire_version})
umpire_install_dir=$(ospath ${install_dir}/umpire-${umpire_version}/)
umpire_tarball=$(ospath ${source_dir}/umpire-${umpire_version}.tar.gz)
umpire_windows_cmake_flags="-DBLT_CXX_STD=c++17 -DCMAKE_CXX_STANDARD=17 -DUMPIRE_ENABLE_FILESYSTEM=On -DCMAKE_WINDOWS_EXPORT_ALL_SYMBOLS=On"
umpire_extra_cmake_args=""
if [[ "$build_windows" == "ON" ]]; then
umpire_extra_cmake_args="${umpire_windows_cmake_flags}"
fi
if [[ "$enable_cuda" == "ON" ]]; then
umpire_extra_cmake_args="${umpire_extra_cmake_args} -DENABLE_CUDA=ON"
umpire_extra_cmake_args="${umpire_extra_cmake_args} -DCMAKE_CUDA_ARCHITECTURES=${CUDA_ARCH}"
fi
if [[ "$enable_hip" == "ON" ]]; then
umpire_extra_cmake_args="${umpire_extra_cmake_args} -DENABLE_HIP=ON"
umpire_extra_cmake_args="${umpire_extra_cmake_args} -DCMAKE_HIP_ARCHITECTURES=${ROCM_ARCH}"
umpire_extra_cmake_args="${umpire_extra_cmake_args} -DROCM_PATH=${ROCM_PATH}"
fi
if [[ "$enable_sycl" == "ON" ]]; then
umpire_extra_cmake_args="${umpire_extra_cmake_args} -DENABLE_SYCL=ON"
fi
# build only if install doesn't exist
if [ ! -d ${umpire_install_dir} ]; then
if ${build_umpire}; then
if [ ! -d ${umpire_src_dir} ]; then
echo "**** Downloading ${umpire_tarball}"
curl -L https://github.com/LLNL/Umpire/releases/download/v${umpire_version}/umpire-${umpire_version}.tar.gz -o ${umpire_tarball}
tar ${tar_extra_args} -xzf ${umpire_tarball} -C ${source_dir}
fi
echo "**** Configuring Umpire ${umpire_version}"
cmake -S ${umpire_src_dir} -B ${umpire_build_dir} ${cmake_compiler_settings} \
-DCMAKE_VERBOSE_MAKEFILE:BOOL=${enable_verbose} \
-DCMAKE_BUILD_TYPE=${build_config} \
-DBUILD_SHARED_LIBS=${build_shared_libs} \
-Dcamp_DIR=${camp_install_dir} \
-DENABLE_OPENMP=${enable_openmp} \
-DENABLE_TESTS=OFF \
-DUMPIRE_ENABLE_TOOLS=Off \
-DUMPIRE_ENABLE_BENCHMARKS=OFF ${umpire_extra_cmake_args} \
-DCMAKE_INSTALL_PREFIX=${umpire_install_dir}
echo "**** Building Umpire ${umpire_version}"
cmake --build ${umpire_build_dir} --config ${build_config} -j${build_jobs}
echo "**** Installing Umpire ${umpire_version}"
cmake --install ${umpire_build_dir} --config ${build_config}
fi
else
echo "**** Skipping Umpire build, install found at: ${umpire_install_dir}"
fi # build_umpire
################
# MFEM
################
mfem_version=4.7
mfem_src_dir=$(ospath ${source_dir}/mfem-${mfem_version})
mfem_build_dir=$(ospath ${build_dir}/mfem-${mfem_version})
mfem_install_dir=$(ospath ${install_dir}/mfem-${mfem_version}/)
mfem_tarball=$(ospath ${source_dir}/mfem-${mfem_version}.tar.gz)
mfem_windows_cmake_flags="-DCMAKE_WINDOWS_EXPORT_ALL_SYMBOLS=ON"
mfem_extra_cmake_args=""
if [[ "$build_windows" == "ON" ]]; then
mfem_extra_cmake_args="${mfem_windows_cmake_flags}"
fi
# build only if install doesn't exist
if [ ! -d ${mfem_install_dir} ]; then
if ${build_mfem}; then
if [ ! -d ${mfem_src_dir} ]; then
echo "**** Downloading ${mfem_tarball}"
curl -L https://github.com/mfem/mfem/archive/refs/tags/v${mfem_version}.tar.gz -o ${mfem_tarball}
tar ${tar_extra_args} -xzf ${mfem_tarball} -C ${source_dir}
fi
#
# Note: MFEM MPI requires Hypre and Metis
# -DMFEM_USE_MPI=${enable_mpi} \
echo "**** Configuring MFEM ${mfem_version}"
cmake -S ${mfem_src_dir} -B ${mfem_build_dir} ${cmake_compiler_settings} \
-DCMAKE_VERBOSE_MAKEFILE:BOOL=${enable_verbose}\
-DCMAKE_BUILD_TYPE=${build_config} \
-DBUILD_SHARED_LIBS=${build_shared_libs} \
-DMFEM_USE_CONDUIT=ON ${mfem_extra_cmake_args} \
-DCMAKE_PREFIX_PATH="${conduit_install_dir}" \
-DMFEM_ENABLE_TESTING=OFF \
-DMFEM_ENABLE_EXAMPLES=OFF \
-DCMAKE_INSTALL_PREFIX=${mfem_install_dir}
echo "**** Building MFEM ${mfem_version}"
cmake --build ${mfem_build_dir} --config ${build_config} -j${build_jobs}
echo "**** Installing MFEM ${mfem_version}"
cmake --install ${mfem_build_dir} --config ${build_config}
fi
else
echo "**** Skipping MFEM build, install found at: ${mfem_install_dir}"
fi # build_mfem
################
# Catalyst
################
catalyst_version=2.0.0-rc4
catalyst_src_dir=$(ospath ${source_dir}/catalyst-v${catalyst_version})
catalyst_build_dir=$(ospath ${build_dir}/catalyst-v${catalyst_version})
catalyst_install_dir=$(ospath ${install_dir}/catalyst-v${catalyst_version}/)
catalyst_cmake_dir=${catalyst_install_dir}lib64/cmake/catalyst-2.0/
catalyst_tarball=$(ospath ${source_dir}/catalyst-v${catalyst_version}.tar.gz)
# build only if install doesn't exist
if [ ! -d ${catalyst_install_dir} ]; then
if ${build_catalyst}; then
if [ ! -d ${catalyst_src_dir} ]; then
echo "**** Downloading ${catalyst_tarball}"
curl -L https://gitlab.kitware.com/paraview/catalyst/-/archive/v${catalyst_version}/catalyst-v${catalyst_version}.tar.gz -o ${catalyst_tarball}
tar ${tar_extra_args} -xzf ${catalyst_tarball} -C ${source_dir}
fi
echo "**** Configuring Catalyst ${catalyst_version}"
cmake -S ${catalyst_src_dir} -B ${catalyst_build_dir} ${cmake_compiler_settings} \
-DCMAKE_VERBOSE_MAKEFILE:BOOL=${enable_verbose}\
-DCMAKE_BUILD_TYPE=${build_config} \
-DCATALYST_BUILD_TESTING=OFF \
-DCATALYST_USE_MPI=${enable_mpi} \
-DCMAKE_INSTALL_PREFIX=${catalyst_install_dir} \
echo "**** Building Catalyst ${catalyst_version}"
cmake --build ${catalyst_build_dir} --config ${build_config} -j${build_jobs}
echo "**** Installing Catalyst ${catalyst_version}"
cmake --install ${catalyst_build_dir} --config ${build_config}
fi
else
echo "**** Skipping Catalyst build, install found at: ${catalyst_install_dir}"
fi # build_catalyst
################
# Ascent
################
# if we are in an ascent checkout, use existing source
ascent_checkout_dir=$(ospath ${script_dir}/../../src)
ascent_checkout_dir=$(abs_path ${ascent_checkout_dir})
echo ${ascent_checkout_dir}
if [ -d ${ascent_checkout_dir} ]; then
ascent_version=checkout
ascent_src_dir=$(abs_path ${ascent_checkout_dir})
echo "**** Using existing Ascent source repo checkout: ${ascent_src_dir}"
else
ascent_version=develop
ascent_src_dir=$(ospath ${source_dir}/ascent/src)
fi
# otherwise use ascent develop
ascent_build_dir=$(ospath ${build_dir}/ascent-${ascent_version}/)
ascent_install_dir=$(ospath ${install_dir}//ascent-${ascent_version}/)
echo "**** Creating Ascent host-config (ascent-config.cmake)"
#
echo '# host-config file generated by build_ascent.sh' > ${root_dir}/ascent-config.cmake
# capture compilers if they are provided via env vars
if [ ! -z ${CC+x} ]; then
echo 'set(CMAKE_C_COMPILER ' ${CC} ' CACHE PATH "")' >> ${root_dir}/ascent-config.cmake
fi
if [ ! -z ${CXX+x} ]; then
echo 'set(CMAKE_CXX_COMPILER ' ${CXX} ' CACHE PATH "")' >> ${root_dir}/ascent-config.cmake
fi
if [ ! -z ${FTN+x} ]; then
echo 'set(CMAKE_Fortran_COMPILER ' ${FTN} ' CACHE PATH "")' >> ${root_dir}/ascent-config.cmake
fi
# capture compiler flags if they are provided via env vars
if [ ! -z ${CFLAGS+x} ]; then
echo 'set(CMAKE_C_FLAGS "' ${CFLAGS} '" CACHE PATH "")' >> ${root_dir}/ascent-config.cmake
fi
if [ ! -z ${CXXFLAGS+x} ]; then
echo 'set(CMAKE_CXX_FLAGS "' ${CXXFLAGS} '" CACHE PATH "")' >> ${root_dir}/ascent-config.cmake
fi
if [ ! -z ${FFLAGS+x} ]; then
echo 'set(CMAKE_F_FLAGS "' ${FFLAGS} '" CACHE PATH "")' >> ${root_dir}/ascent-config.cmake
fi
if [[ "$enable_mpicc" == "ON" ]]; then
echo 'set(MPI_C_COMPILER ' ${mpicc_exe} ' CACHE PATH "")' >> ${root_dir}/ascent-config.cmake
echo 'set(MPI_CXX_COMPILER ' ${mpicxx_exe} ' CACHE PATH "")' >> ${root_dir}/ascent-config.cmake
fi
echo 'set(CMAKE_VERBOSE_MAKEFILE ' ${enable_verbose} ' CACHE BOOL "")' >> ${root_dir}/ascent-config.cmake
echo 'set(CMAKE_BUILD_TYPE ' ${build_config} ' CACHE STRING "")' >> ${root_dir}/ascent-config.cmake
echo 'set(BUILD_SHARED_LIBS ' ${build_shared_libs} ' CACHE STRING "")' >> ${root_dir}/ascent-config.cmake
echo 'set(CMAKE_INSTALL_PREFIX ' ${ascent_install_dir} ' CACHE PATH "")' >> ${root_dir}/ascent-config.cmake
echo 'set(ENABLE_TESTS ' ${enable_tests} ' CACHE BOOL "")' >> ${root_dir}/ascent-config.cmake
echo 'set(ENABLE_MPI ' ${enable_mpi} ' CACHE BOOL "")' >> ${root_dir}/ascent-config.cmake
echo 'set(ENABLE_FIND_MPI ' ${enable_find_mpi} ' CACHE BOOL "")' >> ${root_dir}/ascent-config.cmake
echo 'set(ENABLE_FORTRAN ' ${enable_fortran} ' CACHE BOOL "")' >> ${root_dir}/ascent-config.cmake
echo 'set(ENABLE_PYTHON ' ${enable_python} ' CACHE BOOL "")' >> ${root_dir}/ascent-config.cmake
if ${build_pyvenv}; then
echo 'set(PYTHON_EXECUTABLE ' ${venv_python_exe} ' CACHE PATH "")' >> ${root_dir}/ascent-config.cmake
echo 'set(PYTHON_MODULE_INSTALL_PREFIX ' ${venv_python_site_pkgs_dir} ' CACHE PATH "")' >> ${root_dir}/ascent-config.cmake
echo 'set(ENABLE_DOCS ON CACHE BOOL "")' >> ${root_dir}/ascent-config.cmake
echo 'set(SPHINX_EXECUTABLE ' ${venv_sphinx_exe} ' CACHE PATH "")' >> ${root_dir}/ascent-config.cmake
fi
if ${build_caliper}; then
echo 'set(CALIPER_DIR ' ${caliper_install_dir} ' CACHE PATH "")' >> ${root_dir}/ascent-config.cmake
fi
echo 'set(CONDUIT_DIR ' ${conduit_install_dir} ' CACHE PATH "")' >> ${root_dir}/ascent-config.cmake
echo 'set(VTKM_DIR ' ${vtkm_install_dir} ' CACHE PATH "")' >> ${root_dir}/ascent-config.cmake
echo 'set(CAMP_DIR ' ${camp_install_dir} ' CACHE PATH "")' >> ${root_dir}/ascent-config.cmake
echo 'set(RAJA_DIR ' ${raja_install_dir} ' CACHE PATH "")' >> ${root_dir}/ascent-config.cmake
echo 'set(UMPIRE_DIR ' ${umpire_install_dir} ' CACHE PATH "")' >> ${root_dir}/ascent-config.cmake
echo 'set(MFEM_DIR ' ${mfem_install_dir} ' CACHE PATH "")' >> ${root_dir}/ascent-config.cmake
echo 'set(ENABLE_VTKH ON CACHE BOOL "")' >> ${root_dir}/ascent-config.cmake
echo 'set(ENABLE_APCOMP ON CACHE BOOL "")' >> ${root_dir}/ascent-config.cmake
echo 'set(ENABLE_DRAY ON CACHE BOOL "")' >> ${root_dir}/ascent-config.cmake
if ${build_catalyst}; then
echo 'set(CATALYST_DIR ' ${catalyst_cmake_dir} ' CACHE PATH "")' >> ${root_dir}/ascent-config.cmake
fi
if [[ "$enable_cuda" == "ON" ]]; then
echo 'set(ENABLE_CUDA ON CACHE BOOL "")' >> ${root_dir}/ascent-config.cmake
echo 'set(CMAKE_CUDA_ARCHITECTURES ' ${CUDA_ARCH} ' CACHE PATH "")' >> ${root_dir}/ascent-config.cmake
fi
if [[ "$enable_hip" == "ON" ]]; then
echo 'set(ENABLE_HIP ON CACHE BOOL "")' >> ${root_dir}/ascent-config.cmake
echo 'set(BLT_CXX_STD c++17 CACHE STRING "")' >> ${root_dir}/ascent-config.cmake
echo 'set(CMAKE_HIP_ARCHITECTURES ' ${ROCM_ARCH} ' CACHE STRING "")' >> ${root_dir}/ascent-config.cmake
echo 'set(ROCM_PATH ' ${ROCM_PATH} ' CACHE PATH "")' >> ${root_dir}/ascent-config.cmake
echo 'set(KOKKOS_DIR ' ${kokkos_install_dir} ' CACHE PATH "")' >> ${root_dir}/ascent-config.cmake
fi
# build only if install doesn't exist
if [ ! -d ${ascent_install_dir} ]; then
if ${build_ascent}; then
if [ ! -d ${ascent_src_dir} ]; then
echo "**** Cloning Ascent"
git clone --recursive https://github.com/Alpine-DAV/ascent.git ${source_dir}/ascent
fi
echo "**** Configuring Ascent"
cmake -S ${ascent_src_dir} -B ${ascent_build_dir} -C ${root_dir}/ascent-config.cmake
echo "**** Building Ascent"
cmake --build ${ascent_build_dir} --config ${build_config} -j${build_jobs}
echo "**** Installing Ascent"
cmake --install ${ascent_build_dir} --config ${build_config}
if ${build_catalyst}; then
mv ${ascent_install_dir}/lib/libcatalyst-ascent.so ${catalyst_install_dir}lib64/catalyst/libcatalyst-ascent.so
fi
fi
else
echo "**** Skipping Ascent build, install found at: ${ascent_install_dir}"
fi # build_ascent
It supports supports building on linux, macOS, and windows – without device support, and with CUDA or ROCm/HIP.
uberenv + spack¶
We also use Spack (http://spack.io) to help build Ascent’s third party dependencies on OSX and Linux.
Uberenv (scripts/uberenv/uberenv.py
) automates fetching spack, building and installing third party dependencies, and can optionally install Ascent as well. To automate the full install process, Uberenv uses the Ascent Spack package along with extra settings such as Spack compiler and external third party package details for common HPC platforms.
Uberenv Options for Building Third Party Dependencies¶
uberenv.py
has a few options that allow you to control how dependencies are built:
Option
Description
Default
–prefix
Destination directory
uberenv_libs
–spec
Spack spec
linux: %gcc osx: %clang
–spack-config-dir
Folder with Spack settings files
linux: (empty) osx:
scripts/uberenv_configs/spack_configs/darwin/
-k
Ignore SSL Errors
False
–install
Fully install ascent not just dependencies
False
The -k
option exists for sites where SSL certificate interception undermines fetching
from github and https hosted source tarballs. When enabled, uberenv.py
clones spack using:
git -c http.sslVerify=false clone https://github.com/llnl/spack.git
And passes -k
to any spack commands that may fetch via https.
Default invocation on Linux:
python scripts/uberenv/uberenv.py --prefix uberenv_libs \
--spec %gcc
Default invocation on OSX:
python scripts/uberenv/uberenv.py --prefix uberenv_libs \
--spec %clang \
--spack-config-dir scripts/uberenv_configs/spack_configs/darwin/
The uberenv –install installs ascent@develop (not just the development dependencies):
python scripts/uberenv/uberenv.py --install
For details on Spack’s spec syntax, see the Spack Specs & dependencies documentation.
Compiler Settings for Third Party Dependencies¶
You can edit yaml files under scripts/uberenv_configs/spack_configs/configs/{platform}
or use the –spack-config-dir option to specify a directory with compiler and packages yaml files to use with Spack. See the Spack Compiler Configuration
and Spack System Packages
documentation for details.
For macOS, the defaults in scripts/uberenv_configs/spack_configs/configs/darwin/compilers.yaml
are X-Code’s clang and gfortran from https://gcc.gnu.org/wiki/GFortranBinaries#MacOS.
Note
The bootstrapping process ignores ~/.spack/compilers.yaml
to avoid conflicts
and surprises from a user’s specific Spack settings on HPC platforms.
When run, uberenv.py
checkouts a specific version of Spack from github as spack
in the
destination directory. It then uses Spack to build and install Conduit’s dependencies into
spack/opt/spack/
. Finally, it generates a host-config file {hostname}.cmake
in the
destination directory that specifies the compiler settings and paths to all of the dependencies.
Building Third Party Dependencies for Development¶
You can use scripts/uberenv/uberenv.py
to help setup your development environment on OSX and Linux. uberenv.py
leverages Spack (https://spack.io/) to build the external third party libraries and tools used by Ascent.
Fortran support in is optional, dependencies should build without fortran.
After building these libraries and tools, it writes an initial host-config file and adds the Spack built CMake binary to your PATH, so can immediately call the config-build.sh
helper script to configure a ascent build.
#build third party libs using spack
python scripts/uberenv/uberenv.py
#copy the generated host-config file into the standard location
cp uberenv_libs/`hostname`*.cmake host-configs/
# run the configure helper script
./config-build.sh
# or you can run the configure helper script and give it the
# path to a host-config file
./config-build.sh uberenv_libs/`hostname`*.cmake
Building with Spack¶
Currently, we maintain our own fork of Spack for stability. As part of the uberenv python script, we automatically clone our Spack fork.
Warning
Installing Ascent from the Spack develop branch will most likely fail. We build and test spack installations with uberenv.py.
To install Ascent and also build all of its dependencies as necessary run:
spack install ascent
The Ascent Spack package provides several variants that customize the options and dependencies used to build Ascent.
Please see the Ascent Spack package source (or use spack info ascent
) to learn about variants.
Uberenv Spack Configurations¶
See the Spack configs we use to build our CI Containers for concrete examples of using pacakges.yaml
and compilers.yaml
to specify system packages and compiler details to Spack.
Using Ascent in Another Project¶
Under src/examples
there are examples demonstrating how to use Ascent in a CMake-based build system (using-with-cmake
, using-with-cmake-mpi
) and via a Makefile (using-with-make
, using-with-make-mpi
). You can read more details about these examples Using Ascent in Your Project.
Under src/examples/proxies
you can find example integrations using ascent in the Lulesh, Kripke, and Cloverleaf3D proxy-applications.
In src/examples/synthetic/noise
you can find an example integration using our synthetic smooth noise application.
Building Ascent in a Docker Container¶
Under src/examples/docker/master/ubuntu
there is an example Dockerfile
which can be used to create an ubuntu-based docker image with a build of the Ascent github master branch. There is also a script that demonstrates how to build a Docker image from the Dockerfile (example_build.sh
) and a script that runs this image in a Docker container (example_run.sh
). The Ascent repo is cloned into the image’s file system at /ascent
, the build directory is /ascent/build-debug
, and the install directory is /ascent/install-debug
.
Special Case Builds¶
BabelFlow Support¶
Here is script that shows how to build additional dependencies for bflow-stats (babelflow+pmt+streamstats+topo_reader):
root_dir=$(pwd)
# babelflow v1.0.1
git clone --recursive https://github.com/sci-visus/BabelFlow.git
git checkout v1.0.1
# pmt v1.0.2
git clone https://bitbucket.org/cedmav/parallelmergetree.git
git checkout v1.0.2
# STREAMSTATS
git clone https://github.com/xuanhuang1/STREAMSTAT.git
# topo_reader
git clone https://github.com/xuanhuang1/topo_reader.git
# build
# build babelflow 1.0.1
babelflow_src_dir=${root_dir}/BabelFlow
babelflow_build_dir=${root_dir}/BabelFlow/build
babelflow_install_dir=${root_dir}/BabelFlow/install
cmake -S ${babelflow_src_dir} -B ${babelflow_build_dir} \
-DCMAKE_VERBOSE_MAKEFILE:BOOL=ON\
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=${babelflow_install_dir} \
-DBUILD_SHARED_LIBS=ON \
-DCRAYPE_LINK_TYPE=dynamic \
-DENABLE_MPI=ON \
-DENABLE_FIND_MPI=OFF
cmake --build ${babelflow_build_dir} -j6
cmake --install ${babelflow_build_dir}
# build parallelmergetree 1.0.2
parallelmergetree_src_dir=${root_dir}/parallelmergetree
parallelmergetree_build_dir=${root_dir}/parallelmergetree/build
parallelmergetree_install_dir=${root_dir}/parallelmergetree/install
cmake -S ${parallelmergetree_src_dir} -B ${parallelmergetree_build_dir} \
-DCMAKE_VERBOSE_MAKEFILE:BOOL=ON\
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=${parallelmergetree_install_dir} \
-DBUILD_SHARED_LIBS=ON \
-DCRAYPE_LINK_TYPE=dynamic \
-DLIBRARY_ONLY=ON\
-DBabelFlow_DIR=${babelflow_install_dir}
cmake --build ${parallelmergetree_build_dir} -j6
cmake --install ${parallelmergetree_build_dir}
# build topo_reader
topo_reader_src_dir=${root_dir}/topo_reader/TopologyFileParser
topo_reader_build_dir=${root_dir}/topo_reader/TopologyFileParser/build
topo_reader_install_dir=${root_dir}/topo_reader/TopologyFileParser/install
cmake -S ${topo_reader_src_dir} -B ${topo_reader_build_dir} \
-DCMAKE_VERBOSE_MAKEFILE:BOOL=ON\
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=${topo_reader_install_dir} \
-DBUILD_SHARED_LIBS=ON \
-DCRAYPE_LINK_TYPE=dynamic \
-DFUNCTION_TYPE=double
cmake --build ${topo_reader_build_dir} -j6
cmake --install ${topo_reader_build_dir}
# build STREAMSTAT
STREAMSTAT_src_dir=${root_dir}/STREAMSTAT/
STREAMSTAT_build_dir=${root_dir}/STREAMSTAT/build
STREAMSTAT_install_dir=${root_dir}/STREAMSTAT/install
cmake -S ${STREAMSTAT_src_dir} -B ${STREAMSTAT_build_dir} \
-DCMAKE_VERBOSE_MAKEFILE:BOOL=ON\
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=${STREAMSTAT_install_dir} \
-DBUILD_SHARED_LIBS=ON \
-DCRAYPE_LINK_TYPE=dynamic
cmake --build ${STREAMSTAT_build_dir} -j6
cmake --install ${STREAMSTAT_build_dir}
# add the path in ascent build
#ascent_src_dir=${root_dir}/ascent/src
#ascent_build_dir=${root_dir}/ascent/build
#ascent_install_dir=${root_dir}/ascent/install
#cmake -S ${ascent_src_dir} -B ${ascent_build_dir} \
# -DBABELFLOW_DIR=${babelflow_install_dir} \
# -DPMT_DIR=${parallelmergetree_install_dir} \
# -DStreamStat_DIR=${STREAMSTAT_install_dir}\
# -DTopoFileParser_DIR=${topo_reader_install_dir}
ParaView Support¶
Ascent ParaView support is in src/examples/paraview-vis directory.
This section describes how to configure, build and run the example
integrations provided with Ascent and visualize the results insitu
using ParaView. ParaView pipelines are provided for all example
integrations. We describe in details the ParaView pipeline for
cloverleaf3d
in the ParaView Visualization section.
Setup spack¶
Install spack, modules and shell support.
Clone the spack repository:
git clone https://github.com/spack/spack.git cd spack source share/spack/setup-env.sh
If the
module
command does not exist:install
environment-modules
using the package manager for your system.run
add.modules
to add themodule
command to your.bashrc
fileAlternatively run
spack bootstrap
Install ParaView and Ascent¶
For MomentInvariants (optional module in ParaView for pattern detection) visualization patch ParaView to enable this module:
Download the MomentInvariants patch
Patch paraview:
patch -p1 < paraview-package-momentinvariants.patch
Install ParaView (any version >= 5.7.0). When running on Linux we prefer
mpich
, which can be specified by using^mpich
.spack install paraview+python3+mpi+osmesa
for CUDA use:
spack install paraview+python3+mpi+osmesa+cuda
Install Ascent
spack install ascent~vtkh+python
If you need ascent built with vtkh you can use
spack install ascent+python
. Note that you need specific versions ofvtkh
andvtkm
that work with the version of Ascent built. Those versions can be read fromscripts/uberenv/project.json
by cloningspack_url
, branchspack_branch
.paraview-package-momentinvariants.patch
is already setup to patchvtkh
andvthm
with the correct versions, but make sure it is not out of date.
Load required modules:
spack load conduit;spack load python;spack load py-numpy;spack load py-mpi4py;spack load paraview
Setup and run example integrations¶
You can test Ascent with ParaView support by running the available
integrations. Visualization images will be generated in the current
directory. These images can be checked against the images in
src/examples/paraview-vis/tests/baseline_images
.
Test
proxies/cloverleaf3d
Go to a directory where you intend to run cloverleaf3d integration (for
summit.olcf.ornl.gov
use a member work directory such ascd $MEMBERWORK/csc340
) so that the compute node can write there.Create links to required files for cloverleaf3d:
ln -s $(spack location --install-dir ascent)/examples/ascent/paraview-vis/paraview_ascent_source.py
ln -s $(spack location --install-dir ascent)/examples/ascent/paraview-vis/paraview-vis-cloverleaf3d.py paraview-vis.py
for surface visualization.Or
ln -s $(spack location --install-dir ascent)/examples/ascent/paraview-vis/paraview-vis-cloverleaf3d-momentinvariants.py paraview-vis.py
for MomentInvariants visualization (Optional)ln -s $(spack location --install-dir ascent)/examples/ascent/paraview-vis/ascent_actions.json ln -s $(spack location --install-dir ascent)/examples/ascent/paraview-vis/expandingVortex.vti ln -s $(spack location --install-dir ascent)/examples/ascent/proxies/cloverleaf3d/clover.in
Run the simulation
$(spack location --install-dir mpi)/bin/mpiexec -n 2 $(spack location --install-dir ascent)/examples/ascent/proxies/cloverleaf3d/cloverleaf3d_par > output.txt 2>&1
Examine the generated images
Similarily test
proxies/kripke
,proxies/laghos
,proxies/lulesh
,synthetic/noise
. After you create the apropriate links similarily withcloverleaf3d
you can run these simulations with:$(spack location --install-dir mpi)/bin/mpiexec -np 8 ./kripke_par --procs 2,2,2 --zones 32,32,32 --niter 5 --dir 1:2 --grp 1:1 --legendre 4 --quad 4:4 > output.txt 2>&1
$(spack location --install-dir mpi)/bin/mpiexec -n 8 ./laghos_mpi -p 1 -m data/cube01_hex.mesh -rs 2 -tf 0.6 -visit -pa > output.txt 2>&1
$(spack location --install-dir mpi)/bin/mpiexec -np 8 ./lulesh_par -i 10 -s 32 > output.txt 2>&1
$(spack location --install-dir mpi)/bin/mpiexec -np 8 ./noise_par --dims=32,32,32 --time_steps=5 --time_delta=1 > output.txt 2>&1
Setup and run on summit.olcf.ornl.gov¶
Execute section Setup spack
Configure spack
add a file
~/.spack/packages.yaml
with the following content as detailed next. This insures that we use spectrum-mpi as the MPI runtime.packages: spectrum-mpi: buildable: false externals: - modules: - spectrum-mpi/10.3.1.2-20200121 spec: spectrum-mpi@10.3.1.2-20200121 cuda: buildable: false externals: - modules: - cuda/10.1.168 spec: cuda@10.1.168
Load the correct compiler:
module load gcc/7.4.0 spack compiler add spack compiler remove gcc@4.8.5
Compile spack packages on a compute node
For busy summit I got
internal compiler error
when compilingllvm
andparaview
on the login node. To fix this, move the spack installation on$MEMBERWORK/csc340
and compile everything on a compute node.First login to a compute node:
bsub -W 2:00 -nnodes 1 -P CSC340 -Is /bin/bash
Install all spack packages as in Install ParaView and Ascent with
-j80
option (there are 84 threads)Disconnect from the compute node:
exit
.
Continue with Setup and run example integrations but run the integrations as described next:
Execute cloverleaf
bsub $(spack location --install-dir ascent)/examples/ascent/paraview-vis/summit-moment-invariants.lsf
To check if the integration finished use:
bjobs -a
Nightly tests¶
We provide a docker file for Ubuntu 18.04 and a script that installs
the latest ParaView and Ascent, runs the integrations provided with
Ascent, runs visualizations using ParaView pipelines and checks the
results. See tests/README-docker.md
for how to create the docker
image, run the container and execute the test script.
Notes¶
Global extents are computed for uniform and rectilinear topologies but they are not yet computed for a structured topology (lulesh). This means that for lulesh and datasets that have a structured topology we cannot save a correct parallel file that represents the whole dataset.
For the
laghos
simulation accessed through Python extracts interface, only the higher order mesh is accessible at the moment, which is a uniform dataset. The documentation shows a non-uniform mesh but that is only available in thevtkm
pipelines.