Extracts are an abstraction that enables the user to specify how they want to capture their data. In terms of Ascent, data capture sends data outside the Ascent infrastructure. Examples include writing out the raw simulation data to the file system, creating HDF5 files, or sending the data off node (e.g., ADIOS).
Currently supported extracts include:
- Python : use a python script with NumPy to analyze mesh data
- Relay : leverages Conduit’s Relay library to do parallel I/O
- ADIOS : use ADIOS to send data to a separate resource
Python extracts can execute arbitrary python code. Python code uses Conduit’s python interface to interrogate and retrieve mesh data. Code is executed on each MPI rank, and mpi4py can be used for collective communication.
conduit::Node extracts; extracts["e1/type"] = "python"; extracts["e1/params/source"] = py_script;
Python source code is loaded into Ascent via a string that could be loaded from the file system
import numpy as np from mpi4py import MPI # obtain a mpi4py mpi comm object comm = MPI.Comm.f2py(ascent_mpi_comm_id()) # get this MPI task's published blueprint data mesh_data = ascent_data().child(0) # fetch the numpy array for the energy field values e_vals = mesh_data["fields/energy/values"] # find the data extents of the energy field using mpi # first get local extents e_min, e_max = e_vals.min(), e_vals.max() # declare vars for reduce results e_min_all = np.zeros(1) e_max_all = np.zeros(1) # reduce to get global extents comm.Allreduce(e_min, e_min_all, op=MPI.MIN) comm.Allreduce(e_max, e_max_all, op=MPI.MAX) # compute bins on global extents bins = np.linspace(e_min_all, e_max_all) # get histogram counts for local data hist, bin_edges = np.histogram(e_vals, bins = bins) # declare var for reduce results hist_all = np.zeros_like(hist) # sum histogram counts with MPI to get final histogram comm.Allreduce(hist, hist_all, op=MPI.SUM)
The example above shows how a python script could be used to create a distributed-memory histogram of a mesh variable that has been published by a simulation.
import conduit import ascent.mpi # we treat everything as a multi_domain in ascent so grab child 0 n_mesh = ascent_data().child(0) ascent_opts = conduit.Node() ascent_opts['mpi_comm'].set(ascent_mpi_comm_id()) a = ascent.mpi.Ascent() a.open(ascent_opts) a.publish(n_mesh) actions = conduit.Node() scenes = conduit.Node() scenes['s1/plots/p1/type'] = 'pseudocolor' scenes['s1/plots/p1/params/field'] = 'radial_vert' scenes['s1/image_prefix'] = 'tout_python_mpi_extract_inception' add_act =actions.append() add_act['action'] = 'add_scenes' add_act['scenes'] = scenes actions.append()['action'] = 'execute' a.execute(actions) a.close()
In addition to performing custom python analysis, your can create new data sets and plot them through a new instance of Ascent. We call this technique Inception.
Relay extracts save data to the file system. Currently, Relay supports saving data to Blueprint HDF5, YAML, or JSON files. By default, Relay saves the published mesh data to the file system, but is a pipeline is specified, then the result of the pipeline is saved. Relay extracts can be opened by post-hoc tools such as VisIt.
conduit::Node pipelines; // pipeline 1 pipelines["pl1/f1/type"] = "contour"; // filter knobs conduit::Node &contour_params = pipelines["pl1/f1/params"]; contour_params["field"] = "radial_vert"; contour_params["iso_values"] = 250.; conduit::Node extracts; extracts["e1/type"] = "relay"; extracts["e1/pipeline"] = "pl1"; extracts["e1/params/path"] = output_file;
In this example, a contour of a field is saved to the file system in json form. To save the files in HDF5 format:
extracts["e1/params/protocol"] = "hdf5";
json are also valid
By default, the relay extract creates one file per mesh domain saved. You can control
the number of files written (aggregating multiple domains per file) using the
extracts["e1/params/num_files"] = 2;
Additionally, Relay supports saving out only a subset of the data. The
fields parameters is a list of
strings that indicate which fields should be saved.
The current ADIOS extract is experimental and this section is under construction.