Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “task parallelism”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Accelerating Neutrino Event Generation in MARLEY Using CUDA-Based RNG and GPU Parallelization

MARLEY is a simulation tool that helps scientists study how low-energy neutrinos interact with matter. To work properly, MARLEY uses random numbers thousands of times in each simulation. These random numbers are important for modeling things like how neutrinos collide with atoms and what particles they produce. Right now, MARLEY runs on a regular computer processor (CPU) and uses a built-in random number generator called the Mersenne Twister. This setup works, but it can be slow, especially when trying to simulate many events. This research focuses on making MARLEY run faster by moving the random number generation and some of the repetitive calculations from the CPU to a graphics processing unit (GPU), which can handle many tasks at the same time. We use CUDA (a tool for programming NVIDIA GPUs) and cuRAND (a GPU-based random number library) to test faster alternatives to the current random number system. We compare different GPU-based generators, like curand_mtgp32, xorwow, and philox, to see which ones are the quickest and still give reliable results. Early tests show that using the GPU can make MARLEY simulations much faster. This project not only helps improve current simulation performance but also moves closer to a full simulation chain where all stages can run on modern GPU hardware.

Dunkley, Kimieka [Florida A-M]↗
DOE OSTI2024

HydraGNN_Predictive_GFM_2024 - Ensemble of predictive graph foundation models for ground state atomistic materials modeling

We provide the ensemble of fifteen pre-trained graph foundation models (GFMs) for atomistic materials modeling applications. Each one of the fifteen GFMs has been trained on five open-source datasets that (once aggregated) amount to over 154 million atomistic structures, which cover over two-thirds of the natural elements of the periodic table and that comprises a broad set of organic and inorganic compounds. This vast set of atomistic structures comprises ground state configurations that are dynamically stable (i.e., equilibrated structures with atomic forces approximately close to zero values) as well as dynamically unstable structures (i.e., non-equilibrium structures with non-negligible non-zero values of atomic forces). The ensemble of datasets aggregated does NOT include excited states. The datasets have been curated to remove atomistic structures with spectral norm of the force tensor above 100 eV/angstrom. Moreover, a linear term of the energy was computed for each dataset using a linear regression model that uses the chemical concentration of each natural element as regressor. The linear term predicted by the linear regression model has been subtracted from each original energy value to perform a re-alignment of the energy values across different electronic structures approximation theories performed to generate the diverse multi-source, multi-fidelity datasets. The folder "ADIOS_files" contains the set of pre-processed datasets in Adaptable I/O System (ADIOS) format (https://www.exascaleproject.org/research-project/adios/) that have been used for the development and training of GFMs in this work. The "ADIOS_files" directory contains 6 sub-directories named as follows: - ANI1x-v3.bp - MPTrj-v3.bp - OC2020-20M-v3.bp - OC2020-v3.bp - OC2022-v3.bp - qm7x-v3.bp Each sub-directory contains the pre-processed datasets converted in Adaptable I/O System (ADIOS) format (https://www.exascaleproject.org/research-project/adios/) that have been used to the development, training, and performance testing of the ensemble go predictive graph foundation models. Each GFM was developed using HydraGNN (https://github.com/ORNL/HydraGNN) as underlying graph neural network (GNN) architecture. The multi-task learning (MTL) capability of HydraGNN was used to simultaneously train the GFMs on labeled values for direct predictions of energy (a total system property of an atomistic structure that measures the chemical stability) and atomic forces (an atomic level property of an atomistic structure that measures the dynamical stability). The hyper parameters of the GFM have been tuned using scalable hyperparameter optimization (HPO) algorithms implemented in the software DeepHyper (https://github.com/deephyper/deephyper). The pre-training of each HPO trial was performed using distributed data parallelism (DDP) to scale the training across 128 compute nodes of the exascale OLCF supercomputer Frontier. Each HPO trial was trained only for 10 epochs and an early stopping was performed to avoid wasting significant computational resources on GNN architectures that were clearly underperforming. For each HPO trial, the 'omnistat' tool developed by (AMD Research - Advanced Micro Device) was used to measure the total energy consumption in kWh. The ensemble of GFMs was obtained by selecting the fifteen best performing HPO trials. Four models have been selected for their clear advantage in accuracy, and these are the GFMs with IDs 229, 156, 147, 260. Additional eleven models have been selected based on judicious balance between accuracy and energy consumption needed for training, and these are the GFMs with IDs 165, 78, 137, 1, 175, 171, 181, 67, 179, 167, 351. Each selected GFM of the ensemble was continued to cumulate a total of at most 30 epochs. In some cases, the total number of epochs actually performed was les than 30 due to two combined factors: (1) the size of the GFM (i.e., the number of model parameters to train) and (2) the total wall-clock time for which the computational resources could be allocated on OLCF-Frontier. The "Ensemble_of_models" directory contains 15 sub-directories named as follows: - gfm_0.229 - gfm_0.156 - gfm_0.147 - gfm_0.260 - gfm_0.165 - gfm_0.78 - gfm_0.137 - gfm_0.1 - gfm_0.175 - gfm_0.171 - gfm_0.181 - gfm_0.67 - gfm_0.179 - gfm_0.167 - gfm_0.351 Each one of these sub-directories refers to one of the fifteen HPO trials that have been selected to continue the pre-training with at most 30 epochs. With each sub-directory associated with a specific HPO trial, the following files can be found: - config.json: file for argument parsing to develop and train an HydraGNN architecture - gfm_0.ID_epoch_N.pk: file with model parameters for HPO ID trial after N epochs of training The ensemble of fifteen GFM architectures was used for (1) ensemble averaging to stabilize the predictions of energy and atomic forces after pre-training for post-processing analysis and (2) ensemble uncertainty quantification (UQ). The code used to develop, pre-train, and load the pre-trained models for post-processing analysis is available on the ORNL-GitHub at the following link: https://github.com/ORNL/HydraGNN/tree/Predictive_GFM_2024

36 MATERIALS SCIENCE↗
DOE OSTI2025

Geometric GNNs for charged particle tracking at GlueX

Nuclear physics experiments are aimed at uncovering the fundamental building blocks of matter. The experiments involve high-energy collisions that produce complex events with many particle trajectories. Tracking charged particles resulting from collisions in the presence of a strong magnetic field is critical to enable the reconstruction of particle trajectories and precise determination of interactions. It is traditionally achieved through combinatorial approaches that scale worse than linearly as the number of hits grows. Since particle hit data naturally form a point cloud and can be structured as graphs, graph neural networks (GNNs) emerge as an intuitive and effective choice for this task. In this study, we evaluate the GNN model for track finding on the data from the GlueX experiment at Jefferson Lab. We use simulation data to train the model and test on both simulation and real GlueX measurements. We demonstrate that GNN-based track finding outperforms the currently used traditional method at GlueX in terms of segment-based efficiency at a fixed purity while providing faster inferences. We show that the GNN model can achieve significant speedup by processing multiple events in batches, which exploits the parallel computation capability of graphical processing units (GPUs). Finally, we compare the GNN implementation on GPU and field-programmable gate array and describe the trade-off.

batched GNN pipeline↗
DOE OSTI2023

Scaling Resolution of Gigapixel Whole Slide Images Using Spatial Decomposition on Convolutional Neural Networks

Gigapixel images are prevalent in scientific domains ranging from remote sensing, and satellite imagery to microscopy, etc. However, training a deep learning model at the natural resolution of those images has been a challenge in terms of both, overcoming the resource limit (e.g. HBM memory constraints), as well as scaling up to a large number of GPUs. In this paper, we trained Residual neural Networks (ResNet) on 22,528 x 22,528-pixel size images using a distributed spatial decomposition method on 2,304 GPUs on the Summit Supercomputer. We applied our method on a Whole Slide Imaging (WSI) dataset from The Cancer Genome Atlas (TCGA) database. WSI images can be in the size of 100,000 x 100,000 pixels or even larger, and in this work we studied the effect of image resolution on a classification task, while achieving state-of-the-art AUC scores. Moreover, our approach doesn't need pixel-level labels, since we're avoiding patching from the WSI images completely, while adding the capability of training arbitrary large-size images. This is achieved through a distributed spatial decomposition method, by leveraging the non-block fat-tree interconnect network of the Summit architecture, which enabled GPU-to-GPU direct communication. Finally, detailed performance analysis results are shown, as well as a comparison with a data-parallel approach when possible.

Tsaris, Aristeidis (aris)↗
DOE OSTI2026

Model-based, in-situ, non-destructive qualification and certification of parts made by autonomous additive manufacturing

To address the significant productivity challenges associated with the qualification and certification (Q&C) tasks of additively manufactured (AM) parts, which have traditionally relied on rigorous post‐build inspection and testing, we propose an integrated framework that combines model‐based qualification and certification (MBQ&C) with autonomous additive manufacturing (AAM). MBQ&C employs high‐fidelity predictive models, developed within the Integrated Computational Materials Engineering (ICME) paradigm, to simulate process–structure–property–performance relationships for assessing a part’s fitness for use. Since predictive models are commonly machine learning (ML)-based or reduced-order surrogates of validated physics models, they run efficiently, enabling timely inference. In parallel, the self-driving AAM utilises ML-based adaptive, closed‐loop control strategies to avoid, mitigate, or repair defects and anomalies during fabrication, thereby increasing the likelihood of producing acceptable parts. A key feature of the combined AAM-MBQ&C framework is that predictive models explicitly incorporate defects or anomalies that persist after the build, using instance-specific data captured via in-situ sensing. This customisation enables a build‐specific assessment of fitness for use, rather than relying on nominal or generic parameters. Such individualised evaluation provides a robust basis for Q&C-related acceptance decisions relating to each build. Additionally, the rapid solution capabilities of ML or reduced-order models enable the determination of a part’s suitability for service shortly after build completion. As the framework matures, it has the potential to substantially reduce reliance on conventional point‐design approaches—such as time‐consuming post‐build computed tomography scanning and costly destructive testing. Thus, the AAM-MBQ&C framework represents a transformative, scalable strategy for quality assurance of AM components, as parts produced within a stable, validated, and certified envelope can be certified with reduced testing. Key benefits include: (1) significant gains in Q&C productivity through efficient, model-centric assessment; (2) performance-based classification of defects into critical and non-critical categories; (3) the ability to predict potential deviations in the performance of parts affected by real-time, adaptive process control interventions relative to those produced under a certified process, and (4) the enabling of virtual Q&C for service environments that are difficult, hazardous, or impractical to access or reproduce experimentally. Collectively, these capabilities strengthen the business case for AM, particularly for high‐consequence and mission‐critical applications. Finally, although this work focuses on powder-based AM, the proposed techniques could be extended to AM processes employing alternative feedstock forms.

Gunasegaram, Dayalan↗
DOE OSTI2020

Materials response under hypersonic flow conditions probed by multi-modal diagnostics in a benchtop wind tunnel

In this LDRD “Materials response under hypersonic flow conditions probed by multi-modal diagnostics in a benchtop wind tunnel” a peer-reviewed lab-scale hypersonic wind tunnel (WT) was planned to be assembled in-house, along with time and space resolved and related flow and materials diagnostics to derive high quality measurements and the science that is able to inform the development of our modeling and simulation codes for the hypersonic regime. The baseline capability to achieve Mach 3 flow was scheduled to be completed in Q4 2020, in the first year of the LDRD. However due to the COVID-19 delays and travel restrictions starting in March, 2020 this task could not be completed to carry out the materials response studies in-house, or those planned with external partners that were engaged during this LDRD (NASA Langley, NASA Ames, Texas A&M university, and Virginia Tech). Still, using our hypersonic computational fluid simulation codes, many of the hypersonic components have been designed (for Mach 3, 5, and 7) and procured (or in procurement phase), and a new dedicated hypersonic laboratory was facilitized, 2 WT test sections have been manufactured, and a Mach 3 nozzle was 3D printed for prototyping. In parallel, unique materials diagnostics have been tested under static conditions and fully specified and awaiting validation in our partner facilities. Due to the new hypersonic initiative announced in January, 2020 we proposed a related scope increase in hypersonics research requiring energy interactions and new diagnostics well beyond what could be supported by an LDRD ER. A decision was made to end the current LDRD ER and formally propose a new LDRD SI that leverages the current effort, which will be starting in FY21. As a result of the uniqueness of this LDRD work, our early efforts have been encouraged and well received in the hypersonic research community, and is now part of multiple unsolicited internal and external project proposals expected to be funded in the coming years, which is encouraging to make LLNL a leader in experimentally probing extreme physics of local energetics critical to hypersonics.

36 MATERIALS SCIENCE↗
DOE OSTI2023

CSPlib: A performance portable parallel software toolkit for analyzing complex kinetic mechanisms

Computational singular perturbation (CSP) is a method to analyze dynamical systems. It targets the decoupling of fast and slow dynamics using an alternate linear expansion of the right-hand side of the governing equations based on eigenanalysis of the associated Jacobian matrix. This representation facilitates diagnostic analysis, detection and control of stiffness, and the development of simplified models. For this work, we have implemented CSP in a C++ open-source library CSPlib using the Kokkos parallel programming model to address portability across diverse heterogeneous computing platforms, i.e., multi/many-core CPUs and GPUs. We describe the CSPlib implementation and present its computational performance across different computing platforms using several test problems. Specifically, we test the CSPlib performance for a constant pressure ignition reactor model on different architectures, including IBM Power 9, Intel Xeon Skylake, and NVIDIA V100 GPU. The size of the chemical kinetic mechanism is varied in these tests. As expected, the Jacobian matrix evaluation, the eigensolution of the Jacobian matrix, and matrix inversion are the most expensive computational tasks. When considering the higher throughput characteristic of GPUs, GPUs performs better for small matrices with higher occupancy rate. CPUs gain more advantages from the higher performance of well-tuned and optimized linear algebra libraries such as OpenBLAS.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Distributed Outage Detection in Power Distribution Networks

Real time topology knowledge is essential for situational awareness of power distribution networks. Line outages change the topology of a distribution network. Hence, outage detection is an important task. Most of the existing outage detection algorithms are centralized, in which sensors communicate their data to a control center which performs outage detection using the received data. However, with the increasing size of the distribution network and with different areas of the network being monitored by different operators, communication is a bottleneck and scalability is a major concern. To address these issues, we propose a novel outage detection algorithm using a divide and conquer approach. First, we divide a distribution network into sub-networks, such that outage detection can be run in parallel in each sub-network independently ensuring scalability to large networks. Further, to reduce the latency, bandwidth and attenuation challenges associated with communications in a large network, we divide each sub-network into multiple control areas which communicate only with their neighbors. We employ a distributed iterative load estimation across the control areas of each sub-network and then use the load estimate for local outage detection in each control area. Here, the performance of our algorithm is evaluated for multiple feeder models and compared against traditional centralized outage detection algorithms.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2022

HiPACE++: A portable, 3D quasi-static particle-in-cell code

Modeling plasma accelerators is a computationally challenging task and the quasi-static particle-in-cell algorithm is a method of choice in a wide range of situations. In this work, we present the first performance-portable, quasi-static, three-dimensional particle-in-cell code HiPACE++. By decomposing all the computation of a 3D domain in successive 2D transverse operations and choosing appropriate memory management, HiPACE++ demonstrates orders-of-magnitude speedups on modern scientific GPUs over CPU-only implementations. The 2D transverse operations are performed on a single GPU, avoiding time-consuming communications. The longitudinal parallelization is done through temporal domain decomposition, enabling near-optimal strong scaling from 1 to 512 GPUs. HiPACE++ is a modular, open-source code enabling efficient modeling of plasma accelerators from laptops to state-of-the-art supercomputers.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Proximity Portability and in Transit , M-to-N Data Partitioning and Movement in SENSEI [Book Chapter]

In high-performance parallel in situ processing, the term in transit processing refers to those configurations where data must move from a producer to a consumer that runs on separate resources. In the context of parallel and distributed computing on an HPC platform one of the central challenges is to determine a mapping of data from producer ranks to consumer ranks. This problem is complicated by the heterogeneity that arises in producer-consumer pairs, such as when producer and consumer codes have different levels of concurrency, different scaling characteristics, or different data models. The resulting mapping and movement of data from M producer to N consumer ranks can have a significant impact on aggregate application performance, particularly when the data consumer requires only a subset of the overall data for its task. This chapter focuses on the design considerations that underlie SENSEI’s implementation to this challenging problem. These design considerations extend the core SENSEI architecture and include ideas like the need to accommodate flexibility in the choice of different partitioning methods, the ability for a data consumer to request and receive only the subset of data needed for its particular operation, and the ability to leverage any of several different data transport tools. The idea of proximity portability, being able to use different data transport methods as part of an in transit workflow, is illustrated through the use of three different transport layers where switching from one transport tool to another is accomplished with only a configuration file change. Here, the chapter also includes a performance analysis summary showing the performance gains that are possible in terms of multiple metrics, such as memory footprint, time to solution, and amount of data moved, when using optimized partitioners in an in transit setting, gains that are made possible by the implementation shaped by specific design considerations.

Bethel, E. Wes↗
DOE OSTI2022

Implementation of a practical Markov chain Monte Carlo sampling algorithm in PyBioNetFit

Abstract Summary Bayesian inference in biological modeling commonly relies on Markov chain Monte Carlo (MCMC) sampling of a multidimensional and non-Gaussian posterior distribution that is not analytically tractable. Here, we present the implementation of a practical MCMC method in the open-source software package PyBioNetFit (PyBNF), which is designed to support parameterization of mathematical models for biological systems. The new MCMC method, am, incorporates an adaptive move proposal distribution. For warm starts, sampling can be initiated at a specified location in parameter space and with a multivariate Gaussian proposal distribution defined initially by a specified covariance matrix. Multiple chains can be generated in parallel using a computer cluster. We demonstrate that am can be used to successfully solve real-world Bayesian inference problems, including forecasting of new Coronavirus Disease 2019 case detection with Bayesian quantification of forecast uncertainty. Availability and implementation PyBNF version 1.1.9, the first stable release with am, is available at PyPI and can be installed using the pip package-management system on platforms that have a working installation of Python 3. PyBNF relies on libRoadRunner and BioNetGen for simulations (e.g. numerical integration of ordinary differential equations defined in SBML or BNGL files) and Dask.Distributed for task scheduling on Linux computer clusters. The Python source code can be freely downloaded/cloned from GitHub and used and modified under terms of the BSD-3 license (https://github.com/lanl/pybnf). Online documentation covering installation/usage is available (https://pybnf.readthedocs.io/en/latest/). A tutorial video is available on YouTube (https://www.youtube.com/watch?v=2aRqpqFOiS4&t=63s). Supplementary information Supplementary data are available at Bioinformatics online.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

A Length Adaptive Algorithm-Hardware Co-design of Transformer on FPGA Through Sparse Attention and Dynamic Pipelining

Transformers are considered one of the most important deep learning models since 2018, in part because it establishes state-of-the-art (SOTA) records and could potentially replace existing Deep Neural Networks (DNNs). Despite the remarkable triumphs, the prolonged turnaround time of Transformer models is a widely recognized roadblock. The variety of sequence lengths imposes additional computing overhead where inputs need to be zero-padded to the maximum sentence length in the batch to accommodate the parallel computing platforms. This paper targets the field-programmable gate array (FPGA) and proposes a coherent sequence length adaptive algorithm–hardware co-design for Transformer acceleration. Particularly, we develop a hardware-friendly sparse attention operator and a length-aware hardware resource scheduling algorithm. The proposed sparse attention operator brings the complexity of attention-based models down to linear complexity and alleviates the off-chip memory traffic. The proposed length-aware resource hardware scheduling algorithm dynamically allocates the hardware resources to fill up the pipeline slots and eliminates bubbles for NLP tasks. Experiments show that our design has very small accuracy loss and has 80.2 × and 2.6 × speedup compared to CPU and GPU implementation, and 4 × higher energy efficiency than state-of-the-art GPU accelerator optimized via CUBLAS GEMM.

Peng, Hongwu↗
DOE OSTI2020

P SI4 1.4: Open-source software for high-throughput quantum chemistry

PSI4 is a free and open-source ab initio electronic structure program providing implementations of Hartree–Fock, density functional theory, many-body perturbation theory, configuration interaction, density cumulant theory, symmetry-adapted perturbation theory, and coupled-cluster theory. Most of the methods are quite efficient, thanks to density fitting and multi-core parallelism. The program is a hybrid of C++ and Python, and calculations may be run with very simple text files or using the Python API, facilitating post-processing and complex workflows; method developers also have access to most of PSI4 's core functionalities via Python. Job specification may be passed using The Molecular Sciences Software Institute (MolSSI) QCSCHEMA data format, facilitating interoperability. We announce a rewrite of our top-level computation driver, and concomitant adoption of the MolSSI QCARCHIVE INFRASTRUCTURE project, makes the latest version of PSI4 well suited to distributed computation of large numbers of independent tasks. The project has fostered the development of independent software components that may be reused in other quantum chemistry programs.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Capacity Investment under Bayesian Information Updates at Reporting Periods: Model and Application

We consider capacity addition decisions by a new product manufacturer faced with uncertain technology alternatives. The manufacturing capacity addition and technology development occurs in parallel, with preliminary results from a technology project's success providing valuable information to the manufacturer in adding capacity. We solve a stochastic dynamic program with Bayesian updates to obtain the manufacturer's expected profit‐maximizing capacity investment decision. Our model and applications are motivated by the Critical Materials Institute (CMI) (funded by the Department of Energy), which manages research projects focused on mitigating critical material constraints, vital to renewable energy technologies such as direct‐drive wind turbines, electric vehicles, and energy‐efficient lighting. We capture three unique aspects of the problem: first, the learning from project progress depends on task‐based stochastic outcomes and a project's percent‐done. Second, the underlying technology's profitability is based on a model of competition. Third, we evaluate the impact of progress across a portfolio of projects based on a manufacturer's capacity addition. We develop a heuristic that produces results that are close to optimal and can thus be used for large problem sizes. The managerial insights from an application of our model to CMI projects include: (i) technology projects that report the percent‐done of a project earlier increase expected manufacturer profit; (ii) careful choice of “safe bets,” that is, technologies with low profitability but a high probability of success, can increase expected manufacturer profit; (iii) a portfolio of projects can increase profits significantly over separate project evaluation; and (iv) dynamic management of project resources can increase overall profit.

Vedantam, Aditya↗
DOE OSTI2024

FLARE: field line analysis and reconstruction for 3D boundary plasma modeling

The FLARE code is a magnetic mesh generator that is integrated within a suite of tools for the analysis of the magnetic geometry in toroidal fusion devices. A magnetic mesh is constructed from field line segments and permits fast reconstruction of field lines in 3D boundary plasma codes such as EMC3-EIRENE. Both intrinsically non-axisymmetric configurations (stellarators) and those with symmetry breaking perturbations of an axisymmetric equilibrium (tokamaks) are supported. The code itself is written in Modern Fortran with MPI support for parallel computing, and it incorporates object-oriented programming for the definition of the magnetic field and the material surface geometry. Extended derived types for a number of different magnetohydrodynamic equilibrium and plasma response models are implemented. The core element of FLARE is a field line tracer with adaptive step-size control, and this is integrated into tools for the construction of Poincaré maps and invariant manifolds of X-points. A collection of high-level procedures that generate output files for visualization is build on top of that. The analysis modules are build with Python frontends that facilitate customization of tasks and/or scripting of parameter scans.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Enabling machine learning-ready HPC ensembles with Merlin

With the growing complexity of computational and experimental facilities, many scientific researchers are turning to machine learning (ML) techniques to analyze large scale ensemble data. With complexities such as multi-component workflows, heterogeneous machine architectures, parallel file systems, and batch scheduling, care must be taken to facilitate this analysis in a high performance computing (HPC) environment. Here, we present Merlin, a workflow framework to enable large ML-friendly ensembles of scientific HPC simulations. By augmenting traditional HPC with distributed compute technologies, Merlin aims to lower the barrier for scientific subject matter experts to incorporate ML into their analysis. As a producer–consumer workflow model, Merlin enables multi-machine, cross-batch job, dynamically allocated yet persistent workflows capable of utilizing surge-compute resources. Key features of Merlin are a flexible HPC-centric interface, low per-task overhead, multi-tiered fault recovery, and a hierarchical sampling algorithm that allows for $\mathscr{O}$(N) task execution and $\mathscr{O}$(N ln N) task queuing to ensembles of millions of tasks. In addition to Merlin’s design, we test the algorithm’s performance in an HPC center and demonstrate the ability to enqueue 40 million simulations in 100 s, with a 30 millisecond per-task overhead that is independent of ensemble size. Finally, we describe some example applications that Merlin has enabled on leadership-class HPC resources, such as the ML-augmented optimization of nuclear fusion experiments and the calibration of infectious disease models to study the progression of and possible mitigation strategies for COVID-19.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Next-Generation Marine Energy Software Needs Assessment

Over the past decade the marine energy industry has continued to grow and evolve, with new concepts and technologies constantly being pursued. Additionally, the field of computing is vastly different today than it was five or ten years ago. By utilizing advanced software and hardware architectures, like graphics processing units as well as parallelization and high-performance computing resources, software can produce higher quality outputs and a higher volume of outputs. These software and hardware resources can enable the marine energy community to exploit computational advancements from other research fields, which can include machine learning, differentiable programming, and controls co-design. Better integration of existing software and development of potential new software is necessary to take advantage of trends in modern computing and respond to the current and future needs of the marine energy community. In order to better understand the existing marine energy software landscape and industry needs, DOE's Water Power Technologies Office (WPTO) tasked Sandia National Laboratories and the National Renewable Energy Laboratory to update the needs assessment by identifying existing software gaps and software needs, and assisting WPTO in planning the next wave of marine energy software development. The proposed effort involved cataloguing and analyzing the available data on existing software related to marine energy. The marine energy software landscape has vastly changed in the last ten years. There are now nearly 230 different software packages utilized by the marine energy sector, compared to a decade ago when the Cardinal Engineering survey identified approximately 40 software packages. In 2012, the marine energy software landscape was captured in two tables, whereas the current marine energy software landscape required development of a software database to collect and categorize software.

16 TIDAL AND WAVE POWER↗
DOE OSTI2021

Progress on Optimizing Wind Farms and Rotor Designs Using Adjoints

Modern wind plants are increasingly tasked with multiple performance objectives. In addition to designing plants that maximize power output and minimize the levelized cost of energy (LCOE), the design and operation of wind plants is increasingly influenced by challenges regarding grid integration of variable generation renewables. This places a growing emphasis on making wind plants more controllable and predictable. WindSE is a Reynolds-averaged Navier-Stokes (RANS) model designed around analytical gradient and adjoint methods, with the ability to capture terrain-induced effects, as shown in Figure 1. The recent addition of an unsteady solver with an actuator line method (ALM) and ongoing work to enable massively parallel optimizations gives it a unique niche to explore coupled plant-level controls and design problems. This code is an open source python package built on the FEniCS framework that utilizes fast, parallel PETSc solvers to model fluid flow throughout wind-farm scale domains. Two recent studies performed using WindSE demonstrate the capability to optimize under a wide variety of flow conditions and objective functions. In the first, we present an optimization focused on modifying the layout of a wind farm with a fixed number of turbines for maximum total power output [1]. This study highlights the ability to quickly perform simulations using the steady Navier-Stokes solver combined with rotors represented as actuator disks while also stressing the importance of capturing terrain-induced effects. Gradient-based optimization using the RANS equations is viable due to the inclusion of efficiently computed adjoint derivatives. We interpret the physical results of the optimal layout and also discuss the computational cost of scaling to larger problems. In the second study, we present the capabilities of the unsteady Navier-Stokes solver, where rotor-blade profiles represented by actuator lines are optimized to enhance wake steering effects and overall power production [2]. We quantify the wind plant performance gains obtained from this type of simultaneous control co-design optimization as compared to optimizing the blade design and yaw independently. Figure 2 shows the differences between a baseline two-turbine system and an optimized system where we fine-tune the blade chord profile. Results and challenges from each study are quickly summarized and used to motivate the current development efforts within WindSE. Current and future work is focused on enabling higher-resolution studies with more degrees of freedom through parallelization of both the simulation and optimization algorithms. We present benchmarking results to show that WindSE performs well in both weak- and strong-scaling tests and further demonstrate that the optimizer obtains the same convergence rates in both shared- and distributed-memory environments. Using larger wind farms, we can study deep-array effects within an optimization context, allowing the use of objective functions that have been previously unstudied. As an example, we present ongoing work on a blockage metric which characterizes the loss of available kinetic energy due to wake effects from multiple upstream turbines.

adjoint optimization↗