Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Open source software”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,081 records · Page 60

Software engineering to sustain a high-performance computing scientific application: QMCPACK

We provide an overview of the software engineering efforts and their impact in QMCPACK, a production-level ab-initio Quantum MonteCarlo open-source code targeting high-performance computing (HPC) systems. Aspects included are: (i) strategic expansion ofcontinuous integration (CI) targeting CPU, using GitHub Actions runners, and graphics processing units (GPU) in pre-exascalesystems, using self-hosted hardware; (ii) incremental reduction of memory leaks using sanitizers, (iii) incorporation of Dockercontainers for CI and reproducibility, and (iv) refactoring efforts to improve maintainability, testing coverage, and memory lifetime management. We quantify the value of these improvements by providing metrics to illustrate the shift towards a predictive, rather than reactive, sustainable maintenance approach. Our goal, in documenting the impact of these efforts on QMCPACK, is to contribute to the body of knowledge on the importance of research software engineering (RSE) for the sustainability of community HPC codes and scientific discovery at scale.

Godoy, William↗

XScan: An Integrated Tool for Understanding Open Source Community-Based Scientific Code

Many scientific communities have adopted community-based models that integrate multiple components to simulate whole system dynamics. The community software projects’ complexity, stems from the integration of multiple individual software components that were developed under different application requirements and various machine architectures, has become a challenge for effective software system understanding and continuous software development. The paper presents an integrated software toolkit called X-ray Software Scanner (in abbreviation, XScan) for a better understanding of large-scale community-based scientific codes. Our software tool provides support to quickly summarize the overall information of scientific codes, including the number of lines of code, programming languages, external library dependencies, as well as architecture-dependent parallel software features. The XScan toolkit also realizes a static software analysis component to collect detailed structural information and provides an interactive visualization and analysis of the functions. We use a large-scale community-based Earth System Model to demonstrate the workflow, functions and visualization of the toolkit. We also discuss the application of advanced graph analytics techniques to assist software modular design and component refactoring.

Zheng, Weijian↗

MetaboDirect: an analytical pipeline for the processing of FT-ICR MS-based metabolomic data

Background: Microbiomes are now recognized as the main drivers of ecosystem function ranging from the oceans and soils to humans and bioreactors. However, a grand challenge in microbiome science is to characterize and quantify the chemical currencies of organic matter (i.e., metabolites) that microbes respond to and alter. Critical to this has been the development of Fourier transform ion cyclotron resonance mass spectrometry (FT-ICR MS), which has drastically increased molecular characterization of complex organic matter samples, but challenges users with hundreds of millions of data points where readily available, user-friendly, and customizable software tools are lacking. Results: Here, we build on years of analytical experience with diverse sample types to develop MetaboDirect, an open-source, command-line-based pipeline for the analysis (e.g., chemodiversity analysis, multivariate statistics), visualization (e.g., Van Krevelen diagrams, elemental and molecular class composition plots), and presentation of direct injection high-resolution FT-ICR MS data sets after molecular formula assignment has been performed. When compared to other available FT-ICR MS software, MetaboDirect is superior in that it requires a single line of code to launch a fully automated framework for the generation and visualization of a wide range of plots, with minimal coding experience required. Among the tools evaluated, MetaboDirect is also uniquely able to automatically generate biochemical transformation networks (ab initio) based on mass differences (mass difference network-based approach) that provide an experimental assessment of metabolite connections within a given sample or a complex metabolic system, thereby providing important information about the nature of the samples and the set of microbial reactions or pathways that gave rise to them. Finally, for more experienced users, MetaboDirect allows users to customize plots, outputs, and analyses. Conclusion: Application of MetaboDirect to FT-ICR MS-based metabolomic data sets from a marine phage-bacterial infection experiment and a Sphagnum leachate microbiome incubation experiment showcase the exploration capabilities of the pipeline that will enable the research community to evaluate and interpret their data in greater depth and in less time. It will further advance our knowledge of how microbial communities influence and are influenced by the chemical makeup of the surrounding system. The source code and User’s guide of MetaboDirect are freely available through (https://github.com/Coayala/MetaboDirect) and (https://metabodirect.readthedocs.io/en/latest/), respectively.

54 ENVIRONMENTAL SCIENCES↗

NeuroCoreX: An Open-Source FPGA-Based Spiking Neural Network Emulator with On-Chip Learning

Spiking Neural Networks (SNNs) are computational models inspired by the event-driven communication and connectivity patterns of biological neural circuits. They enable high energy efficiency and natural support for diverse architectures ranging from layered networks to small-world and graphstructured topologies. In this work, we introduce NeuroCoreX, an open-source, FPGA-based spiking neural network emulator that provides real-time, on-chip learning and flexible network organization. NeuroCoreX supports both feedforward sensory inputs streamed directly from sensors or PCs via UART and recurrent on-chip connectivity, enabling simultaneous processing and learning from external stimuli and internal network dynamics-capabilities rarely available in existing FPGA SNN platforms. The system implements a Leaky Integrate-and-Fire (LIF) neuron model with current-based synapses and supports pair-based STDP learning on both feedforward and recurrent synapses. A lightweight Python interface enables interactive configuration, live monitoring, weight read-back, and experiment control. Importantly, NeuroCoreX is tightly integrated with the SuperNeuroMAT simulator, allowing SNN models to be transferred seamlessly from software to hardware for hardware-in-the-loop development. By combining real-time plasticity, flexible connectivity, and an open-source VHDL implementation, NeuroCoreX provides an extensible and accessible platform for neuromorphic research, algorithm-hardware co-design, and energy-efficient edge intelligence.

Gautam, Ashish [ORNL]↗

Establishing model credibility for process-microstructure-property relationships in additive manufacturing using exascale computing

Additive Manufacturing (AM) of alloys holds significant promise as a disruptive technology in various industries, yet its adoption is often hindered by challenges in achieving consistent part quality. These issues are primarily due to the complex process-microstructure-property (PSP) relationships inherent to AM. Computational models can greatly aid in understanding these relationships, but their widespread impact and adoption has been limited by a lack of validated, open-source, and computationally efficient PSP modeling frameworks and hardware limitations. Here, this study leverages the ExaAM software suite and data from the AMBench-2018 series of laser powder bed fusion (LPBF) benchmark experiments to perform a comprehensive model assessment, including verification, validation, sensitivity analysis, and uncertainty quantification. The RADICAL-EnTK workflow manager was used to perform an ensemble of heat transport, solidification, and mechanical response simulations on the exascale computer Frontier, considering uncertainties in critical model inputs such as laser spot size and nucleation parameters, and consisting of 125 explicit grain structure simulations and 7875 crystal plasticity simulations. For a selected location within the Inconel 625 AMBench-2018 test artifact, sensitivity analysis and uncertainty quantification were performed using the predicted distributions of grain structure and mechanical properties. Qualitative agreement was found between the predicted grain size and texture and the observed AMBench-2018 microstructure, the mean predicted yield stress was within 5% of the experimental measurement mean, and the mean predicted engineering stress at 5% strain was within 10% of the experimental measurement mean. The insights gained from development and validation of the ExaAM PSP modeling framework will help guide future directions for enhancing the credibility and reliability of PSP models in AM, thereby accelerating the adoption of AM technologies in various industries.

Additive manufacturing↗

Agile Components for Modeling Internal Flows

The purpose of this project was to rapidly demonstrate the capabilities of Sandia’s Agile Components toolkit by building a prototype simulator for modeling internal fluid flows and similar technologies of interest to GE Global Research. In doing so, this allowed the CRADA partner to learn about the Agile Components from the Sandia experts that designed and developed the toolkit. Sandia and DOE benefited from the application of Agile Components to another type of modeling and simulation application that helped to further generalize and harden the Agile Components software. As per the CRADA/PTS agreement, the resulting demonstration application can now be distributed under an open source license so that it can be made available for others to also learn from.

97 MATHEMATICS AND COMPUTING↗

Open-Source Architecture for Multi-Party Update Verification for Data Acquisition Devices

Power grids are integral parts of modern daily life and are increasingly under cyberattack. Common software update processes for grid control devices rely on only one organization to provide verification. This paper roposes a multi-signature software update process to help better secure data acquisition devices from malicious actions by using standard cryptosystems such as TLS. A prototype system build on Linux and off-the-shelf hardware has shown successful update and attack prevention with our customized multi-party update software.

Newberg, Benjamin↗

Software stewardship and advancement of a high-performance computing scientific application: QMCPACK

Here, we provide an overview of the software engineering efforts and their impact in QMCPACK, a production-level ab-initio Quantum Monte Carlo open-source code targeting high-performance computing (HPC) systems. Aspects included are: (i) strategic expansion of continuous integration (CI) targeting CPUs, using GitHub Actions own runners, and NVIDIA and AMD GPUs used in pre-exascale systems, (ii) incremental reduction of memory leaks using sanitizers, (iii) incorporation of Docker containers for CI and reproducibility, and (iv) refactoring efforts to improve maintainability, testing coverage, and memory lifetime management. We quantify the value of these improvements by providing metrics to illustrate the shift towards a predictive, rather than reactive, maintenance approach. Our goal, in documenting the impact of these efforts on QMCPACK, is to contribute to the body of knowledge on the importance of research software engineering (RSE) for the stewardship and advancement of community HPC codes to enable scientific discovery at scale.

97 MATHEMATICS AND COMPUTING↗

Geospatial Data from the Alpine Treeline Warming Experiment (ATWE) on Niwot Ridge, Colorado, USA

This is a collection of all GPS- and computer-generated geospatial data specific to the Alpine Treeline Warming Experiment (ATWE), located on Niwot Ridge, Colorado, USA. The experiment ran between 2008 and 2016, and consisted of three sites spread across an elevation gradient. Geospatial data for all three experimental sites and cone/seed collection locations are included in this package. –––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––Geospatial files include cone collection, experimental site, seed trap, and other GPS location/terrain data. File types include ESRI shapefiles, ESRI grid files or Arc/Info binary grids, TIFFs (.tif), and keyhole markup language (.kml) files. Trimble-imported data include plain text files (.txt), Trimble COR (CorelDRAW) files, and Trimble SSF (Standard Storage Format) files. Microsoft Excel (.xlsx) and comma-separated values (.csv) files corresponding to the attribute tables of many files within this package are also included. A complete list of files can be found in this document in the “Data File Organization” section in the included Data User's Guide. Maps are also included in this data package for reference and use. These maps are separated into two categories, 2021 maps and legacy maps, which were made in 2010. Each 2021 map has one copy in portable network graphics (.png) format, and the other in .pdf format. All legacy maps are in .pdf format. .png image files can be opened with any compatible programs, such as Preview (Mac OS) and Photos (Windows). All GIS files were imported into geopackages (.gpkg) using QGIS, and double-checked for compatibility and data/attribute integrity using ESRI ArcGIS Pro. Note that files packaged within geopackages will open in ArcGIS Pro with “main.” preceding each file name, and an extra column named “geom” defining geometry type in the attribute table. The contents of each geospatial file remain intact, unless otherwise stated in “niwot_geospatial_data_list_07012021.pdf/.xlsx”. This list of files can be found as an .xlsx and a .pdf in this archive.As an open-source file format, files within gpkgs (TIFF, shapefiles, ESRI grid or “Arc/Info Binary”) can be read using both QGIS and ArcGIS Pro, and any other geospatial softwares. Text and .csv files can be read using TextEdit/Notepad/any simple text-editing software; .csv’s can also be opened using Microsoft Excel and R. .kml files can be opened using Google Maps or Google Earth, and Trimble files are most compatible with Trimble’s GPS Pathfinder Office software. .xlsx files can be opened using Microsoft Excel. PDFs can be opened using Adobe Acrobat Reader, and any other compatible programs. A selection of original shapefiles within this archive were generated using ArcMap with associated FGDC-standardized metadata (xml file format). We are including these original files because they contain metadata only accessible using ESRI programs at this time, and so that the relationship between shapefiles and xml files is maintained. Individual xml files can be opened (without a GIS-specific program) using TextEdit or Notepad. Since ESRI’s compatibility with FGDC metadata has changed since the generation of these files, many shapefiles will require upgrading to be compatible with ESRI’s latest versions of geospatial software. These details are also noted in the “niwot_geospatial_data_list_07012021” file.

54 ENVIRONMENTAL SCIENCES↗

Python ARM Radar Toolkit (Py-ART) Roadmap

Radar software is key for producing scientific insight from radar data. There are several platforms for interacting with radar data; the open-source variants are well documented in Heistermann et al. 2014. The Python ARM Radar Toolkit (Py-ART; Helmus and Collis, 2016) is one of these. This document is a guiding roadmap for development for Py-ART. It sets priorities for U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) Climate Research Facility-funded work and it advises the community which contributions are desired and will receive the most assistance from ARM-funded software engineers.

Python ARM Radar Toolkit, linear programming, valu↗

Automated Energy-Dispersive X-ray Spectroscopy Analysis for Multi-Modal Few-Shot Learning

Scanning transmission electron microscopy (STEM) is a powerful tool that allows for the atomic-scale analysis of a materials’ structure, chemistry, and defect domains (Akers et al. 2021). The current generation of microscopes generate vast amounts of data, surpassing the limits of effective manual analysis traditionally performed by domain experts (Spurgeon et al. 2021). While recent strides in machine learning have significantly enhanced the processing of large and intricate datasets acquired through electron microscopy, the prevalent use of proprietary software packages for initial data collection poses a challenge. In many cases, these software packages act as a ‘black box’, constraining user functionality and hindering the output of data in a format that is conducive to seamless integration into machine learning models. This work addresses these challenges by adapting HyperSpy, an open-source Python library, for the analysis and quantification of raw energy dispersive spectroscopy (EDS) data acquired through STEM. The modified HyperSpy code successfully facilitates user-defined segmentation of the data, enabling the integration of atomic %, weight %, and raw EDS spectra for each segmented region into an existing few-shot machine learning model. While initial results reveal discrepancies in quantified atomic and weight percentages when compared to proprietary software, ongoing efforts aim to rectify this issue by refining the fit of the HyperSpy model to the EDS spectra. Overall, this research underscores the potential of open-source tools like HyperSpy to enhance the accessibility of analytical tools, fostering a transparent and user-friendly environment for seamlessly incorporating electron microscopy data into machine learning models.

36 MATERIALS SCIENCE↗

Development, validation, and verification of multi-pass thermo-mechanical welding simulations using the open-source MOOSE framework: NeT TG4 benchmark weldment

This study develops and validates a sequentially coupled thermo-mechanical welding simulation for the three-pass 316L stainless steel NeT TG4 benchmark weldment using the open-source Multiphysics Object-Oriented Simulation Environment (MOOSE) and the Nuclear Engineering Material model Library (NEML). A diffused ellipsoidal heat source was calibrated against thermocouple data and weld macrographs to accurately model the fusion zone geometry and transient thermal fields. Material hardening is represented using the Lemaitre-Chaboche mixed isotropic-kinematic hardening model, while four annealing models - no annealing, single-stage at 1050 °C and 1300 °C, and two-stage at 800 °C/1300 °C - were implemented to assess the impact of annealing models on the accuracy of the predicted welding-induced plasticity, distortions, and residual stresses. The predictions were validated against experimental measurements and benchmarked against results from commercial software, demonstrating that thermo-mechanical MOOSE welding simulations achieve comparable accuracy with enhanced computational efficiency. This work highlights the potential of using open-source finite element frameworks like MOOSE for advanced manufacturing simulations.

Ji, Wendy [Australian Nuclear Science and Technolo↗

Making Uintah Performance Portable for Department of Energy Exascale Testbeds

To help ease ports to forthcoming Department of Energy (DOE) exascale systems, testbeds have been made available to select users. These testbeds are helpful for preparing codes to run on the same hardware and similar software as in their respective exascale systems. This paper describes how the Uintah Computational Framework, an open-source asynchronous many-task (AMT) runtime system, has been modified to be performance portable across the DOE Crusher, DOE Polaris, and DOE Sunspot testbeds in preparation for portable simulations across the exascale DOE Frontier and DOE Aurora systems. The Crusher, Polaris, and Sunspot testbeds feature the AMD MI250X, NVIDIA A100, and Intel PVC GPUs, respectively. This performance portability has been made possible by extending Uintah’s intermediate portability layer [18] to additionally support the Kokkos::HIP, Kokkos::OpenMPTarget, and Kokkos::SYCL back-ends. This paper also describes notable updates to Uintah’s support for Kokkos, which were required to make this extension possible. Results are shown for a challenging radiative heat transfer calculation, central to the University of Utah’s predictive boiler simulations. These results demonstrate single-source portability across AMD-, NVIDIA-, and Intel-based GPUs using various Kokkos back-ends.

Holmen, John↗

Coupling Subsurface and Above-Surface Models for Optimizing the Design of Borefields and District Heating and Cooling Systems

Accurate dynamic energy simulation is important for the design and sizing of district heating and cooling systems with geothermal heat exchange for seasonal energy storage. Current modeling approaches in building and district energy simulation tools typically consider heat conduction through the ground between boreholes without flowing groundwater. While detailed simulation tools for subsurface heat and mass transfer exist, these fall short in simulating above-surface energy systems. To support the design and operation of such systems, the study developed a coupled model including a software package for building and district energy simulation, and software for detailed heat and mass transfer in the subsurface. For the first, it uses the open-source Modelica Buildings Library, which includes dynamic simulation models for building and district energy and control systems. For the heat and mass transfer in the soil, it uses the TOUGH simulator. The TOUGH family of codes can model heat and multi-phase, multi-component mass transport for a variety of fluid systems, as well as chemical reactions, in fractured porous media. The study validated the coupled modeling approach by comparing the simulation results with one from the g-function based ground response model. It then looked into effects when the water table and the regional groundwater flow are considered in the ground, from the perspective of heat exchange between borehole and ground, and the electrical consumption of the district heating and cooling systems. To access the simulation models, please find the links in the submission: -- For coupled approach validation: see model Buildings.Fluid.Geothermal.Borefields.Examples.BorefieldsWithTough and Buildings.Examples.DistrictReservoirNetworks.Examples.Reservoir3Variable_TOUGH from the "Modelica Building Library" resource, branch issue1495_tough_interface, commit a2667c0. -- For the study of the effect of water table: see model Buildings.Examples.DistrictReservoirNetworks.Examples.Reservoir3Variable_TOUGH from he "Modelica Building Library" resource, branch issue1495_tough_interface_moreIO, commit 760de49. -- For the study of the effect of regional groundwater flow: see Buildings.Examples.DistrictReservoirNetworks.Examples.Reservoir3Variable_TOUGH from he "Modelica Building Library" resource, branch issue1495_tough_interface_moreIO_3D, commit c2a2d2a. The coupling interface script "GrounResponse.py" can be found from the above links in the folder Buildings/Resources/Python-Sources. Also, the needed files for TOUGH simulation are in the folder Buildings/Resources/Python-Sources/ToughFiles that can be accessed through the above links. A brief description of these files is given below; detailed specifications for the first three files may be found in the TOUGH3 Users Guide (Jung et al., 2018) https://tough.lbl.gov/documentation/tough-manuals/. (1) INCON - initial conditions for each grid block (2) INFILE - main input file with material properties and control parameters (3) MESH - description of the computational grid (4) readsave - Modelica/TOUGH interface program: read the final output of TOUGH simulation after TOUGH time step and prepare for transfer to Modelica for next Modelica time step (5) readsave.inp - input parameters for program readsave (6) writeincon - Modelica/TOUGH interface program: write the output of Modelica after Modelica time step and prepare for transfer to TOUGH as initial conditions for the next TOUGH step (7) writeincon.inp - input parameters for program writeincon

15 GEOTHERMAL ENERGY↗

Generic and ML Workloads in an HPC Datacenter: Node Energy, Job Failures, and Node-Job Analysis

HPC datacenters offer a backbone to the modern digital society. Increasingly, they run Machine Learning (ML) jobs next to generic, compute-intensive workloads, supporting science, business, and other decision-making processes. However, understanding how ML jobs impact the operation of HPC datacenters, relative to generic jobs, remains desirable but understudied. In this work, we leverage long-term operational data, collected from a national-scale production HPC datacenter, and statistically compare how ML and generic jobs can impact the performance, failures, resource utilization, and energy consumption of HPC datacenters. Our study provides key insights, e.g., ML-related power usage causes GPU nodes to run into temperature limitations, median/mean runtime and failure rates are higher for ML jobs than for generic jobs, both ML and generic jobs exhibit highly variable arrival processes and resource demands, significant amounts of energy are spent on unsuccessfully terminating jobs, and concurrent jobs tend to terminate in the same state. We open-source our cleaned-up data traces on Zenodo (https://doi. org/10.5281/zenodo.13685426), and provide our analysis toolkit as software hosted on GitHub (https://github.com/atlarge-research/2024-icpads-hpc-workload-characterization). This study offers multiple benefits for data center administrators, who can improve operational efficiency, and for researchers, who can further improve system designs, scheduling techniques, etc.

crossanalysis↗

Leveraging Shore-Side, Building Energy Simulation Tools for Use in the Shipboard Environment

Within the confines of the Maritime Prescreening Assessment of Conservation Technologies program, the U.S. Department of Energy's National Renewable Energy Laboratory, in collaboration with the Naval Surface Warfare Center Carderock and Philadelphia Divisions, the U.S. Department of Transportation's Maritime Administration, and the Massachusetts Maritime Academy, recently developed a physics-based modeling engine to evaluate non-propulsive ship energy consumption under different operational scenarios and technology applications, in order to enhance the Navy's ability to optimize ship operational reach and tactical performance. To accomplish this, a mature thermophysical modeling software, EnergyPlus TM , was modified for ship-related applications. This software, originally developed for use in whole-building energy consumption studies, is a free and open-source simulation program that has been used for more than two decades. The team evaluated the accuracy of the modified version of EnergyPlus by comparing results to direct measurements taken over a 6-week underway period onboard the Maritime Administration's Training Ship Kennedy. The team developed an energy model of a prescribed area of the ship and compared its performance to measured results obtained from 235 sensor points. EnergyPlus behaved well with respect to solar loading and convective heat transfer to seawater while the ship was underway. However, results suggest that additional work remains to accurately simulate seawater heat transfer while the ship is stationary.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Cardinal: Seismic and Geoacoustic Array Processing

Data collected via seismic and infrasound array deployments are leveraged in the geosciences to detect and characterize a myriad of natural and anthropogenic sources. These deployments consist of numerous sensors placed in a predetermined configuration to amplify signal strength and improve the efficacy of array processing techniques used to measure signal directionality and waveform coherence. High‐fidelity feature extraction is often predicated on interstation distance as well as the frequency content and wavelength of an incident signal. Numerous array processing softwares analyze data in sequential frequency bands to obtain a more detailed characterization of a signal. However, current algorithms are limited in their ability to determine optimal array configuration for each band. We introduce an open‐source Python code, called Cardinal, to process seismic and infrasound array data in discretized time–frequency space with the option of applying an adaptive array design to determine optimal subarray configuration for each frequency band. To reduce computational time, the array processing step can be run in parallel using multithreading. Furthermore, the software has the capability to aggregate array processing results from different time–frequency pixels to produce separate sets of detections, or families, with added utility via the application of an adaptive semblance threshold, which aids in isolating signals‐of‐interest from coherent background noise. Upon appropriate configuration, Cardinal exhibits the potential to combine distinct seismic and infrasound phases into separate families.

Adaptive Array↗

FAST Modularization Framework for Wind Turbine Simulation: Full-System Linearization: Preprint

The wind engineering community relies on multiphysics engineering software to run nonlinear time-domain simulations e.g. for design-standards-based loads analysis. Although most physics involved in wind energy are nonlinear, linearization of the underlying nonlinear system equations is often advantageous to understand the system response and exploit well-established methods and tools for analyzing linear systems. This paper presents the development and verification of the new linearization functionality of the open-source engineering tool FAST v8 for land-based wind turbines, as well as the concepts and mathematical background needed to understand and apply it correctly.

17 WIND ENERGY↗