Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Software pipelines”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

The Future of a Myriad of Accelerated Biodiscoveries Lies in AI‐Powered Mass Spectrometry and Multiomics Integration

The intersection of modern artificial intelligence (AI) and mass spectrometry (MS) is set to transform the MS‐based “omics” research fields, particularly proteomics, metabolomics, lipidomics, and glycomics, enabling advancements across a wide range of domains, from health to environment and industrial biotechnology. Beginning with an overview of key challenges inherent in MS software pipelines, this personal perspective explores how AI‐driven solutions can address them to enhance data processing, integration and interpretation. It proposes a paradigm shift in molecular identification and quantitation algorithms, leveraging AI to enable holistic interpretation of MS‐based multiomics data. While centered on MS‐based omics, this holistic AI‐driven paradigm is also critical for connecting dynamic biochemical changes to genomics and transcriptomics contexts, reinforcing the integrative value of MS in multiomics research. Ultimately, this AI‐driven approach could enhance efficiency, accuracy, and molecular breadth of coverage, deepening our systems‐level understanding of biological processes and accelerating a myriad of biodiscoveries.

47 OTHER INSTRUMENTATION↗
DOE OSTI2020

Rapid Annotation of Photosynthetic Systems (RAPS): automated algorithm to generate genome-scale metabolic networks from algal genomes

Algae have mounting potential to serve as platform strains for engineering efforts, but tools for these species lag behind traditional model systems like E. coli and yeast. Metabolic models are one such tool that would enhance our ability to rationally engineer algae but current automated algorithms to build models do not perform well for algal genomes. We introduce a novel software pipeline, Rapid Annotation of Photosynthetic Systems (RAPS), which leverages manual curation efforts of published models to create high quality first draft metabolic networks for new species. We compared models produced by our pipeline to published models and found that these models are able to capture more genes than the published models and are more predictive of experimentally determined growth rate. We used RAPS to automate the creation of 8 first draft metabolic models of new species to enable the further study of metabolism in these species.

09 BIOMASS FUELS↗
DOE OSTI2022

Inductive predictions of hydrologic events using a Long Short-Term Memory network and the Soil and Water Assessment Tool

We present machine learning methods to predict hydrologic features such as streamflow and soil moisture from spatially and temporally varying hydrological and meteorological data. Here, we used a temporal reduction technique to reduce computation and memory requirements and trained a Long Short-Term Memory (LSTM) network to predict soil moisture and streamflow over multiple watersheds. We show LSTM networks can be trained in a fraction of the time required by complex process-based and attention-based models such as Soil and Water Assessment Tool (SWAT) and GeoMAN without sacrificing accuracy. We also demonstrate that outside data - sourced from a watershed other than the target - can be used to train LSTM to comparable or even superior prediction accuracy. The success of LSTM in such spatially-inductive settings shows hydrologic features can be predicted with minimal prior knowledge of the watershed in question. Finally, we make all methodologies of this work publicly available as an end-to-end software pipeline that facilitates rapid prototyping of hydrologic learners.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Progress in end-to-end optimization of fundamental physics experimental apparata with differentiable programming

In this article we examine recent developments in the research area concerning the creation of end-to-end models for the complete optimization of measuring instruments. The models we consider rely on differentiable programming methods and on the specification of a software pipeline including all factors impacting performance — from the data-generating processes to their reconstruction and the inference on the parameters of interest — along with the careful specification of a utility function well aligned with the end goals of the experiment. Building on previous studies originated within the MODE Collaboration, we focus specifically on applications involving instruments for particle physics experimentation, as well as industrial and medical applications that share the detection of radiation as their data-generating mechanism. This report illustrates the most recent advancements in the area, and outlines, for each of the discussed applications as well as for automatic differentiation itself, ongoing and future work.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2020

Constructing Self-Labeled Materials Imaging Datasets from Open Access Scientific Journals with EXSCLAIM!

Due to recent improvements in image resolution and acquisition speeds, materials microscopy is experiencing an explosion in imaging data. Yet, despite the volume of images generated, the overall accessibility landscape is highly fragmented, as researchers who do release images to the public, often only do so as snapshots of their larger private dataset in context of scientific journal publications. The effort to automatically consolidate images and descriptive information from web-based platforms has garnered broad attention from the computer vision, language technologies, and chemistry/materials informatics communities. However, these methods are problematic for scientific figures because over 30% of figures are compound in nature, and it is the individual images themselves, paired with relevant context, that are necessary for construction a proper labeled dataset. To this end, we outline in this paper the design of a software pipeline for the automatic EXtraction, Separation, and Caption-based natural Language Annotation of IMages from scientific figures (EXSCLAIM!). Successful consolidation of materials imaging across literature sources will enhance navigation and searchability of materials microscopy images for both novice and experienced researchers, as well as establish the framework necessary for users to search by images, text, or some combination of both.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Optical emissivity dataset of multi-material heterogeneous designs generated with automated figure extraction

Optical device design is typically an iterative optimization process based on a good initial guess from prior reports. Optical properties databases are useful in this process but difficult to compile because their parsing requires finding relevant papers and manually converting graphical emissivity curves to data tables. Here, we present two contributions: one is a dataset of thermal emissivity records with design-related parameters, and the other is a software tool for automated colored curve data extraction from scientific plots. We manually collected 64 papers with 176 figures reporting thermal emissivity and automatically retrieved 153 colored curve data records. The automated figure analysis software pipeline uses Faster R-CNN for axes and legend object detection, EasyOCR for axes numbering recognition, and k-means clustering for colored curve retrieval. Additionally, we manually extracted geometry, materials, and method information from the text to add necessary metadata to each emissivity curve. Finally, we analyzed the dataset to determine the dominant classes of emissivity curves and determine the underlying design parameters leading to a type of emissivity profile.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Boosting RDataFrame performance with transparent bulk event processing

RDataFrame is ROOT’s high-level interface for Python and C++ data analysis. Since it first became available, RDataFrame adoption has grown steadily and it is now poised to be a major component of analysis software pipelines for LHC Run 3 and beyond. Thanks to its design inspired by declarative programming principles, RDataFrame enables the development of highperformance, highly parallel analyses without requiring expert knowledge of multi-threading and I/O: user logic is expressed in terms of self-contained, small computation kernels tied together by a high-level API. This design completely decouples analysis logic from its actual execution, and opens several interesting avenues for workflow optimization. In particular, in this work we explore the benefits of moving internal data processing from an event-by-event to a bulkby-bulk loop. This refactoring dramatically reduces the framework’s runtime overheads; in collaboration with the I/O layer it improves data access patterns; it exposes information that optimizing compilers might use to auto-vectorize the invocation of user-defined computations; finally, while existing user-facing interfaces remain unaffected, it becomes possible to additionally offer interfaces that explicitly expose bulks of events, useful e.g. for the injection of GPU kernels into the analysis workflow. In order to inform similar future R&D, design challenges will be presented, as well as an investigation of the relevant timememory trade-off backed by novel performance benchmarks.

Guiraud, Enrico↗
DOE OSTI2023

Agilent CRADA (Abstract)

The CRADA between Agilent Technologies Inc. and Battelle will focus on five software components as listed below: Prototype 4D Feature Finding functionality with a particular focus on recovering low level features and extending the bottom end dynamic range of IM-MS technology. Compare and contrast developments to current 4D Feature Finding capabilities. Highlight important algorithmic aspects employed. Implement the PNNL saturation correction algorithm. Agilent will give PNNL the needed data file access API and assistance in understanding it implementation and any needed instrumental aspects. Supported high resolution products to include Agilent’s TOF, QTOF and IM-QTOF mass spectrometers. PNNL will then work with Agilent to benchmark performance. Implementation of the PNNL Hadamard de-multiplexing algorithm. Agilent will give provide PNNL the needed date file access API access and as needed assistance in understanding the current Agilent multiplexed IM offering. PNNL will then work with Agilent on benchmark performance. Add ion mobility collision cross sections to existing and new metabolomic libraries for data analysis with Agilent’s informatics program MPP/ID Browser. PNNL will work with Agilent to create a software pipeline that takes data from chemical and metabolic standards and properly formats it for inclusion in MPP accessible libraries, using the collision cross section as a new separation dimension. Improvements of MPP multidimensional matching to identify metabolomic features using multiple characteristics beyond retention time and accurate mass. Most significantly matching will include analyte collision cross section with proposed support for sample fraction or RapidFire cartridge and fragmentation spectra. PNNL will work with Agilent to modify and improve the current MPP analysis pipeline to allow for creating, aligning, and identifying MS features defined by accurate mass, collision cross section and chromatographic retention time. As additional criteria such as fraction or RapidFire cartridge type are supported in the identification process, then they also will become part of the automation workflow. This includes the automation of said system to work with command line program (i.e. not a GUI) sufficient for programmatic execution in a pipeline.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Scalable Truck Charging Demand Simulation for Cost-Optimized Infrastructure Planning

This project developed a scalable, high-resolution model to simulate medium- and heavy-duty (MHD) electric truck charging demand and assess its impact on grid infrastructure. Using generative modeling, simulation, and cost optimization, the project delivered an end-to-end software pipeline and a library of 96 real-world scenarios for the Dallas–Houston megaregion. We demonstrated a modular architecture for transportation and grid modeling, implemented cost-optimized infrastructure planning methods, and quantified grid capital, operational, and environmental costs across a wide range of truck electrification scenarios. The results have been adopted by major utility stakeholders and contributed to regional planning efforts.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

AI-Driven Detector Design for the EIC (Final Technical Report)

We developed an optimization workflow based on DNN-based fast-simulation and reconstruction algorithms. We used these methods to advance the design of calorimeter systems for the Electron-Ion Collider (EIC). This DNN-driven optimization provides a blueprint for integrating gradient-based methods into detector-design workflows. All software pipelines and methods have been released publicly and incorporated into the EIC collaboration’s physics studies, broadening their impact. Three journal articles detailing the methods developed here serve as a reference for the design and optimal use of next generation high-granularity calorimeter systems in nuclear and particle physics.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2025

Hybrid learning techniques for scientific data reduction with performance guarantees

The research initiatives supported by the U.S. Department of Energy (DOE) Grant DE-SC0022265 are fundamentally aimed at pioneering advanced machine learning (ML) techniques for scientific data compression within high-performance computing (HPC) environments. This comprehensive body of work addresses the critical challenge posed by the exponential growth of data generated by scientific simulations in domains such as fusion energy, climate modeling, and computational fluid dynamics (CFD). A core objective is to develop compression algorithms that achieve substantial data reduction—often by orders of magnitude—while rigorously ensuring the fidelity of both the primary data (PD) and scientifically crucial derived quantities of interest (QoI). The methodologies deployed under this grant integrate sophisticated deep learning architectures, prominently featuring autoencoders, advanced generative models like conditional diffusion, and hybrid learning techniques. Key innovations include the development of Guaranteed Autoencoders (GAE) and the Guaranteed Conditional Diffusion with Tensor Correction (GCDTC) framework, which provide explicit, instance-level error bounds on reconstructed data. Furthermore, specialized strategies such as nonlinear constraint satisfaction are employed to preserve the integrity of QoI, a vital requirement for the trustworthiness of downstream scientific analyses. This research also focuses on the design and implementation of scalable, GPU-accelerated software pipelines that seamlessly integrate into existing HPC workflows, ensuring both computational efficiency and practical applicability. The CAESAR framework, for example, unifies foundation and generative models to create an adaptive and efficient compression solution for spatio-temporal scientific data. Collectively, these efforts represent a significant advancement in mitigating the scientific data deluge, enabling more effective data management, accelerated scientific discovery, and optimized utilization of HPC resources.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Final report- UFL - RAPIDS2: A SciDAC Institute for Computer Science, Data, and Artificial Intelligence

The research initiatives supported by the U.S. Department of Energy (DOE) Grant DE-SC0022265 are fundamentally aimed at pioneering advanced machine learning (ML) techniques for scientific data compression within high-performance computing (HPC) environments. This comprehensive body of work addresses the critical challenge posed by the exponential growth of data generated by scientific simulations in domains such as fusion energy, climate modeling, and computational fluid dynamics (CFD). A core objective is to develop compression algorithms that achieve substantial data reduction—often by orders of magnitude—while rigorously ensuring the fidelity of both the primary data (PD) and scientifically crucial derived quantities of interest (QoI). The methodologies deployed under this grant integrate sophisticated deep learning architectures, prominently featuring autoencoders, advanced generative models like conditional diffusion, and hybrid learning techniques. Key innovations include the development of Guaranteed Autoencoders (GAE) and the Guaranteed Conditional Diffusion with Tensor Correction (GCDTC) framework, which provide explicit, instance-level error bounds on reconstructed data. Furthermore, specialized strategies such as nonlinear constraint satisfaction are employed to preserve the integrity of QoI, a vital requirement for the trustworthiness of downstream scientific analyses. This research also focuses on the design and implementation of scalable, GPU-accelerated software pipelines that seamlessly integrate into existing HPC workflows, ensuring both computational efficiency and practical applicability. The CAESAR framework, for example, unifies foundation and generative models to create an adaptive and efficient compression solution for spatio-temporal scientific data. Collectively, these efforts represent a significant advancement in mitigating the scientific data deluge, enabling more effective data management, accelerated scientific discovery, and optimized utilization of HPC resources.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Pose Estimation for the Inspecta Project

In this addendum report we take the lessons learned from our initial generic object pose estimation research described in (Smartt, et al., 2025) and apply it to metal cup seals, which are attached to items after an international nuclear safeguards inspection (i.e., metal cup seals may be attached to containers of spent nuclear fuel after an inspector has measured and verified the material). We have also refined our software pipeline and training techniques since the final Inspecta report and will describe that process here. The objective of this task is to allow a robot to acquire images of the engraved seal characters to confirm that the correct seal is attached to the correct container. However, the seal may be attached in any orientation and therefore the robot manipulator arm with a camera needs to align with the normal face of the seal.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2024

Improving the Discovery Power of the Dark Energy Survey Gravitational Wave Program in Its Fourth Observing Campaign

The universe is growing at an accelerated rate, yet the several experiments and science cases used to measure this property yield conflicting results for the rate. Known colloquially as the Hubble tension problem, this discrepancy calls for a novel measurement of the Hubble Constant, H0, in order to be resolved. The Dark Energy Survey Gravitational Wave group (DESGW) aims to relieve this tension by performing a standard sirens measurement of H0 independent of traditional methods. To do so in the current era of data abundance and limited telescope time demands a robust, optimized, and complete science software pipeline to carefully balance the transition from gravitational wave data to state-of-the-art images, images to photometric data, and photometric data to identification and analysis of electromagnetic counterparts to gravitational wave signals. This thesis details the contributions of the author to the DESGW Search & Discovery Pipeline for the fourth observing run of the LIGO/Virgo/KAGRA gravitational wave detector network, as well as the science applications that have been pursued.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

BlendPATH (Blending Pipeline Analysis Tool for Hydrogen) [SWR-24-10]

BlendPATH provides case-by-case techno-economic analysis for potential projects where hydrogen is blended into a natural gas transmission pipeline. BlendPATH estimates 1) the transmission pipeline modifications and operating conditions necessary to blend hydrogen to a user specified volume faction of hydrogen in pipeline gas and 2) the incremental capital and operating expenses to prepare said transmission pipeline for hydrogen blending. BlendPATH is developed in Python and requires SAInt, a underlying commercial natural gas pipeline network modeling software, to run. Pipeline modification and operation condition estimation is guided by ASME B31.12. The intent of this software is to target application for projects in the initial project assessment stage and provide the user with the capability of assess promising opportunities before the use proceeds with further detailed pipeline evaluation based on a probable economic outcome.

Kee, Jamie↗
DOE OSTI2024

Mbin v1.0

The Mbin software, is a software toolkit that implements the IMG metagenome binning pipeline. The software allows the user to process input metagenome contigs, and produces metagenome assembled genomes (metagenome bins) and valuation metrics per bin including completion and contamination estimates, quality assignment, predicted lineage and eukaryotic potential. It is currently packed as a portable docker container and provides the advantage of running the process of binning and analysis of the bins generated, using a suite of tools run sequentially with controls in place to capture errors and optional arguments to run a modified version depending on individual needs and capabilities.

Varghese, Neha↗
DOE OSTI2022

Tsdat: An Open-Source Data Standardization Framework for Marine Energy and Beyond: Preprint

Many organizations are tasked with the collection and processing of large quantities of data from various measurement devices. Data reported from these sources are often not interoperable with datasets and software used by analysts and other organizations in the same field, introducing barriers for collaboration on large-scale projects. This poses a particular problem for cross-device comparisons and machine learning applications. To address these challenges, the open source Time-Series Data Pipelines (Tsdat) software was developed by a joint collaboration between Pacific Northwest National Laboratory, the National Renewable Energy Laboratory, and Sandia National Laboratories to facilitate collaboration and accelerate advancements in the Marine Energy domain through the development of an open-source ecosystem of tools. This paper will describe the Tsdat software and the data standards within which the framework operates. A beta version of the framework has been released and is currently being used by several projects in marine energy, wind energy, and building energy systems.

big data↗
DOE OSTI2021

Implementing the LSST Software Stack for DESGW processing and the Integration of Convolutional Neural Networks into the DESGW Pipeline

The Dark Energy Survey Gravitational Wave (DESGW) group strives to understand thelargescale structure of the universe and galaxies by looking for electromagnetic signatures intelescope images following gravitational wave detections. The DESGW group uses a processing pipeline to perform difference imaging to look for potential candidates. In order to perform these searches more effectively, we first explored using an alternative processing pipeline, the LSST Software Stack to analyze the telescope images. Because the LSST software stack is currently transitioning between its generation 2 and generation 3 system, we decided that while the software will be usable in the future once generation 3 is complete, at the moment its incompleteness makes it impractical to use. We then decided to work on improving the current pipeline by developing a module that integrates a Convolutional Neural Network (CNN) to test difference imaging products for bad subtractions due to misalignment.

Navarro, Alexander↗