SEARCH · Engineering Papers

Results for “Computational pipelines”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27

All sources NASA NTRS DOE OSTI

DOE OSTI2026

The Impact of Time-Aware Design Choices in ICS Anomaly Detection

Industrial control systems (ICS) remain vulnerable to increasingly sophisticated cyberattacks, yet evaluating anomaly detection models in these environments is challenging due to temporal dependencies, missing-not-at-random patterns, and extremely imbalanced datasets. These factors make common practices—especially random data splits and na¨ıve imputation— prone to severe temporal leakage, which can inflate reported performance and obscure real-world limitations. In this work, we systematically examine classical machine learning models, temporal deep learning architecture, and tensordecomposition– based methods on a gas-pipeline dataset using a fully temporally separated evaluation pipeline designed to mimic realistic deployment conditions. Our findings show that proper temporal handling and MNAR-aware preprocessing significantly alter the relative performance of popular anomaly-detection methods, providing practical guidance for designing reliable, leakage-resistant ICS intrusion-detection systems.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

SZ3: A Modular Framework for Composing Prediction-Based Error-Bounded Lossy Compressors

Today's scientific simulations require a significant reduction of data volume because of extremely large amounts of data they produce and the limited I/O bandwidth and storage space. Here, error-bounded lossy compression has been considered one of the most effective solutions to the above problem. In practice, however, the best-fit compression method often needs to be customized or optimized in particular because of diverse characteristics in different datasets and various user requirements on the compression quality and performance. In this paper, we address this issue with a novel modular, composable compression framework named SZ3. Our contributions are four-folds. (1) We develop SZ3 which features an innovative modular abstraction for the prediction-based compression framework, such that compression modules can be plugged in easily to create new compressors based on characteristics of data and user requirements. (2) We create a new compression pipeline by SZ3 for GAMESS data, which significantly improves the compression ratios over state-of-the-art compressors. (3) We develop an adaptive compression pipeline by SZ3 for APS data with minimal efforts, which leads to the best rate-distortion among all existing error-bounded lossy compressors for any bit-rate. (4) We compare the sustainability of SZ3 with leading error-bounded prediction-based compressors, and then demonstrate the necessity of diverse pipelines by integrating and evaluating several compression pipelines on diverse scientific datasets from multiple disciplines. Experiments show that SZ3 incurs very limited overhead in compressor integration and our customized compression pipelines lead to up to 20% improvement in compression ratios under the same data distortion, when compared with the best existing approach.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Rational design and testing of abiotic stress‐inducible synthetic promoters from poplar cis ‐regulatory elements

Summary Abiotic stress resistance traits may be especially crucial for sustainable production of bioenergy tree crops. Here, we show the performance of a set of rationally designed osmotic‐related and salt stress‐inducible synthetic promoters for use in hybrid poplar. De novo motif‐detecting algorithms yielded 30 water‐deficit (SD) and 34 salt stress (SS) candidate DNA motifs from relevant poplar transcriptomes. We selected three conserved water‐deficit stress motifs (SD18, SD13 and SD9) found in 16 co‐expressed gene promoters, and we discovered a well‐conserved motif for salt response (SS16). We characterized several native poplar stress‐inducible promoters to enable comparisons with our synthetic promoters. Fifteen synthetic promoters were designed using various SD and SS subdomains, in which heptameric repeats of five‐to‐eight subdomain bases were fused to a common core promoter downstream, which, in turn, drove a green fluorescent protein (GFP) gene for reporter assays. These 15 synthetic promoters were screened by transient expression assays in poplar leaf mesophyll protoplasts and agroinfiltrated Nicotiana benthamiana leaves under osmotic stress conditions. Twelve synthetic promoters were induced in transient expression assays with a GFP readout. Of these, five promoters (SD18‐1, SD9‐2, SS16‐1, SS16‐2 and SS16‐3) endowed higher inducibility under osmotic stress conditions than native promoters. These five synthetic promoters were stably transformed into Arabidopsis thaliana to study inducibility in whole plants. Herein, SD18‐1 and SD9‐2 were induced by water‐deficit stress, whereas SS16‐1, SS16‐2 and SS16‐3 were induced by salt stress. The synthetic biology design pipeline resulted in five synthetic promoters that outperformed endogenous promoters in transgenic plants.

09 BIOMASS FUELS↗

DOE OSTI2023

Digital Twin for Optimizing Real-time Economy of the Integrated Energy Systems

Economic and safe operation of integrated energy systems (IES) requires real-time optimization (RTO) of the control and actions conducted on each system component. In this regard, digital twins (DTs), which consist of a physical system, a virtual system, and the data communication that occurs between the two, are essential for effective RTO. Through the data warehouse, the virtual system is constantly updated with real-time data from the physical system, and functions as the model in the optimization framework. The reduced-order model of the dynamic process model in the virtual system is used in the optimization framework. The optimization results are then returned, via the data warehouse, as control actions to the physical system. This work demonstrates the software capabilities of DT assets for an IES in the context of preparing a DT for an experimental system comprised of Idaho National Laboratory (INL)’s Thermal Energy Delivery System and battery system. For the virtual demonstration, the DTs encompass (1) a physical system, including the Modelica models of the Thermal Energy Delivery System and the battery system; (2) virtual optimization via the Optimization of Real-Time Capacity Allocation (ORCA) platform; and (3) the open-source data warehouse software DeepLynx. This work assesses the performance of ORCA, which utilizes a reduced-order model built using the Risk Analysis Virtual Environment (RAVEN) and trained on the Modelica models and real-time data pipeline through the graph database hosted in DeepLynx. The proposed optimization workflow will be an RTO model based on DTs and the data they generate.

25 ENERGY STORAGE↗

DOE OSTI2025

NREL ASSIST Barge / Thermodynamic retrievals TROPoe v0.19

This dataset contains daily files with thermodynamic profiles retrieved with the optimal estimation physical retrieval TROPoe v0.19 (Turner and Löhnert, 2014; Turner and Blumberg, 2019; Turner and Löhnert, 2021). The profiles are retrieved every 10 minutes from instantaneous observations from the NREL ASSIST-II (SN 12) infrared spectrometer. Observations are noise-filtered but not averaged in time to minimize errors due to non-uniform clouds. Additional input data in TROPoe are cloud base height from a collocated NREL Vaisala CL51 ceilometer (when available) or vertically staring Halo XR lidar, and surface temperature, relative humidity, and pressure from the collocated Oregon State University met tower. The full pipeline to run the retrieval is available at https://github.com/StefanoWind/TROPoe_processor. In addition to these temporally resolved input data, TROPoe requires an a priori dataset (prior) that provides mean climatological estimates of thermodynamic profiles and specifies how temperature and humidity covary with height as an input (for details see, e.g., Djalalova et al., 2022). The prior is a key component of the retrieval and provides a constraint on the ill-posed inversion problem. A monthly prior was computed from operational radiosonde launches at Upton, NY.

17 WIND ENERGY↗

DOE OSTI2025

Rhode Island Site - NREL ASSIST Thermodynamic Retrievals TROPoe v0.19 / Derived Data

This dataset contains daily files with thermodynamic profiles retrieved with the optimal estimation physical retrieval TROPoe v0.19 (Turner and Löhnert, 2014; Turner and Blumberg, 2019; Turner and Löhnert, 2021). The profiles are retrieved every 10 minutes from instantaneous observations from the NREL ASSIST-II (SN 12) infrared spectrometer. Observations are noise-filtered but not averaged in time to minimize errors due to non-uniform clouds. Additional input data in TROPoe are cloud base height from the collocated NREL upgraded Galion lidar (Newsom et al., 2019), and surface temperature, relative humidity, and pressure from the collocated PNNL met tower. The full pipeline to run the retrieval is available at https://github.com/StefanoWind/TROPoe_processor. In addition to these temporally resolved input data, TROPoe requires an a priori dataset (prior) that provides mean climatological estimates of thermodynamic profiles and specifies how temperature and humidity covary with height as an input (for details see, e.g., Djalalova et al., 2022). The prior is a key component of the retrieval and provides a constraint on the ill-posed inversion problem. A monthly prior was computed from operational radiosonde launches at Upton, NY.

17 WIND ENERGY↗

DOE OSTI2026

NREL ASSIST Rhode Island / Thermodynamic retrievals TROPoe

This dataset contains daily files with thermodynamic profiles retrieved with the optimal estimation physical retrieval TROPoe (Turner and Löhnert 2014; Turner and Blumberg 2019; Turner and Löhnert 2021). The profiles are retrieved every 10 minutes from instantaneous observations from the NREL ASSIST-II (SN 11) infrared spectrometer. Observations are noise-filtered but not averaged in time to minimize errors due to non-uniform clouds. Additional input data in TROPoe are cloud base height from a collocated NREL Upgraded Galion lidar #1212-60 and temperature, RH, and pressure from collocated PNNL met station. The lidar beta is used to estimate the cloud base height through a modified algorithm by Newsom et al., 2019 used at ARM SGP. The full pipeline for running the retrieval is available at https://github.com/StefanoWind/TROPoe_processor. In addition to these temporally resolved input data, TROPoe requires an a priori dataset (prior) which provides mean climatological estimates of thermodynamic profiles and specifies how temperature and humidity covary with height as an input (for details see e.g. Djalalova et al. 2022). The prior is a key component of the retrieval and provides a constraint on the ill-posed inversion problem. A monthly prior was computed from operational radiosonde launches at Upton, NY.

17 WIND ENERGY↗

DOE OSTI2025

Optimizing inference of segmentation on high-resolution images in MLExchange

MLExchange is a machine learning (ML) operations platform providing web user-interfaces (UIs) for data visualization and analysis pipelines at synchrotron facilities. Among these UIs is the segmentation app which helps synchrotron users utilize ML algorithms to automatically segment high-resolution scientific images with minimal manual annotation effort. In this work, we share code optimizations that significantly speed up the segmentation inference workflow of large data in short time. By optimizing the sequence of CPU-GPU data transfers and introducing CPU parallelization to key operations, we improve the per-device, per-image frame computational efficiency and observe close to 3×$$\times$$ speedup over the original segmentation inference workflow run time when utilizing a single GPU. Further adaptations enabling multi-GPU inference yield more than 40×$$\times$$ speedup with 100 GPUs compared to the optimized single GPU inference workflow. This acceleration of the segmentation inference workflow will provide MLExchange users with easy access to segmentation results with little wait time.

Lu, Shizhao↗

DOE OSTI2023

From BEYONDPLANCK to COSMOGLOBE: Preliminary WMAP Q -band analysis

We present the first application of the COSMOGLOBE analysis framework by analyzing nine-year WMAP time-ordered observations that uses similar machinery to that of BEYONDPLANCK for the Planck Low Frequency Instrument (LFI). We analyzed only the Q-band (41 GHz) data and report on the low-level analysis process based on uncalibrated time-ordered data to calibrated maps. Most of the existing BEYONDPLANCK pipeline may be reused for WMAP analysis with minimal changes to the existing codebase. The main modification is the implementation of the same preconditioned biconjugate gradient mapmaker used by the WMAP team. Producing a single WMAP Q1-band sample requires 22 CPU-hrs, which is slightly more than the cost of a Planck 44 GHz sample of 17 CPU-hrs; this demonstrates that a full end-to-end Bayesian processing of the WMAP data is computationally feasible. In general, our recovered maps are very similar to the maps released by the WMAP team, although with two notable differences. In terms of temperature, we find a ~2 μK quadrupole difference that most likely is caused by different gain modeling, while in polarization we find a distinct 2.5 μK signal that has been previously referred to as poorly measured modes by the WMAP team. In the COSMOGLOBE processing, this pattern arises from temperature-to-polarization leakage from the coupling between the CMB Solar dipole, transmission imbalance, and sidelobes. No traces of this pattern are found in either the frequency map or TOD residual map, suggesting that the current processing has succeeded in modeling these poorly measured modes within the assumed parametric model by using Planck information to break the sky-synchronous degeneracies inherent in the WMAP scanning strategy.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2025

Large Language Model Integration for Knowledge Retrieval and Interaction for the DUNE Experiment

The Deep Underground Neutrino Experiment (DUNE) is a next-generation neutrino experiment that will generate an unprecedented volume of heterogeneous information-from documentation and technical notes to experimental data and reconstruction pipelines. Efficient knowledge retrieval and contextual understanding are increasingly critical for collaboration-wide productivity and onboarding. In this work, we present DUNE-GPT, a prototype framework that leverages large language models (LLMs) and retrieval-augmented generation (RAG) to enable natural-language querying of DUNE's internal documentation and technical resources. The system provides an intelligent interface for DUNE collaborators to interact with experiment-specific knowledge while maintaining data privacy and infrastructure compliance within Fermilab computing resources.

Rafique, A. [Argonne (main)]↗

DOE OSTI2025

Hierarchical Network Partitioning for Solution of Potential-Driven, Steady-State Nonlinear Network Flow Equations

The solution of potential-driven steady-state flow in large networks is a task which manifests in various engineering applications, such as transport of natural gas or water through pipeline networks. The resultant system of nonlinear equations depends on the network topology, and in general, there is no numerical algorithm that offers guaranteed convergence to the solution (assuming a solution exists). Some methods offer guarantees in cases where the network topology satisfies certain assumptions, but these methods fail for larger networks. On the other hand, the Newton-Raphson algorithm offers a convergence guarantee if the starting point lies close to the (unknown) solution. It would be advantageous to compute the solution of the large nonlinear system through the solution of smaller nonlinear sub-systems wherein the solution algorithms (Newton-Raphson or otherwise) are more likely to succeed. Here, this letter proposes and describes such a procedure, a hierarchical network partitioning algorithm that enables the solution of large nonlinear systems corresponding to potential-driven steady-state network flow equations.

42 ENGINEERING↗

DOE OSTI2023

CI/CD Pipeline and DevSecOps Integration for Security and Load Testing

Dominic D’Onofrio is currently a Junior studying Information and Technology at New Mexico Institute of Mining and Technology. He recently secured an Internship with NMCCoE, where he is involved with the TracerFIRE 12 project. Additionally, he is contributing to the load and security testing team by researching ways to implement pipelining and DevSecOps; this is his main project while he is at part time capacity for TracerFIRE 12. He is doing these projects to enhance his knowledge as a system administrator and gain a deeper understating of cybersecurity practices within national labs.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Identifying Critical Infrastructure in Imagery Data Using Explainable Convolutional Neural Networks

To date, no method utilizing satellite imagery exists for detailing the locations and functions of critical infrastructure across the United States, making response to natural disasters and other events challenging due to complex infrastructural interdependencies. This paper presents a repeatable, transferable, and explainable method for critical infrastructure analysis and implementation of a robust model for critical infrastructure detection in satellite imagery. This model consists of a DenseNet-161 convolutional neural network, pretrained with the ImageNet database. The model was provided additional training with a custom dataset, containing nine infrastructure classes. The resultant analysis achieved an overall accuracy of 90%, with the highest accuracy for airports (97%), hydroelectric dams (96%), solar farms (94%), substations (91%), potable water tanks (93%), and hospitals (93%). Critical infrastructure types with relatively low accuracy are likely influenced by data commonality between similar infrastructure components for petroleum terminals (86%), water treatment plants (78%), and natural gas generation (78%). Local interpretable model-agnostic explanations (LIME) was integrated into the overall modeling pipeline to establish trust for users in critical infrastructure applications. The results demonstrate the effectiveness of a convolutional neural network approach for critical infrastructure identification, with higher than 90% accuracy in identifying six of the critical infrastructure facility types.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

PDBx/mmCIF Ecosystem: Foundational Semantic Tools for Structural Biology

PDBx/mmCIF, Protein Data Bank Exchange (PDBx) macromolecular Crystallographic Information Framework (mmCIF), has become the data standard for structural biology. With its early roots in the domain of small-molecule crystallography, PDBx/mmCIF provides an extensible data representation that is used for deposition, archiving, remediation, and public dissemination of experimentally determined three-dimensional (3D) structures of biological macromolecules by the Worldwide Protein Data Bank (wwPDB, wwpdb.org). Extensions of PDBx/mmCIF are similarly used for computed structure models by ModelArchive (modelarchive.org), integrative/hybrid structures by PDB-Dev (pdb-dev.wwpdb.org), small angle scattering data by Small Angle Scattering Biological Data Bank SASBDB (sasbdb.org), and for models computed generated with the AlphaFold 2.0 deep learning software suite (alphafold.ebi.ac.uk). Community-driven development of PDBx/mmCIF spans three decades, involving contributions from researchers, software and methods developers in structural sciences, data repository providers, scientific publishers, and professional societies. Having a semantically rich and extensible data framework for representing a wide range of structural biology experimental and computational results, combined with expertly curated 3D biostructure data sets in public repositories, accelerates the pace of scientific discovery. Herein, we describe the architecture of the PDBx/mmCIF data standard, tools used to maintain representations of the data standard, governance, and processes by which data content standards are extended, plus community tools/software libraries available for processing and checking the integrity of PDBx/mmCIF data. Use cases exemplify how the members of the Worldwide Protein Data Bank have used PDBx/mmCIF as the foundation for its pipeline for delivering Findable, Accessible, Interoperable, and Reusable (FAIR) data to many millions of users worldwide.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2022

A Conditional Autoencoder for Galaxy Photometric Parameter Estimation

Astronomical photometric surveys routinely image billions of galaxies, and traditionally infer the parameters of a parametric model for each galaxy. This approach has served us well, but the computational expense of deriving a full posterior probability distribution function is a challenge for increasingly ambitious surveys. In this paper, we use deep learning methods to characterize galaxy images, training a conditional autoencoder on mock data. The autoencoder can reconstruct and denoise galaxy images via a latent space engineered to include semantically meaningful parameters, such as brightness, location, size, and shape. Our model recovers galaxy fluxes and shapes on mock data with a lower variance than the Hyper Suprime-Cam photometry pipeline, and returns reasonable answers even for inputs outside the range of its training data. When applied to data in the training range, the regression errors on all extracted parameters are nearly unbiased with a variance near the Cramr-Rao bound.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2021

Scalable FBP decomposition for cone-beam CT reconstruction

Filtered Back-Projection (FBP) is a fundamental compute intense algorithm used in tomographic image reconstruction. Cone-Beam Computed Tomography (CBCT) devices use a cone-shaped X-ray beam, in comparison to the parallel beam used in older CT generations. Distributed image reconstruction of cone-beam datasets typically relies on dividing batches of images into different nodes. This simple input decomposition, however, introduces limits on input/output sizes and scalability.We propose a novel decomposition scheme and reconstruction algorithm for distributed FPB. This scheme enables arbitrarily large input/output sizes, eliminates the redundancy arising in the end-to-end pipeline and improves the scalability by replacing two communication collectives with only one segmented reduction. Finally, we implement the proposed decomposition scheme in a framework that is useful for all current-generation CT devices (7th gen). In our experiments using up to 1024 GPUs, our framework can construct 40963 volumes, for real-world datasets, in under 16 seconds (including I/O).

Chen, Peng↗

DOE OSTI2023

Distributed Rules-Based Deconfliction of ADMS Applications: Part 1: Requirements & Decomposition

This work introduces a rules-based deconfliction methodology for resolving conflicting device control commands issued by advanced power applications considering a range of technical, economic, environmental, and social objectives. The methodology is designed to serve as one of multiple alternative implementations (along with application cooperation and global optimization) for the numerical component of the Deconfliction Pipeline. Development of the methodology is divided into two parts. This first document introduces the requirements, context, and methods for decomposing the deconfliction problem using the Laminar Coordination Framework and Variable Grid Structures. The deconfliction problem is decomposed into a distributed optimization problem based on the concept of quasi-static grid segments, which form independent distributed areas for control and coordination. Selection of the optimal number of decompositions of the deconfliction problem should be made based on a tradeoff analysis between computational speed and global optimality. This second document will define an initial set of technical, economic, and environmental criteria, as well as thirty specific qualitative rules that are used as part of the deconfliction methodology to eliminate non-viable setpoint alternatives. The deconfliction optimization problem is converted into a ranking of individual discrete setpoints, which are scored by the extent to which they satisfy specific decision criteria. The ranking is determined through the concepts of deconfliction exclusivity, priority, and preference. Several multi-criteria decision-making frameworks are examined with the simple multi-attribute rating technique exploiting ranks (SMARTER) recommended as a simple implementation alternative that aligns with the steps of the rules-based deconfliction methodology.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Transcript and blood-microbiome analysis towards a blood diagnostic tool for goats affected by Haemonchus contortus

The Alpine goat (Capra aegagrus hircus) is parasitized by the barber pole worm (Haemonchus contortus). Hematological parameters from transcript and metagenome analysis in the host are reflective of infestation. We explored comparisons between blood samples of control, infected, infected zoledronic acid-treated, and infected antibody (anti-γδ T cells) treated wethers under controlled conditions. Seven days post-inoculation (dpi), we identified 7,627 transcripts associated with the different treatment types. Microbiome measurements at 7 dpi revealed fewer raw read counts across all treatments and a less diverse microbial flora than at 21 dpi. This study identifies treatment specific transcripts and an increase in microflora abundance and diversity as wethers age. Further, F/B ratio reflect health, based on depression or elevation above thresholds defined by the baseline of non-infected controls. Forty Alpine wethers were studied where blood samples were collected from five goats in four treatment groups on 7 dpi and 21 dpi. Transcript and microbiome profiles were obtained using the Partek Flow (St. Louis, Missouri, USA) software suites pipelines. Inflammation comparisons were based on the Firmicutes/Bacteriodetes ratios that are calculated as well as the reduction of microbial diversity.

60 APPLIED LIFE SCIENCES↗