Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data acquisition workflows”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

A Review of Visualization Methods for Cyber-Physical Security: Smart Grid Case Study

Cyber-Physical Systems (CPSs) are becoming increasingly complex and interconnected as they attempt to meet the demands of evolving society. As a result, monitoring and maintaining them becomes a more complex and demanding task for control system operators and cyber defenders. While the literature on visualization techniques in the context of cybersecurity is extensive, the same cannot be said for studies on visualization for the security of cyber-physical systems. This paper aims to fill that gap by: 1) defining the main features of a visualizations workflow for security visualizations in cyber-physical systems. The workflow includes the acquisition of cyber and physical data, processing of data, selection, and configuration of both visualization tools and end-user interactions. 2) Providing an overview of cyber-physical security visualization systems, with a focus on smart grids as a case study. Finally, we use the perspectives gained from this analysis to provide insights and directions for future research and design of cyber-physical visualization techniques.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Beyond the Hype: Navigating the Promise and Pitfalls of Multi-Modal Models for Materials Science

Multi-modal models offer great potential for accelerating discovery in materials and chemical systems, but their adoption raises crucial questions: What materials science challenges are best addressed by multi-modal approaches? How do we weigh the benefits against the resource investment required for multi-modal data acquisition? And critically, how can we optimize experimental workflows to leverage these models effectively? In this presentation, I will delve into the development of multi-modal characterization and analytics, focusing on their application in the demanding fields of next-generation microelectronics and energy storage materials. I will share challenges encountered in designing these workflows, highlighting lessons learned and posing questions that remain unanswered.

AI↗
DOE OSTI2024

Massive all-atom analysis of 2D materials with quantum properties (Final report)

Improvements in microscopy have enabled the acquisition of data at a scale that is difficult to process manually, making automated machine learning approaches to analyzing experimental images essential. In this project, we developed and applied machine learning (ML) workflows for atomic resolution scanning transmission electron microscopy (STEM) images. This development included improving both methodology as well as generating user-friendly codes. We developed machine learning architectures which, after training, automatically identify the location and types of defects throughout a material. We used these data to produce class-averaged images of 2D atomic coordinates with up to 0.3 pm precision, uncovering the structure and oscillations of long-range strain fields around point defects in WSe 2-2x Te 2x . We also resolved a long-standing problem in this field in the training of ML models, a lack of labeled experimental data, by developing a cycle-GAN that transformed simulated-generated labeled data into labeled data indistinguishable from experiment and therefore suitable for training. This removed the remaining parts of the ML data processing workflow where human intervention was still critical and therefore a bottleneck to working at scale. Codes have been developed and released for this full machine learning workflow. ML approaches to partially automate STEM acquisition were also developed. Finally we applied ML and other advanced data processing methods to several materials science problems in two-dimensional materials, including studying the evolution of hyperuniformity with defect concentration in WSe2, understanding phase transformations in transition metal dichalcogenides during in-situ heating in the STEM, and exploring how 2D interfaces transform from twisted into aligned structures.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2021

Development of Time Lapse VSP Integration Workflow: A Case Study at Farnsworth CO2-EOR Project

Abstract This study aims to develop a 4D Vertical Seismic Profile (VSP) integration workflow to improve the prediction of subsurface stress changes. The selected study site is a 5-spot pattern within the ongoing CO2-EOR operations at the Farnsworth Field Unit FWU in Ochiltree County, Texas. The specific pattern has undergone extensive geological and geomechanical characterization through the acquisition of 3D seismic data, geophysical well logs, and core. This workflow constrains a numerical hydromechanical model by applying a penalty function formed between "modeled" versus "observed" time-lapse compressional and shear seismic velocity changes. Analyses of geophysical logs and ultra-sonic measurements on core exhibit measurable sensitivities to changes in both fluid saturation and mean effective stress. These data are used to develop a site-specific rock physics model and stress-velocity relationship, which inform the numerical models used to generate the "modeled" portion of the penalty function. The "observed" portion of the penalty function is provided by a novel elastic full-waveform inversion of the available 3D baseline and three monitor surveys to produce high-quality estimates of time-lapse compressional and shear seismic velocity changes. The modeling workflow accounts sequentially for fluid substitution and stress impacts. Hydrodynamic and geomechanical properties of the 3D coupled numerical model are estimated through geostatistical integration of well log and core data with 3D seismic inversion products. Changes in seismic velocities due to fluid substitution are computed using the Biot-Gassmann workflow and site-specific rock physics. Stress impacts on time-lapse seismic velocity changes are modeled from the effective stress output of the hydromechanical model and are initially based on the velocity versus effective stress relationship extracted from core mechanical testing. Based on the principle of superposition of seismic wavefields, seismic velocity changes attributed to fluid substitution and that due to changes in mean effective stress are treated as linearly additive. The modeled results are upscaled using Backus averaging to reconcile scale discrepancies between the modeled and measured datasets to formulate the penalty function. This manuscript presents the forward modeling process and concludes that for the base case, the seismic velocity changes due to mean effective stress dominates over the seismic velocity changes attributed to fluid substitution because of the extensive range of the pressure perturbations. Successful minimization of this penalty function calibrates the coupled hydrodynamic geomechanical numerical model and affirms the suitability of acoustic time-lapse measurements such as 4D-VSP for geomechanical calibration.

02 PETROLEUM↗
DOE OSTI2025

eCounter: Inline Per-IP Network Monitoring at Millisecond Resolution via eBPF

Scientific data acquisition (SciDAQ) systems are shifting from archive-based workflows to streaming paradigms, where real-time, fine-grained network monitoring becomes essential. While P4-enabled devices offer per-packet in-band observability, they require specialized switches and routers. Host-side tools like Prometheus exporters lack sufficient temporal granularity. To bridge this gap, we present eCounter, a lightweight, hardware-agnostic, inline telemetry agent built on extended Berkeley Packet Filter (eBPF). eCounter captures per-interface ingress and egress traffic, categorized by IP address and protocol, at millisecond to sub-millisecond resolution. In a 100 Gbps environment, it continuously exports up to 3,257 time-series bins per second with only 4% CPU utilization at a 35¿KiB/s data rate. We evaluate eCounter across diverse NIC MTU settings, hook types, CPU architectures and operating systems, and observed negligible impact on concurrent high-throughput streaming applications. Complexity analysis confirms that it can be readily scaled to distributed SciDAQ deployments.

Mei, Xinxin [Computational Sciences and Technology↗
DOE OSTI2026

Supervisory Control and Data Acquisition for Electrochemical Separation Experimentation

The Python-based program is a laboratory automation tool designed to control and monitor electrochemical systems. The tool was developed for capacitive deionization (CDI) experiments, but it can be used for any system that requires controlled voltage or current segments and multi-parameter monitoring. The program integrates hardware components to run user-defined experimental parameters, providing operational control of a programmable power supply, peristaltic pump, and data acquisition devices. Currently, the program is structured with a workflow that includes an initialization (or pre-run) phase, a main loop, and a post-experiment stabilization (or post-run) phase. The initialization phase prepares and stabilizes the cell, ensuring that the electrodes and solution reach a baseline state before the experiment begins. The main loop consists of multiple voltage segments that repeat, controlling the experiment while recording key parameters such as time, voltage, current, pH, and conductivity. Finally, the post-experiment stabilization phase allows the system to stabilize after the experiment, returning the cell and solution to equilibrium conditions before ending the sequence. The program is designed with four variations, each tailored to different experimental needs. All variations include both the initialization and post-experiment stabilization stages, which run for a set amount of time, voltage, current, and flow rate before and after the main experiment block. The main loop runs for a set number of cycles, as defined by the user input, and each cycle is composed of 2 or 4 segments. The 4 program variations are described as follows: Program 1: The main program includes 2 segments. Each segment is defined to have a set duration, flow rate, voltage, and current. This program measures conductivity, flow rate, voltage, and current. Program 2: The main program expands Program 1 to include 4 segments. Each segment has a specified duration, flow rate, voltage, and current. Like Program 1, it measures conductivity, flow rate, voltage, and current. Program 3: The main program consists of 2 segments, each defined by time, flow rate, voltage, and current. In addition to conductivity, flow rate, voltage, and current, Program 3 collects pH and temperature data through a 4-channel data acquisition device. Program 4: This program independently controls two channels of a multi-channel power supply simultaneously. While conductivity can only be measured for one cell at a time, the dual-channel control makes it possible to operate two cells simultaneously under different voltage/current conditions. The main program includes 2 segments.For each program, all measurements are automatically logged and integrated into a single Excel output file. Data are displayed in numerical format and plotted, both in real time, to track system performance. A key feature of the program is its ability to synchronize all outputs so that every measurement shares a single timestamp, ensuring accurate alignment of voltage, current, pH, conductivity, and pH data.By combining hardware control, real-time monitoring, and unified data collection, this program significantly reduces manual workload and minimizes errors, making it a reliable platform for researchers, engineers, and laboratory technicians conducting CDI experiments, among other electrochemical tests.

Valentino, Lauren [Argonne National Laboratory (AN↗
DOE OSTI2022

Watershed Workflow: A toolset for parameterizing data-intensive, integrated hydrologic models

Integrated, distributed hydrologic models leverage advances in computational power and data accessibility to improve predictive understanding of the water cycle. While impressive advances in this area of environmental modeling have been accomplished, such models are still rarely used, partially because of difficulty integrating model and data. This research describes the release of Watershed Workflow version 1.2, a new library aiming to automate and enable complex workflows defining inputs to high resolution, integrated, distributed hydrologic models. Watershed Workflow provides tools enabling the discovery, acquisition, mapping, and coordination of watershed geometry, land cover, soil properties, and meteorological data. It enables the construction of unstructured meshes that incorporate this data, and provides tools for automating a “first” simulation on any watershed in the United States. We present the design of the workflow tool, and describe best practices for its usage, culminating in a final example from watershed specification to simulation at the Coweeta Hydrologic Laboratory.

Integrated hydrologic modeling↗
DOE OSTI2024

Human-in-the-Loop: The Future of Machine Learning in Automated Electron Microscopy

Machine learning (ML) methods are progressively gaining acceptance in the electron microscopy community for de-noising, semantic segmentation, and dimensionality reduction of data post-acquisition. The introduction of the application programming interfaces (APIs) by major instrument manufacturers now allows the deployment of ML workflows in microscopes, not only for data analytics but also for real-time decision-making and feedback for microscope operation. However, the number of use cases for real-time ML remains remarkably small. Furthermore, we discuss some considerations in designing ML-based active experiments and pose that the likely strategy for the next several years will be human-in-the-loop automated experiments (hAE). In this paradigm, the ML learning agent directly controls beam position and image and spectroscopy acquisition functions, and a human operator monitors experiment progression in real and feature space of the system and tunes the policies of the ML agent to steer the experiment toward specific objectives.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

PeakDecoder enables machine learning-based metabolite annotation and accurate profiling in multidimensional mass spectrometry measurements

Multidimensional measurements using state-of-the-art separations and mass spectrometry provide advantages in untargeted metabolomics analyses for studying biological and environmental bio-chemical processes. However, the lack of rapid analytical methods and robust algorithms for these heterogeneous data has limited its application. Here, we develop and evaluate a sensitive and high-throughput analytical and computational workflow to enable accurate metabolite profiling. Our workflow combines liquid chromatography, ion mobility spectrometry and data-independent acquisition mass spectrometry with PeakDecoder, a machine learning-based algorithm that learns to distinguish true co-elution and co-mobility from raw data and calculates metabolite identification error rates. We apply PeakDecoder for metabolite profiling of various engineered strains of Aspergillus pseudoterreus, Aspergillus niger, Pseudomonas putida and Rhodosporidium toruloides. Results, validated manually and against selected reaction monitoring and gas-chromatography platforms, show that 2683 features could be confidently annotated and quantified across 116 microbial sample runs using a library built from 64 standards.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

A method for crystallographic mapping of an alpha-beta titanium alloy with nanometre resolution using scanning precession electron diffraction and open-source software libraries

An approach for the crystallographic mapping of two-phase alloys on the nanoscale using a combination of scanned precession electron diffraction and open-source python libraries is introduced in this paper. This method is demonstrated using the example of a two-phase α/β titanium alloy. The data were recorded using a direct electron detector to collect the patterns, and recently developed algorithms to perform automated indexing and analyse the crystallography from the results. Very high-quality mapping is achieved at a 3 nm step size. The results show the expected Burgers orientation relationships between the α laths and β matrix, as well as the expected misorientations between α laths. A minor issue was found that one area was affected by 180° ambiguities in indexing occur due to this area being aligned too close to a zone axis of the α with twofold projection symmetry (not present in 3D) in the zero-order Laue Zone, and this should be avoided in data acquisition in the future. Nevertheless, this study demonstrates a good workflow for the analysis of nanocrystalline two- or multi-phase materials, which will be of widespread use in analysing two-phase titanium and other systems and how they evolve as a function of thermomechanical treatments.

36 MATERIALS SCIENCE↗
DOE OSTI2025

ESnet-JLab FPGA Accelerated Transport (data plane) [EJFAT (udplb)] v1.0

The ESnet-JLab FPGA Accelerated Transport system is a solution for streaming high-speed scientific measurement data from Data Acquisition Systems (DAQs) to high-performance computing facilties. It is generally compatible with many science workflows, and makes no assumptions about the specifics of any particular experiment. This program (udplb) implements the data plane portion of the EJFAT system. It is an FPGA design that rewrites and forwards data packets from a UDP-based scientific workflow to high-performance compute nodes. It depends on another program (udplbd, disclosed separately) to implement the control system.

Bengough, Peter [Malleable Networks, Inc.]↗
DOE OSTI2020

Seismology in the Cloud: A New Streaming Workflow

Data-intensive research in seismology is experiencing a recent boom, driven in part by large volumes of available data and advances in the growing field of data science. However, there are significant barriers to processing large data volumes, such as long retrieval times from data repositories, complex data management, and limited computational resources. New tools and platforms have reduced the barriers to entry for scientific cluster computing, including the maturation of the commercial cloud as an accessible instrument for research. Here, we build a customized research cluster in the cloud to test a new workflow for large-scale seismic analysis, in which data are processed as a stream (retrieved on-the-fly and acted upon without storing), with data from the Incorporated Research Institutions for Seismology Data Management Center. We use this workflow to deploy a spectral peak detection algorithm over 5.6 TB of compressed continuous seismic data from 2074 stations of the USArray Transportable Array EarthScope network. Using a 50-node cluster in the cloud, we completed the noise survey in 80 hr, with an average data throughput of 1.7 GB per minute. By varying cluster sizes, we find the scaling of our analysis to be sublinear, due to a combination of algorithmic limitations and data center response times. The cloud-based streaming workflow represents an order-of-magnitude increase in acquisition and processing speed compared to a traditional download-store-process workflow, and offers the additional benefits of employing a flexible, accessible, and widely used computing architecture. It is limited, however, due to its reliance on Internet transfer speeds and data center service capacity, and may not work well for repeated analyses or those for which even higher data throughputs are needed. These research applications will require a new class of cloud-native approaches in which both data and analysis are in the cloud.

58 GEOSCIENCES↗
DOE OSTI2023

Uncertainty-aware molecular dynamics from Bayesian active learning for phase transformations and thermal transport in SiC

Abstract Machine learning interatomic force fields are promising for combining high computational efficiency and accuracy in modeling quantum interactions and simulating atomistic dynamics. Active learning methods have been recently developed to train force fields efficiently and automatically. Among them, Bayesian active learning utilizes principled uncertainty quantification to make data acquisition decisions. In this work, we present a general Bayesian active learning workflow, where the force field is constructed from a sparse Gaussian process regression model based on atomic cluster expansion descriptors. To circumvent the high computational cost of the sparse Gaussian process uncertainty calculation, we formulate a high-performance approximate mapping of the uncertainty and demonstrate a speedup of several orders of magnitude. We demonstrate the autonomous active learning workflow by training a Bayesian force field model for silicon carbide (SiC) polymorphs in only a few days of computer time and show that pressure-induced phase transformations are accurately captured. The resulting model exhibits close agreement with both ab initio calculations and experimental measurements, and outperforms existing empirical models on vibrational and thermal properties. The active learning workflow readily generalizes to a wide range of material systems and accelerates their computational understanding.

36 MATERIALS SCIENCE↗
NASA NTRS2019

2018 NISAR Applications Workshop: Forest and Disturbance; Workshop Report

Forest lands cover the globe and are important sources for providing ecosystem services including: carbon sequestration, biodiversity, timber, air and water quality. As such, counties around the world have dedicated programs for managing them. Accurate and timely information concerning the status of these forests (moisture, biomass, disturbance type, etc.) is essential to those Nations’ human and ecological health as well as economy. The joint NASA/US Forest Service workshop focused on arming forest land managers with observations and remote sensing information from the upcoming NASA-ISRO (Indian Space Research Organization) SAR (Synthetic Aperture Radar) (NISAR) satellite mission (expected to launch early 2022). Participants included representatives from different US Federal Agencies, private sector, and non-governmental organizations (NGO) that are key players in facilitating integration of Earth Observations (EO) into forest management and decision support workflows. They included scientists, technicians, and program managers with a responsibility for data acquisition and exploitation such as product development, delivery, and use, and capacity building. Discussions were held over two days to convey the broader forest and disturbance community information needs for various representative participants and programs and to facilitate the delivery of NISAR mission geospatial products and observational capabilities. Case studies were presented to demonstrate the current state of practice in the use of SAR remote sensing for applications of direct importance to forest and disturbance land management community. Eleven organizations presented their information requirements in response to a set of questions provided by the NASA team, then the NASA team responded by describing the degree to which NISAR could meet these requirements. Discussion ensued about needed data product specifications to increase utility (e.g., projection, latency, etc.), tools and capacity building. The general findings of this workshop were that (a) NISAR observations will be particularly useful to the global forest carbon and disturbance monitoring applications, but that certain data product design decisions (projections and radiometric and terrain corrections) need to be considered to increase utility; b) the biomass and disturbance detection algorithms meet many of the community needs, however there are other information products of value (e.g., soil moisture or disturbance classification, not just detection) and all products should be compliant with existing community standards for reporting uncertainty; c) providing SAR education to the community will be key specifically thinking about putting the information first and the SAR theory second, providing a simple guide of standard data processing steps (e.g., dB (decibel) to power conversion and speckle filtering); d) the community needs a user-friendly interface for finding free, archived data over their geographic regions of interest; e) user-friendly tools that connect to open-sources GIS (Global Information System) software (e.g., QGIS (Quantum GIS)) that include a graphical user interface (GUI) for SAR processing that enables both download and cloud processing. To integrate these findings and prepare the community before NISAR launches, it was suggested that there be a dedicated NISAR Forest and Disturbance Applications Working Group (as per the specifications in the NISAR Utilization Plan). After launch, it was decided that the community continue capacity building activities.

Stavros, Natasha↗
DOE OSTI2024

Exploring Ion Mobility Mass Spectrometry Data File Conversions to Leverage Existing Tools and Enable New Workflows

Ion mobility (IM) is often combined with LC-MS experiments to provide an additional dimension of separation for complex sample analysis. While highly complex samples are better characterized by the full dimensionality of LC-IM-MS experiments to uncover new information, downstream data analysis workflows are often not equipped to properly mine the additional IM dimension. For many samples the data acquisition benefits of including IM separations are all that is necessary to uncover sample information and the full dimensionality of the data is not required for data analysis. Post-acquisition reduction and adaptation of the dimensions of LC-IM-MS and IM-MS experiments into an LC-MS format opens the possibility to use a plethora of existing software tools. In this work, we developed data file conversion tools to reduce the complexity of IM data analysis. Three data file transformations are introduced in the PNNL PreProcessor software: 1) mapping the IM axis to the LC axis for IM-MS data, 2) converting the drift time vs. m/z space to CCS/z vs m/z space, and 3) transforming All Ions IM/MS mobility aligned fragmentation data to a standard LC-MS DDA data file format. Finally, these new data file conversions are demonstrated with corresponding lipidomics and proteomics workflows that leverage existing LC-MS data analysis software to highlight the benefits of the data transformations.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

ESnet-JLab FPGA Accelerated Transport (control plane) [EJFAT (udplbd2)] v2.0

The ESnet-JLab FPGA Accelerated Transport system is a solution for streaming high-speed scientific measurement data from Data Acquisition Systems (DAQs) to high-performance computing facilties. It is generally compatible with many science workflows, and makes no assumptions about the specifics of any particular experiment. This program (udplbd version 2) implements the control plane for the system. It is responsible for programming network forwarding rules into the data plane (implemented by the hardware designed named udplb, described separately). It also implements the control loop necessary to match up offered workload with available capacity on high-performance compute nodes.

Howard, Derek [Lawrence Berkeley National Laborato↗
DOE OSTI2022

CSEM Fluid Monitoring Methodology Using Real Data Examples

Conference presentation at International Meeting for Applied Geoscience & Energy (IMAGE), Houston, Texas, August 28 – September 1, 2023. Using field data from hydrocarbon and CO 2 applications, we illustrate the importance of a workflow and adaption to the target on hand. Verifying the geophysical acquisition and processing steps with 3D modeling and checking them against a 3D anisotropic log-derived model maintains confidence in the workflow and minimizes the influence on the data. This allows us to predict data validity and to certify the data with respect to the borehole logs.

20 FOSSIL-FUELED POWER PLANTS↗
DOE OSTI2026

Single-cell proteomics of Arabidopsis leaf mesophyll reveals dynamic protein responses to water-deficit stress

Background The application of single-cell omics tools to biological systems can provide unique insights into diverse cellular populations and their heterogeneous responses to internal and external perturbations. Thus far, most single-cell studies in plant systems have been limited to RNA-sequencing approaches, which only provide indirect readouts of cellular functions. Results Here, we present a single-cell proteomics workflow for plant cells that integrates tape-sandwich protoplasting, piezoelectric cell sorting, nanoPOTS sample preparation, and ion mobility-based MS data acquisition method for label-free single-cell proteomics analysis of Arabidopsis leaf mesophyll cells. From a single leaf protoplast, over 3,000 proteins were quantified with high precision. The workflow is demonstrated to identify stress associated changes in protein abundance by analyzing 117 protoplasts from well-watered and water-deficit stressed plants. Additionally, we describe a new approach for constructing covarying protein networks at the single-cell level and demonstrate how single-cell protein covariation analysis can reveal previously unrecognized protein functions while also capturing stress-induced changes in protein–protein dynamics. Conclusions The label-free scProteomic approach presented here represents a significant advance through the demonstration of a facile protoplast isolation method combined with deep and precise proteomic coverage of Arabidopsis leaf mesophyll cell types. We believe this study will serve as an informative reference to future plant scProteomic investigations.

Arabidopsis↗