Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,027 records · Page 57

Analysis of High-Throughput ELISA Microarray Data

Our research group develops analytical methods and software for the high-throughput analysis of quantitative enzyme-linked immunosorbent assay (ELISA) microarrays. ELISA microarrays differ from DNA microarrays in several fundamental aspects and most algorithms for analysis of DNA microarray data are not applicable to ELISA microarrays. In this review, we provide an overview of the steps involved in ELISA microarray data analysis and how the statistically sound algorithms we have developed provide an integrated software suite to address the needs of each data-processing step. The algorithms discussed are available in a set of open-source software tools (http://www.pnl.gov/statistics/ProMAT).

59 BASIC BIOLOGICAL SCIENCES↗

Phased array compaction cell for measurement of the transversely isotropic elastic properties of compacting sediments

Sediments undergoing compaction typically exhibit transversely isotropic (TI) elastic properties. We present a new experimental apparatus, the phased array compaction cell, for measuring the TI elastic properties of clay-rich sediments during compaction. This apparatus uses matched sets of P- and S-wave ultrasonic transducers located along the sides of the sample and an ultrasonic P-wave phased array source, together with a miniature P-wave receiver on the top and bottom ends of the sample. The phased array measurements are used to form plane P-waves that provide estimates of the phase velocities over a range of angles. From these measurements, the five TI elastic constants can be recovered as the sediment is compacted, without the need for sample unloading, recoring, or reorienting. This paper provides descriptions of the apparatus, the data processing, and an application demonstrating recovery of the evolving TI properties of a compacting marine sediment sample.

54 ENVIRONMENTAL SCIENCES↗

A systematic review of machine learning in groundwater monitoring

With increasing concerns about water scarcity, groundwater has become crucial since this resource provides most of the freshwater needs. However, various human and natural activities often contaminate the groundwater, making it unsuitable for use. Over the years, scientists and engineers have used many methods to predict and track groundwater contamination as part of environmental monitoring. Consequently, there is an urgent need for improved methods, particularly in the face of increasing contamination. Machine learning has sometimes been used to monitor groundwater, air quality, and climate. Traditional methods must be improved due to the complexity and large amount of environmental data. This includes using hybrid models that combine traditional and new techniques. Despite the use of machine learning in many scientific areas, there is a lack of comprehensive reviews focusing on its use in environmental monitoring, especially groundwater monitoring. We aim to fill this gap by exploring machine-learning applications in groundwater monitoring. We discuss relevant methods, their limitations, and future potential. We summarize research on automating data processing and model training using groundwater sensor data. Our research underscores the transformative potential of machine learning to revolutionize long-term groundwater monitoring and contamination detection, providing valuable insights for future research and practical applications.

AI/ML↗

Combinatorial Exploration and Mapping of Phase Transformation in a Ni–Ti–Co Thin Film Library

Combinatorial synthesis and high-throughput characterization of a Ni–Ti–Co thin film materials library are reported for exploration of reversible martensitic transformation. The library was prepared by magnetron co-sputtering, annealed in vacuum at 500 °C without atmospheric exposure, and evaluated for shape memory behavior as an indicator of transformation. Composition, structure, and transformation behavior of the 177 pads in the library were characterized using high-throughput wavelength dispersive spectroscopy (WDS), X-ray photoelectron spectroscopy (XPS), X-ray diffraction (XRD), and four-point probe temperature-dependent resistance (R(T)) measurements. A new, expanded composition space having phase transformation with low thermal hysteresis and Co > 10 at. % is found. Unsupervised machine learning methods of hierarchical clustering were employed to streamline data processing of the large XRD and XPS data sets. Through cluster analysis of XRD data, we identified and mapped the constituent structural phases. Finally, composition–structure–property maps for the ternary system are made to correlate the functional properties to the local microstructure and composition of the Ni–Ti–Co thin film library.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Enabling real-time multi-messenger astrophysics discoveries with deep learning

Multi-messenger astrophysics is a fast-growing, interdisciplinary field that combines data, which vary in volume and speed of data processing, from many different instruments that probe the Universe using different cosmic messengers: electromagnetic waves, cosmic rays, gravitational waves and neutrinos. In this Expert Recommendation, we review the key challenges of real-time observations of gravitational wave sources and their electromagnetic and astroparticle counterparts, and make a number of recommendations to maximize their potential for scientific discovery. These recommendations refer to the design of scalable and computationally efficient machine learning algorithms; the cyber-infrastructure to numerically simulate astrophysical sources, and to process and interpret multi-messenger astrophysics data; the management of gravitational wave detections to trigger real-time alerts for electromagnetic and astroparticle follow-ups; a vision to harness future developments of machine learning and cyber-infrastructure resources to cope with the big-data requirements; and finally, the need to build a community of experts to realize the goals of multi-messenger astrophysics.

79 ASTRONOMY AND ASTROPHYSICS↗

Accounting for electron-beam-induced warping of molecular nanocrystals in MicroED structure determination

High-energy electrons induce sample damage and motion at the nanoscale to fundamentally limit the determination of molecular structures by electron diffraction. Using a fast event-based electron counting (EBEC) detector, we characterize beam-induced, dynamic, molecular crystal lattice reorientations (BIRs). These changes are sufficiently large to bring reciprocal lattice points entirely in or out of intersection with the sphere of reflection, occur as early events in the decay of diffracted signal due to radiolytic damage, and coincide with beam-induced migrations of crystal bend contours within the same fluence regime and at the same illuminated location on a crystal. These effects are observed in crystals of biotin, a series of amino acid metal chelates, and a six-residue peptide, suggesting that incident electrons inevitably warp molecular lattices. The precise orientation changes experienced by a given microcrystal are unpredictable but are measurable by indexing individual diffraction patterns during beam-induced decay. Reorientations can often tilt a crystal lattice several degrees away from its initial position before irradiation, and for an especially beam-sensitive Zn(II)-methionine chelate, are associated with dramatic crystal quakes prior to 1 e − Å −2 electron beam fluence accumulates. Since BIR coincides with the early stages of beam-induced damage, it echoes the beam-induced motion observed in single-particle cryoEM. As with motion correction for cryoEM imaging experiments, accounting for BIR-induced errors during data processing could improve the accuracy of MicroED data.

Vlahakis, Niko (ORCID:0000000250920265)↗

HPDR: High-Performance Portable Scientific Data Reduction Framework

The rapid growth in scientific data generation is outpacing advancements in computing systems necessary for efficient storage, transfer, and analysis, particularly in the context of exascale computing. With the deployment of first-generation exascale computing systems and next-generation experimental facilities, this gap is widening and necessitates effective data reduction techniques to manage enormous data volumes. Over the past decade, various data reduction methods, including lossless compression, error-controlled lossy compression, and data refactoring, have been developed to accelerate I/O in scientific workflows. Despite significant reductions in data volume, these methods introduce considerable computational overhead, which can become the new bottleneck in data processing. To mitigate this, GPU-accelerated data reduction algorithms have been introduced. However, challenges remain in their integration into exascale workflows, including limited portability across different GPU architectures, substantial memory transfer overhead, and reduced scalability on dense multi-GPU systems. To address these challenges, we propose HPDR, a high-performance and portable data reduction framework. HPDR is designed to enable the execution of state-of-the-art reduction algorithms across diverse processor architectures while reducing memory transfer overhead to 2.3 % of the original, resulting in up to 3.5× faster throughput compared to existing solutions. It also achieves up to 96% of the theoretical speedup in multi-GPU settings. In addition, evaluations on accelerating I/O operations at scale up to 1,024 nodes of the Frontier supercomputer demonstrate that HPDR can achieve up to 103 TB/s reduction throughput, providing up to 4× acceleration in parallel I/O performance compared to existing data reduction routines. This work highlights the potential of HPDR to significantly enhance data reduction efficiency in exascale computing environments.

Chen, Jieyang [University of Oregon]↗

Use of Convolutional Neural Network Image Classification and High-Speed Ion Probe Data Toward Real-Time Detonation Characterization in a Water-Cooled Rotating Detonation Engine

As rotating detonation engines (RDEs) progress in maturity, the importance of monitoring advancements toward development of active control becomes more critical. Experimental RDE data processing at time scales which satisfy real-time diagnostics will likely require the use of machine learning. This study aims to develop and deploy a novel real-time monitoring technique capable of determining detonation wave number, direction, frequency, and individual wave speeds throughout experimental RDE operational windows. To do so, the diagnostic integrates image classification by a convolutional neural network (CNN) and ionization current signal analysis. Wave mode identification through single-image CNN classification bypasses the need to evaluate sequential images and offers instantaneous identification of the wave mode present in the RDE annulus. Here, real-time processing speeds are achieved due to low data volumes required by the methodology, namely one short-exposure image and a short window of sensor data to generate each diagnostic output. The diagnostic acquires live data using a modified experimental setup alongside Pylon and PyDAQmx libraries within a python data acquisition environment. Lab-deployed diagnostic results are presented across varying wave modes, operating conditions, and data quality, currently executed at 3–4 Hz with a variety of iteration speed optimization options to be considered as future work. These speeds exceed that of conventional techniques and offer a proven structure for real-time RDE monitoring. The demonstrated ability to analyze detonation wave presence and behavior during RDE operation will certainly play a vital role in the development of RDE active control, necessary for RDE technology maturation toward industrial integration.

42 ENGINEERING↗

AEGIS (Air Emissions Grouped by Industrial Sectors) [SWR-24-49]

AEGIS is a robust framework designed to build and compile emissions inventories for various industrial sectors. It integrates multiple emissions databases provided by the USEPA – including GHGRP, NEI, and TRI – and leverages the STEWI and STEWICOMBO tools to retrieve, merge, and process data. The framework produces facility- and process-level inventories, identifies discrepancies (e.g., NAICS or FRS mismatches), and performs exploratory analysis including emission concentration calculations and visualization.

Atnoorkar, Swaroop↗

Spatially Resolved Raman Spectroscopic Investigation of Uranyl Fluoride: A Case Study in the Importance of Instrument Optimization

Raman spectroscopy is an emerging technique for rapid and nondestructive analysis of nuclear materials for forensic and nonproliferation applications as it is a powerful tool for distinguishing multiple chemical forms of materials with similar stoichiometries. Recent developments in spectroscopic software have enabled rapid data collection with high-speed Raman spectroscopic mapping capabilities. However, some uranium-rich materials are susceptible to degradation in humid air and/or laser-induced phase transformations. To mitigate environmental or measurement-related sample degradation of potential samples of interest, we have taken a systematic approach to define optimized data collection parameters for high-throughput measurements of uranyl fluoride (UO 2 F 2 ), which is an important intermediate material in the nuclear fuel cycle. First, we systematically describe the influence of optical magnification (5× to 100×), laser power, and exposure time on obtained signal for identical particles of UO 2 F 2 and find that at low laser power and exposure times, comparable signal is obtained regardless of optical magnification. Second, we ensure sample integrity during data collection, and third, collect spectroscopic maps that employ optimized parameters to reduce the time required to obtain spatially resolved spectroscopic information. Reductions of 90% and 99% in measurement times are discussed as they relate to differences in resolving spectroscopic features of particles in identical mapping areas. Finally, during this work, we found that additional data processing options were needed and thus developed a customized Python script for importing, processing, analyzing, and visualizing Raman spectroscopic map data.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Road Lidar Dataset for the TxDOT Austin District

This is a road lidar data collection for developing road elevation models and road inundation mapping methodologies, a joint work between ORNL and The University of Texas at Austin. This dataset is generated as part of the flood transportation infrastructure, partly funded by the NOAA CIROH project. ORNL is a project partner for high-performance computing-empowered flood inundation mapping methodology R&D. The dataset is computed using a GPU-accelerated lidar data processing workflow developed at ORNL. The lidar data source is from TxGIO, the state lidar data collection site. The output dataset is in two formats: laz and copc. It is organized by TxDOT's maintenance sections, covering the Austin District. Data size: 3.86 billion road lidar points, 1.67% of the entire lidar data input Projection: EPSG:32614 (WGS84/UTM zone 14N) Website: https://web.corral.tacc.utexas.edu/nfiedata/road3d/austin_district/AustinMaintenanceSections_H_epsg6343_V_epsg5703/ LICENSE FOR USE -- MAPS AND DATA DISCLAIMER This resource is shared under the Creative Commons Attribution CC BY, http://creativecommons.org/licenses/by/4.0/ MAPS AND DATA DISCLAIMER The Oak Ridge National Laboratory (ORNL) shall not be held liable for improper or incorrect use of the data described or information contained on this map or associated series of maps. The data and related map graphics are not legal, land survey or engineering documents and are not intended to be used as such. ORNL gives no warranty, express or implied, as to the accuracy, reliability, utility or completeness of this information. The user of these maps and data assumes all responsibility and risk for the use of the maps and data. ORNL disclaims all warranties, representations or endorsements either express or implied, with regard to the information contained in this map product, including, but not limited to, all implied warranties of merchantability, fitness for a particular purpose or non-infringement. This preliminary map product is for research and review purposes only. It is not intended to be used for emergency management operational or life safety decisions at the local or regional governmental level or by the general public. Users requiring information regarding hazardous conditions or meteorological conditions for specific geographic areas should consult directly with their city or county emergency management office.

54 ENVIRONMENTAL SCIENCES↗

Photosynthetic and stomatal response, and leaf traits, of two pine species, Talladega National Forest, Alabama, 2023

Photosynthetic response to irradiance (AQ), CO2 (ACi) and stomatal response to irradiance measurements on two pine species common to the southeast United States, long leaf pine, Pinus palustris Mill., and loblolly pine, Pinus taeda L.. Data were collected in June 2023 within the Oakmulgee District of the Talladega National Forest in west-central Alabama at the NEON site (TALL). Measurements were made with four LI-COR LI-6800 and one LI-COR LI-6400XT gas exchange systems. ACi and AQ curves enable the estimation of photosynthetic parameters (e.g., Vcmax, Jmax, Rd, aQY), while stomatal response curves enable the estimation of stomatal slope and intercept parameters. Foliar trait data (needle dimensions, leaf mass per area, leaf nitrogen concentration) is provided for all gas exchange samples. The data package files include multiple raw and processed data files (csv), metadata files (csv), the complete instrument output for all measurements in .zip format, and a pdf describing the experimental protocol.

54 ENVIRONMENTAL SCIENCES↗

A New Decade in Seismoacoustics (2010–2022)

Several sources of interest often generate both low‐frequency acoustic and seismic signals due to energy propagation through the atmosphere and the solid Earth. Seismic and acoustic observations are associated with a wide range of sources, including earthquakes, volcanoes, bolides, chemical and nuclear explosions, ocean noise, and others. The fusion of seismic and acoustic observations contributes to a better understanding of the source, both in terms of constraining source location and physics, as well as the seismic to acoustic coupling of energy. Here in this review, we summarize progress in seismoacoustic data processing, including recent developments in open‐source data availability, low‐cost seismic and acoustic sensors, and large‐scale deployments of collocated sensors from 2010 to 2022. Similarly, we outline the recent advancements in modeling efforts for both source characteristics and propagation dynamics. Finally, we highlight the advantages of fusing multiphenomenological signals, focusing on current and future techniques to improve source detection, localization, and characterization efforts. This review aims to serve as a reference for seismologists, acousticians, and others within the growing field of seismoacoustics and multiphenomenology research.

58 GEOSCIENCES↗

SPE3 Far-field Quicklook

The purpose of this report is to provide an overview of the seismic data collected during the SPE3 chemical shot. Farfield seismic data includes data collected at distances of more than 100 m from the shot and instruments deployed for the experiments. This report does not address infrasound, remote sensing, or seismic data collected by regional permanent stations. The primary goal is a fast data check before extensive analysis. The SPE3 shot was conducted on July 24, 2012 (DOY 206) at the SPE test site and in the same borehole as SPE1 and SPE2 (table 1). Data processing at LLNL primarily used mini-­seed data volumes downloaded from the UNR server along with associated metadata. This is in contrast to earlier shots, where LLNL data was based on reformatted Reftek data. Sensor deployment was similar to previous shots, with a few changes. Parts of lines 1 (13, 14, 15, 16), 3 (23,26,36), and 5 (1-­12) were recorded by telemetry as well as to disk, which provided immediate access to the data. A small array (6 station) geophone array was deployed southeast of the shot near AF9 to measure particle motions. A new rotational sensor (ATA ARS-­16) was deployed at station L3-­20.

58 GEOSCIENCES↗

Addressing the data and real-time challenges in large scale particle physics experiments through AI and in situ computing technologies

Modern high energy physics experiments are faced not only with the challenge of having to deal with extremely high data rates but with the need to process data quickly to meet real time constraints. At Fermilab, we explore the use of novel computing technologies and techniques to address these challenges. I will discuss my R&D efforts in applying such computing solutions to enhance the multi-messenger astronomy capabilities and improve the overall physics performance of large-scale LArTPC based neutrino experiments. These efforts offer excellent opportunities for fruitful collaboration.

43 PARTICLE ACCELERATORS↗

PNNL Soil Microbiome SFA

PNNL's Soil Microbiome Science Focus Area (SFA) is focused on understanding the basic biology underpinning how interactions among various soil microbial community members, across trophic levels, lead to the emergence of community functions. Moisture, in particular, drives microbial interactions and influences everything from cell function to substrate fate within soils. The group predicts this results in repeatable, predictable phenotypes. The sum of these phenotypes comprises the “soil metaphenome”. Understanding how the soil metaphenome shifts in response to moisture will provide a basis for modeling and predicting these shifts in reaction network responses. PNNL’s Soil Microbiome SFA project repository on DataHub allows for exploring and downloading integrated experimental omics data, experimental metadata, pre- and post-processed data files, and other associated materials directly related to experimental project publication data. Visit the PNNL Soil Microbiome Science Focus Area Program homepage for more information.

54 ENVIRONMENTAL SCIENCES↗

A Data Analysis Framework for Earth System Simulation within an In-Situ Infrastructure

This paper presents a generic procedure to implement a scalable and high performance data analysis framework for large-scale scientific simulation within an in-situ infrastructure. It demonstrates a unique capability for global Earth system simulations using advanced computing technologies (i.e ., automated code analysis and instrumentation), in-situ infrastructure (i.e ., ADIOS) and big data analysis engines (i.e ., SciKit-learn). This paper also includes a useful case that analyzes a globe Earth System simulations with the integration of scalable in-situ infrastructure and advanced data processing package. Finally, the in-situ data analysis framework can provides new insights on scientific discoveries in multiscale modeling paradigms.

97 MATHEMATICS AND COMPUTING↗

Memory synchronization filter

Data synchronization between memories of a data processing system is achieved by transferring the data blocks from a first memory to a second memory, forming a hash list from addresses of data blocks that are written to the second memory or modified in the second memory. The hash list may be to identify a set of data blocks that are possibly written to or modified. Data blocks that are possibly modified may be written back from the second memory to the first memory in response to a synchronization event. The hash list may be updated by computing, in hardware or software, hash functions of an address of the transferred or modified data block to determine bit positions to be set. The hash list may be queried by computing hash functions of an address to determine bit positions, and checking bits in the hash list at those bit positions.

97 MATHEMATICS AND COMPUTING↗