SEARCH · Engineering Papers

Results for “science data processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 955 records · Page 53

All sources NASA NTRS DOE OSTI

DOE OSTI2015

UV-Vis Spectroscopy as a Tool for Safeguards; Instrumentation installation and fundamental data collection

Two spectrophotometric process monitors, one optimized for high concentration (approximately 10 g/L) and one for trace levels (approximately 10 ppm),were developed at Argonne and installed at the SRS H-Canyon facility for field testing. These systems were built of Commercial-Off-The-Shelf components utilizing a custom, facility-specific hardware interface. The systems directly provide a qualitative measurement of process chemistry (i.e. valence state). With appropriate calibrations the systems could provide quantitative data. Laboratory tests were performed to determine the spectrophotometric molar absorptivity coefficients for relevant actinide and transition metals of interest.

36 MATERIALS SCIENCE↗

DOE OSTI2021

Geophysical Retrievals in an Artificial Intelligence (AI) Framework for Illuminating Processes Controlling Water Cycle

Focal Area(s): This white paper responds to Focal Area #3: Insight gleaned from complex data (both observed and simulated) using AI, big data analytics, and other advanced methods, including explainable AI and physics- or knowledge-guided AI. Science Challenge: This white paper addresses the water-cycle and data-model integration grand challenge. It leverages data from the Atmospheric Radiation Measurement (ARM) Climate Research Facility, and Next-Generation Ecosystem Experiment (NGEE), and Science Focus Area (SFA). The white paper focuses on controlling cloud, precipitation, and radiative properties as observed and simulated by the Earth System Models (ESM). The described framework can be readily applied to any other ensemble of instruments, including satellites and other ground-based networks.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

CROCUS Urban Canyons - Space Science and Engineering Center (SPARC) Doppler lidar data

This is the netCDF format output from the Halo Photonics Streamline XR Doppler lidar that was deployed next to the Space Science and Engineering Center (SPARC) trailer at the University of Illnois-Chicago greenhouse parking lot during CROCUS Urban Canyons. The purpose of collecting this dataset is to provide vertical and horizontal wind profiles for studying the characteristics of turbulence over the urban canyon of Chicago. This data contains the radial velocity, intensity, and backscatter from the vertical profile, range height indicator, and sector scans that were performed over both Intensive Operating Period 1 and 2 of CROCUS Urban Canyons. There are four different types of files: * The Range Height Indicator (RHI) files contain scans that are along a constant azimuth, spanning the entire hemisphere of elevation values above the surface. * The Velocity Azimuth Display (VAD) files contain the raw radial velocity data from the 6-beam, 60 degree scans. * The User1 files contain stacked Plan Position Indicator scans over a 45 degree quadrant over downtown Chicago. * The Stare files contain vertically pointing scans. These are standard netCDF files that can be opened using xarray. The VAD scans can be processed from their raw radial velocities to horizontal wind speeds with the Atmospheric data Community Toolkit (https://arm-doe.github.io/ACT/).

EARTH SCIENCE > ATMOSPHERE > ATMOSPHERIC WINDS↗

DOE OSTI2020

4D imaging of lithium-batteries using correlative neutron and X-ray tomography with a virtual unrolling technique

The temporally and spatially resolved tracking of lithium intercalation and electrode degradation processes are crucial for detecting and understanding performance losses during the operation of lithium-batteries. Here, high-throughput X-ray computed tomography has enabled the identification of mechanical degradation processes in a commercial Li/MnO 2 primary battery and the indirect tracking of lithium diffusion; furthermore, complementary neutron computed tomography has identified the direct lithium diffusion process and the electrode wetting by the electrolyte. Virtual electrode unrolling techniques provide a deeper view inside the electrode layers and are used to detect minor fluctuations which are difficult to observe using conventional three dimensional rendering tools. Moreover, the 'unrolling' provides a platform for correlating multi-modal image data which is expected to find wider application in battery science and engineering to study diverse effects e.g. electrode degradation or lithium diffusion blocking during battery cycling.

25 ENERGY STORAGE↗

DOE OSTI2017

Improved neutron-gamma discrimination for a 3 He neutron detector using subspace learning methods

3 He gas based neutron linear-position-sensitive detectors (LPSDs) have been applied for many neutron scattering instruments. Traditional Pulse-Height Analysis (PHA) for Neutron-Gamma Discrimination (NGD) resulted in the neutron-gamma efficiency ratio on the orders of 10 5 -10 6 . The NGD ratios of 3 He detectors need to be improved for even better scientific results from neutron scattering. Digital Signal Processing (DSP) analyses of waveforms were proposed for obtaining better NGD ratios, based on features extracted from rise-time, pulse amplitude, charge integration, a simplified Wiener filter, and the cross-correlation between individual and template waveforms of neutron and gamma events. Fisher linear discriminant analysis (FLDA) and three multivariate analyses (MVAs) of the features were performed. The NGD ratios are improved by about 10 2 -10 3 times compared with the traditional PHA method. Finally, our results indicate the NGD capabilities of 3 He tube detectors can be significantly improved with subspace-learning based methods, which may result in a reduced data-collection time and better data quality for further data reduction.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2014

Optical system for liquid level measurements

This work describes the development of a novel laser instrument for non-contact liquid level measurements. The physical principle of operation is based on the self-mixing effect induced in the laser diode by modulating the laser wavelength. The frequency main tone of the resulting fringes is proportional to the absolute target distance. A high speed FPGA (Field Programmable Gate Array) electronic performs the data acquisition and elaboration. The high elaboration/acquisition speed improves the chance of acquiring good signal also on a moving liquid target. The optical characteristics of the liquid surface have been studied in order to identify the optimal optical configuration when the surface is flat as well as when the surface is rippled. The final low-cost instrument works in real time and is able to track a filling process with 2.5 cm/s of speed, with resolution better than 1 mm for distances up to 50 cm.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2022

Applications of Autonomous Data Collection and Active Learning

Advances in sensors and robotics have dramatically improved the diversity of experimental approaches available to the materials community. Autonomous data collection platforms, either custom-made or commercially available, provide researchers with novel tools with which to probe materials behavior and perform advanced materials characterization. The application of novel control algorithms and active learning approaches can create much more robust experimental data, or can be used to improve the performance of existing characterization tools. Five papers within this special topic focus on experimental and computational methodologies for use in automatic data collection routines for materials characterization. From novel platforms for materials discovery to new statistical frameworks for assessing the autonomous experimentation process, these five papers highlight the diverse range of applications of automation for advancing materials science.

36 MATERIALS SCIENCE↗

DOE OSTI2021

Latent map Gaussian processes for mixed variable metamodeling

Gaussian processes (GPs) are ubiquitously used in sciences and engineering as metamodels. Standard GPs, however, can only handle numerical or quantitative variables. Here we introduce latent map Gaussian processes (LMGPs) that inherit the attractive properties of GPs and are also applicable to mixed data which have both quantitative and qualitative inputs. The core idea behind LMGPs is to learn a continuous, low-dimensional latent space or manifold which encodes all qualitative inputs. To learn this manifold, we first assign a unique prior vector representation to each combination of qualitative inputs. We then use a low-rank linear map to project these priors on a manifold that characterizes the posterior representations. As the posteriors are quantitative, they can be directly used in any standard correlation function such as the Gaussian or Matern. Hence, the optimal map and the corresponding manifold, along with other hyperparameters of the correlation function, can be systematically learned via maximum likelihood estimation. Through a wide range of analytic and real-world examples, we demonstrate the advantages of LMGPs over state-of-the-art methods in terms of accuracy and versatility. In particular, we show that LMGPs can handle variable-length inputs, have an explainable neural network interpretation, and provide insights into how qualitative inputs affect the response or interact with each other. We also employ LMGPs in Bayesian optimization and illustrate that they can discover optimal compound compositions more efficiently than conventional methods that convert compositions to qualitative variables via manual featurization.

42 ENGINEERING↗

DOE OSTI2022

Deep-learning-guided high-resolution subsurface reflectivity imaging with application to ground-penetrating radar data

Subsurface reflectivity imaging is one of the most important geophysical characterization methods for revealing subsurface structures. In many cases, accurate subsurface reflectivity imaging is challenging because of, for example, random or coherent noise in the data and sparse source-receiver observation geometry. Here, we develop a deep-learning-guided iterative imaging method to improve subsurface structure imaging. Specifically, we train a supervised neural network to infer a noise-free, high-resolution image from a noisy, low-resolution image and use this estimated image as guidance to regularize least-squares imaging. We develop a systematic method to generate high-quality synthetic training data (data-label pairs) to train the guidance neural network. The trained neural network can provide high-fidelity predictions even for field-data images that are not in the training data. We validate our new imaging method using one synthetic and two field ground-penetrating radar data examples, and find that our method can produce clean, high-resolution subsurface reflectivity images where existing single-pass and least-squares imaging methods fail due to noise and insufficient data coverage.

58 GEOSCIENCES↗

DOE OSTI2024

Enhancing Discoverability and Management of Atmospheric Data at Scale: Solutions from the ARM Data Center

The Atmospheric Radiation Measurement (ARM) is a multi-laboratory and multi-institutional U.S. Department of Energy (DOE) Office of Science National User Facility. The ARM Data Center (ADC), located at Oak Ridge National Laboratory, collects, archives, and shares vast atmospheric data crucial for climate research. The ADC manages over 7 PB of data from 460 instruments worldwide, processing it into more than 11,000 diverse data products using the Network Common Data Form (NetCDF) for machine-independent accessibility. The primary challenge addressed in this paper is the efficient management and distribution of vast and diverse datasets essential for the climate research community, enhancing accessibility through advanced tools like Data Discovery. The ADC has developed advanced infrastructure and software architecture to handle the continuous influx of heterogeneous data to enhance data discoverability, resulting in increased scientific collaboration. In 2023, users from over 34 countries downloaded and utilized ARM data, resulting in 1,455 publications. The ADC’s efforts have significantly improved the discoverability and usability of atmospheric data, fostering extensive scientific research and collaboration. This paper details the solutions implemented by the ADC team for efficient data discovery and distribution, and it demonstrates ARM’s capability of staging processed data for scientific analysis.

Shah, Chirag [ORNL] (ORCID:0000000203145737)↗

DOE OSTI2015

HPCCloud: A Cloud/Web-Based Simulation Environment

Advanced modeling and simulation has enabled the design of a variety of innovative products and the analysis of numerous complex phenomenon. There are significant barriers to widespread adoption of these tools. In particular, advanced modeling and simulation: (1) is considered complex to use, (2) needs in-house expertise, and (3) requires high capital costs. In this paper, we describe the development of an end-to-end, advanced modeling and simulation cloud platform that encapsulates best practices for scientific computing in the cloud, and demonstrate using Hydra-TH as a prototypical application. As an alternative to traditional advanced modeling and simulation workflows, our Web-based approach simplifies the processes, decreases the need for in-house computational science and engineering experts, and lowers the capital investments. In addition to providing significantly improved, intuitive software, the environment offers reproducible workflows where the full lifecycle of data from input to final analyzed results can be saved, shared, and even published.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Ensemble Simulations on Leadership Computing Systems

Scientific productivity can be enhanced through workflow management tools, relieving large High Performance Computing (HPC) system users from the tedious tasks of scheduling and designing the complex computational execution of scientific applications. This paper presents a study on the usage of ensemble workflow tools to accelerate science using the Summit and Frontier supercomputing systems. The research aims to connect science domain simulations using Oak Ridge Leadership Computing Facility (OLCF) supercomputing platforms with ensemble workflow methods in order to accelerate HPC-enabled discovery and boost scientific impact. We present the coupling, porting and optimization of Radical-Cybertools on three applications: Chroma, NAMD and LAMMPS. The tools augment traditional HPC monolithic runs with a pilot scheduler. Lessons-learned are discussed for physics, biology and materials science applications. We discuss intrinsic limitations of coupling and porting ensemble workflow tools to applications that run on large HPC systems. The origins of technical challenges and their solutions developed during the implementation process are discussed. Data management strategies, OLCF’s policies for ensembles, and natively supported workflow tools are also summarized.

Georgiadou, Antigoni [ORNL] (ORCID:000000020977631↗

DOE OSTI2025

Cosmological Hydrodynamics at Exascale: A Trillion-Particle Leap in Capability

Resolving the most fundamental questions in cosmology requires simulations that match the scale, fidelity, and physical complexity demanded by next-generation sky surveys. To achieve the realism needed for this critical scientific partnership, detailed gas dynamics must be treated self-consistently with gravity for end-to-end modeling of structure formation. Exascale computing enables simulations that span survey-scale volumes while incorporating key astrophysical processes that shape complex cosmic structures. We present results from CRK-HACC, a cosmological hydrodynamics code built for extreme scalability. Using separation-of-scale techniques, GPU-resident tree solvers, in situ analysis pipelines, and multi-tiered I/O, CRK-HACCexecuted Frontier-E: a four trillion particle full-sky simulation, over an order of magnitude larger than previous efforts. The run achieved 513.1 PFLOPs peak performance, processing 46.6 billion particles per second and writing more than 100 PB of data in just over one week of runtime. Frontier-E marks a significant advance in predictive modeling for next-generation cosmological science.

Frontiere, Nicholas [Argonne National Laboratory (↗

DOE OSTI2015

2015 IC Report: w14_biosurveillance [Slides]

Our primary use of the institutional computing clusters has been for Wikipedia-based disease surveillance and forecasting work. We've expanded breadth: the number of diseases and locations that can be surveilled using internet data. We've expanded forecasting: surprisingly little work towards forecasting disease spread with quantified uncertainty. We've expanded big data processing techniques: the Wikipedia access logs are 5TB of compressed text; we're working on efficient ways to process and store time series. Our work received significant media coverage and was highlighted as LANL's best science story of 2014.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2017

Automated Blazar Light Curves Using Machine Learning

Every night in a remote clearing called Fenton Hill high in the Jemez Mountains of central New Mexico, a bank of robotically controlled telescopes tilt their lenses to the sky for another round of observation through digital imaging. Los Alamos National Laboratory’s Thinking Telescopes project is watching for celestial transients including high-power cosmic flashes called, and like all science, it can be messy work. To keep the project clicking along, Los Alamos scientists routinely install equipment upgrades, maintain the site, and refine the sophisticated machinelearning computer programs that process those images and extract useful data from them. Each week the system amasses 100,000 digital images of the heavens, some of which are compromised by clouds, wind gusts, focus problems, and so on. For a graduate student at the Lab taking a year’s break between master’s and Ph.D. studies, working with state-of-the-art autonomous telescopes that can make fundamental discoveries feels light years beyond the classroom.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2019

Evaluation of Machine Learning Approaches to Estimate Aerosol Mixing State Metrics in Atmospheric Models

Aerosol mixing state describes how aerosol compositions are distributed among atmospheric aerosol particles in a population. Oversimplified assumptions of aerosol mixing state in atmospheric modeling can introduce errors in estimations of weather and climate-relevant aerosol microphysical properties. A more comprehensive representation of the aerosol mixing state can be achieved in principle with a Particle-resolved Monte Carlo (PartMC) model but at added computational cost that may be prohibitive for direct invocation in operational numerical weather prediction or multi-year climate simulations. The aim of our research is to explore the machine learning (ML) methodologies for estimating aerosol mixing state metrics, which we define here in three different ways: with respect to (a) hygroscopicity; (b) optical properties; and (c) chemical species abundance. We adopted a data-driven approach, leveraging deep learning and statistical learning techniques, to take advantage of massive PartMC model simulations. First, we performed particle-resolved simulations by PartMC to create a series of scenarios considering a range of global environmental conditions. Each scenario consists of aerosol populations and corresponding mixing state metrics. The gas concentration, aerosol mass concentration, environmental variables and mixing state metrics of each population constitute the datasets for machine learning implementations. We have adopted and evaluated various configurations of machine learning methodologies in this investigation, embracing deep learning, Extreme Gradient Boosting (XGBoost) algorithm, and the ensemble approaches. After a rigorous model selection process, we identified an appropriate model to derive estimates of aerosol mixing state metrics. We used the computational and data resources of the Oak Ridge Leadership Computing Facility (OLCF) and the ORNL Compute and Data Environment for Science (CADES). The NVIDIA DGX-1 hardware was used for the prototyping of the ML models. Our approach has allowed us to gain a new understanding of how machine learning methodologies can be applied to improve the representation of aerosol mixing state in atmospheric models and benefit the atmospheric research community. Next, we plan to extend our research and methodology to quantify some of the aerosol-related uncertainties in the E3SM Atmospheric Model (EAM) Version 1.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

Deployment and Evaluation of SciStream on OLCF's Advanced Computing Ecosystem (ACE)

The growing demand for real-time analysis, experimental steering, and decision-making in scientific workflows has created a need for tightly coupled integrations between experimental facilities and high-performance computing (HPC) systems. The Department of Energy’s Integrated Research Infrastructure (IRI) initiative highlights data streaming as a key capability for enabling memory-to-memory data transfers, bypassing the limitations of traditional store-and-forward models. SciStream is a toolkit developed by researchers at Argonne National Laboratory (ANL) to support such streaming by addressing cross-domain security, delegated authentication, and application transparency. We deployed and evaluated SciStream on the Oak Ridge Leadership Computing Facility’s (OLCF) Advanced Computing Ecosystem (ACE) infrastructure, leveraging the Olivine OpenShift cluster and its high-bandwidth Data Streaming Nodes (DSNs) as gateway nodes. Our evaluation included synthetic streaming workloads derived from IRI science workflows, a streaming simulator, and integration with RabbitMQ to handle low-level messaging. This report documents the deployment process, performance evaluation, and challenges encountered, along with opportunities for future improvements.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2026

Lessons Learned from Ecosystem-Scale Experimental Field Studies (Workshop Report)

Efforts to understand and predict ecosystem responses to environmental change require long-term, large-scale, spatially representative experiments and observations that capture natural variability, test predictive models, and generate transferable knowledge. Such studies are indispensable for unraveling the complexities of terrestrial ecosystems and their responses to disturbances and evolving environmental conditions, while generating the data necessary for developing mechanistic models and predictive tools that inform decision-making processes. Having a rich history of designing and executing large-scale ecosystem experiments, the U.S. Department of Energy’s Environmental System Science program convened a workshop in January 2025 that brought together leaders in the field to distill critical lessons from decades of experience in large-scale experiments. The workshop aimed to (1) provide an ecosystem experiment primer for best practices, thus ensuring a high scientific return on investment for funding agencies, and (2) offer a robust framework for the design and management of future research initiatives. This report synthesizes insights and experiences from workshop participants and is structured to capture the entire research life cycle, from goal setting and design to operations, adaptive management, team dynamics, collaborations, and the often overlooked aspect of decommissioning. By synthesizing decision-making and lessons learned across diverse research approaches, the report aims to provide a template of essential factors to consider when designing successful long-term, large-scale ecosystem experiments.

54 ENVIRONMENTAL SCIENCES↗