Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “sparse data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Demonstration of an AI-driven workflow for autonomous high-resolution scanning microscopy

Abstract Modern scanning microscopes can image materials with up to sub-atomic spatial and sub-picosecond time resolutions, but these capabilities come with large volumes of data, which can be difficult to store and analyze. We report the Fast Autonomous Scanning Toolkit (FAST) that addresses this challenge by combining a neural network, route optimization, and efficient hardware controls to enable a self-driving experiment that actively identifies and measures a sparse but representative data subset in lieu of the full dataset. FAST requires no prior information about the sample, is computationally efficient, and uses generic hardware controls with minimal experiment-specific wrapping. We test FAST in simulations and a dark-field X-ray microscopy experiment of a WSe 2 film. Our studies show that a FAST scan of <25% is sufficient to accurately image and analyze the sample. FAST is easy to adapt for any scanning microscope; its broad adoption will empower general multi-level studies of materials evolution with respect to time, temperature, or other parameters.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Accelerated Nano-Optical Imaging through Sparse Sampling

The integration time and signal-to-noise ratio are inextricably linked when performing scanning probe microscopy based on raster scanning. This often yields a large lower bound on the measurement time, for example, in nano-optical imaging experiments performed using a scanning near-field optical microscope (SNOM). Here, in this study, we utilize sparse scanning augmented with Gaussian process regression to bypass the time constraint. We apply this approach to image charge-transfer polaritons in graphene residing on ruthenium trichloride (α-RuCl 3 ) and obtain key features such as polariton damping and dispersion. Critically, nano-optical SNOM imaging data obtained via sparse sampling are in good agreement with those extracted from traditional raster scans but require 11 times fewer sampled points. As a result, Gaussian process-aided sparse spiral scans offer a major decrease in scanning time.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Using AI to build a hydrobiogeochemical soil model

Soil water content is a function of inputs from precipitation and outputs via evaporation, transpiration, lateral flow, and vertical percolation, and is sensitive to biogeochemical processes. As such, soils serve as an ideal integrator of atmospheric, hydrological, and biogeochemical processes affecting the water cycle. In addition, soil water retention capacity, infiltration rates, and hydraulic conductivity can buffer or exacerbate the effects of extreme precipitation events (e.g., flooding, runoff, subsurface transport, erosion, greenhouse gas emissions) and mitigate the impact of droughts and heat waves on land systems (e.g., fire, crop failure). However, integrating water cycle measurements spanning different land atmosphere compartments across scales is a fundamental barrier for numerical model predictability. A significant challenge is that each domain (soil, hydrology, biology, and atmosphere) typically collects different sets of data at different temporal and spatial frequencies/scales, and even different dimensionalities (2D vs 3D). To implement soil as an integrator of the water cycle in land models, we suggest that novel machine learning (ML) tools can be developed to effectively simulate complex landscapes across various domains and scales, extended to regions with sparse or no data. The ultimate goals are to improve predictive understanding of land-atmosphere interactions and to extend the predictability of current Earth System Models (ESMs) through better integration of hydrological and biogeochemical data. We envision a framework in which: (1) ML-aided data reconstructions enable the merger of data sources into a unified geospatial product; (2) automated detection techniques are used to improve the knowledge of complex soil processes and interactions; and (3) this knowledge is leveraged and incorporated into models through AI-based emulators to distinctly connect the land and atmospheric compartments of the water cycle in models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2019

Data Integration Aggregated Model and Ontology for Nuclear Deployment (DIAMOND): Preliminary Model and Ontology

The data of a nuclear utility or plant are stored in various isolated forms in different systems. These data have different structures and tools and are, therefore, used independently. The integration of data sources is often performed manually and on an as-needed basis to achieve a targeted cost saving. A holistic approach to the integration of the data would enable creating a data warehouse for a single nuclear power plant (NPP) or multiple plants. A data warehouse results in direct and indirect cost saving because it automates the manual search for data, enables sharing and comparison of data from various tools of a single or multiple plants, enable digital transformation of data, increase frequencies of needed data that are sparse in nature (such as failures signatures), reduces the need for training on various tools for plant staff, enables a holistic staff perception of plant activities, reveals cost-saving opportunities, and improves the visual perception of all parts a plant necessary for operations and maintenance. One of the main challenges to deploy a data warehouse is associated with the integration of data sources into a consistent data model using a standard ontology that is developed by and for the nuclear power industry. This model is non-existent and is a necessity to enable the nuclear data warehouse role. A data model contains objects that make up a given domain along with their attributes and relationships. It provides domain standardization and maintainability, setting forth the objects, attributes, and relationships within that domain and acting as the guide by which the various applications within that domain will structure their data and interchange. An ontology is defined as “a set of concepts and categories in a subject area or domain that shows their properties and the relations between them”. Ontologies consist of a taxonomy or directed rooted tree. A survey of the models and ontologies available to support a data integration aggregated model and ontology for nuclear deployment (DIAMOND) was performed and different standards that address specific aspects of the needed model and ontology were leveraged and integrated in this effort. In specific, the Basic Formal Ontology (BFO) and Lifecycle Modeling Language (LML) were leveraged for the base ontology. The ISO 4224 and 15926 were leveraged in the development of the overlapping nuclear and fossil data aspects. The electric power research institute (EPRI) power distribution common information model (CIM) was also reviewed and the experience gained was leveraged in the power generation scope of DIAMOND. The initial aim targeted by this effort was to develop the ontology definitions and model structure. This report documents the resulting model and ontology to develop DIAMOND. The report describes the various elements of the model development based on a growing tree development approach, i.e. DIAMOND’s tree structure is established with the core data aspects and is populated with a preliminary level of details. Fifteen sources of data in a nuclear power plant were considered and incorporated into DIAMOND at this stage of development, representing the majority of operations data sources in nuclear power plants. Additional data sources, scope, and a higher level of fidelity will be incorporated into DIAMOND next in collaboration with the nuclear industry and relevant stakeholders.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2018

Exploring parasite heterogeneity using single-cell RNA-seq reveals a gene signature among sexual stage Plasmodium falciparum parasites

The malaria parasite has a complex lifecycle, including several events of differentiation and stage progression, while actively evading immunity in both its mosquito and human hosts. Important parasite gene expression and regulation during these events remain hidden in rare populations of cells. Here, we combine a capillary-based platform for cell isolation with single-cell RNA-sequencing to transcriptionally profile 165 single infected red blood cells (iRBCs) during the intra-erythrocytic developmental cycle (IDC). Unbiased analyses of single-cell data grouped the cells into eight transcriptional states during IDC. Interestingly, we uncovered a gene signature from the single iRBC analyses that can successfully discriminate between developing asexual and sexual stage parasites at cellular resolution, and we verify five, previously undefined, gametocyte stage specific genes. Moreover, we show the capacity of detecting expressed genes from the variable gene families in single parasites, despite the sparse nature of data. In total, the single parasite transcriptomics holds promise for molecular dissection of rare parasite phenotypes throughout the malaria lifecycle.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2026

A ModEx Framework for Watershed Subsurface Investigation With Limited Geophysical Data Using Machine Learning and Hydrologic Modeling

Abstract Subsurface heterogeneity influences watershed hydrology strongly but remains difficult to characterize at catchment scales with sparse and costly field data. Geophysical surveys such as electromagnetic induction (EMI) provide local spatial subsurface images yet scaling them to watershed scales and converting EMI‐derived resistivity into hydraulic properties remains a challenge. We present a Model–Experiment (ModEx) framework that integrates limited EMI data with machine learning (ML) and hydrologic modeling to improve process representation and guide field investigations. Sparse EMI surveys were scaled to the catchment scale using a Random Forest model, and the resulting resistivity fields were combined with nearby borehole constraints to parameterize a hydrologic model. The EMI‐informed hydrological simulations improved predictions of streamflow sustained by subsurface flow and shallow saturation patterns. By combining EMI data and ML with hydrologic modeling, the ModEx framework guides future subsurface surveys, providing a transferable and efficient strategy for data–model integration across diverse watersheds. Plain Language Summary Mapping the underground network of soil and rock that controls water is essential for predicting floods and droughts, but seeing underground is difficult and expensive. We cannot drill everywhere, so scientists use geophysical tools to scan broad areas. There are two key challenges: these geophysical scans are often sparse across the whole watershed, and the geophysical data is hard to translate into water‐related properties. We used artificial intelligence to solve these problems. We taught a computer to find patterns linking the limited geophysical data to the land surface properties. This allowed it to fill in the gaps and create a complete, useful subsurface map for the entire watershed. This new map improves hydrologic simulations, leading to more accurate predictions of water movement in the watershed. It also helps scientists build better models with less data and generates a priority map showing where to measure next, making future investigations more efficient. Key Points Limited EMI scaled with ML improves catchment‐scale subsurface parameterization for hydrologic models The framework integrates hydrologic modeling with limited geophysical data to support subsurface investigation design ModEx framework offers a transferable data–model integration strategy that quantifies and reduces uncertainty guiding watershed studies

Chen, Hang↗
DOE OSTI2021

PLANC: Parallel Low-rank Approximation with Nonnegativity Constraints

In this work, we consider the problem of low-rank approximation of massive dense nonnegative tensor data, for example, to discover latent patterns in video and imaging applications. As the size of data sets grows, single workstations are hitting bottlenecks in both computation time and available memory. We propose a distributed-memory parallel computing solution to handle massive data sets, loading the input data across the memories of multiple nodes, and performing efficient and scalable parallel algorithms to compute the low-rank approximation. We present a software package called Parallel Low-rank Approximation with Nonnegativity Constraints, which implements our solution and allows for extension in terms of data (dense or sparse, matrices or tensors of any order), algorithm (e.g., from multiplicative updating techniques to alternating direction method of multipliers), and architecture (we exploit GPUs to accelerate the computation in this work). We describe our parallel distributions and algorithms, which are careful to avoid unnecessary communication and computation, show how to extend the software to include new algorithms and/or constraints, and report efficiency and scalability results for both synthetic and real-world data sets.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

A novel coupling of noise reduction algorithms for particle flow simulations

Proper orthogonal decomposition (POD) and its extension based on time-windows have been shown to greatly improve the effectiveness of recovering smooth ensemble solutions from noisy particle data. However, to successfully de-noise any molecular system, a large number of measurements still need to be provided. In order to achieve a better efficiency in processing time-dependent fields, we have combined POD with a well-established signal processing technique, wavelet-based thresholding. In this novel hybrid procedure, the wavelet filtering is applied within the POD domain and referred to as WAVinPOD. The algorithm exhibits promising results when applied to both synthetically generated signals and particle data. In this work, the simulations compare the performance of our new approach with standard POD or wavelet analysis in extracting smooth profiles from noisy velocity and density fields. Numerical examples include molecular dynamics and dissipative particle dynamics simulations of unsteady force- and shear-driven liquid flows, as well as phase separation phenomenon. Simulation results confirm that WAVinPOD preserves the dimensionality reduction obtained using POD, while improving its filtering properties through the sparse representation of data in wavelet basis. This paper shows that WAVinPOD outperforms the other estimators for both synthetically generated signals and particle-based measurements, achieving a higher signal-to-noise ratio from a smaller number of samples. The new filtering methodology offers significant computational savings, particularly for multi-scale applications seeking to couple continuum informations with atomistic models. It is the first time that a rigorous analysis has compared de-noising techniques for particle-based fluid simulations.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2016

Inter-Disciplinary Collaboration in Support of the Post-Standby TREAT Mission

Although analysis methods have advanced significantly in the last two decades, high fidelity multi- physics methods for reactors systems have been under development for only a few years and are not presently mature nor deployed. Furthermore, very few methods provide the ability to simulate rapid transients in three dimensions. Data for validation of advanced time-dependent multi- physics is sparse; at TREAT, historical data were not collected for the purpose of validating three-dimensional methods, let alone multi-physics simulations. Existing data continues to be collected to attempt to simulate the behavior of experiments and calibration transients, but it will be insufficient for the complete validation of analysis methods used for TREAT transient simulations. Hence, a 2018 restart will most likely occur without the direct application of advanced modeling and simulation methods. At present, the current INL modeling and simulation team plans to work with TREAT operations staff in performing reactor simulations with MAMMOTH, in parallel with the software packages currently being used in preparation for core restart (e.g., MCNP5, RELAP5, ABAQUS). The TREAT team has also requested specific measurements to be performed during startup testing, currently scheduled to run from February to August of 2018. These startup measurements will be crucial in validating the new analysis methods in preparation for ultimate application for TREAT operations and experiment design. This document describes the collaboration between modeling and simulation staff and restart, operations, instrumentation and experiment development teams to be able to effectively interact and achieve successful validation work during restart testing.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

FAST INVERSION OF SOLAR Ca II SPECTRA

We present a fast (<<1 s per profile) inversion code for solar Ca II lines. The code uses an archive of spectra that are synthesized prior to the inversion under the assumption of local thermodynamic equilibrium (LTE). We show that it can be successfully applied to spectrograph data or more sparsely sampled spectra from two-dimensional spectrometers. From a comparison to a non-LTE inversion of the same set of spectra, we derive a first-order non-LTE correction to the temperature stratifications derived in the LTE approach. The correction factor is close to unity up to log τ ∼ –3 and increases to values of 2.5 and 4 at log τ = –6 in the quiet Sun and the umbra, respectively.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2018

Robust Period Estimation Using Mutual Information for Multiband Light Curves in the Synoptic Survey Era

The Large Synoptic Survey Telescope (LSST) will produce an unprecedented amount of light curves using six optical bands. Robust and efficient methods that can aggregate data from multidimensional sparsely sampled time-series are needed. Here, we present a new method for light curve period estimation based on quadratic mutual information (QMI). The presented method does not assume a particular model for the light curve nor its underlying probability density and it is robust to non-Gaussian noise and outliers. By combining the QMI from several bands the true period can be estimated even when no single-band QMI yields the period. Period recovery performance as a function of average magnitude and sample size is measured using 30,000 synthetic multiband light curves of RR Lyrae and Cepheid variables generated by the LSST Operations and Catalog simulators. The results show that aggregating information from several bands is highly beneficial in LSST sparsely sampled time-series, obtaining an absolute increase in period recovery rate up to 50%. We also show that the QMI is more robust to noise and light curve length (sample size) than the multiband generalizations of the Lomb–Scargle and AoV periodograms, recovering the true period in 10%–30% more cases than its competitors. A python package with efficient Cython implementations of the QMI and other methods is provided.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Artificial Intelligence Guided Studies of van der Waals Magnets

A materials informatics framework to explore a large number of candidate van der Waals (vdW) materials is developed. In particular, in this study a large space of monolayer transition metal halides is investigated by combining high-throughput density functional theory calculations and artificial intelligence (AI) to accelerate the discovery of stable materials and the prediction of their magnetic properties. Here, the formation energy is used as a proxy for chemical stability. Semi-supervised learning is harnessed to mitigate the challenges of sparsely labeled materials data in order to improve the performance of AI models. This approach creates avenues for the rapid discovery of chemically stable vdW magnets by leveraging the ability of AI to recognize patterns in data, to learn mathematical representations of materials from data and to predict materials properties. Using this approach, previously unexplored vdW magnetic materials with potential applications in data storage and spintronics are identified.

36 MATERIALS SCIENCE↗
DOE OSTI2017

A comparative study of discrete fracture network and equivalent continuum models for simulating flow and transport in the far field of a hypothetical nuclear waste repository in crystalline host rock

One of the major challenges of simulating flow and transport in the far field of a geologic repository in crystalline host rock is related to reproducing the properties of the fracture network over the large volume of rock with sparse fracture characterization data. Various approaches have been developed to simulate flow and transport through the fractured rock. The approaches can be broadly divided into Discrete Fracture Network (DFN) and Equivalent Continuum Model (ECM). The DFN explicitly represents individual fractures, while the ECM uses fracture properties to determine equivalent continuum parameters. In this paper, we compare DFN and ECM in terms of upscaled observed transport properties through generic fracture networks. The major effort was directed on making the DFN and ECM approaches similar in their conceptual representations. This allows for separating differences related to the interpretation of the test conditions and parameters from the differences between the DFN and ECM approaches. The two models are compared using a benchmark test problem that is constructed to represent the far field (1 × 1 × 1 km 3 ) of a hypothetical repository in fractured crystalline rock. The test problem setting uses generic fracture properties that can be expected in crystalline rocks. The models are compared in terms of the: 1) effective permeability of the domain, and 2) nonreactive solute breakthrough curves through the domain. The principal differences between the models are mesh size, network connectivity, matrix diffusion and anisotropy. We demonstrate how these differences affect the flow and transport. Finally, we identify the factors that should be taken in consideration when selecting an approach most suitable for the site-specific conditions.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2026

Machine Learning Eliminates Reanalysis Warm Bias and Reveals Weaker Winter Surface Cooling Over Arctic Sea Ice

The surface energy budget governs Arctic sea-ice growth/melt, yet observations are sparse, and reanalysis data sets suffer from systematic biases. Here, we train a neural network with observational data to bias-correct hourly ERA5 fluxes over Arctic ice-covered regions (≥70°N; sea-ice concentration >80%) for 1994–2024. Training data cover two full seasonal cycles and different sea-ice regimes. The neural network reduces RMSE for net shortwave radiation by ∼40%, downward longwave radiation by ∼16% and the total surface energy budget by ∼55%, eliminating the wintertime warm bias of ∼4 K in ERA5. Wintertime surface cooling is reduced by ∼50%, yielding thermodynamic ice-growth estimates of ∼80–120 cm, consistent with SMOS–CryoSat satellite thickness increases and in contrast to the 150–200 cm growth implied by ERA5. Our bias-corrected data capture the observed clear/cloudy states of the winter boundary layer and can be used to study Arctic climatology, evaluate climate models and drive sea-ice-ocean models.

Hossain, Akil [Alfred Wegener Institute for Polar ↗
DOE OSTI2024

Efficient learning of accurate surrogates for simulations of complex systems

Machine learning methods are increasingly deployed to construct surrogate models for complex physical systems at a reduced computational cost. However, the predictive capability of these surrogates degrades in the presence of noisy, sparse or dynamic data. Here, we introduce an online learning method empowered by optimizer-driven sampling that has two advantages over current approaches: it ensures that all local extrema (including endpoints) of the model response surface are included in the training data, and it employs a continuous validation and update process in which surrogates undergo retraining when their performance falls below a validity threshold. We find, using benchmark functions, that optimizer-directed sampling generally outperforms traditional sampling methods in terms of accuracy around local extrema even when the scoring metric is biased towards assessing overall accuracy. Finally, the application to dense nuclear matter demonstrates that highly accurate surrogates for a nuclear equation-of-state model can be reliably autogenerated from expensive calculations using few model evaluations.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2015

Acoustic emission during the ferroelectric transition Pm3{sup ¯}m to P4mm in BaTiO{sub 3} and the ferroelastic transition R3{sup ¯}m-C2/c in Pb{sub 3}(PO{sub 4}){sub 2}

Acoustic emission (AE) spectroscopy without frequency filtering (∼broadband AE) and moderate time integration is shown to be sensitive enough to allow the investigation of subtle nano-structural changes in ferroelectric BaTiO{sub 3} and ferroelastic Pb{sub 3}(PO{sub 4}){sub 2}. AE signals during weak phase transitions are compatible with avalanche statistics as observed previously in large-strain systems. While the data are too sparse to determine avalanche exponents, they are well suited to determine other thermodynamic parameters such as transition temperatures and critical stresses.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2016

Theoretical evaluation of two dimensional electron gas characteristics of quaternary Al{sub x}In{sub y}Ga{sub 1–x–y}N/GaN hetero-junctions

The two dimensional electron gas (2DEG) characteristics of gated metal-face wurtzite AlInGaN/GaN hetero-junctions including positions of subband energy levels, fermi energy level, and the 2DEG concentration as functions of physical and compositional properties of the hetero-junction (i.e., barrier thickness and metal mole-fractions) are theoretically evaluated using the variational method. The calculated values of the 2DEG concentration are in good agreement with the sparsely available experimental data reported in the literature. According to our simulation results, a considerable shift in the positive direction of threshold voltage of AlInGaN/GaN hetero-junction field-effect transistors can be achieved by engineering both the spontaneous and the piezoelectric polarizations using a quaternary AlInGaN barrier-layer of appropriate mole-fractions.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Bayesian Gaussian process inference for neutron spin echo measurement

Neutron spin echo (NSE) spectroscopy provides unique access to microscopic dynamics, but its application is often constrained by low neutron flux, long acquisition times, and significant noise. Here, we present a Bayesian inference approach based on Gaussian process regression (GPR) to reconstruct high-quality spin echo signals from sparse and noisy data by exploiting correlations in reciprocal space. Benchmarks on synthetic datasets and validation with experimental NSE measurements of dendrimers show that GPR suppresses noise, interpolates missing intensity values, and accommodates irregular observations. The method improves accuracy, shortens acquisition times, and enables high-throughput and real-time studies. Beyond NSE, the framework is broadly applicable to other low signal-to-noise ratio scattering techniques, thereby extending the scope of neutron spectroscopy.

Tung, Chi-Huan [Oak Ridge National Laboratory (ORN↗