Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “High dimensional data,”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29
All sourcesNASA NTRSDOE OSTI
DOE OSTI2013

Exploration of High-Dimensional Scalar Function for Nuclear Reactor Safety Analysis and Visualization

The next generation of methodologies for nuclear reactor Probabilistic Risk Assessment (PRA) explicitly accounts for the time element in modeling the probabilistic system evolution and uses numerical simulation tools to account for possible dependencies between failure events. The Monte-Carlo (MC) and the Dynamic Event Tree (DET) approaches belong to this new class of dynamic PRA methodologies. A challenge of dynamic PRA algorithms is the large amount of data they produce which may be difficult to visualize and analyze in order to extract useful information. We present a software tool that is designed to address these goals. We model a large-scale nuclear simulation dataset as a high-dimensional scalar function defined over a discrete sample of the domain. First, we provide structural analysis of such a function at multiple scales and provide insight into the relationship between the input parameters and the output. Second, we enable exploratory analysis for users, where we help the users to differentiate features from noise through multi-scale analysis on an interactive platform, based on domain knowledge and data characterization. Our analysis is performed by exploiting the topological and geometric properties of the domain, building statistical models based on its topological segmentations and providing interactive visual interfaces to facilitate such explorations. We provide a user’s guide to our software tool by highlighting its analysis and visualization capabilities, along with a use case involving dataset from a nuclear reactor safety simulation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

On‐site surrogates for large‐scale calibration

Abstract Motivated by a computer model calibration problem from the oil and gas industry, involving the design of a honeycomb seal, we develop a new Bayesian methodology to cope with limitations in the canonical apparatus stemming from several factors. We propose a new strategy of on‐site design and surrogate modeling for a computer simulator acting on a high‐dimensional input space that, although relatively speedy, is prone to numerical instabilities, missing data, and nonstationary dynamics. Our aim is to strike a balance between data‐faithful modeling and computational tractability in a calibration framework—tailoring the computer model to a limited field experiment. Situating our on‐site surrogates within the canonical calibration apparatus requires updates to that framework. We describe a novel yet intuitive Bayesian setup that carefully decomposes otherwise prohibitively large matrices by exploiting the sparse blockwise structure. Empirical illustrations demonstrate that this approach performs well on toy data and our motivating honeycomb example.

Huang, Jiangeng↗
NASA NTRS2018

Climatespark: an In-Memory Distributed Computing Framework for Big Climate Data Analytics

The unprecedented growth of climate data creates new opportunities for climate studies, and yet big climate data pose a grand challenge to climatologists to efficiently manage and analyze big data. The complexity of climate data content and analytical algorithms increases the difficulty of implementing algorithms on high performance computing systems. This paper proposes an in-memory, distributed computing framework, ClimateSpark, to facilitate complex big data analytics and time-consuming computational tasks. Chunking data structure improves parallel I/O efficiency, while a spatiotemporal index is built for the chunks to avoid unnecessary data reading and preprocessing. An integrated, multi-dimensional, array-based data model (ClimateRDD) and ETL operations are developed to address big climate data variety by integrating the processing components of the climate data lifecycle. ClimateSpark utilizes Spark SQL and Apache Zeppelin to develop a web portal to facilitate the interaction among climatologists, climate data, analytic operations and computing resources (e.g., using SQL query and Scala/Python notebook). Experimental results show that ClimateSpark conducts different spatiotemporal data queries/analytics with high efficiency and data locality. ClimateSpark is easily adaptable to other big multiple- dimensional, array-based datasets in various geoscience domains.

Hu, Fei↗
DOE OSTI2026

A score-based diffusion model approach for adaptive learning of stochastic partial differential equation solutions

In this paper, we propose a novel framework for adaptively learning the time-evolving solutions of stochastic partial differential equations (SPDEs) using score-based diffusion models within a recursive Bayesian inference setting. SPDEs play a central role in modeling complex physical systems under uncertainty, but their numerical solutions often suffer from model errors and reduced accuracy due to incomplete physical knowledge and environmental variability. To address these challenges, we encode the governing physics into the score function of a diffusion model using simulation data and incorporate observational information via a likelihood-based correction in a reverse-time stochastic differential equation. This enables adaptive learning through iterative refinement of the solution as new data becomes available. To improve computational efficiency in high-dimensional settings, we introduce the ensemble score filter, a training-free approximation of the score function designed for real-time inference. Numerical experiments on benchmark SPDEs demonstrate the accuracy and robustness of the proposed method under sparse and noisy observations.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1998

Two-Dimensional High-Lift Aerodynamic Optimization Using Neural Networks

The high-lift performance of a multi-element airfoil was optimized by using neural-net predictions that were trained using a computational data set. The numerical data was generated using a two-dimensional, incompressible, Navier-Stokes algorithm with the Spalart-Allmaras turbulence model. Because it is difficult to predict maximum lift for high-lift systems, an empirically-based maximum lift criteria was used in this study to determine both the maximum lift and the angle at which it occurs. The 'pressure difference rule,' which states that the maximum lift condition corresponds to a certain pressure difference between the peak suction pressure and the pressure at the trailing edge of the element, was applied and verified with experimental observations for this configuration. Multiple input, single output networks were trained using the NASA Ames variation of the Levenberg-Marquardt algorithm for each of the aerodynamic coefficients (lift, drag and moment). The artificial neural networks were integrated with a gradient-based optimizer. Using independent numerical simulations and experimental data for this high-lift configuration, it was shown that this design process successfully optimized flap deflection, gap, overlap, and angle of attack to maximize lift. Once the neural nets were trained and integrated with the optimizer, minimal additional computer resources were required to perform optimization runs with different initial conditions and parameters. Applying the neural networks within the high-lift rigging optimization process reduced the amount of computational time and resources by 44% compared with traditional gradient-based optimization procedures for multiple optimization runs.

Greenman, Roxana M.↗
DOE OSTI2023

Navigating the Expansive Landscapes of Soft Materials: A User Guide for High-Throughput Workflows

Synthetic polymers are highly customizable with tailored structures and functionality, yet this versatility generates challenges in the design of advanced materials due to the size and complexity of the design space. Thus, exploration and optimization of polymer properties using combinatorial libraries has become increasingly common, which requires careful selection of synthetic strategies, characterization techniques, and rapid processing workflows to obtain fundamental principles from these large data sets. Herein, we provide guidelines for strategic design of macromolecule libraries and workflows to efficiently navigate these high-dimensional design spaces. We describe synthetic methods for multiple library sizes and structures as well as characterization methods to rapidly generate data sets, including tools that can be adapted from biological workflows. We further highlight relevant insights from statistics and machine learning to aid in data featurization, representation, and analysis. This Perspective acts as a “user guide” for researchers interested in leveraging high-throughput screening toward the design of multifunctional polymers and predictive modeling of structure–property relationships in soft materials.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Trajectory Optimization via Unsupervised Probabilistic Learning On Manifolds

This report investigates the use of unsupervised probabilistic learning techniques for the analysis of hypersonic trajectories. The algorithm first extracts the intrinsic structure in the data via a diffusion map approach. Using the diffusion coordinates on the graph of training samples, the probabilistic framework augments the original data with samples that are statistically consistent with the original set. The augmented samples are then used to construct conditional statistics that are ultimately assembled in a path-planing algorithm. In this framework the controls are determined stage by stage during the flight to adapt to changing mission objectives in real-time. A 3DOF model was employed to generate optimal hypersonic trajectories that comprise the training datasets. The diffusion map algorithm identfied that data resides on manifolds of much lower dimensionality compared to the high-dimensional state space that describes each trajectory. In addition to the path-planing worflow we also propose an algorithm that utilizes the diffusion map coordinates along the manifold to label and possibly remove outlier samples from the training data. This algorithm can be used to both identify edge cases for further analysis as well as to remove them from the training set to create a more robust set of samples to be used for the path-planing process.

42 ENGINEERING↗
DOE OSTI2024

MULTI-LEADER: MULTI-source LEarning-Accelerated Design of high-Efficiency multi-stage compRessor (Final Technical Report)

The objective of MULTI-LEADER is to cut design costs by 80% while generating more energy-efficient designs of multi-stage compressors by developing and implementing novel machine learning (ML) techniques, which enable faster and fewer design iterations, improved solver performance, and concurrent multi-disciplinary design. Current industrial practices for the design of multi-stage compressors involve simulation-based design optimization with successive levels of model fidelity, iteratively evaluated between distinct disciplines, one stage at a time to tackle the high dimensional design variations. This project addresses these key design challenges: (1) concurrent optimization of multiple stages under many non-linear constraints; (2) multitude of evaluation of high-fidelity and expensive solvers and their gradients during optimization convergence in high-dimensional design; (3) multi-disciplinary design to maximize aerodynamic performance while guaranteeing structural integrity and additive manufacturability; (4) utilization of multiple fidelity of solvers with disparate parameterization and modeling assumptions. MULTI-LEADER achieved more than 5x speed up in detailed design of more energy-efficient compressors via these machine learning (ML) innovations: (i) rapid design surrogates by multi-source learning from diverse fidelities across multiple disciplines, (ii) physics-constrained data-augmented modeling for improved empiricism, (iii) generative manifold embedding for high dimensional concurrent design without gradient information; (iv) budget-constrained fidelity-adaptive sampling towards fewer design iterations.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2024

A Comprehensive Review of Latent Space Dynamics Identification Algorithms for Intrusive and Non-Intrusive Reduced-Order-Modeling

Numerical solvers of partial differential equations (PDEs) have been widely employed for simulating physical systems. However, the computational cost remains a major bottleneck in various scientific and engineering applications, which has motivated the development of reduced-order models (ROMs). Recently, machine-learning-based ROMs have gained significant popularity and are promising for addressing some limitations of traditional ROM methods, especially for advection dominated systems. In this chapter, we focus on a particular framework known as Latent Space Dynamics Identification (LaSDI), which transforms the high-fidelity data, governed by a PDE, to simpler and low-dimensional latent-space data, governed by ordinary differential equations (ODEs). These ODEs can be learned and subsequently interpolated to make ROM predictions. Each building block of LaSDI can be easily modulated depending on the application, which makes the LaSDI framework highly flexible. In particular, we present strategies to enforce the laws of thermodynamics into LaSDI models (tLaSDI), enhance robustness in the presence of noise through the weak form (WLaSDI), select high-fidelity training data efficiently through active learning (gLaSDI, GPLaSDI), and quantify the ROM prediction uncertainty through Gaussian processes (GPLaSDI). We demonstrate the performance of different LaSDI approaches on Burgers equation, a non-linear heat conduction problem, and a plasma physics problem, showing that LaSDI algorithms can achieve relative errors of less than a few percent and up to thousands of times speed-ups.

Computational Engineering, Finance, and Science (c↗
NASA NTRS1992

Tropospheric chemical models

The differences in atmospheric composition over the globe and the short- and long-term variations in this composition are the net effect of several atmospheric and biospheric processes: biospheric emissions, atmospheric circulation, atmospheric chemical transformations and finally deposition back to the surface. Accurate and realistic atmospheric chemistry and circulation models are essential to interpret the observed global distributions and trends of atmospheric species in terms of these underlying processes. Comparisons between model predictions and observations test current understanding of these processes and models used in conjunction with inverse methods allow deductions of the rates of these processes from the observations. With the planned inclusion of at least CO and CH4 observations on the Earth Observing System (EOS) satellites, together with the large global data set expected from in situ observations under the International Global Atmospheric Chemistry (IGAC) Project, the further development of global three-dimensional high-resolution atmospheric chemistry and circulation models in order to interpret this new data is a high-priority endeavor.

Prinn, R. G.↗
DOE OSTI2022

High Throughput Data-Driven Design of Laser-Crystallized 2D MoS 2 Chemical Sensors: A Demonstration for NO 2 Detection

High throughput characterization and processing techniques are becoming increasingly necessary to navigate multivariable, data-driven design challenges for sensors and electronic devices. For two-dimensional materials, device performance is highly dependent upon a vast array of material properties including the number of layers, lattice strain, carrier concentration, defect density, and grain structure. In this work, laser crystallization was used to locally pattern and transform hundreds of regions of amorphous MoS 2 thin films into 2D 2H-MoS 2 . Here a high throughput Raman spectroscopy approach was subsequently used to assess the process-dependent structural and compositional variations for each illuminated region, yielding over 6000 distinct nonresonant, resonant, and polarized Raman spectra. The rapid generation of a comprehensive library of structural and compositional data elucidated important trends between structure–property processing relationships involving laser-crystallized MoS 2 , including the relationships between grain size, grain orientation, and intrinsic strain. Moreover, extensive analysis of structure/property relationships allowed for intelligent design and evaluation of major contributions to device performance in MoS 2 chemical sensors. In particular, it is found that NO 2 sensor performance is strongly dependent on the orientation of the MoS 2 grains relative to the crystal plane.

36 MATERIALS SCIENCE↗
DOE OSTI2023

A Graphical Model for Fusing Diverse Microbiome Data

This paper develops a Bayesian graphical model for fusing disparate types of count data. The motivating application is the study of bacterial communities from diverse high-dimensional features, in this case, transcripts, collected from different treatments. In such datasets, there are no explicit correspondences between the communities and each corresponds to different factors, making data fusion challenging. We introduce a flexible multinomial-Gaussian generative model for jointly modeling such count data. This latent variable model jointly characterizes the observed data through a common multivariate Gaussian latent space that parameterizes the set of multinomial probabilities of the transcriptome counts. The covariance matrix of the latent variables induces a covariance matrix of co-dependencies between all the transcripts, effectively fusing multiple data sources. We present a computationally scalable variational Expectation-Maximization (EM) algorithm for inferring the latent variables and the parameters of the model. Here, the inferred latent variables provide a common dimensionality reduction for visualizing the data and the inferred parameters provide a predictive posterior distribution. In addition to simulation studies that demonstrate the variational EM procedure, we apply our model to a bacterial microbiome dataset.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2018

Final Scientific/Technical Report: Integrating Advanced Software and Statistical Methods for Spectroscopic Dark Energy Surveys

“Dark energy” is a phenomenon of unknown origin that pervades the vacuum of space and is causing the present-day expansion of the universe to accelerate. Understanding the nature of dark energy is one of the most fundamental challenges to modern physics, having the potential to revolutionize our theories of both gravity and elementary particle physics. The effects of dark energy are currently only accessible to astronomical survey experiments that probe large volumes of the cosmos. One of the most powerful forms of experiment for studying dark energy is the cosmological “redshift survey,” which maps the large-scale structure of the universe as traced by the positions of galaxies, quasars, and intergalactic hydrogen clouds. The signature of dark energy and other cosmological parameters is imprinted on this large-scale structure as it evolves over cosmic time. This project focused on maximizing the accurate extraction of information from redshift surveys by improving the algorithms and statistics applied across the chain of inference from raw pixels to high-level cosmological parameter estimation. At the low level, this project developed new techniques for calibrating and analyzing high-background, low signal-to-noise data from astronomical survey spectrographs to provide reliable three-dimensional maps of galaxies and quasars throughout the universe. At the high level, this project developed new statistical frameworks for understanding the completeness of redshift surveys and the underlying demographics of the galaxies that they observe, thereby allowing for a more accurate connection between survey results and theoretical models. This work contributes to maximizing the scientific reach of existing and planned surveys, and opens new potential feasibility for more ambitious surveys in the future.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2016

High-throughput and automated SAXS/USAXS experiment for industrial use at BL19B2 in SPring-8

A highly automated system combining a sample transfer robot with focused SR beam has been established for small-angle and ultra small-angle X-ray scattering (SAXS/USAXS) measurement at BL19B2 for industrial use of SPring-8. High-throughput data collection system can be realized by means of X-ray beam of high photon flux density concentrated by a cylindrical mirror, and a two-dimensional pixel detector PILATUS-2M. For SAXS measurement, we can obtain high-quality data within 1 minute for one exposure using this system. The sample transfer robot has a capacity of 90 samples with a large variety of shapes. The fusion of high-throughput and robotic system has enhanced the usability of SAXS/USAXS capability for industrial application.

43 PARTICLE ACCELERATORS↗
DOE OSTI2024

A deep learning-based workflow for fast prediction of 3D state variables in geological carbon storage: A dimension reduction approach

Deep learning (DL) models are extensively used as surrogate models for high-fidelity simulations of multiphase fluid flow in porous media at large scales, enabling fast forecasts of the spatial–temporal evolution of three-dimensional (3D) state variables in geological carbon storage (GCS). However, training these models in high-dimensional space remains computationally demanding and prone to overfitting because of limited training data. This paper presents a novel workflow to address these challenges by integrating dimension reduction (DR) methods. Here, the proposed workflow employed pre-trained DR models to extract the latent variables of geological models and state variables and utilized the multi-layer perceptron (MLP) for constructing mapping functions between the input and output variables in latent spaces. Subsequently, the pre-trained reconstruction models converted the MLP-predicted latent state variables to their original high-dimensional form. Furthermore, we proposed a novel strategy for the DR and reconstruction of 3D saturation fields to account for the unique data characteristics of sparsity, nonuniformity, and discontinuity. The proposed strategy applied PCA and inverse PCA for 2D average saturation fields and developed a DL-based 3D reconstruction model, leveraging three 2D average saturation fields as input to produce a 3D saturation field as output. The pre-training of DR and reconstruction models and training of MLP models were conducted on 84 Gulf of Mexico (GoM) simulations and evaluated on 12 testing simulations. Each simulation contained 720 monthly time steps, with the first 360 months as the injection period and the rest as the post-injection period. The proposed workflow, incorporating DR and DL models, accurately predicts the normalized 3D pressure fields, achieving mean square error (MSE) of 2.92 × 10 -7 compared to the ground truth obtained from a full-physics simulator. Furthermore, the proposed strategy outperformed PCA and convolutional autoencoder (CAE) models on 3D saturation fields, resulting in minor workflow prediction errors with an MSE of 2.93 × 10 -5 . The results suggest the proposed workflow provides sufficient predictive fidelity across temporal and spatial scales, and enables a speedup of 160 times compared to the full-physics simulator, facilitating improved decision-making and risk assessment for large-scale GCS management in real-time scenarios.

3D reconstruction model↗
NASA NTRS1980

Two-dimensional VLA maps of solar bursts at 15 and 23 GHz with arcsec resolution

Three small, impulsive solar-microwave bursts (peak fluxes 4.9, 3.0, and 0.4 sfu) were observed during 1979 September 7-9, using the Very Large Array (VLA) at 15.05 or 22.5 GHz. The data from 10 antennas distributed on three arms of the array provide the first two-dimensional burst images with spatial resolution as high as 1.0 x 0.75 arcsec. Comparison with optical data showed in the impulsive phase of all three flares, the microwave emission was dominated by a compact source located between the H-alpha kernels. In the post-impulsive phase, the microwave source was larger and elongated in a direction consistent with the orientation of the magnetic field lines joining the H-alpha kernels. These results are interpreted to imply that the initial energy release occurs near the top of the magnetic arch joining the H-alpha kernels.

Marsh, K. A.↗
DOE OSTI2016

Spectral characteristics of background error covariance and multiscale data assimilation

The spatial resolutions of numerical atmospheric and oceanic circulation models have steadily increased over the past decades. Horizontal grid spacing down to the order of 1 km is now often used to resolve cloud systems in the atmosphere and sub-mesoscale circulation systems in the ocean. These fine resolution models encompass a wide range of temporal and spatial scales, across which dynamical and statistical properties vary. In particular, dynamic flow systems for small scales can become spatially localized and temporarily intermittent. An analysis shows that the background correlation length scale is larger than 75 km for streamfunctions, even for a 2-km resolution model, and larger than 25 km for water vapor mixing ratios. The theoretical analyses suggest that such correlation length scales prevent the currently used data assimilation schemes from constraining spatial scales smaller than 150 km for streamfunctions and 50 km for water vapor mixing ratios. These results highlight the necessity of fundamentally modifying the currently used data assimilation algorithm for assimilating high-resolution observations into the aforementioned fine resolution models. A multiscale methodology based on scale decomposition is suggested, and challenges are discussed.

Li, Zhijin↗