Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “sparse data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,117 records · Page 62
All sourcesNASA NTRSDOE OSTI
DOE OSTI2013

Do Diurnal Aerosol Changes Affect Daily Average Radiative Forcing?

Strong diurnal variability of aerosol has been observed frequently for many urban/industrial regions. How this variability may alter the direct aerosol radiative forcing (DARF), however, is largely unknown. To quantify changes in the time-averaged DARF, we perform an assessment of 29 days of high temporal resolution ground-based data collected during the Two-Column Aerosol Project (TCAP) on Cape Cod, which is downwind of metropolitan areas. We demonstrate that strong diurnal changes of aerosol loading (about 20% on average) have a negligible impact on the 24-h average DARF, when daily averaged optical properties are used to find this quantity. However, when there is a sparse temporal sampling of aerosol properties, which may preclude the calculation of daily averaged optical properties, large errors (up to 100%) in the computed DARF may occur. We describe a simple way of reducing these errors, which suggests the minimal temporal sampling needed to accurately find the forcing.

aerosol↗
DOE OSTI2023

Navier: Dataflow Architecture for Computation Chemistry

Navier’s objectives were two evaluate the use of emerging technologies, especially dataflow accelerators, for high-performance computing (HPC) applications, specifically in the domain of chemistry, and to develop a prototype software stack to support such applications. Navier builds on capabilities previously developed by synergistic projects, such as PNNL Data Model Convergence (DMC) LDRD Hardware Advanced Workflows (HAW) and DuOMO, as well as DOE ARIAA. Throughout its 18 months, the Navier team developed new capabilities and artifacts at all levels of the HW/SW stack, provided a seamless way to integrate novel computing architectures (Sambanova SN10 and Xilinx Versal AI) into an existing software stack, developed chemistry workflows, data analytics tools, and HPC molecular dynamics workflows that leverage the developed stack and PNNL institutional investments in emerging architectures. Navier also explored the use of active learning to accelerate a computational chemistry workflow for organic molecules on PNNL Junction cluster (in collaboration with AMD/Xilinx). Navier developed tools, methodologies, and studies for hardware software co-design and (sparse) dataflow accelerators that are composable and can be used together or separately. These methodologies are now used in other projects, such as DOE AMAIS and HPDA. This report describes Navier’s achievement, the developed tools and methodologies, and the research findings and conclusions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Assimilation of citizen science data in snowpack modeling using a new snow data set: Community Snow Observations

A physically based snowpack evolution and redistribution model was used to test the effectiveness of assimilating crowd-sourced snow depth measurements collected by citizen scientists. The Community Snow Observations project gathers, stores, and distributes measurements of snow depth recorded by recreational users and snow professionals in high mountain environments. These citizen science measurements are valuable since they come from terrain that is relatively undersampled and can offer in situ snow information in locations where snow information is sparse or nonexistent. The present study investigates (1) the improvements to model performance when citizen science measurements are assimilated, and (2) the number of measurements necessary to obtain those improvements. Model performance is assessed by comparing time series of observed (snow pillow) and modeled snow water equivalent values, by comparing spatially distributed maps of observed (remotely sensed) and modeled snow depth, and by comparing fieldwork results from within the study area. The results demonstrate that few citizen science measurements are needed to obtain improvements in model performance, and these improvements are found in 62 % to 78 % of the ensemble simulations, depending on the model year. Model estimations of total water volume from a subregion of the study area also demonstrate improvements in accuracy after CSO measurements have been assimilated. These results suggest that even modest measurement efforts by citizen scientists have the potential to improve efforts to model snowpack processes in high mountain environments, with implications for water resource management and process-based snow modeling.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Accurate and rapid acoustic damage characterization in complex structures using sparse sensor networks and deep learning models

Damage diagnosis in critical components is essential for ensuring the safety and reliability of operations across industries, spanning manufacturing, aerospace, and energy. Traditional acoustic nondestructive testing methods primarily focus on detecting defects through the direct scattering of single-mode incident waves from the damage, which limit their applicability to simple structures and small inspection areas. Our earlier research demonstrated that machine learning algorithms combined with sparse sensor networks can identify critical defect signatures even from multiply scattered, multi-mode acoustic signals, indicating the potential for improved defect inspection in complex, real-world structures. In this work, we demonstrate the successful implementation of this approach in a fixed sensor configuration to rapidly and accurately detect simulated defects in a geometrically complex, real-world structure, a brake rotor hub. Three different types of defects were physically simulated on the surface of the hub, and the collected data were used to train an autoencoder-based deep learning model. Two models were tested, one using single measurements and the other using multiple measurements taking advantage of the spatial distribution of the sensor network. After training, the multi-measurement model achieved 100 % accuracy in identifying, classifying, and locating unseen, unique damages. This work illustrates the potential of the proposed method for a wide range of industrial applications.

36 MATERIALS SCIENCE↗
DOE OSTI2013

Mineral Association Changes the Secondary Structure and Dynamics of Murine Amelogenin

Biomineralization proteins, present during the formation of hard tissues including bones, teeth, egg shells and nacre, result in the exquisite structures and properties of the resulting materials.[1] The structure of these proteins is often implicated in the control of the mineral properties, however very little structural data is available for the bulk of these proteins due to the difficulty in determining structures of immobilized proteins. Solid-state NMR is uniquely suited to the study of the structure of proteins bound to surfaces, demonstrated with the structural and orientation insights provided for the hydroxyapatite mineralization proteins statherin and the amelogenin, LRAP.[2] While these data are some of the only structural data available for this important class of protein, the experiments are often expensive and time consuming, due to the need to prepare and measure samples with isolated spin pairs, and are limited to a size of ~60 residues. In this work, we utilized a combination of 1D and recent 2D[3] solid-state NMR techniques along with a sparsely labelled sample to characterize the structure and dynamics of potential HAP binding residues of the 180 residue enamel protein, amelogenin. Amelogenin nanospheres and mineral bound amelogenin were investigated and a shift from unstructured to β-sheet structure was observed, along with a decrease in protein flexibility. This work provides the first molecular level structure and dynamic information of full-length amelogenin on the surface of hydroxyapatite (HAP) and within nanospheres, and demonstrates the ability to evaluate structural characteristics of large biomineralization proteins bound to their physiologically relevant surface. The research was performed at the Pacific Northwest National Laboratory (PNNL), a facility operated by Battelle for the U.S. Department of Energy, with a portion of it performed at the W.R. Wiley Environmental Molecular Sciences Laboratory (EMSL), a national scientific user facility sponsored by the U.S. DOE Biological and Environmental Research program.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Fast Emulation of Expensive Simulations using Approximate Gaussian Processes [Slides]

Nuclear Computational Low-Energy Initiative (NUCLEI) collaboration uses Density Functional Theory (DFT) simulations to predict the structure and binding energies of nuclei over a wide range of proton (Z) and neutron (N) numbers. The DFT simulations utilize a particular parameterization of a Skyrme energy density functional called UNEDF1 which depends on 12 free parameters that must be fit to data (M Kortelainen et al 2014). Fitting involves comparing (e.g.) predicted binding energies of nuclei to experimentally measured values. We use only binding energies as observables, but DFT with UNEDF1 will predict structure (shape) observables as well. In this work, assessing the capability of approximate GP emulators to balance emulator accuracy with computational speed to facilitate improved UNEDF1 calibration. Sparse GPs are straightforward to train and accurate. Calibration is not straightforward with MCMC (using MH or HMC/NUTS). We produced reusable software for continuing and building on this work as well as accessing and using Darwin cluster compute resources

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

DeepM&Mnet: Inferring the electroconvection multiphysics fields based on operator approximation by neural networks

Electroconvection is a multiphysics problem involving coupling of the flow field with the electric field as well as the cation and anion concentration fields. Here, we use electroconvection as a benchmark problem to put forward a new data assimilation framework, the DeepM&Mnet, for simulating multiphysics and multiscale problems at speeds much faster than standard numerical methods using pre-trained neural networks. We first pre-train DeepONets that can predict independently each field, given general inputs from the rest of the fields of the coupled system. DeepONets can approximate nonlinear operators and are composed of two sub-networks, a branch net for the input fields and a trunk net for the locations of the output field. DeepONets, which are extremely fast, are used as building blocks in the DeepM&Mnet and form constraints for the multiphysics solution along with some sparse available measurements of any of the fields. We demonstrate the new methodology and document the accuracy of each individual DeepONet, and subsequently we present two different DeepM&Mnet architectures that infer accurately and efficiently 2D electroconvection fields for unseen electric potentials. Furthermore, the DeepM&Mnet framework is general and can be applied for building any complex multiphysics and multiscale models based on very few measurements using pre-trained DeepONets in a “plug-and-play” mode.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Bayesian chain graph models to characterize microbe-environment dynamics

Microbiome data require statistical models that can simultaneously decode microbes' reaction to the environment and interactions among microbes. While a multiresponse linear regression model seems like a straight-forward solution, we argue that treating it as a graphical model is problematic given that the regression coefficient matrix does not encode the conditional dependence structure between response and predictor nodes. This observation is especially important in biological settings when we have prior knowledge on the edges from specific experimental interventions that can only be properly encoded under a conditional dependence model. Here, we propose a chain graph model with two sets of nodes (predictors and responses) whose solution yields a graph with edges that indeed represent conditional dependence, thus agreeing with the experimenter's intuition on the average behavior of nodes under treatment. The solution to our model is sparse via the Bayesian linear regression (LASSO). In addition, we propose an adaptive extension so that different shrinkages can be applied to different edges to incorporate edge-specific prior knowledge. Our model is computationally inexpensive through an efficient Gibbs sampling algorithm and can account for binary, counting, and compositional responses via an appropriate hierarchical structure. We test the performance of our model in a variety of simulated datasets, thereby showing superior performance to state-of-the-art approaches. We further apply our model to human gut and soil microbial compositional datasets, and we highlight that CG-LASSO can estimate biologically meaningful network structures in the data.

compositional data↗
DOE OSTI2014

WE-G-18A-05: Cone-Beam CT Reconstruction with Deformed Prior Image

Purpose: Prior image can be incorporated into image reconstruction process to improve the quality of on-treatment cone-beam CT (CBCT) from sparseview or low-dose projections. However, the deformation between the prior image and on-treatment CBCT are not considered in current prior image based reconstructions (e.g., prior image constrained compressed sensing (PICCS)). The purpose of this work is to develop a deformed-prior-imagebased- reconstruction strategy (DPIR) to address the mismatch problem between the prior image and target image. Methods: The deformed prior image is obtained by a projection based registration approach. Specifically, the deformation vector fields (DVF) used to deform the prior image is estimated through matching the forward projection of the prior image and the measured on-treatment projection. The deformed prior image is then used as the prior image in the standard PICCS algorithm. Simulation studies on the XCAT phantom was conducted to evaluate the performance of the projection based registration procedure and the proposed DPIR strategy. Results: The deformed prior image matches the geometry of on-treatment CBCT closer as compared to the original prior image. Using the deformed prior image, the quality of the image reconstructed by DPIR from few-view projection data is greatly improved as compared to the standard PICCS algorithm. The relative image reconstruction error is reduced to 11.13% in the proposed DPIR from 17.57% in the original PICCS. Conclusion: The proposed DPIR approach can solve the mismatch problem between the prior image and target image, which overcomes the limitation of the original PICCS algorithm for CBCT reconstruction from sparse-view or low-dose projections.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2019

Cloud, Aerosol and Boundary Layer Structure across the Northeast Pacific Stratocumulus-Cumulus Transition as observed during CSET

During the Cloud System Evolution in the Trades (CSET) field study, 14 research flights of the National Science Foundation G-V sampled the stratocumulus-cumulus transition between Northern California and Hawaii and its synoptic variability. The G-V made vertically resolved measurements of turbulence, cloud microphysics, aerosol characteristics, and trace gases. It also carried dropsondes and a vertically pointing W-band radar and lidar. This paper summarizes these observations with the goals of fostering novel comparisons with theory, models and reanalyses, and satellite-derived products. A longitude-height binning and compositing strategy mitigates limitations of sparse sampling and spatiotemporal variability. Typically, a 1-km-deep decoupled stratocumulus-capped boundary layer near California evolved into 2-km-deep precipitating cumulus clusters surrounded by patches of thin stratus that dissipated toward Hawaii. Low cloud cover was correlated with estimated inversion strength more than with cloud droplet number, even though the thickest clouds were generally precipitating and ultraclean layers indicative of aerosol-cloud-precipitation interaction were common west of 140 degrees W. Accumulation-mode aerosol concentration correlated well with collocated cloud droplet number concentration and was typically largest near the surface. Aitken mode aerosol concentration was typically larger in the free troposphere. Wildfire smoke produced spikes of aerosol and trace gases on some flights. CSET data are compared with space-time collocated output from MERRA-2 reanalysis and from the CAM6 climate model run with winds and temperature nudged toward this reanalysis. The reanalysis compares better with the observed relative humidity than does nudged CAM6. Both vertically diffuse the stratocumulus cloud layer versus observations. MERRA-2 slightly underestimates in situ carbon monoxide measurements and underestimates ozone depletion within the boundary layer.

Bretherton, Christopher S.↗
DOE OSTI2024

Ground Heat Flux Reconstruction Using Bayesian Uncertainty Quantification Machinery and Surrogate Modeling

Ground heat flux (G 0 ) is a key component of the land-surface energy balance of high-latitude regions. Despite its crucial role in controlling permafrost degradation due to global warming, G 0 is sparsely measured and not well represented in the outputs of global scale model simulation. In this study, an analytical heat transfer model is tested to reconstruct G 0 across seasons using soil temperature series from field measurements, Global Climate Model, and climate reanalysis outputs. The probability density functions of ground heat flux and of model parameters are inferred using available G 0 data (measured or modeled) for snow-free period as a reference. When observed G 0 is not available, a numerical model is applied using estimates of surface heat flux (dependent on parameters) as the top boundary condition. These estimates (and thus the corresponding parameters) are verified by comparing the distributions of simulated and measured soil temperature at several depths. Aided by state-of-the-art uncertainty quantification methods, the developed G 0 reconstruction approach provides novel means for assessing the probabilistic structure of the ground heat flux for regional permafrost change studies.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Machine learning for detection of 3D features using sparse x-ray tomographic reconstruction

In many inertial confinement fusion (ICF) experiments, the neutron yield and other parameters cannot be completely accounted for with one and two dimensional models. This discrepancy suggests that there are three dimensional effects that may be significant. Sources of these effects include defects in the shells and defects in shell interfaces, the fill tube of the capsule, and the joint feature in double shell targets. Due to their ability to penetrate materials, x rays are used to capture the internal structure of objects. Methods such as computational tomography use x-ray radiographs from hundreds of projections, in order to reconstruct a three dimensional model of the object. In experimental environments, such as the National Ignition Facility and Omega-60, the availability of these views is scarce, and in many cases only consists of a single line of sight. Mathematical reconstruction of a 3D object from sparse views is an ill-posed inverse problem. These types of problems are typically solved by utilizing prior information. Neural networks have been used for the task of 3D reconstruction as they are capable of encoding and leveraging this prior information. We utilize half a dozen, different convolutional neural networks to produce different 3D representations of ICF implosions from the experimental data. Deep supervision is utilized to train a neural network to produce high-resolution reconstructions. These representations are used to track 3D features of the capsules, such as the ablator, inner shell, and the joint between shell hemispheres. Machine learning, supplemented by different priors, is a promising method for 3D reconstructions in ICF and x-ray radiography, in general.

Wolfe, Bradley T. (ORCID:0000000268301614)↗
DOE OSTI2024

Estimating Sparse Direct Effects in Multivariate Regression With the Spike-and-Slab LASSO

The multivariate regression interpretation of the Gaussian chain graph model simultaneously parametrizes (i) the direct effects of p predictors on q outcomes and (ii) the residual partial covariances between pairs of outcomes. We introduce a new method for fitting sparse versions of these models with spike-and-slab LASSO (SSL) priors. We develop an Expectation Conditional Maximization algorithm to obtain sparse estimates of the p × q matrix of direct effects and the q × q residual precision matrix. Our algorithm iteratively solves a sequence of penalized maximum likelihood problems with self-adaptive penalties that gradually filter out negligible regression coefficients and partial covariances. Because it adaptively penalizes individual model parameters, our method is seen to outperform fixed-penalty competitors on simulated data. We establish the posterior contraction rate for our model, buttressing our method’s excellent empirical performance with strong theoretical guarantees. Using our method, we estimated the direct effects of diet and residence type on the composition of the gut microbiome of elderly adults.

EM algorithm↗
DOE OSTI2015

OPEN CLUSTERS AS PROBES OF THE GALACTIC MAGNETIC FIELD. I. CLUSTER PROPERTIES

Stars in open clusters are powerful probes of the intervening Galactic magnetic field via background starlight polarimetry because they provide constraints on the magnetic field distances. We use 2MASS photometric data for a sample of 31 clusters in the outer Galaxy for which near-IR polarimetric data were obtained to determine the cluster distances, ages, and reddenings via fitting theoretical isochrones to cluster color–magnitude diagrams. The fitting approach uses an objective χ{sup 2} minimization technique to derive the cluster properties and their uncertainties. We found the ages, distances, and reddenings for 24 of the clusters, and the distances and reddenings for 6 additional clusters that were either sparse or faint in the near-IR. The derived ranges of log(age), distance, and E(B−V) were 7.25–9.63, ∼670–6160 pc, and 0.02–1.46 mag, respectively. The distance uncertainties ranged from ∼8% to 20%. The derived parameters were compared to previous studies, and most cluster parameters agree within our uncertainties. To test the accuracy of the fitting technique, synthetic clusters with 50, 100, or 200 cluster members and a wide range of ages were fit. These tests recovered the input parameters within their uncertainties for more than 90% of the individual synthetic cluster parameters. These results indicate that the fitting technique likely provides reliable estimates of cluster properties. The distances derived will be used in an upcoming study of the Galactic magnetic field in the outer Galaxy.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Cosmological cross-correlations and nearest neighbour distributions

ABSTRACT Cross-correlations between data sets are used in many different contexts in cosmological analyses. Recently, k-nearest neighbour cumulative distribution functions (kNN-CDF) were shown to be sensitive probes of cosmological (auto) clustering. In this paper, we extend the framework of NN measurements to describe joint distributions of, and correlations between, two data sets. We describe the measurement of joint kNN-CDFs, and show that these measurements are sensitive to all possible connected N-point functions that can be defined in terms of the two data sets. We describe how the cross-correlations can be isolated by combining measurements of the joint kNN-CDFs and those measured from individual data sets. We demonstrate the application of these measurements in the context of Gaussian density fields, as well as for fully non-linear cosmological data sets. Using a Fisher analysis, we show that measurements of the halo-matter cross-correlations, as measured through NN measurements are more sensitive to the underlying cosmological parameters, compared to traditional two-point cross-correlation measurements over the same range of scales. Finally, we demonstrate how the NN cross-correlations can robustly detect cross-correlations between sparse samples – the same regime where the two-point cross-correlation measurements are dominated by noise.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2014

ACDT: Architected Composite Data Types Trading-in Unfettered Data Access for Improved Execution

With Exascale performance and its challenges in mind, one ubiquitous concern among architects is energy efficiency. Petascale systems projected to Exascale systems are unsustainable at current power consumption rates. One major contributor to system-wide power consumption is the number of memory operations leading to data movement and management techniques applied by the runtime system. To address this problem, we present the concept of the Architected Composite Data Types (ACDT) framework. The framework is made aware of data composites, assigning them a specific layout, transformations and operators. Data manipulation overhead is amortized over a larger number of elements and program performance and power efficiency can be significantly improved. We developed the fundamentals of an ACDT framework on a massively multithreaded adaptive runtime system geared towards Exascale clusters. Showcasing the capability of ACDT, we exercised the framework with two representative processing kernels - Matrix Vector Multiply and the Cholesky Decomposition – applied to sparse matrices. As transformation modules, we applied optimized compress/decompress engines and configured invariant operators for maximum energy/performance efficiency. Additionally, we explored two different approaches based on transformation opaqueness in relation to the application. Under the first approach, the application is agnostic to compression and decompression activity. Such approach entails minimal changes to the original application code, but leaves out potential applicationspecific optimizations. The second approach exposes the decompression process to the application, hereby exposing optimization opportunities that can only be exploited with application knowledge. The experimental results show that the two approaches have their strengths in HW and SW respectively, where the SW approach can yield performance and power improvements that are an order of magnitude better than ACDT-oblivious, hand-optimized implementations.We consider the ACDT runtime framework an important component of compute nodes that will lead towards power efficient Exascale clusters.

exascale↗
DOE OSTI2020

Long short-term memory embedded nudging schemes for nonlinear data assimilation of geophysical flows

Reduced rank nonlinear filters are increasingly utilized in data assimilation of geophysical flows, but often require a set of ensemble forward simulations to estimate forecast covariance. On the other hand, predictor-corrector type nudging approaches are still attractive due to their simplicity of implementation when more complex methods need to be avoided. However, optimal estimate of nudging gain matrix might be cumbersome. In this paper, we put forth a fully nonintrusive recurrent neural network approach based on a long short-term memory (LSTM) embedding architecture to estimate the nudging term, which plays a role not only to force the state trajectories to the observations but also acts as a stabilizer. Furthermore, our approach relies on the power of archival data and the trained model can be retrained effectively due to power of transfer learning in any neural network applications. In order to verify the feasibility of the proposed approach, we perform twin experiments using Lorenz 96 system. Our results demonstrate that the proposed LSTM nudging approach yields more accurate estimates than both extended Kalman filter (EKF) and ensemble Kalman filter (EnKF) when only sparse observations are available. With the availability of emerging AI-friendly and modular hardware technologies and heterogeneous computing platforms, we articulate that our simplistic nudging framework turns out to be computationally more efficient than either the EKF or EnKF approaches.

42 ENGINEERING↗
DOE OSTI2014

Photometric and structural properties of NGC 6544: A combined VVV-Hubble space telescope study

We combine archival Hubble Space Telescope imaging with wide-field near-infrared photometry to study the neglected metal-poor Galactic globular cluster NGC 6544. A high spatial resolution map of differential reddening over the inner portion of the cluster is constructed, revealing variations of up to half of the total reddening, and the resulting corrected color-magnitude diagrams reveal a sparse blue horizontal branch and centrally concentrated blue straggler population, verified via relative proper motions. Using the corrected photometry to investigate the cluster distance, reddening, and age via direct comparison to well-calibrated photometry of clusters with similar metallicities, we estimate (m – M){sub 0} = 11.96, E(B – V) = 0.79, and an age coeval with M13 to within the relevant uncertainties. Although our data are insufficient to place tight constraints on the reddening law toward NGC 6544, we find no strong evidence that it is non-standard at optical or near-infrared wavelengths. We also provide near-infrared fiducial sequences extending nearly 2 mag below the cluster main sequence turnoff, generated from a statistically decontaminated sample of cluster stars. Lastly, we redetermine the cluster center and construct a radial number density profile which is well fit by an atypically flat power law with a slope of about 1.7. We discuss this result, together with a flattened main sequence luminosity function and inverted mass function, in the context of mass segregation and tidal stripping via interactions with Milky Way potential.

79 ASTRONOMY AND ASTROPHYSICS↗