Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “large data sets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Virtual Volumetric Additive Manufacturing (VirtualVAM)

Abstract Tomographic volumetric additive manufacturing (VAM) produces arbitrary 3D geometries by exposure of a rotating volume of photopolymer resin to tomographically‐patterned illumination. This enables high speed, layer‐less printing of parts from a wide range of photopolymers not amenable to layer‐by‐layer processes. Since the entire geometry is produced at once over the course of a few seconds to minutes, molecular diffusion length scales become significant to the printing process. Understanding these molecular reaction and diffusion processes is imperative for advancing VAM to a usable technology. These processes are experimentally very difficult to monitor and measure. Herein, VirtualVAM ‐ a simulation framework for modeling the tomographic VAM process, is developed and experimentally validated. VirtualVAM simulates reaction, diffusion, and heat generation processes over the course of a print with single‐voxel resolution. From a few experimentally‐determined input parameters and a set of images for projection, VirtualVAM is able to generate a large spatio‐temporal data set for any given tomographic VAM print. Using VirtualVAM, a number of experimentally‐unattainable aspects of the VAM process are investigated such as single‐voxel conversion profiles, effect of molecular oxygen, and stopping time determination. VirtualVAM also enables the optimization of exposure patterns to further improve contrast between in‐part and out‐of‐part delivered dose.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Multifrontal Non-negative Matrix Factorization

Non-negative matrix factorization (Nmf) is an important tool in high-performance large scale data analytics with applications ranging from community detection, recommender system, feature detection and linear and non-linear unmixing. While traditional Nmf works well when the data set is relatively dense, however, it may not extract sufficient structure when the data is extremely sparse. Specifically, traditional Nmf fails to exploit the structured sparsity of the large and sparse data sets resulting in dense factors. We propose a new algorithm for performing Nmf on sparse data that we call multifrontal Nmf (Mf-Nmf) since it borrows several ideas from the multifrontal method for unconstrained factorization (e.g. LU and QR). We also present an efficient shared memory parallel implementation of Mf-Nmf and discuss its performance and scalability. We conduct several experiments on synthetic and realworld datasets and demonstrate the usefulness of the algorithm by comparing it against standard baselines. We obtain a speedup of 1.2x to 19.5x on 24 cores with an average speed up of 10.3x across all the real world datasets.

Sao, Piyush↗
NASA NTRS1992

On the use of stochastic process-based methods for the analysis of hyperspectral data

Further development in remote sensing technology requires refinement of information system design aspects, i.e., the ability to specify precisely the data to collect and the means to extract increasing amounts of information from the increasingly rich and complex data stream created. One of the principal directions of advance is that data from much larger numbers of spectral bands can be collected, but with significantly increased signal-to-noise ratio. The theory of stochastic or random processes may be applied to the modeling of second-order variations. A multispectral data set with a large number of spectral bands is analyzed using standard pattern recognition techniques. The data were classified using first a single spectral feature, then two, and continuing on with greater and greater numbers of features. Three different classification schemes are used: a standard maximum likelihood Gaussian scheme; the same approach with the mean values of all classes adjusted to be the same; and the use of a minimum distance to means scheme such that mean differences are used.

Landgrebe, David A.↗
DOE OSTI2025

Long‐Term Large‐Scale Atmospheric Forcing Data From Three‐Dimensional Constrained Variational Analysis for the ARM SGP Site

Here, this study presents a long‐term three‐dimensional large‐scale forcing data set (VARANAL3D) derived from the three‐dimensional constrained variational analysis (3DCVA) method at the Atmospheric Radiation Measurement (ARM) program Southern Great Plains (SGP) site from 2004 to 2018. Building on the same input data sets as the conventional continuous forcing data set (VARANAL), VARANAL3D maintains overall consistency in domain‐averaged fields while introducing spatial variability, offering critical insights into the influence of mesoscale synoptic systems on cloud‐related processes. Evaluations are conducted across four cloud and precipitation regimes: Clear‐sky, Shallow‐clouds, Afternoon‐precipitation, and Nocturnal‐precipitation, presenting high consistency of the domain‐mean forcing data sets while emphasizing the role of subdomain forcing variability particularly in precipitating regimes. Single column model (SCM) simulations demonstrate that subdomain VARANAL3D forcing improves cloud and precipitation representation, with the ensemble outperforming domain‐mean forcing in three cloudy and precipitating regimes. Overall, these results highlight VARANAL3D's value for investigating the impacts of spatial variability of large‐scale forcing on atmospheric processes. The VARANAL3D data set provides new opportunities for evaluating model physics, advancing the development of scale‐aware parameterizations and deepening our understanding of cloud and precipitation dynamics.

Environmental sciences↗
DOE OSTI2022

Adapting Reinforcement Learning Treatment Policies Using Limited Data to Personalize Critical Care

We report reinforcement learning (RL) demonstrates promise for developing effective treatment policies in critical care settings. However, existing RL methods often require large and comprehensive patient data sets and do not readily lend themselves to settings in which certain patient subpopulations are severely underrepresented. In this study, we develop a new method, noisy Bayesian policy updates (NBPU), for selecting high-performing reinforcement learning–based treatment policies for underrepresented patient subpopulations using limited observations. Our method uses variational inference to learn a probability distribution over treatment policies based on a reference patient subpopulation for which sufficient data are available. It then exploits limited data from an underrepresented patient subpopulation to update this probability distribution and adapts its recommendations to this subpopulation. We demonstrate our method’s utility on a data set of ICU patients receiving intravenous blood anticoagulant medication. Our results show that NBPU outperforms state-of-the-art methods in terms of both selecting effective treatment policies for patients with nontypical clinical characteristics and predicting the corresponding policies’ performance for these patients.

60 APPLIED LIFE SCIENCES↗
NASA NTRS2005

Impact of CAMEX-4 Data Sets for Hurricane Forecasts using a Global Model

This study explores the impact on hurricane data assimilation and forecasts from the use of dropsondes and remote-sensed moisture profiles from the airborne Lidar Atmospheric Sensing Experiment (LASE) system. We show that the use of these additional data sets, above those from the conventional world weather watch, has a positive impact on hurricane predictions. The forecast tracks and intensity from the experiments show a marked improvement compared to the control experiment where such data sets were excluded. A study of the moisture budget in these hurricanes showed enhanced evaporation and precipitation over the storm area. This resulted in these data sets making a large impact on the estimate of mass convergence and moisture fluxes, which were much smaller in the control runs. Overall this study points to the importance of high vertical resolution humidity data sets for improved model results. We note that the forecast impact from the moisture profiling data sets for some of the storms is even larger than the impact from the use of dropwindsonde based winds.

Kamineni, Rupa↗
DOE OSTI2025

Randomized Algorithms for Symmetric Nonnegative Matrix Factorization

Symmetric Nonnegative Matrix Factorization (SymNMF) is a technique in data analysis and machine learning that approximates a matrix with a product of a nonnegative, low-rank matrix and it transpose. To design faster and more scalable algorithms for SymNMF we develop two randomized algorithms for its computation. The first method uses randomized matrix sketching to compute an initial low-rank approximation to the input matrix and proceeds to uses this as a low-rank input to rapidly compute a SymNMF. The second methods uses randomized leverage score sampling to approximately solve constrained least squares problems. Many successful methods for SymNMF rely on (approximately) solving sequences of constrained least squares problems. Here, we prove theoretically that leverage score sampling can approximately solve constrained least squares problems to e-accuracy. Finally we demonstrate both methods work in practice by applying them to graph clustering tasks on large real world data sets. These experiments show that our methods approximately maintain solution quality and achieve significant speed ups for both large dense and large sparse problems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Machine Learning Benchmarks for the Classification of Equivalent Circuit Models from Electrochemical Impedance Spectra

Analysis of Electrochemical Impedance Spectroscopy (EIS) data for electrochemical systems often consists of defining an Equivalent Circuit Model (ECM) using expert knowledge and then optimizing the model parameters to deconvolute various resistance, capacitive, inductive, or diffusion responses. For small data sets, this procedure can be conducted manually; however, it is not feasible to manually define a proper ECM for extensive data sets with a wide range of EIS responses. Automatic identification of an ECM would substantially accelerate the analysis of large sets of EIS data. We showcase machine learning methods to classify the ECMs of 9,300 impedance spectra provided by QuantumScape for the BatteryDEV hackathon. The best-performing approach is a gradient-boosted tree model utilizing a library to automatically generate features, followed by a random forest model using the raw spectral data. A convolutional neural network using boolean images of Nyquist representations is presented as an alternative, although it achieves a lower accuracy. We publish the data and open source the associated code. The approaches described in this article can serve as benchmarks for further studies. A key remaining challenge is the identifiability of the labels, underlined by the model performances and the comparison of misclassified spectra.

25 ENERGY STORAGE↗
DOE OSTI2020

The Power Curve Working Group's assessment of wind turbine power performance prediction methods

Wind turbine power production deviates from the reference power curve in real-world atmospheric conditions. Correctly predicting turbine power performance requires models to be validated for a wide range of wind turbines using inflow in different locations. The Share-3 exercise is the most recent intelligence-sharing exercise of the Power Curve Working Group, which aims to advance the modeling of turbine performance. The goal of the exercise is to search for modeling methods that reduce error and uncertainty in power prediction when wind shear and turbulence digress from design conditions. Herein, we analyze data from 55 wind turbine power performance tests from nine contributing organizations with statistical tests to quantify the skills of the prediction-correction methods. We assess the accuracy and precision of four proposed trial methods against the baseline method, which uses the conventional definition of a power curve with wind speed and air density at hub height. The trial methods reduce power-production prediction errors compared to the baseline method at high wind speeds, which contribute heavily to power production; however, the trial methods fail to significantly reduce prediction uncertainty in most meteorological conditions. For the meteorological conditions when a wind turbine produces less than the power its reference power curve suggests, using power deviation matrices leads to more accurate power prediction. We also determine that for more than half of the submissions, the data set has a large influence on the effectiveness of a trial method. Overall, this work affirms the value of data-sharing efforts in advancing power curve modeling and establishes the groundwork for future collaborations.

17 WIND ENERGY↗
NASA NTRS2015

Simulations of Cloud-Radiation Interaction Using Large-Scale Forcing Derived from the CINDY/DYNAMO Northern Sounding Array

The recently completed CINDY/DYNAMO field campaign observed two Madden-Julian oscillation (MJO) events in the equatorial Indian Ocean from October to December 2011. Prior work has indicated that the moist static energy anomalies in these events grew and were sustained to a significant extent by radiative feedbacks. We present here a study of radiative fluxes and clouds in a set of cloud-resolving simulations of these MJO events. The simulations are driven by the large-scale forcing data set derived from the DYNAMO northern sounding array observations, and carried out in a doubly periodic domain using the Weather Research and Forecasting (WRF) model. Simulated cloud properties and radiative fluxes are compared to those derived from the S-PolKa radar and satellite observations. To accommodate the uncertainty in simulated cloud microphysics, a number of single-moment (1M) and double-moment (2M) microphysical schemes in the WRF model are tested. The 1M schemes tend to underestimate radiative flux anomalies in the active phases of the MJO events, while the 2M schemes perform better, but can overestimate radiative flux anomalies. All the tested microphysics schemes exhibit biases in the shapes of the histograms of radiative fluxes and radar reflectivity. Histograms of radiative fluxes and brightness temperature indicate that radiative biases are not evenly distributed; the most significant bias occurs in rainy areas with OLR less than 150 W/ cu sq in the 2M schemes. Analysis of simulated radar reflectivities indicates that this radiative flux uncertainty is closely related to the simulated stratiform cloud coverage. Single-moment schemes underestimate stratiform cloudiness by a factor of 2, whereas 2M schemes simulate much more stratiform cloud.

Indian Ocean↗
NASA NTRS1988

Error dynamics and relative orbital uncertainty of the Magellan spacecraft

Relative error is discussed in light of a proposed orbiter mission to Venus. It is hypothesized that the relative (position) error of the orbiting spacecraft between disjoint data sets will be large because of residual uncertainties of the absolute estimates and their decorrelation caused by succeeding estimates. A method is proposed to limit relative error by inducing correlations between the estimates on successive data arcs. These correlations are induced by overlapping the data arcs. Covariance analyses of the data sets are undertaken at two distinct mapping times, yielding the relative error between those mapping times. The technique is applied to a mosaicking problem on a specific day in the mission.

Haw, R. J.↗
DOE OSTI2022

Standardizing Platinum Dainotti-correlated gamma-ray bursts, and using them with standardized Amati-correlated gamma-ray bursts to constrain cosmological model parameters

ABSTRACT We show that the Platinum gamma-ray burst (GRB) data compilation, probing the redshift range 0.553 ≤ z ≤ 5.0, obeys a cosmological-model-independent three-parameter Fundamental Plane (Dainotti) correlation and so is standardizable. While they probe the largely unexplored z ∼ 2.3–5 part of cosmological redshift space, the GRB cosmological parameter constraints are consistent with, but less precise than, those from a combination of baryon acoustic oscillation (BAO) and Hubble parameter [H(z)] data. In order to increase the precision of GRB-only cosmological constraints, we exclude common GRBs from the larger Amati-correlated A118 data set composed of 118 GRBs and jointly analyse the remaining 101 Amati-correlated GRBs with the 50 Platinum GRBs. This joint 151 GRB data set probes the largely unexplored z ∼ 2.3–8.2 region; the resulting GRB-only cosmological constraints are more restrictive, and consistent with, but less precise than, those from H(z) + BAO data.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS2001

A strategy for compression and analysis of massive geophysical data sets

This paper describes a method for summaraizing data in a way that approximately preserves high-resolution data structure while reducing data volume and maintaining global integrity of very large, remote sensing data sets. The method is under development for one of Terra's instruments, the Multi-angle Imaging SpectroRadiometer (MISR).

geophysical data sets compression↗
DOE OSTI2021

Rotational Equivariance for Object Classification Using xView

With the recent addition of large, curated and labeled data sets to the remote sensing discipline, deep learning models have largely surpassed the performance of classical techniques. These deep models, typically Convolutional Neural Networks, are invariant to translation through the use of successive convolution layers which are themselves equivariant to translation. Further, the combination of multiple convolution and pooling layers means that in practice, the model is also approximately invariant to translation. However, until recently these models could only approach rotational invariance through data augmentation. Here we propose using a new model formulation which achieves rotational equaivariance without data augmentation for overhead imagery classification. We utilize the popular xView data set to compare the rotational equivariance formalization against a regular CNN and CNN with rotational data augmentation for the task of image classification.

Bynum, Lucius EJ↗
DOE OSTI2024

Coincidence anomaly detection for unsupervised locating of edge localized modes in the DIII-D tokamak dataset

Using supervised learning to train a machine learning model to predict an on-coming edge localized mode (ELM) requires a large number of labeled samples. Creating an appropriate data set from the very large database of discharges at a long-running tokamak, such as DIII-D, would be a very time-consuming process for a human. Considering this need and difficulty, we use coincidence anomaly detection, an unsupervised learning technique, to train an ELM-identifier to identify and label ELMs in the DIII-D discharge database. This ELM-identifier shows, simultaneously, a precision of 0.68 and a recall of 0.63 (AUC is 0.73) on identifying ELMs in example time series pulled from thousands of discharges spanning five years. In a test set of 50 discharges, the algorithm finds over 26 thousand ELM candidates, more than 5 times the existing catalog of ELMs labeled by humans.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Light-Duty Vehicle Trip Classification Using One-Class Novelty Detection and Exhaustive Feature Extraction

Travel mode classification within travel survey data sets, especially light-duty vehicle (LDV) trips, is foundational, though nontrivial, to emerging mobility systems, travel behavior analysis, and fuel consumption estimation. Current travel mode detection approaches require well-sampled and balanced data sets with ground truth travel mode labels. The detection approaches are rarely applied and validated on large-scale, real-world data sets, which may not satisfy the dataset requirements. This work proposes an LDV trip detection model as a supplement to current travel mode detection methods, for the case when the training set is highly (and/or completely) unbalanced, to the extent that classical machine-learning approaches become difficult or impossible to deploy. The proposed model uses a novelty detection technique - one-class support vector machines (OCSVMs) - and a novel exhaustive feature extraction (EFE) technique on continuous time series data (i.e., Global Positioning System [GPS] speed profiles) for single-mode trip trajectories. Training and validation of the model are conducted on a large-scale, real-world data set. The proposed method accurately identifies LDV trips from a broad set of multimodal trips by leveraging a wealth of preexisting in-vehicle GPS travel data. Additional sensitivity analysis sheds light on the optimal training size and feature selection, which will benefit applications limited by highly imbalanced data. The paper also discusses performance comparison with regular machine-learning approaches, the model's robustness, and the potential to extend the proposed model to multi-modal trip prediction.

33 ADVANCED PROPULSION SYSTEMS↗
NASA NTRS2006

Status and Plans for the WCRP/GEWEX Global Precipitation Climatology Project (GPCP)

Status and plans for GPCP are presented along with scientific findings from the current data set. Global and large regional rainfall variations and possible long-term changes are examined using the 26-year (1979-2004) monthly dataset from the Global Precipitation Climatology Project (GPCP). One emphasis is to discriminate among the variations due to ENSO, volcanic events and possible long-term changes. Although the global change of precipitation in the data set is near zero, the data set does indicate an upward trend (0.13 mm/day/25yr) and a downward trend (-0.06 mm/day/25yr) over tropical oceans and lands (25S-25N), respectively. This corresponds to a 4% increase (ocean) and 2% decrease (land) during this time period. Simple techniques are derived to attempt to eliminate variations due to ENSO and major volcanic eruptions in the Tropics. Using only annual values two "volcano years" are determined by examining ocean-land coupled variations in precipitation related to ENSO and other phenomena. The outlier years coincide with Pinatubo and El Chicon eruptions. The ENSO signal is reduced by deriving mean ocean and land values for El Nino, La Nina and neutral conditions based on Nino 3.4 SST and normalizing the annual ocean and land precipitation to the neutral set of cases. The impact of the two major volcanic eruptions over the past 25 years is estimated to be about a 5% reduction in tropical rainfall. The modified data set (with ENSO and volcano effect at least partially removed) retains the same approximate linear change slopes over the data set period, but with reduced variance leading to significance tests with results in the 90-95% range. Intercomparisons between the GPCP, SSM/I (1988-2004), and TRMM (1998-2004) satellite rainfall products and alternate gauge analyses over land are made to attempt to increase or decrease confidence in the changes seen in the GPCP analysis.

Adkerm Robert F.↗
DOE OSTI2025

Unique & challenging aspects of plutonium metal standards exchange program for actinide measurements

The Los Alamos National Laboratory exchange program is the only program of its kind for the distribution of plutonium (Pu) standards materials with a range of impurity contents to multiple laboratories for destructive measurements of elemental concentration. This paper discusses statistical methods used to address challenges in Pu metal exchange data by way of two case studies. Challenges include how to evaluate a data set when a large fraction of the values are minimum detection limits (MDLs), and how to determine potential outliers with limited in-formation on the true spread of the data.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗