Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “large data sets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Coincidence anomaly detection for unsupervised locating of edge localized modes in the DIII-D tokamak dataset

Using supervised learning to train a machine learning model to predict an on-coming edge localized mode (ELM) requires a large number of labeled samples. Creating an appropriate data set from the very large database of discharges at a long-running tokamak, such as DIII-D, would be a very time-consuming process for a human. Considering this need and difficulty, we use coincidence anomaly detection, an unsupervised learning technique, to train an ELM-identifier to identify and label ELMs in the DIII-D discharge database. This ELM-identifier shows, simultaneously, a precision of 0.68 and a recall of 0.63 (AUC is 0.73) on identifying ELMs in example time series pulled from thousands of discharges spanning five years. In a test set of 50 discharges, the algorithm finds over 26 thousand ELM candidates, more than 5 times the existing catalog of ELMs labeled by humans.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Light-Duty Vehicle Trip Classification Using One-Class Novelty Detection and Exhaustive Feature Extraction

Travel mode classification within travel survey data sets, especially light-duty vehicle (LDV) trips, is foundational, though nontrivial, to emerging mobility systems, travel behavior analysis, and fuel consumption estimation. Current travel mode detection approaches require well-sampled and balanced data sets with ground truth travel mode labels. The detection approaches are rarely applied and validated on large-scale, real-world data sets, which may not satisfy the dataset requirements. This work proposes an LDV trip detection model as a supplement to current travel mode detection methods, for the case when the training set is highly (and/or completely) unbalanced, to the extent that classical machine-learning approaches become difficult or impossible to deploy. The proposed model uses a novelty detection technique - one-class support vector machines (OCSVMs) - and a novel exhaustive feature extraction (EFE) technique on continuous time series data (i.e., Global Positioning System [GPS] speed profiles) for single-mode trip trajectories. Training and validation of the model are conducted on a large-scale, real-world data set. The proposed method accurately identifies LDV trips from a broad set of multimodal trips by leveraging a wealth of preexisting in-vehicle GPS travel data. Additional sensitivity analysis sheds light on the optimal training size and feature selection, which will benefit applications limited by highly imbalanced data. The paper also discusses performance comparison with regular machine-learning approaches, the model's robustness, and the potential to extend the proposed model to multi-modal trip prediction.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2025

Unique & challenging aspects of plutonium metal standards exchange program for actinide measurements

The Los Alamos National Laboratory exchange program is the only program of its kind for the distribution of plutonium (Pu) standards materials with a range of impurity contents to multiple laboratories for destructive measurements of elemental concentration. This paper discusses statistical methods used to address challenges in Pu metal exchange data by way of two case studies. Challenges include how to evaluate a data set when a large fraction of the values are minimum detection limits (MDLs), and how to determine potential outliers with limited in-formation on the true spread of the data.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

Automated and High-Throughput Phase Separation Control for Supramolecular Polymer Blends Enabled by Machine Learning

Supramolecular polymer blends (SPBs) offer tunable morphologies that dictate their macroscopic properties, yet their rational design is limited by the absence of predictive structure−morphology models. Here, we introduce a data-driven highthroughput workflow that integrates modular polymer synthesis, robotic formulation, automated morphology characterization, and machine learning (ML) for accelerated SPB discovery. Using a plug-and-play synthetic strategy, 33 hydrogen-bonding endfunctional homopolymers were prepared and orthogonally combined to generate 260 SPBs in 1 day. A fully automated atomic force microscopy (AFM) pipeline enabled systematic imaging, producing 2340 morphology data sets with minimal human intervention. Domain spacings were extracted through complementary imageprocessing methods and used to train ML models. A support vector regression (SVR) model accurately predicted target phase-separation sizes (50, 100, and 150 nm), which were experimentally validated. This work demonstrates the power of coupling high-throughput experimentation with ML to accelerate morphology discovery and provides one of the first large-scale experimental data sets for supramolecular polymer systems.

ML-guided polymer design↗
DOE OSTI2024

Development of the ARM Lagrangian Large-Scale Forcing Data (ARMLAGTRAJ) Value-Added Product Based on the lagtraj Framework

The Atmospheric Radiation Measurement (ARM) large-scale forcing data developed based on the constrained variational analysis (VARANAL) value-added product (VAP) (Zhang and Lin 1997, Zhang et al. 2001, Xie et al. 2004, Tang et al. 2019) has been widely used for single-column models (SCMs), cloud-resolving models (CRMs), and large-eddy simulation models (LESs) to understand and improve physical processes in models. Recently, the U.S. Department of Energy (DOE) ARM user facility conducted several major field campaigns using ship-based moving observational platforms. For example, the Marine ARM GPCI Investigation of Clouds (MAGIC) field campaign focused on the role of subtropical marine-boundary layer (MBL) clouds, and the Multidisciplinary Drifting Observatory for the Study of Arctic Climate (MOSAiC) field campaign aimed to improve understanding of the coupled climate systems in the Arctic. Observations from moving platforms are critical to provide a comprehensive characterization of coupled-system processes associated with all stages of the cloud and/or sea-ice life cycle. Traditional ARM large-scale forcing data have been developed at fixed locations. They need to be extended to include these moving platforms to address data needs for ship-based field campaigns or to support LES modeling in a Lagrangian framework. With these considerations in mind, we develop ARM-type Lagrangian large-scale forcing data sets based on the lagtraj framework (Boeing et al. 2020) with notable enhancements in generating forcings that are more suitable for ARM field campaigns. The lagtraj is a novel tool that generates forcings for LES and SCM simulation in both Lagrangian and Eulerian perspective. This technical report focuses on the major changes we performed on the lagtraj algorithm and provides an overview of the ARM Lagrangian Large-Scale Forcing Data (ARMLAGTRAJ) value-added products.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

PSY (PowerSystems.jl) [SWR-23-105]

PowerSystems.jl is a package to organize and manipulate data for the study of energy systems with diverse modeling requirements. This software serves two main purposes: to reduce the burden of large power system data set development, and to promote reproducible research and simulation. PowerSystems.jl implements an abstract hierarchy to represent and customize power systems data and includes data containers for quasi-static and dynamic simulation applications. Key features include efficient management of large quantities of time series data, optimized serialization, and comprehensive validation capabilities. For further information regarding the Sienna modeling framework at NREL, see: https://www.nrel.gov/analysis/sienna.html

Lara Aguilar, Jose Daniel↗
DOE OSTI2023

Data Analytics for Residential PV from Permit to Interconnect (Final Technical Report)

The main objective of this research is to provide novel insights into the effects of permitting, inspection, and interconnection (PII) processes on PV system installations—and in particular, into the relationship between PII processes and adoption timelines. This research can then be used to clarify the potential effect of various process changes on reducing PII timelines, customer cancellation rates, and related costs nationwide. NREL completed this research by assembling a data set of distributed, largely residential rooftop solar systems less than 50 kilowatts in size from participating solar installers. NREL produced five publications describing the effects that PII processes can have on adoption timelines nationwide, in addition to publishing an interactive data viewer with five years of PII cycle time data. This tool can be used by stakeholders to identify potential adoption timelines by local government.

14 SOLAR ENERGY↗
DOE OSTI2022

On the Statistics of Elsasser Increments in Solar Wind and Magnetohydrodynamic Turbulence

In this Letter we investigate the dependency with scale of the empirical probability distribution functions (PDF) of Elsasser increments using large sets of WIND data (collected between 1995 and 2017) near 1 au. The empirical PDF are compared to the ones obtained from high-resolution numerical simulations of steadily driven, homogeneous reduced MHD turbulence on a 2048 3 rectangular mesh. A large statistical sample of Alfvénic increments is obtained by using conditional analysis based on the solar wind average properties. The PDF tails obtained from observations and numerical simulations are found to have exponential behavior in the inertial range, with an exponential decrement that satisfies power laws of the form α l ∝ l –μ , where l is the scale size, with μ between 0.17 and 0.25 for observations and 0.43 for simulations. PDF tails were extrapolated assuming their exponential behavior extends to arbitrarily large increments in order to determine structure function scaling laws at very high orders. Our results point to potentially universal scaling laws governing the PDF of Elsasser increments and to an alternative approach to investigate high-order statistics in solar wind observations.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

3D Continuous Forcing Dataset from 3D Constrained Variational Analysis at SGP

The continuous 3D large-scale forcing (VARANAL3D) data set derived from 3D constrained variational analysis (3DCVA) extends the conventional constrained variational analysis method by incorporating multiple sub-columns within the analysis domain. This advancement introduces spatial variability into the large-scale forcing fields, thereby enriching the data set’s applicability. The VARANAL3D data set spans from 2004 to 2018 and covers a region of 5˚×4.5˚ domain around the ARM SGP site. The analysis domain is divided into 10×9 sub-columns with 0.5˚ resolution. The 3D large-scale forcing data provides necessary variables to drive and evaluate single-column models (SCM), cloud-resolving models (CRM) ,and large-eddy simulations (LES), as well as information for testing model sensitivity to spatial variability of the large-scale forcing data, facilitating more rigorous testing and refinement of physical processes in SCM/CRM/LES.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Serial crystallography with multi-stage merging of thousands of images

KAMO and BLEND provide particularly effective tools to automatically manage the merging of large numbers of data sets from serial crystallography. The requirement for manual intervention in the process can be reduced by extending BLEND to support additional clustering options such as the use of more accurate cell distance metrics and the use of reflection-intensity correlation coefficients to infer `distances' among sets of reflections. This increases the sensitivity to differences in unit-cell parameters and allows clustering to assemble nearly complete data sets on the basis of intensity or amplitude differences. If the data sets are already sufficiently complete to permit it, one applies KAMO once and clusters the data using intensities only. When starting from incomplete data sets, one applies KAMO twice, first using unit-cell parameters. In this step, either the simple cell vector distance of the original BLEND or the more sensitive NCDist is used. This step tends to find clusters of sufficient size such that, when merged, each cluster is sufficiently complete to allow reflection intensities or amplitudes to be compared. One then uses KAMO again using the correlation between reflections with a common hkl to merge clusters in a way that is sensitive to structural differences that may not have perturbed the unit-cell parameters sufficiently to make meaningful clusters. Many groups have developed effective clustering algorithms that use a measurable physical parameter from each diffraction still or wedge to cluster the data into categories which then can be merged, one hopes, to yield the electron density from a single protein form. Since these physical parameters are often largely independent of one another, it should be possible to greatly improve the efficacy of data-clustering software by using a multi-stage partitioning strategy. Here, one possible approach to multi-stage data clustering is demonstrated. The strategy is to use unit-cell clustering until the merged data are sufficiently complete and then to use intensity-based clustering. Using this strategy, it is demonstrated that it is possible to accurately cluster data sets from crystals that have subtle differences.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Tutorial: Lessons Learned for Behavior Analysts from Data Scientists

Big data is a computing term used to refer to large and complex data sets, typically consisting of terabytes or more of diverse data that is produced rapidly. The analysis of such complex data sets requires advanced analysis techniques with the capacity to identify patterns and abstract meanings from the vast data. The field of data science combines computer science with mathematics/statistics and leverages artificial intelligence, in particular machine learning, to analyze big data. This field holds great promise for behavior analysis, where both clinical and research studies produce large volumes of diverse data at a rapid pace (i.e., big data). This article presents basic lessons for the behavior analytic researchers and clinicians regarding integration of data science into the field of behavior analysis. We provide guidance on how to collect, protect, and process the data, while highlighting the importance of collaborating with data scientists to select a proper machine learning model that aligns with the project goals and develop models with input from human experts. Here, we hope this serves as a guide to support the behavior analysts interested in the field of data science to advance their practice or research, and helps them avoid some common pitfalls.

42 ENGINEERING↗
DOE OSTI2022

Accelerating cosmological inference with Gaussian processes and neural networks – an application to LSST Y1 weak lensing and galaxy clustering

ABSTRACT Studying the impact of systematic effects, optimizing survey strategies, assessing tensions between different probes and exploring synergies of different data sets require a large number of simulated likelihood analyses, each of which cost thousands of CPU hours. In this paper, we present a method to accelerate cosmological inference using emulators based on Gaussian process regression and neural networks. We iteratively acquire training samples in regions of high posterior probability which enables accurate emulation of data vectors even in high dimensional parameter spaces. We showcase the performance of our emulator with a simulated 3×2 point analysis of LSST-Y1 with realistic theoretical and systematics modelling. We show that our emulator leads to high-fidelity posterior contours, with an order of magnitude speed-up. Most importantly, the trained emulator can be re-used for extremely fast impact and optimization studies. We demonstrate this feature by studying baryonic physics effects in LSST-Y1 3×2 point analyses where each one of our MCMC runs takes approximately 5 min. This technique enables future cosmological analyses to map out the science return as a function of analysis choices and survey strategy.

Astronomy & Astrophysics↗
DOE OSTI2023

Three-dimensional nanoscale reduced-angle ptycho-tomographic imaging with deep learning (RAPID)

X-ray ptychographic tomography is a nondestructive method for three dimensional (3D) imaging with nanometer-sized resolvable features. The size of the volume that can be imaged is almost arbitrary, limited only by the penetration depth and the available scanning time. Here we present a method that rapidly accelerates the imaging operation over a given volume through acquiring a limited set of data via large angular reduction and compensating for the resulting ill-posedness through deeply learned priors. The proposed 3D reconstruction method “RAPID” relies initially on a subset of the object measured with the nominal number of required illumination angles and treats the reconstructions from the conventional two-step approach as ground truth. It is then trained to reproduce equal fidelity from much fewer angles. After training, it performs with similar fidelity on the hitherto unexamined portions of the object, previously not shown during training, with a limited set of acquisitions. In our experimental demonstration, the nominal number of angles was 349 and the reduced number of angles was 21, resulting in a x140 aggregate speedup over a volume of 4.48 x 93.18 x 3.92 μm 3 and with (14nm) 3 feature size, i.e. ~ 10 8 voxels. RAPID’s key distinguishing feature over earlier attempts is the incorporation of atrous spatial pyramid pooling modules into the deep neural network framework in an anisotropic way. We found that adjusting the atrous rate improves reconstruction fidelity because it expands the convolutional kernels’ range to match the physics of multi-slice ptychography without significantly increasing the number of parameters.

47 OTHER INSTRUMENTATION↗
DOE OSTI2020

Leveraging Real-World Driving Data for Design and Impact Evaluation of Energy Efficient Control Strategies

Modeling and simulation are crucial in the development of advanced energy efficient control strategies. Utilizing real-world driving data as the underlying basis for control design and simulation lends veracity to projected real-world energy savings. Standardized drive cycles are limited in their utility for evaluating advanced driving strategies that utilize connectivity and on-vehicle sensing, primarily because they are typically intended for evaluating emissions and fuel economy under controlled conditions. Real-world driving data, because of its scale, is a useful representation of various road types, driving styles, and driving environments. The scale of real-world data also presents challenges in effectively using it in simulations. A fast and efficient simulation methodology is necessary to handle the large number of simulations performed for design analysis and impact evaluation of control strategies. In this study, two methods are presented of leveraging real-world data in both design optimization of energy efficient control strategies and in evaluating the real-world impact of those control strategies upon large-scale deployment. Through these methodologies, strategies with highest impact on energy savings were selected to be implemented as control algorithms. The developed algorithms were incorporated into a vehicle dynamics and powertrain control architecture implemented on a Cadillac CT6 demonstration vehicle. The control algorithms were then exercised on real-world driving scenarios to determine their impact on collective energy savings. The methodology utilizes the large-scale driving data sets maintained by the National Renewable Energy Laboratory to extract real-world driving scenarios and efficient simulation software tools. The insights obtained through this research help in guiding technology selection for energy efficient driving controls.

ARPA-E↗
DOE OSTI2024

Comparison of Global Aboveground Biomass Estimates From Satellite Observations and Dynamic Global Vegetation Models

The global forest carbon stocks represent the amount of carbon stored in woody vegetation and are important for quantifying the ability of the global forests to sequester atmospheric CO 2 and to provide ecosystem services (e.g., timber) under climate change. The forest ecosystem carbon pool estimates are highly variable and poorly quantified in areas lacking forest inventory estimates. Here, we compare and analyze aboveground biomass (AGB) estimates from five satellite-based global data sets and nine dynamic global vegetation models (DVGMs). We find that across the data sets, mean AGB exhibits the largest variability around the tropical area. In addition, AGB shows a similar latitudinal trend but large variability among the data sets. Satellite-based AGB estimates are lower than those simulated by DVGMs. The divergence among the satellite-based AGB estimates can be driven by the methodology, input satellite products, and the forested areas used to estimate AGB. The modeled NPP, autotrophic respiration, and carbon allocation mostly drive the variability of AGB simulated by DGVMs. The future availability of a high-quality global forest area map is anticipated to improve AGB estimate accuracy and to reduce the discrepancies among different satellite- and model-based AGB estimates. Furthermore, we suggest the carbon-modeling community reexamine the methodology used to estimate AGB and forested areas for a more robust global forest carbon stock estimation.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

The Open Cluster Chemical Abundances and Mapping Survey. VI. Galactic Chemical Gradient Analysis from APOGEE DR17

The goal of the Open Cluster Chemical Abundances and Mapping (OCCAM) survey is to constrain key Galactic dynamic and chemical evolution parameters by the construction and analysis of a large, comprehensive, uniform data set of infrared spectra for stars in hundreds of open clusters. This sixth contribution from the OCCAM survey presents analysis of SDSS/APOGEE Data Release 17 (DR17) results for a sample of stars in 150 open clusters, 94 of which we designate to be "high-quality" based on the appearance of their color–magnitude diagram. We find the APOGEE DR17-derived [Fe/H] values to be in good agreement with those from previous high-resolution spectroscopic open cluster abundance studies. Using a subset of the high-quality sample, the Galactic abundance gradients were measured for 16 chemical elements, including [Fe/H], for both Galactocentric radius ($R$ GC ) and guiding center radius ($R$ guide ). We find an overall Galactic [Fe/H] versus RGC gradient of -0.073 ± 0.002 dex kpc -1 over the range of 6 > $R$ GC < 11.5 kpc, and a similar gradient is found for [Fe/H] versus $R$ guide . Significant Galactic abundance gradients are also noted for O, Mg, S, Ca, Mn, Na, Al, K, and Ce. Our large sample additionally allows us to explore the evolution of the gradients in four age bins for the remaining 15 elements.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Physical, socio-psychological, and behavioural determinants of household energy consumption in the UK

Determining which attitudes and behaviours predict household energy consumption can help accelerate the low-carbon energy transition. Conventional approaches in this domain are limited, often relying on survey methods that produce data on individuals’ motivations and self-reported activities without pairing these with actual energy consumption records, which are particularly hard to collect for large, nationally representative samples. This challenge precludes the development of empirical evidence on which attitudes and behaviours influence patterns of energy consumption, thus limiting the extent to which these can inform energy interventions or conservation programs. This study demonstrates a novel methodology for estimating energy consumption in the absence of actual energy records by using a large, publicly available data set of energy consumption in the UK. We develop a predictive model using the Smart Energy Research Laboratory (SERL) data portal (with records from nearly 13,000 UK households) and then use this model to predict energy consumption (both electric and gas) for a sample of 1,000 UK householders for which we separately collect over 200 variables relating to climate change attitudes and practices. Our approach uses a set of over 50 independent variables that are shared between the data sets, allowing us to train a model on the SERL data and use it to analyse the relationship between energy consumption and the opinions, motivations, and daily practices of survey respondents. Results show that electricity consumption is influenced by a broader range of factors compared to gas. Household energy use is best explained by physical dwelling characteristics, socio-demographic variables, and certain behavioural and attitudinal measures. Notably, pro-environmental attitudes, frugality, and conscientiousness correlate with lower energy use, while income and consumerism are linked to higher consumption. We discuss how these findings can inform efforts to decarbonise home energy use in the UK.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2020

Modeling occupancy-driven building loads for large and diversified building stocks through the use of parametric schedules

Building energy modeling provides a fundamental tool to assess the potential for energy efficiency to contribute to reducing world energy consumption and global emissions. Occupancy-related operations are a key source of uncertainty for building energy analysis, particularly for aggregated building stocks. At a district or city level, it is critical to estimate aggregated power load profiles for sizing power grid infrastructure, power plant capacity allocation, and energy efficiency measures. The stochastic nature of behavior-related operations complicates the creation of models that accurately capture building load profiles for entire building stocks. This research introduces a new methodology called parametric schedules to model occupancy-driven schedules for large and diverse building stocks. In contrast to computationally expensive methodologies proposed in the literature, our work does not use a recursive time-consuming step. Occupancy is estimated by the extrapolation of operation times directly from metered electric consumption data; occupancy-related schedules are stochastically assigned to each building model, guaranteeing diversity of operation times in the stock. Our procedure has been tested on a large, diversified data-set of 25,000 commercial buildings in Los Angeles, California. It proved to be able to adequately represent the stochastic schedules diversity of the stock and to refine the stock calibration process by 1%. This innovative approach represents a useful asset for utility companies, grid operators, urban planners, and balancing authorities, seeking to improve building stock modeling and better estimate the impact of energy conservation measures. – This work is part of a larger stock modeling tool called ComStock, which is under development by NREL.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗