Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “large data sets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 505 records · Page 28
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Unique & challenging aspects of plutonium metal standards exchange program for actinide measurements

The Los Alamos National Laboratory exchange program is the only program of its kind for the distribution of plutonium (Pu) standards materials with a range of impurity contents to multiple laboratories for destructive measurements of elemental concentration. This paper discusses statistical methods used to address challenges in Pu metal exchange data by way of two case studies. Challenges include how to evaluate a data set when a large fraction of the values are minimum detection limits (MDLs), and how to determine potential outliers with limited in-formation on the true spread of the data.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS2013

Mapping Impervious Surfaces Globally at 30m Resolution Using Global Land Survey Data

Impervious surfaces, mainly artificial structures and roads, cover less than 1% of the world's land surface (1.3% over USA). Regardless of the relatively small coverage, impervious surfaces have a significant impact on the environment. They are the main source of the urban heat island effect, and affect not only the energy balance, but also hydrology and carbon cycling, and both land and aquatic ecosystem services. In the last several decades, the pace of converting natural land surface to impervious surfaces has increased. Quantitatively monitoring the growth of impervious surface expansion and associated urbanization has become a priority topic across both the physical and social sciences. The recent availability of consistent, global scale data sets at 30m resolution such as the Global Land Survey from the Landsat satellites provides an unprecedented opportunity to map global impervious cover and urbanization at this resolution for the first time, with unprecedented detail and accuracy. Moreover, the spatial resolution of Landsat is absolutely essential to accurately resolve urban targets such a buildings, roads and parking lots. With long term GLS data now available for the 1975, 1990, 2000, 2005 and 2010 time periods, the land cover/use changes due to urbanization can now be quantified at this spatial scale as well. In the Global Land Survey - Imperviousness Mapping Project (GLS-IMP), we are producing the first global 30 m spatial resolution impervious cover data set. We have processed the GLS 2010 data set to surface reflectance (8500+ TM and ETM+ scenes) and are using a supervised classification method using a regression tree to produce continental scale impervious cover data sets. A very large set of accurate training samples is the key to the supervised classifications and is being derived through the interpretation of high spatial resolution (approx. 2 m or less) commercial satellite data (Quickbird and Worldview2) available to us through the unclassified archive of the National Geospatial Intelligence Agency (NGA). For each continental area several million training pixels are derived by analysts using image segmentation algorithms and tools and then aggregated to the 30m resolution of Landsat. Here we will discuss the production/testing of this massive data set for Europe, North and South America and Africa, including assessments of the 2010 surface reflectance data. This type of analysis is only possible because of the availability of long term 30m data sets from GLS and shows much promise for integration of Landsat 8 data in the future.

global land survey↗
DOE OSTI2026

Automated and High-Throughput Phase Separation Control for Supramolecular Polymer Blends Enabled by Machine Learning

Supramolecular polymer blends (SPBs) offer tunable morphologies that dictate their macroscopic properties, yet their rational design is limited by the absence of predictive structure−morphology models. Here, we introduce a data-driven highthroughput workflow that integrates modular polymer synthesis, robotic formulation, automated morphology characterization, and machine learning (ML) for accelerated SPB discovery. Using a plug-and-play synthetic strategy, 33 hydrogen-bonding endfunctional homopolymers were prepared and orthogonally combined to generate 260 SPBs in 1 day. A fully automated atomic force microscopy (AFM) pipeline enabled systematic imaging, producing 2340 morphology data sets with minimal human intervention. Domain spacings were extracted through complementary imageprocessing methods and used to train ML models. A support vector regression (SVR) model accurately predicted target phase-separation sizes (50, 100, and 150 nm), which were experimentally validated. This work demonstrates the power of coupling high-throughput experimentation with ML to accelerate morphology discovery and provides one of the first large-scale experimental data sets for supramolecular polymer systems.

ML-guided polymer design↗
NASA NTRS1985

Dynamics explorer guest investigator

The use of Dynamics Explorer (DE) data sets to model the auroral inputs for the time dependent ionospheric model (TDIM) is reported. The modelling requires DE-1 SAI images and simultaneous DE-2 LAPI particle data. The data sets allow the large scale relative auroral variations and local absolute energy flexes and characteristics energies to be defined. The images enabled global scale auroral modelling with 12 min. time resolution and the LAPI data presented a detailed energy flux and characteristic energy calibration of the image model. The auroral model is used as an input to the TDIM and studies ionospheric storms.

Sojka, J. J.↗
DOE OSTI2024

Development of the ARM Lagrangian Large-Scale Forcing Data (ARMLAGTRAJ) Value-Added Product Based on the lagtraj Framework

The Atmospheric Radiation Measurement (ARM) large-scale forcing data developed based on the constrained variational analysis (VARANAL) value-added product (VAP) (Zhang and Lin 1997, Zhang et al. 2001, Xie et al. 2004, Tang et al. 2019) has been widely used for single-column models (SCMs), cloud-resolving models (CRMs), and large-eddy simulation models (LESs) to understand and improve physical processes in models. Recently, the U.S. Department of Energy (DOE) ARM user facility conducted several major field campaigns using ship-based moving observational platforms. For example, the Marine ARM GPCI Investigation of Clouds (MAGIC) field campaign focused on the role of subtropical marine-boundary layer (MBL) clouds, and the Multidisciplinary Drifting Observatory for the Study of Arctic Climate (MOSAiC) field campaign aimed to improve understanding of the coupled climate systems in the Arctic. Observations from moving platforms are critical to provide a comprehensive characterization of coupled-system processes associated with all stages of the cloud and/or sea-ice life cycle. Traditional ARM large-scale forcing data have been developed at fixed locations. They need to be extended to include these moving platforms to address data needs for ship-based field campaigns or to support LES modeling in a Lagrangian framework. With these considerations in mind, we develop ARM-type Lagrangian large-scale forcing data sets based on the lagtraj framework (Boeing et al. 2020) with notable enhancements in generating forcings that are more suitable for ARM field campaigns. The lagtraj is a novel tool that generates forcings for LES and SCM simulation in both Lagrangian and Eulerian perspective. This technical report focuses on the major changes we performed on the lagtraj algorithm and provides an overview of the ARM Lagrangian Large-Scale Forcing Data (ARMLAGTRAJ) value-added products.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1980

Water-ice and water-updraft relationships near -10 C within populations of Florida cumuli

Evidence is presented for a sequential development of cloud water, rainwater, graupel, and crystalline ice with the aging of the cloud. This evidence is based on in-cloud microphysical data set collected in Florida convective towers that were penetrated close to their tops near -10 C; the very rapid onset of graupel that appears on repeat penetrations of some towers is of particular interest. A separate data set shows a large scatter in the relationship between the maximum value of cloud water and vertical velocity which indicates that measurements of cloud water can be misleading as an indication of growth activity. The sequential pass data showing the evolution of ice and water are consistent with a rime-splintering, secondary ice production hypothesis.

Sax, R. I.↗
DOE OSTI2023

PSY (PowerSystems.jl) [SWR-23-105]

PowerSystems.jl is a package to organize and manipulate data for the study of energy systems with diverse modeling requirements. This software serves two main purposes: to reduce the burden of large power system data set development, and to promote reproducible research and simulation. PowerSystems.jl implements an abstract hierarchy to represent and customize power systems data and includes data containers for quasi-static and dynamic simulation applications. Key features include efficient management of large quantities of time series data, optimized serialization, and comprehensive validation capabilities. For further information regarding the Sienna modeling framework at NREL, see: https://www.nrel.gov/analysis/sienna.html

Lara Aguilar, Jose Daniel↗
DOE OSTI2023

Data Analytics for Residential PV from Permit to Interconnect (Final Technical Report)

The main objective of this research is to provide novel insights into the effects of permitting, inspection, and interconnection (PII) processes on PV system installations—and in particular, into the relationship between PII processes and adoption timelines. This research can then be used to clarify the potential effect of various process changes on reducing PII timelines, customer cancellation rates, and related costs nationwide. NREL completed this research by assembling a data set of distributed, largely residential rooftop solar systems less than 50 kilowatts in size from participating solar installers. NREL produced five publications describing the effects that PII processes can have on adoption timelines nationwide, in addition to publishing an interactive data viewer with five years of PII cycle time data. This tool can be used by stakeholders to identify potential adoption timelines by local government.

14 SOLAR ENERGY↗
DOE OSTI2022

On the Statistics of Elsasser Increments in Solar Wind and Magnetohydrodynamic Turbulence

In this Letter we investigate the dependency with scale of the empirical probability distribution functions (PDF) of Elsasser increments using large sets of WIND data (collected between 1995 and 2017) near 1 au. The empirical PDF are compared to the ones obtained from high-resolution numerical simulations of steadily driven, homogeneous reduced MHD turbulence on a 2048 3 rectangular mesh. A large statistical sample of Alfvénic increments is obtained by using conditional analysis based on the solar wind average properties. The PDF tails obtained from observations and numerical simulations are found to have exponential behavior in the inertial range, with an exponential decrement that satisfies power laws of the form α l ∝ l –μ , where l is the scale size, with μ between 0.17 and 0.25 for observations and 0.43 for simulations. PDF tails were extrapolated assuming their exponential behavior extends to arbitrarily large increments in order to determine structure function scaling laws at very high orders. Our results point to potentially universal scaling laws governing the PDF of Elsasser increments and to an alternative approach to investigate high-order statistics in solar wind observations.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS2015

RACORO Continental Boundary Layer Cloud Investigations: 1. Case Study Development and Ensemble Large-Scale Forcings

Observation-based modeling case studies of continental boundary layer clouds have been developed to study cloudy boundary layers, aerosol influences upon them, and their representation in cloud- and global-scale models. Three 60 h case study periods span the temporal evolution of cumulus, stratiform, and drizzling boundary layer cloud systems, representing mixed and transitional states rather than idealized or canonical cases. Based on in situ measurements from the Routine AAF (Atmospheric Radiation Measurement (ARM) Aerial Facility) CLOWD (Clouds with Low Optical Water Depth) Optical Radiative Observations (RACORO) field campaign and remote sensing observations, the cases are designed with a modular configuration to simplify use in large-eddy simulations (LES) and single-column models. Aircraft measurements of aerosol number size distribution are fit to lognormal functions for concise representation in models. Values of the aerosol hygroscopicity parameter, kappa, are derived from observations to be approximately 0.10, which are lower than the 0.3 typical over continents and suggestive of a large aerosol organic fraction. Ensemble large-scale forcing data sets are derived from the ARM variational analysis, European Centre for Medium-Range Weather Forecasts, and a multiscale data assimilation system. The forcings are assessed through comparison of measured bulk atmospheric and cloud properties to those computed in "trial" large-eddy simulations, where more efficient run times are enabled through modest reductions in grid resolution and domain size compared to the full-sized LES grid. Simulations capture many of the general features observed, but the state-of-the-art forcings were limited at representing details of cloud onset, and tight gradients and high-resolution transients of importance. Methods for improving the initial conditions and forcings are discussed. The cases developed are available to the general modeling community for studying continental boundary clouds.

Boundary layers↗
DOE OSTI2025

3D Continuous Forcing Dataset from 3D Constrained Variational Analysis at SGP

The continuous 3D large-scale forcing (VARANAL3D) data set derived from 3D constrained variational analysis (3DCVA) extends the conventional constrained variational analysis method by incorporating multiple sub-columns within the analysis domain. This advancement introduces spatial variability into the large-scale forcing fields, thereby enriching the data set’s applicability. The VARANAL3D data set spans from 2004 to 2018 and covers a region of 5˚×4.5˚ domain around the ARM SGP site. The analysis domain is divided into 10×9 sub-columns with 0.5˚ resolution. The 3D large-scale forcing data provides necessary variables to drive and evaluate single-column models (SCM), cloud-resolving models (CRM) ,and large-eddy simulations (LES), as well as information for testing model sensitivity to spatial variability of the large-scale forcing data, facilitating more rigorous testing and refinement of physical processes in SCM/CRM/LES.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2008

Correlations of Prompt and Afterglow Emission in Swift Long and Short Gamma Ray Bursts

Correlation studies of prompt and afterglow emissions from gamma-ray bursts (GRBs) between different spectral bands has been difficult to do in the past because few bursts had comprehensive and intercomparable afterglow measurements. In this paper we present a large and uniform data set for correlation analysis based on bursts detected by the Swift mission. For the first time, short and long bursts can be analyzed and compared. It is found for both classes that the optical, X-ray and gamma-ray emissions are linearly correlated, but with a large spread about the correlation line; stronger bursts tend to have brighter afterglows, and bursts with brighter X-ray afterglow tend to have brighter optical afterglow. Short bursts are, on average, weaker in both prompt and afterglow emissions. No short bursts are seen with extremely low optical to X-ray ratio as occurs for 'dark' long bursts. Although statistics are still poor for short bursts, there is no evidence yet for a subgroup of short bursts with high extinction as there is for long bursts. Long bursts are detected in the dark category at the same fraction as for pre-Swift bursts. Interesting cases are discovered of long bursts that are detected in the optical, and yet have low enough optical to X-ray ratio to be classified as dark. For the prompt emission, short and long bursts have different average tracks on flux vs fluence plots. In Swift, GRB detections tend to be fluence limited for short bursts and flux limited for long events.

Gehrel, Neil↗
DOE OSTI2022

Serial crystallography with multi-stage merging of thousands of images

KAMO and BLEND provide particularly effective tools to automatically manage the merging of large numbers of data sets from serial crystallography. The requirement for manual intervention in the process can be reduced by extending BLEND to support additional clustering options such as the use of more accurate cell distance metrics and the use of reflection-intensity correlation coefficients to infer `distances' among sets of reflections. This increases the sensitivity to differences in unit-cell parameters and allows clustering to assemble nearly complete data sets on the basis of intensity or amplitude differences. If the data sets are already sufficiently complete to permit it, one applies KAMO once and clusters the data using intensities only. When starting from incomplete data sets, one applies KAMO twice, first using unit-cell parameters. In this step, either the simple cell vector distance of the original BLEND or the more sensitive NCDist is used. This step tends to find clusters of sufficient size such that, when merged, each cluster is sufficiently complete to allow reflection intensities or amplitudes to be compared. One then uses KAMO again using the correlation between reflections with a common hkl to merge clusters in a way that is sensitive to structural differences that may not have perturbed the unit-cell parameters sufficiently to make meaningful clusters. Many groups have developed effective clustering algorithms that use a measurable physical parameter from each diffraction still or wedge to cluster the data into categories which then can be merged, one hopes, to yield the electron density from a single protein form. Since these physical parameters are often largely independent of one another, it should be possible to greatly improve the efficacy of data-clustering software by using a multi-stage partitioning strategy. Here, one possible approach to multi-stage data clustering is demonstrated. The strategy is to use unit-cell clustering until the merged data are sufficiently complete and then to use intensity-based clustering. Using this strategy, it is demonstrated that it is possible to accurately cluster data sets from crystals that have subtle differences.

36 MATERIALS SCIENCE↗
NASA NTRS2011

Development of Monte Carlo Capability for Orion Parachute Simulations

Parachute test programs employ Monte Carlo simulation techniques to plan testing and make critical decisions related to parachute loads, rate-of-descent, or other parameters. This paper describes the development and use of a MATLAB-based Monte Carlo tool for three parachute drop test simulations currently used by NASA. The Decelerator System Simulation (DSS) is a legacy 6 Degree-of-Freedom (DOF) simulation used to predict parachute loads and descent trajectories. The Decelerator System Simulation Application (DSSA) is a 6-DOF simulation that is well suited for modeling aircraft extraction and descent of pallet-like test vehicles. The Drop Test Vehicle Simulation (DTVSim) is a 2-DOF trajectory simulation that is convenient for quick turn-around analysis tasks. These three tools have significantly different software architectures and do not share common input files or output data structures. Separate Monte Carlo tools were initially developed for each simulation. A recently-developed simulation output structure enables the use of the more sophisticated DSSA Monte Carlo tool with any of the core-simulations. The task of configuring the inputs for the nominal simulation is left to the existing tools. Once the nominal simulation is configured, the Monte Carlo tool perturbs the input set according to dispersion rules created by the analyst. These rules define the statistical distribution and parameters to be applied to each simulation input. Individual dispersed parameters are combined to create a dispersed set of simulation inputs. The Monte Carlo tool repeatedly executes the core-simulation with the dispersed inputs and stores the results for analysis. The analyst may define conditions on one or more output parameters at which to collect data slices. The tool provides a versatile interface for reviewing output of large Monte Carlo data sets while preserving the capability for detailed examination of individual dispersed trajectories. The Monte Carlo tool described in this paper has proven useful in planning several Crew Exploration Vehicle parachute tests.

Moore, James W.↗
DOE OSTI2023

Tutorial: Lessons Learned for Behavior Analysts from Data Scientists

Big data is a computing term used to refer to large and complex data sets, typically consisting of terabytes or more of diverse data that is produced rapidly. The analysis of such complex data sets requires advanced analysis techniques with the capacity to identify patterns and abstract meanings from the vast data. The field of data science combines computer science with mathematics/statistics and leverages artificial intelligence, in particular machine learning, to analyze big data. This field holds great promise for behavior analysis, where both clinical and research studies produce large volumes of diverse data at a rapid pace (i.e., big data). This article presents basic lessons for the behavior analytic researchers and clinicians regarding integration of data science into the field of behavior analysis. We provide guidance on how to collect, protect, and process the data, while highlighting the importance of collaborating with data scientists to select a proper machine learning model that aligns with the project goals and develop models with input from human experts. Here, we hope this serves as a guide to support the behavior analysts interested in the field of data science to advance their practice or research, and helps them avoid some common pitfalls.

42 ENGINEERING↗
DOE OSTI2022

Accelerating cosmological inference with Gaussian processes and neural networks – an application to LSST Y1 weak lensing and galaxy clustering

ABSTRACT Studying the impact of systematic effects, optimizing survey strategies, assessing tensions between different probes and exploring synergies of different data sets require a large number of simulated likelihood analyses, each of which cost thousands of CPU hours. In this paper, we present a method to accelerate cosmological inference using emulators based on Gaussian process regression and neural networks. We iteratively acquire training samples in regions of high posterior probability which enables accurate emulation of data vectors even in high dimensional parameter spaces. We showcase the performance of our emulator with a simulated 3×2 point analysis of LSST-Y1 with realistic theoretical and systematics modelling. We show that our emulator leads to high-fidelity posterior contours, with an order of magnitude speed-up. Most importantly, the trained emulator can be re-used for extremely fast impact and optimization studies. We demonstrate this feature by studying baryonic physics effects in LSST-Y1 3×2 point analyses where each one of our MCMC runs takes approximately 5 min. This technique enables future cosmological analyses to map out the science return as a function of analysis choices and survey strategy.

Astronomy & Astrophysics↗
NASA NTRS1984

Large-scale inhomogeneities and galaxy statistics

The density fluctuations associated with the formation of large-scale cosmic pancake-like and filamentary structures are evaluated using the Zel'dovich approximation for the evolution of nonlinear inhomogeneities in the expanding universe. It is shown that the large-scale nonlinear density fluctuations in the galaxy distribution due to pancakes modify the standard scale-invariant correlation function xi(r) at scales comparable to the coherence length of adiabatic fluctuations. The typical contribution of pancakes and filaments to the J3 integral, and more generally to the moments of galaxy counts in a volume of approximately (15-40 per h Mpc)exp 3, provides a statistical test for the existence of large scale inhomogeneities. An application to several recent three dimensional data sets shows that despite large observational uncertainties over the relevant scales characteristic features may be present that can be attributed to pancakes in most, but not all, of the various galaxy samples.

Schaeffer, R.↗
NASA NTRS1996

Are Planetary Regolith Particles Back Scattering? Response to a Paper by M. Mishchenko

In a recent paper Mishchenko asserts that soil particles are strongly forward scattering, whereas particles on the surfaces of objects in the solar system have been inferred to be back scattering. Mishchenko suggests that this apparent discrepancy is an artifact caused by using an approximate light scattering model to analyse the data, and that planetary regolith particles are actually strong forward scatterers. The purpose of the present paper is to point out the errors in Mishchenko's paper and to show from both theoretical arguments and experimental data that inhomogencous composite particles which are large compared to the wavelength of visible light, such as rock fragments and agglutinates, can be strongly back scattering and are the fundamental scatterers in media composed of them. Such particles appear to be abundant in planetary regoliths and can account for the back scattering character of the surfaces of many bodies in the solar system. If the range of phase angles covered by a data set is insufficient, serious errors in retrieving the particle scattering properties can result whether an exact or approximate scattering model is used. However, if the data set includes both large and small phase angles, approximate regolith scattering models can correctly retrieve the sign of the particle scattering asymmetry.

Hapke, Bruce↗