Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “large data sets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

DIRAC current, upcoming and planned capabilities and technologies

DIRAC is the interware for building and operating large scale distributed computing systems. It is adopted by multiple collaborations from various scientific domains for implementing their computing models. DIRAC provides a framework and a rich set of ready-to-use services for Workload, Data and Production Management tasks of small, medium and large scientific communities having different computing requirements. The base functionality can be easily extended by custom components supporting community specific workflows. DIRAC is at the same time an aging project, and a new DiracX project is taking shape for replacing DIRAC in the long term. This contribution will highlight DIRAC’s current, upcoming and planned capabilities and technologies, and how the transition to DiracX will take place. Examples include, but are not limited to, adoption of security tokens and interactions with Identity Provider services, integration of Clouds and High Performance Computers, interface with Rucio, improved monitoring and deployment procedures.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Mock data sets for the Eboss and DESI Lyman-α forest surveys

We present a publicly-available code to generate sets of mock Lyman-α (Lyα) forest data that have realistic large-scale correlations including those due to the Baryonic Acoustic Oscillations (BAO). The primary purpose of these mocks is to test the analysis procedures of the Extended Baryon Oscillation Survey (eBOSS) and the Dark Energy Spectroscopy Instrument (DESI) surveys. The transmitted flux fraction, F(λ), of background quasars due to Lyα absorption in the intergalactic medium (IGM) is simulated using the Fluctuating Gunn-Petterson Approximation (FGPA) applied to Gaussian random fields produced through the use of fast Fourier transforms (FFT). The output includes the IGM-Lyα transmitted flux fraction along quasar lines of sight and a catalog of high-column-density systems appropriately placed at high-density regions of the IGM. This output serves as input to additional code that superimposes the IGM tranmission on realistic quasar spectra, adds absorption by high-column-density systems and metals, and simulates instrumental transmission and noise. Redshift space distortions (RSD) of the flux correlations are implemented by including the large-scale velocity-gradient field in the FGPA resulting in a correlation function of F(λ) that can be accurately predicted. One hundred realizations have been produced over the 14,000 deg 2 DESI survey footprint with 100 quasars per deg 2 . The analysis of these realizations shows that the correlations of F(λ) follows the prediction within the accuracy of eBOSS survey. Here, the most time-consuming part of the mock production occurs before application of the FGPA, and the existing pre-FGPA forests can be used to easily produce new mock sets with modified redshift-dependent bias parameters or observational conditions.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

LSST Undergraduate Internships at Fermilab

Fermi National Accelerator Laboratory (Fermilab) plans to leverage the laboratory’s long tradition of hosting internships in high-energy physics and cosmology by setting up a mentoring laboratory program for undergraduate students to study data and model driven Large Synoptic Survey Telescope (LSST) science at the laboratory’s Center for Particle Astrophysics. The laboratory program will form an intern group that will explore the science of combining of LSST analysis using DC2 simulations and CosmoSIS modeling. Interns will be empowered to learn from each other while under the supervision of experienced cosmic frontier scientists, a model that has been successful in the laboratory setting. In addition, many of these undergraduate interns will form future candidate graduate student classes

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Performance assessment of ensembles of in situ workflows under resource constraints

Summary Scientific breakthroughs in biomolecular methods and improvements in hardware technology have shifted from a long‐running simulation to a large set of shorter simulations running simultaneously, called an ensemble. In an ensemble, simulations are usually coupled with analyses of data produced by the simulations. In situ methods can be used to analyze large volumes of data generated by scientific simulations at runtime (i.e., simulations and analyses are performed concurrently). In this work, we study the execution of ensemble‐based simulations paired with in situ analyses using in‐memory staging methods. Using an ensemble of molecular dynamics in situ workflows with multiple simulations and analyses, we first show that collecting traditional metrics such as makespan, instructions per cycle, memory usage, or cache miss ratio is not sufficient to characterize complex behaviors of ensembles. We propose a method to evaluate the performance of ensembles of workflows that captures multiple resource usage aspects: resource efficiency, resource allocation, and resource provisioning. Experimental results demonstrate that the proposed method can effectively distinguish the performance of different component placements in an ensemble with up to 32 ensemble members. By evaluating different co‐location scenarios, our proposed performance indicators demonstrate benefits of co‐locating simulation and coupled analyses within a compute node.

Do, Tu Mai Anh↗
DOE OSTI2020

Multiscale-Informed Modeling of High Temperature Component Response with Uncertainty Quantification

This report summarizes a joint effort between Argonne National Laboratory, Idaho National Laboratory, and Los Alamos National Laboratory to develop and deploy constitutive models targeted at predicting the life of Grade 91 alloy components subjected to high temperature environments typical of those that structural components in advanced nuclear reactors would experience. Two distinct, but complementary constitutive modeling approaches have been taken here. The first employs a phenomenological viscoplastic model for which parameters have been calibrated based on experimental data for a wide range of Grade 91 alloy that has undergone a variety of processing. A Bayesian approach was used to derive distributions of uncertain parameters for this model based on this data set. The second approach is a reduced order model suitable for engineering-scale analysis that is based on the results of a large set of mesoscale simulations. Mesoscale models allow for the microstructure and composition of a particular alloy to be directly taken into account in the computation of the viscoplastic response, but are computationally expensive, which makes it impractical to directly call those models for the material constitutive response in an engineering-scale simulation. The reduced-order representation of the response of the underlying model used here allows for an engineering-scale model to take into account the characteristics of the underlying microstructure, while only incurring a reasonable computational expense. Both of these approaches have different strengths, and are applicable for different parts of the design/analysis process. The phenomenological models can be readily parameterized based on a set of experimental data for a given class of materials and used for scoping calculations. Once a specific material is chosen and adequately characterized, the reduced order models can accurately predict the response of that specific alloy, and because the models are based on predictive models of the underlying microstructure, they can be used to more confidently predict the response under conditions in regions where there is limited experimental data. Both of these models have been integrated in the Grizzly code, which is used here to perform proof-of-concept uncertainty quantification analyses of a simple component under prototypical conditions. The built- in stochastic analysis capabilities in the MOOSE framework that Grizzly is built on are used here to run large sets of simulations for this uncertainty quantification analysis. As would be expected, because the reduced order models are developed for a much more tightly defined alloy, they predict tighter distributions of the time to failure than the phenomenological models, which are calibrated to a broader set of data. Also important is that these simulations demonstrate that a reduced order modeling approach can be successfully deployed to propagate uncertainties from the material scale to practical engineering-scale component simulations.

42 ENGINEERING↗
DOE OSTI2025

A Universal Augmentation Framework for Long-Range Electrostatics in Machine Learning Interatomic Potentials

Most current machine learning interatomic potentials (MLIPs) rely on short-range approximations, without explicit treatment of long-range electrostatics. To address this, we recently developed the Latent Ewald Summation (LES) method, which infers electrostatic interactions, polarization, and Born effective charges (BECs), just by learning from energy and force training data. Here, in this study, we present LES as a standalone library, compatible with any short-range MLIP, and demonstrate its integration with methods such as MACE, NequIP, Allegro, CACE, CHGNet, and UMA. We benchmark LES-enhanced models on distinct systems, including bulk water, polar dipeptides, and gold dimer adsorption on defective substrates, and show that LES not only captures correct electrostatics but also improves accuracy. Additionally, we scale LES to large and chemically diverse data by training MACELES-OFF on the SPICE set containing molecules and clusters, making a universal MLIP with electrostatics for organic systems, including biomolecules. MACELES-OFF is more accurate than its short-range counterpart (MACE-OFF) trained on the same data set, predicts dipoles and BECs reliably, and has better descriptions of bulk liquids. By enabling efficient long-range electrostatics without directly training on electrical properties, LES paves the way for electrostatic foundation MLIPs.

Kim, Dongjin [University of California, Berkeley, ↗
DOE OSTI2026

Constraining the Aerosol Effects on Deep Convective Clouds by Considering the Coupling Between Clouds and the Planetary Boundary Layer

Several mechanisms have been proposed for the aerosol invigoration effect. Although their principles are well established, their actual magnitudes and roles in cloud development remain uncertain and debatable. This uncertainty partly stems from observational-based studies, in which it has been challenging to separate the co-variability between aerosols and meteorology. Addressing this problem requires large data samples. To this end, this study employs the Atmospheric Radiation Measurement data set expanding to 16 years (some up to 17 years, compared to 10 years in previous work) in the U.S. Southern Great Plains. It also conducts a more careful and rigorous analysis to isolate the influences of convective available potential energy (CAPE) and synoptic patterns to address a previously raised concern. We incorporated a new key process affecting aerosol-cloud interaction: cloud-surface coupling. The state/degree of the coupling relationship turns out to play an important role in the invigoration effect. Our analysis reinforces earlier findings of a robust positive relationship between cloud thickness and aerosol loading across CAPE percentiles—but only under cloud-surface coupled conditions. The increase in cloud thickness with aerosol loading is most pronounced in coupled clouds with high CAPE and bases below 1 km. Coupled clouds with bases below 1 km thicken between 1 and 4 km, depending on the CAPE percentile. Decoupled clouds show no such systematic changes. Synoptic patterns also lead to different strengths of the invigoration effect. Clean and polluted air masses are predominantly associated with northerly and southerly winds, respectively, with a stronger invigoration effect in cleaner air masses.

Geosciences↗
DOE OSTI2025

Performance of the spin-component-scaled methods for energy bands

The performance of various spin-component-scaled parameterisations is examined for the second-order many-body Green's-function [MBGF(2)] calculations of valence energy bands, taking three of the experimentally well-characterised polymers as examples: polyethylene, polytetrafluoroethylene, and polyacetylene. The parameterisations considered are Grimme's original SCS parameter set, Jung et al.'s original SOS set (retaining the opposite-spin component only), Śmiga et al.'s SCS(IP) set (calibrated specifically for ionization energies), and Śmiga et al.'s SOS(IP) set (calibrated for ionization energies with the opposite-spin component only; implicit in the os-D2 model of Opoku et al.). The SCS(IP) and SOS(IP) parameterisations are found to shift both outer and inner valence bands by up to a few electronvolts away from the experimental data. The original SCS and SOS parameter sets do not improve upon, but largely maintain the accuracy of the unscaled MBGF(2) methods. Given that the SOS-MBGF(2) method can be implemented in a quartic-scaling algorithm (for all roots), it is most promising for solid-state applications. Furthermore this observation is consistent with the success of the quartic-scaling GW methods without the vertex correction based on a density-functional theory reference.

Green’s-function theory↗
DOE OSTI2024

Intricate short-range order in GeSn alloys revealed by atomistic simulations with highly accurate and efficient machine-learning potentials

GeSn alloys hold promise for silicon-compatible integrated applications in electronics, photonics, and topological quantum devices. However, understanding their intricate structures using density functional theory (DFT) calculations is hindered by spatiotemporal constraints. To overcome this limitation, we develop highly accurate and efficient machine-learning interatomic potentials based on a neuroevolution potential approach with farthest point sampling on a comprehensive DFT data set. The application of the developed machine-learning potential in large-scale atomistic simulations bridges the spatiotemporal gap between modeling and advanced characterizations, and facilitates the discovery of structural intricacies in GeSn alloys. Through extensive statistical sampling, we identify a type of short-range order (SRO) that is distinguished by both its structural signature and electronic band gap from the SRO structure previously predicted. Modeling based on a large simulation cell reveals the coexistence of nano SRO domains with various degrees of ordering, demonstrating a complex spatial heterogeneity of SRO structure. Finally, our study not only reinforces the significance of fine-level structural information in alloys, but it also constitutes an effective framework for exploring SRO in a broad range of complex alloys based on highly accurate and effective machine-learning potentials.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Open Source Platform for Live Processing of High Data Rate Electron Microscopy (SBIR Phase I Award Topic 15, Subtopic B, Final Scientific / Technical Report)

4D scanning transmission electron microscopy (STEM) is experiencing a revolution both in terms of data size, and data veracity, as are many other areas of experimental data acquisition. This presents enormous opportunities to make new scientific discoveries, but is also hamstrung by aging software that is no longer fit for purpose. Experimental data acquisition platforms must make the same investments in software infrastructure that the simulation community made decades ago to take advantage of high-performance computing (HPC) if they hope to realize the full potential of the latest advances made in detector technology. This requires rethinking the design and deployment of hardware, moving away from small, fixed commodity computing capabilities with closed interfaces to open, modular, scalable systems designed for parallelism and throughput. The technical approach will leverage best-in-class open source technologies, and develop new approaches strategically to build an open source platform for experimental data acquisition targeting high data rate electron microscopy. The project will develop a software platform that uses the power and performance of modern HPC resources to ingest the large volume of data produced by electron microscopes using direct electron detectors. A set of configurable pipelines will be used to enable the processing to be tailored to the problem being investigated with builtin visualization of data at interactive rates. This project will leverage investments made by the DOE and other agencies to provide a powerful software platform for the experimental data community working with most powerful electron microscopes, such as 4D STEM detectors. It will create a platform that serves as a reference for the community. Permissive open source licensing will make the platform available to all, with a simple deployment strategy offering the ability to keep data close to where it is generated. This advanced software platform will require customization for each deployment, creating opportunities for software services that are typically offered to extend open source software platforms.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

How Low Can You Go? Using Synthetic 3D Imagery to Drastically Reduce Real-World Training Data for Object Detection

Deep convolutional neural networks (DCNNs) currently provide state-of-the-art performance on image classification and object detection tasks, and there are many global security mission areas where such models could be extremely useful. Crucially, the success of these models is driven in large part by the widespread availability of high-quality open source data sets such as Image Net, Common Objects in Context (COCO), and KITTI, which contain millions of images with thousands of unique labels. However, global security relevant objects-of-interest can be difficult to obtain: relevant events are low frequency and high consequence; the content of relevant images is sensitive; and adversaries and proliferators seek to obscure their activities. For these cases where exemplar data is hard to come-by, even fine-tuning an existing model with available data can be effectively impossible. Recent work demonstrated that models can be trained using a combination of real-world and synthetic images generated from 3D representations; that such models can exceed the performance of models trained using real-world data alone; and that the generated images need not be perfectly realistic (Tremblay, et al., 2018). However, this approach still required hundreds to thousands of real-world images for training and fine tuning, which for sparse, global security-relevant datasets can be an unrealistic hurdle. In this research, we validate the performance and behavior of DCNN models as we drive the number of real-world images used for training object detection tasks down to a minimal set. We perform multiple experiments to identify the best approach to train DCNNs from an extremely small set of real-world images. In doing so, we: Develop state-of-the-art, parameterized 3D models based on real-world images and sample from their parameters to increase the variance in synthetic image training data; Use machine learning explainability techniques to highlight and correct through targeted training the biases that result from training using completely synthetic images; and Validate our results by comparing the performance of the models trained on synthetic data to one another, and to a control model created by fine-tuning an existing ImageNet-trained model with a limited number (hundreds) of real-world images.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Machine learning to generate gridded extreme precipitation data sets for global land areas with limited in situ measurements

There is a strong need for gridded observational data sets that describe the climatology of extreme precipitation across the globe; such data sets are critical for quantifying precipitation extremes and their corresponding influence on large perturbations on surface and groundwater systems and flooding. Machine learning (ML) methods can be used to generate pseudo in situ measurements of the climatology of extreme precipitation for land regions with poor sampling by matching relevant geographic (e.g., orography) and atmospheric (e.g., surface temperature, precipitation, and pressure) variables for densely sampled regions and those with limited geographic sampling.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Analytical and Computational Advances, Opportunities, and Challenges in Marine Organic Biogeochemistry in an Era of “Omics”

Advances in sampling tools, analytical methods, and data handling capabilities have been fundamental to the growth of marine organic biogeochemistry over the past four decades. There has always been a strong feedback between analytical advances and scientific advances. However, whereas advances in analytical technology were often the driving force that made possible progress in elucidating the sources and fate of organic matter in the ocean in the first decades of marine organic biogeochemistry, today process-based scientific questions should drive analytical developments. Several paradigm shifts and challenges for the future are related to the intersection between analytical progress and scientific evolution. Untargeted “molecular headhunting” for its own sake is now being subsumed into process-driven targeted investigations that ask new questions and thus require new analytical capabilities. However, there are still major gaps in characterizing the chemical composition and biochemical behavior of macromolecules, as well as in generating reference standards for relevant types of organic matter. Field-based measurements are now routinely complemented by controlled laboratory experiments and in situ rate measurements of key biogeochemical processes. And finally, the multidisciplinary investigations that are becoming more common generate large and diverse datasets, requiring innovative computational tools to integrate often disparate data sets, including better global coverage and mapping. Here, we compile examples of developments in analytical methods that have enabled transformative scientific advances since 2004, and we project some challenges and opportunities in the near future. We believe that addressing these challenges and capitalizing on these opportunities will ensure continued progress in understanding the cycling of organic carbon in the ocean.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Detection of Cyclopropenylidene on Titan with ALMA

We report the first detection on Titan of the small cyclic molecule cyclopropenylidene (c-C{sub 3}H{sub 2}) from high-sensitivity spectroscopic observations made with the Atacama Large Millimeter/submillimeter Array. Multiple lines of cyclopropenylidene were detected in two separate data sets: ∼251 GHz in 2016 (Band 6) and ∼352 GHz in 2017 (Band 7). Modeling of these emissions indicates abundances of 0.50 ± 0.14 ppb (2016) and 0.28 ± 0.08 (2017) for a 350 km step model, which may either signify a decrease in abundance, or a mean value of 0.33 ± 0.07 ppb. Inferred column abundances are (3–5) × 10{sup 12} cm{sup −2} in 2016 and (1–2) × 10{sup 12} cm{sup −2} in 2017, similar to photochemical model predictions. Previously the C{sub 3}H{sub 3}{sup +} ion has been measured in Titan’s ionosphere by Cassini’s Ion and Neutral Mass Spectrometer (INMS), but the neutral (unprotonated) species has not been detected until now, and aromatic versus aliphatic structure could not be determined by the INMS. Our work therefore represents the first unambiguous detection of cyclopropenylidene, the second known cyclic molecule in Titan’s atmosphere along with benzene (C{sub 6}H{sub 6}) and the first time this molecule has been detected in a planetary atmosphere. We also searched for the N-heterocycle molecules pyridine and pyrimidine finding nondetections in both cases, and determining 2σ upper limits of 1.15 ppb (c-C{sub 5}H{sub 5}N) and 0.85 ppb (c-C{sub 4}H{sub 4}N{sub 2}) for uniform abundances above 300 km. These new results on cyclic molecules provide fresh constraints on photochemical pathways in Titan’s atmosphere, and will require new modeling and experimental work to fully understand the implications for complex molecule formation.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Results of the Photometric LSST Astronomical Time-series Classification Challenge (PLAsTiCC)

Abstract Next-generation surveys like the Legacy Survey of Space and Time (LSST) on the Vera C. Rubin Observatory (Rubin) will generate orders of magnitude more discoveries of transients and variable stars than previous surveys. To prepare for this data deluge, we developed the Photometric LSST Astronomical Time-series Classification Challenge (PLAsTiCC), a competition that aimed to catalyze the development of robust classifiers under LSST-like conditions of a nonrepresentative training set for a large photometric test set of imbalanced classes. Over 1000 teams participated in PLAsTiCC, which was hosted in the Kaggle data science competition platform between 2018 September 28 and 2018 December 17, ultimately identifying three winners in 2019 February. Participants produced classifiers employing a diverse set of machine-learning techniques including hybrid combinations and ensemble averages of a range of approaches, among them boosted decision trees, neural networks, and multilayer perceptrons. The strong performance of the top three classifiers on Type Ia supernovae and kilonovae represent a major improvement over the current state of the art within astronomy. This paper summarizes the most promising methods and evaluates their results in detail, highlighting future directions both for classifier development and simulation needs for a next-generation PLAsTiCC data set.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Model Evaluation and Intercomparison of Marine Warm Low Cloud Fractions With Neural Network Ensembles

Abstract Low cloud fractions (LCFs) and meteorological factors (MFs) over an oceanic region containing multiple cloud regimes are examined for three data sets: one Energy Exascale Earth System Model (E3SM) simulation with the default 72‐layer vertical grid (E3SM72), another one with 8‐times vertical resolution via the Framework for Improvement by Vertical Enhancement (E3SM 8), and one with MFs from ERA5 reanalysis and LCFs from the CERES SSF product (ERA5‐SSF). Neural networks (NNs) are trained to capture the relationship between MFs and LCF and to select the best‐performing MF subsets for predicting LCF. NN ensembles are used to (a) confirm the performance of selected MF subsets, (b) to serve as proxy models for each data set to predict LCFs for MFs from all data sets, and (c) to classify MFs into those in shared and uniquely occupied MF subspaces. Overall, E3SM72 and E3SM 8 have large fractions of MFs in shared MF subspace, but less so near the Californian and Peruvian stratocumulus decks. E3SM 8 and ERA5 have small fractions of MFs in shared MF subspace but greater than E3SM72 and ERA5, especially in the Southeast Pacific. The differences in LCFs between three pairs of data sets are decomposed into those associated with the differences in the LCF‐MF relationship and those involving different MFs. Given the same MFs, LCFs produced by E3SM 8 are greater than those produced by E3SM72 but are still different from those in ERA5‐SSF. In general, the shift in MFs dominates the difference in the LCFs.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Modeling Spatial Heterogeneity in Surface Turbulent Heat Flux in the U.S. Southern Great Plains

Advances in numerical modeling of cloud dynamics are driving a need for improved land model prediction at convective storm scales. Here, satellite and ground-based vegetation remote sensing data were combined with land model experiments to more accurately characterize land surface spatial heterogeneity in the Community Land Model (CLM4.0). The new subgrid classification of plant functional types (PFT) and leaf area index (LAI) enables consistent comparison between models and ground-based flux measurements in the U.S. southern Great Plains. Errors in vegetation data sets (inferred from comparison between 250 m satellite and ground-based LAI), while large, had less impact on the simulated characteristics of spatial heterogeneity compared to errors in model representation of surface energy partitioning (between latent and sensible heat flux) and its relationship to LAI. Predicted spatial heterogeneity in surface energy partitioning was enhanced after replacing soil and stomatal resistance parameters with a new set that better predicts the observed relationship to LAI. These modifications increase the number of smaller (mesoscale) dry land patches having higher sensible heat flux. The parameter experiments suggest that vegetation state and processes (transpiration) act to broaden the size spectrum of surface heat flux heterogeneity, which can influence clouds and convective initiation. Moreover, improvements in vegetation input data and model parameters had partially compensating effects on surface flux heterogeneity, indicating the importance of evaluating input data and parameterizations together to improve prediction at higher spatial resolutions.

54 ENVIRONMENTAL SCIENCES↗