Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “large data sets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

SIRIUS: Science-Driven Data Management for Multi-Tiered Storage

The data sets being generated by large applications on very large-scale systems are increasing in both size and complexity. At the same time, there are new ways available to store and access these data sets. The goal in this project is to develop software that applications can use to make use of new and existing storage technologies in more sophisticated ways. One challenge in scientific data management is handling ‘hot’ vs ‘cold’ data. Data that is hot is data that is needed (or will be needed soon) in order for the program to continue progressing, while cold data is either output (and so will not be need further during the life of the program) or will not be needed until significantly later in the program’s run. Hot data should be stored in a way that allows fast access. On most systems, economic factors lead to an inverse relationship between storage performance and storage capacity and so fast access storage is limited. This makes it important to correctly place hot and cold data and avoid cold data unnecessarily consuming precious resources. In this reporting period, we addressed this challenge in various ways and at various levels. Data management frameworks offer only limited control to applications in how data is stored. We have added software capabilities for seamlessly moving data between layers of the storage technology using promote and demote functions to existing software frameworks. This gives direct control to applications in deciding what priority data receives. Additionally, we integrated different storage layer management frameworks in order to allow data to be exchanged and moved between storage layers in a consistent way across the application. Further, applications are not always able to directly decide what storage level makes sense for a given piece of data without an understanding of the underlying storage technologies. Data storage frameworks are often positioned to make these sorts of decisions in service of the application. We have added machine-learning based capabilities to data staging frameworks in order to make intelligent decisions about where data should be stored given learning about patterns in previous usage of similar data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Importance of Engineered and Learned Molecular Representations in Predicting Organic Reactivity, Selectivity, and Chemical Properties

Machine-readable chemical structure representations are foundational in all attempts to harness machine learning for the prediction of reactivities, selectivities, and chemical properties directly from molecular structure. The featurization of discrete chemical structures into a continuous vector space is a critical phase undertaken before model selection, and the development of new ways to quantitatively encode molecules is an active area of research. Here, we highlight the application and suitability of different representations, from expert-guided “engineered” descriptors to automatically “learned” features, in different prediction tasks relevant to organic and organometallic chemistry, where differing amounts of training data are available. These tasks include statistical models of stereo- and enantioselectivity, thermochemistry, and kinetics developed using experimental and quantum chemical data. The use of expert-guided molecular descriptors provides an opportunity to incorporate chemical knowledge, domain expertise, and physical constraints into statistical modeling. In applications to stereoselective organic and organometallic catalysis, where data sets may be relatively small and 3D-geometries and conformations play an important role, mechanistically informed features can be used successfully to obtain predictive statistical models that are also chemically interpretable. We provide an overview of several recent applications of this approach to obtain quantitative models for reactivity and selectivity, where topological descriptors, quantum mechanical calculations of electronic and steric properties, along with conformational ensembles, all feature as essential ingredients of the molecular representations used. Alternatively, more flexible, general-purpose molecular representations such as attributed molecular graphs can be used with machine learning approaches to learn the complex relationship between a structure and prediction target. This approach has the potential to out-perform more traditional representation methods such as “hand-crafted” molecular descriptors, particularly as data set sizes grow. One area where this is particularly relevant is in the use of large sets of quantum mechanical data to train quantitative structure–property relationships. A general approach toward curating useful data sets and training highly accurate graph neural network models is discussed in the context of organic bond dissociation enthalpies, where this strategy outperforms regression using precomputed descriptors. Finally, we describe how graph neural network predictions can be incorporated into mechanistically informed statistical models of chemical reactivity and selectivity. Once trained, this approach avoids the expensive computational overhead associated with quantum mechanical calculations, while maintaining chemical interpretability. We illustrate examples for which fast predictions of bond dissociation enthalpy and of the identities of radicals formed through cleavage of a molecule’s weakest bond are used in simple physical models of site-selectivity and reactivity.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

SQL and NoSQL Databases for Cyber Physical Production Systems in Internet of Things for Manufacturing (IoTfM)

Abstract In this paper, the design and performance differences between Relational Database Management Systems (RDBMS) and NoSQL Database Systems are examined, with attention to their applicability for real-world Internet of Things for manufacturing (IoTfM) data. While previous work has extensively compared SQL and NoSQL for both generalized and IoT uses, this work specifically examines the tradeoffs and performance differences for manufacturing applications by using a high-fidelity data set collected from a large US manufacturing firm. Growing an IoT system beyond the pilot stage requires scalable data storage; this work seeks to determine the impact of selected database systems on data write performance at scale. Payload size and message frequency were used as the primary characteristics to maintain model fidelity in simulated clients. As the number of simulated asset clients grow, the data write latency was calculated to determine how both database systems’ performance were affected. To isolate the RDBMS and NoSQL differences, a cloud environment was created using Amazon Web Services (AWS) with two identical data ingestion pipelines: writing data to an RDMBS (1) using AWS Aurora MySQL, and (2) using AWS DynamoDB NoSQL. The findings may provide guidance for further experimentation in large-scale manufacturing IoT implementations.

Gamero, David↗
DOE OSTI2022

Cold gas mass measurements for the era of large optical spectroscopic surveys

ABSTRACT Gas plays an important role in many processes in galaxy formation and evolution, but quantifying the importance of gas has been hindered by the challenge to measure gas masses for large samples of galaxies. Data sets of direct atomic and molecular gas measurements are sufficient to establish simple scaling relations, but often not large enough to quantify three-parameter relations, or second-order dependences. As an alternative approach, we derive here indirect cold gas measurements from optical emission lines using photoionization models for galaxies in the Sloan Digital Sky Survey (SDSS) main galaxy sample and the PHANGS-MUSE survey. We calibrate the gas surface density measurements using xCOLD GASS and PHANGS-ALMA molecular gas measurements to ensure that our measurements are reliable. We demonstrate the importance of taking into account the scale dependence of the relation between optical depth (τ V ) and gas surface density (Σ gas ) and provide a general prescription to estimate Σ gas from τ V , metallicity, and the dust-to-metal ratio, at any arbitrary physical resolution. To demonstrate that the indirect cold gas masses are accurate enough to quantify the role of gas in galaxy evolution, we study the mass–metallicity relation of SDSS galaxies and show that as a third parameter gas mass is better than star formation rate at reducing the scatter of the relation, as predicted by models and simulations.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Measuring Neutron Polarisation in Deuteron Photo-disintegration with the CLAS Start Counter [Thesis]

Deuteron photo-disintegration (γd → γp) is a reaction that represents the simplest case in which nuclear and hadron physics models can be tested. Despite this, associated polarization analyses are limited in terms of angular coverage and energy ranges, especially in observables related to the recoil neutron. This is largely due to a lack in dedicated polarimetry equipment, and represents a roadblock in global progress to understand high-energy phenomena such as hexaquarks, and quark-gluon degrees of freedom. To address this problem, this PhD thesis pioneers a new methodology for the parasitic measurement of nucleon polarization using kinematic reconstruction of (spin-dependent) nucleon-nucleus scattering of reaction products, prior to their detection in large acceptance particle detector apparatus. Following this novel approach, which requires no dedicated polarimeter, a determination of the double polarization observable, $C^n_{x'}$, from deuteron photo-disintegration is presented, using Jefferson Lab’s CLAS detector. The analysis utilizes the (n,p) charge exchange reaction in CLAS’s "start counter" (plastic scintillator) to determine the final state neutron polarizations. The results present the first ever data for this observable above 0.7 GeV (photon beam energy) and significantly extend the angular range of the world data set. This new data is largely statistically consistent with the previous measurement of $C^n_{x'}$ by Bashkanov et al . in the overlapping energy range of 0.4-0.7 GeV. It is planned for the statistical accuracy of the presented result to be increased by the inclusion of additional data. The analysis herein serves as a key proof of concept for future applications, including a recommended similar analysis to be implemented with data from the more modern CLAS12 detector. This paves the way for a plethora of additional analyses using existing data sets that would provide crucial new constraints for hadron and nuclear physics.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

Automated Detection of Instability-Inducing Channel Geometry Transitions in Saint-Venant Simulation of Large-Scale River Networks

A new sweep-search algorithm (SSA) is developed and tested to identify the channel geometry transitions responsible for numerical convergence failure in a Saint-Venant equation (SVE) simulation of a large-scale open-channel network. Numerical instabilities are known to occur at “sharp” transitions in discrete geometry, but the identification of problem locations has been a matter of modeler’s art and a roadblock to implementing large-scale SVE simulations. The new method implements techniques from graph theory applied to a steady-state 1D shallow-water equation solver to recursively examine the numerical stability of each flowpath through the channel network. The SSA is validated with a short river reach and tested by the simulation of ten complete river systems of the Texas–Gulf Coast region by using the extreme hydrological conditions recorded during hurricane Harvey. The SSA successfully identified the problematic channel sections in all tested river systems. Subsequent modification of the problem sections allowed stable solution by an unsteady SVE numerical solver. The new SSA approach permits automated and consistent identification of problem channel geometry in large open-channel network data sets, which is necessary to effectively apply the fully dynamic Saint-Venant equations to large-scale river networks or for city-wide stormwater networks.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Constraints on the Intergalactic Magnetic Field Using Fermi-LAT and H.E.S.S. Blazar Observations

Magnetic fields in galaxies and galaxy clusters are believed to be the result of the amplification of intergalactic seed fields during the formation of large-scale structures in the universe. However, the origin, strength, and morphology of this intergalactic magnetic field (IGMF) remain unknown. Lower limits on (or indirect detection of) the IGMF can be obtained from observations of high-energy gamma rays from distant blazars. Gamma rays interact with the extragalactic background light to produce electron-positron pairs, which can subsequently initiate electromagnetic cascades. The gamma-ray signature of the cascade depends on the IGMF since it deflects the pairs. Here we report on a new search for this cascade emission using a combined data set from the Fermi Large Area Telescope and the High Energy Stereoscopic System. Using state-of-the-art Monte Carlo predictions for the cascade signal, our results place a lower limit on the IGMF of B > 7.1 × 10 -16 G for a coherence length of 1 Mpc even when blazar duty cycles as short as 10 yr are assumed. This improves on previous lower limits by a factor of 2. For longer duty cycles of 10 4 (10 7 ) yr, IGMF strengths below 1.8 × 10 -14 G (3.9 × 10 -14 G) are excluded, which rules out specific models for IGMF generation in the early universe.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Water content of carbon dioxide at hydrate forming conditions

There is an interest to ensure sub-saturated water content in lines containing carbon dioxide in applications such as enhanced oil recovery and carbon sequestration, to reduce risks of hydrate blockage and corrosion. The water content of carbon dioxide at various temperatures and pressures has been measured in the past, but there is no consistent set of measurements that could be used for carbon dioxide storage and transportation design work. The solubility of water in a carbon dioxide rich gas phase at hydrate forming conditions was measured in this work. Pressures ranged from 12.06 to 29.30 bar along two isotherms, 1 °C and -7 °C, all within the gaseous carbon dioxide and hydrate stability zone. For the first time in these types of measurements, the solid phase was also characterized and confirmed to be carbon dioxide hydrate via X-ray computed tomography, simultaneous with water content measurements of the gas phase. Once carbon dioxide hydrate conversion had reached a maximum value (65% estimated by X-ray computed tomography), the equilibrium water content was measured. Prior to reaching this maximum carbon dioxide hydrate conversion, the water content in carbon dioxide was observed to decrease as liquid water converted to carbon dioxide hydrate. Furthermore, this slow conversion to hydrate, metastability of the hydrate phase, or unexpected phases may be responsible for the large discrepancy between prior data sets for similar carbon dioxide water content measurements.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Prompt and non-prompt J/ψ production cross sections at midrapidity in proton-proton collisions at $ \sqrt{\mathrm{s}} $ = 5.02 and 13 TeV

The production of J /ψ is measured at midrapidity ( |y| < 0 . 9) in proton-proton collisions at \( \sqrt{s} \) = 5 . 02 and 13 TeV, through the dielectron decay channel, using the ALICE detector at the Large Hadron Collider. The data sets used for the analyses correspond to integrated luminosities of \( \mathcal{L} \) int = 19.4 ± 0.4 nb - 1 and \( \mathcal{L} \) int = 32.2 ± 0.5 nb - 1 at \( \sqrt{s} \) = 5 . 02 and 13 TeV, respectively. The fraction of non-prompt J /ψ mesons, i.e. those originating from the decay of beauty hadrons, is measured down to a transverse momentum p T = 2 GeV /c (1 GeV /c ) at \( \sqrt{s} \) = 5 . 02 TeV (13 TeV). The p T and rapidity ( y ) differential cross sections, as well as the corresponding values integrated over p T and y , are carried out separately for prompt and non-prompt J /ψ mesons. The results are compared with measurements from other experiments and theoretical calculations based on quantum chromodynamics (QCD). The shapes of the p T and y distributions of beauty quarks predicted by state-of-the-art perturbative QCD models are used to extrapolate an estimate of the \( \mathrm{b}\overline{\mathrm{b}} \) pair cross section at midrapidity and in the total phase space. The total \( \mathrm{b}\overline{\mathrm{b}} \) cross sections are found to be \( {\sigma}_{\mathrm{b}\overline{\mathrm{b}}} \) = 541 ± 45 (stat . ) ± 69 \( {\left(\mathrm{syst}.\right)}_{-12}^{+10} \) (extr . ) μ b and \( {\sigma}_{\mathrm{b}\overline{\mathrm{b}}} \) = 218 ± 37 (stat . ) ± 31 \( {\left(\mathrm{syst}.\right)}_{-9.1}^{+8.2} \) (extr . ) μ b at \( \sqrt{s} \) = 13 and 5.02 TeV, respectively. The value obtained from the combination of ALICE and LHCb measurements in pp collisions at \( \sqrt{s} \) = 13 TeV is also provided.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2020

Combinatorial Exploration and Mapping of Phase Transformation in a Ni–Ti–Co Thin Film Library

Combinatorial synthesis and high-throughput characterization of a Ni–Ti–Co thin film materials library are reported for exploration of reversible martensitic transformation. The library was prepared by magnetron co-sputtering, annealed in vacuum at 500 °C without atmospheric exposure, and evaluated for shape memory behavior as an indicator of transformation. Composition, structure, and transformation behavior of the 177 pads in the library were characterized using high-throughput wavelength dispersive spectroscopy (WDS), X-ray photoelectron spectroscopy (XPS), X-ray diffraction (XRD), and four-point probe temperature-dependent resistance (R(T)) measurements. A new, expanded composition space having phase transformation with low thermal hysteresis and Co > 10 at. % is found. Unsupervised machine learning methods of hierarchical clustering were employed to streamline data processing of the large XRD and XPS data sets. Through cluster analysis of XRD data, we identified and mapped the constituent structural phases. Finally, composition–structure–property maps for the ternary system are made to correlate the functional properties to the local microstructure and composition of the Ni–Ti–Co thin film library.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Synthetic data generation for machine learning model training for energy theft scenarios using cosimulation

Abstract Technical and non‐technical losses in distribution circuits result in significant economic costs to power utilities. One type of non‐technical loss is energy theft by various means including illegal tapping of feeders, bypassing the meter, and billing fraud. These losses are usually hard to detect, and can remain undetected for long periods of time. Machine learning models have been proven effective in detecting these conditions, but rely on the availability of large, good‐quality training data sets. The problem is exacerbated by the imbalanced nature of data related to these conditions—energy theft, though costly, is very rare. The available data sets generally have very few samples of theft with most of the data pertaining to normal operation. Such data sets are generally not suitable to train machine learning models. In this paper, an overview of energy theft detection techniques, the challenges with their data needs, and the limitations of current techniques to bridge such data limitations is presented. A co‐simulation framework is proposed to generate reliable training data for machine learning algorithms for theft detection. An example scenario is presented and a machine learning model is built to detect certain kinds of energy theft.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

A Standardized Analysis Process Using Digital Image Correlation to Calculate In Situ Cladding Strain from Modified Burst Tests for Fuel Performance Code Validation

Historical data collection on nuclear fuel cladding materials has focused on generating a statistically significant amount of data to assess the material and its failure behavior. Furthermore, data generated to support material model and failure criteria development were previously posttest evaluations, so a large number of tests was required to gain new understanding. A way to expedite this process is to develop techniques capable of generating large, high-fidelity data sets from a single test with lower uncertainty or quantified uncertainty. One such example of this approach is Oak Ridge National Laboratory’s use of modified burst tests (MBTs) to analyze the mechanical behavior and failure conditions of cladding during a simulated reactivity-initiated accident (RIA). Each test incorporates digital image correlation (DIC) analysis techniques that are used to assess the accumulated strain in situ, as well as eventual cladding failure. This work has been fruitful in defining strain-to-failure conditions for materials like silicon carbide (SiC) fiber–reinforced/SiC matrix composite tubes (SiC/SiC), iron-chromium-aluminum (FeCrAl) alloy tubes, and chromium-coated Zircaloy-4 tubes. However, there are numerous DIC software available, including open-source and proprietary software. The different DIC software use various algorithms to process images and calculate displacement values. Using these different software and algorithms can lead to varying results, and perhaps larger-than-expected uncertainties. In the present study, previously published MBT data encompassing a variety of test conditions were reanalyzed with two different DIC software to assess the variance in the calculated strain results. The data consisted of SiC/SiC, FeCrAl, and chromium-coated Zircaloy-4 tubes. Plots of the calculated strains during the transient revealed good agreement between the two DIC software. The average root-mean-square errors between the two software was 0.20% strain, which is slightly larger than a previously reported error value for these tests. In conclusion, this variance in results is low enough that this analysis method can be used for code validation.

Reactivity-initiated accident↗
DOE OSTI2024

UMap: An application-oriented user level memory mapping library

Exploiting the prominent role of complex memories in exascale node architecture, the UMap page fault handler offers new capabilities to access large memory-mapped data sets directly. UMap provides flexible configuration options to customize page handling to each application, including analysis of massive observational and simulation data sets. The high-performance design features I/O decoupling, dynamic load balancing, and application-level controls. Page faults triggered by application threads and processes accessing data mapped to a UMapp’ed region are handled via the Linux userfaultfd protocol, an asynchronous message-oriented kernel-user communication mechanism that avoids the context switch penalty of traditional signal fault handlers. UMap is fully open source. In this paper, we give an overview of the UMap library architecture, its extensible plugin architecture, and the use/performance of UMap in emerging heterogeneous memory hierarchies such as near-node Non-volatile Memory (NVM) and network attached memories. We highlight new capabilities in two pagefault management plugins, the NetworkStore and SparseStore. We demonstrate the integration between UMap and multiple ECP products including Caliper, Metall, ZFP, Mochi, and Ripples.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Novel Results Visualization for Dynamic PSA and New Modeling Features in EMRALD

The Event Modeling Risk Assessment Linked Diagram (EMRALD) tool, developed at the Idaho National Laboratory (INL), was designed to simplify the creation of dynamic models and support various research projects. One of the primary goals of EMRALD was to provide visual methods for modeling. EMRALD consists of two main components: a web-based user interface for model development and a solve engine for running model simulations. Over time, it has evolved to meet the diverse needs of its users. Initially, EMRALD's results were simple text outputs with final key state percentages and uncertainty bounds. However, because EMRALD utilizes a three-phase discrete event simulation and tracks the paths of each simulation run leading to a key state, there is significant potential to analyze large sets of path results data, including state paths, events, and timing. Visualizing this data meaningfully posed a challenge. To address this, a novel time-based Sankey diagram was developed. EMRALD exports results data in a format that can be opened in this Sankey viewer, allowing users to visualize paths, occurrences, events, and probability data for the entire simulation run in a single diagram. Moreover, when EMRALD was first created, there were limited tools capable of meeting its graphical requirements, many of which are no longer supported. In 2024, a new web-based interface was developed using modern graphing tools, enabling additional modeling features. This paper discusses the new dynamic PSA results visualization capability and the enhanced modeling tools available in EMRALD.

97 - MATHEMATICS AND COMPUTING↗
DOE OSTI2023

HVAC and Control Templates for the Modelica Buildings Library

This article reports on our experience in creating Modelica models for systems with thousands of configurations and closed-loop controls. The development of such templates required exploration of class parameterization techniques and data structures for handling large sets of equipment parameters. By describing these issues and the approach taken, we show how the Modelica language can support advanced templating logic. The main limitation we encountered relates to parameter assignment and propagation. The interpretation of parameter attributes at user interface runtime, or the handling of non-trivial constructs involving record classes at compile time is not consistently supported by Modelica tools. This leads to choices that are difficult to make when looking for a generic implementation.

Gautier, Antoine↗
DOE OSTI2021

A Comparison of Machine Learning Methods to Forecast Tropospheric Ozone Levels in Delhi

Ground-level ozone is a pollutant that is harmful to urban populations, particularly in developing countries where it is present in significant quantities. It greatly increases the risk of heart and lung diseases and harms agricultural crops. This study hypothesized that, as a secondary pollutant, ground-level ozone is amenable to 24 h forecasting based on measurements of weather conditions and primary pollutants such as nitrogen oxides and volatile organic compounds. We developed software to analyze hourly records of 12 air pollutants and 5 weather variables over the course of one year in Delhi, India. To determine the best predictive model, eight machine learning algorithms were tuned, trained, tested, and compared using cross-validation with hourly data for a full year. The algorithms, ranked by R2 values, were XGBoost (0.61), Random Forest (0.61), K-Nearest Neighbor Regression (0.55), Support Vector Regression (0.48), Decision Trees (0.43), AdaBoost (0.39), and linear regression (0.39). When trained by separate seasons across five years, the predictive capabilities of all models increased, with a maximum R 2 of 0.75 during winter. Bidirectional Long Short-Term Memory was the least accurate model for annual training, but had some of the best predictions for seasonal training. Out of five air quality index categories, the XGBoost model was able to predict the correct category 24 h in advance 90% of the time when trained with full-year data. Separated by season, winter is considerably more predictable (97.3%), followed by post-monsoon (92.8%), monsoon (90.3%), and summer (88.9%). These results show the importance of training machine learning methods with season-specific data sets and comparing a large number of methods for specific applications.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Cataclysmic Variables in the Second Year of the Zwicky Transient Facility

Using a filter in the GROWTH Marshal based on color and the amplitude and timescale of variability, we have identified 372 objects as known or candidate cataclysmic variables (CVs) during the second year of the operation of the Zwicky Transient Facility. From the available difference imaging data, we found that 93 are previously confirmed CVs and 279 are strong candidates. Spectra of four of the candidates confirm them as CVs by the presence of Balmer emission lines, while one of the four has prominent He ii lines indicative of containing a magnetic white dwarf. Gaia EDR3 parallaxes are available for 154 of these systems, resulting in distances from 108–2096 pc and absolute magnitudes in the range of 7.5–15.0, with the largest number of candidates between 10.5 and 12.5. The total numbers are 21% higher than from the previous year of the survey with a greater number of distances available but a smaller percentage of systems close to the Galactic plane. Comparison of these findings with a machine-learning method of searching all the light curves reveals large differences in each data set related to the parameters involved in the search process.

79 ASTRONOMY AND ASTROPHYSICS↗