Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Training Analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,099 records · Page 61
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

A structural equation modeling approach to leveraging the power of extant sentiment analysis tools

Machine-derived sentiment analysis has become a pervasive and useful tool to address a wide array of issues in natural language processing. Leading technology companies such as Google now provide sentiment analysis tools (SATs) as readily accessible online products. Academic researchers develop and make available SATs to support the research enterprise. One of the major challenges with SATs is the inconsistencies in results among the various SATs. Consequently, the selection of a SAT for a specific purpose may significantly impact the application. This study addresses the foregoing problem by utilizing structural equation modeling to merge the outputs of SATs to develop a combined sentiment metric without the need for a labeled training dataset. This method is applicable to a wide range of text-based problems, is data-driven, and replicable. It was tested using three publicly available datasets and compared against seven different SATs. The results indicate that as a continous measure, the proposed method outperformed other SATs in the movie reviews and SemEval datasets, and achieved a tie for first place with IBM Watson on the Sentiment 140 dataset. Also, compared to the published major alternatives, the arithmetic mean solution, this approach performed better across these three datasets.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

ANALYSIS OF MULTIBUNCH SPECTRUM FOR AN UNEVEN BUNCH DISTRIBUTION IN A STORAGE RING

Modern storage-ring designs often require an uneven bunch distribution pattern. An uneven bunch fill pattern can result in complex structures for the beam current spectra. Particularly at high average beam currents, these complex current spectra need to be taken into account in concern of beam-dynamical effects. In this study, we analyze a beam current spectrum for various filling patterns with bunch trains and gaps. The characteristics of the resulting beam current spectra are illustrated and discussed.

Li, Rui↗
DOE OSTI2024

Massive all-atom analysis of 2D materials with quantum properties (Final report)

Improvements in microscopy have enabled the acquisition of data at a scale that is difficult to process manually, making automated machine learning approaches to analyzing experimental images essential. In this project, we developed and applied machine learning (ML) workflows for atomic resolution scanning transmission electron microscopy (STEM) images. This development included improving both methodology as well as generating user-friendly codes. We developed machine learning architectures which, after training, automatically identify the location and types of defects throughout a material. We used these data to produce class-averaged images of 2D atomic coordinates with up to 0.3 pm precision, uncovering the structure and oscillations of long-range strain fields around point defects in WSe 2-2x Te 2x . We also resolved a long-standing problem in this field in the training of ML models, a lack of labeled experimental data, by developing a cycle-GAN that transformed simulated-generated labeled data into labeled data indistinguishable from experiment and therefore suitable for training. This removed the remaining parts of the ML data processing workflow where human intervention was still critical and therefore a bottleneck to working at scale. Codes have been developed and released for this full machine learning workflow. ML approaches to partially automate STEM acquisition were also developed. Finally we applied ML and other advanced data processing methods to several materials science problems in two-dimensional materials, including studying the evolution of hyperuniformity with defect concentration in WSe2, understanding phase transformations in transition metal dichalcogenides during in-situ heating in the STEM, and exploring how 2D interfaces transform from twisted into aligned structures.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2022

Analysis of Inlier and Outlier Compounds with Respect to Artificial Neural Network Cetane Number Prediction Accuracy

Artificial neural networks (ANNs) are exceptional at forming non-linear correlations between multivariate input and target variables; however, they are often seen as a “black box” approach, since how ANNs form these correlations is somewhat ambiguous. Furthermore, the process underlying how ANNs learn from inlier and outlier samples within the input dataset is not fully understood. Intuitively, it is expected that training ANNs with inlier samples will increase prediction accuracy and training with outlier samples will reduce prediction accuracy; though, in practice, this is not always true. The present work identifies and analyzes inliers and outliers of existing experimental cetane number (CN) data encompassing a variety of compounds and compound groups. It also investigates how ANNs trained to predict CN perform with and without outliers included in the training data, and whether a relationship exists between inliers/outliers and ANN prediction accuracy across the whole dataset and for individual samples. Additionally, individual outlier compounds are analyzed, highlighting how they structurally differ from inlier compounds.

09 BIOMASS FUELS↗
DOE OSTI2023

The Field Epidemiology Training Program’s Contribution to Essential Public Health Functions in Seven National Public Health Institutes

Objective: This study explores how Field Epidemiology Training Programs (FETP) whose National Public Health Institutes (NPHI) are supported by U.S. Centers for Disease Control and Prevention (CDC) have contributed to strengthening essential public health functions. Methods: We conducted 96 semi-structured interviews with public health experts including NPHI staff, non-NPHI government staff, and staff from non-governmental and international organizations in Cambodia, Colombia, Liberia, Mozambique, Nigeria, Rwanda, and Zambia. We managed data using MAXQDA and employed direct content analysis to derive themes. Results: Three overarching themes emerged in relation to FETPs’ role within the NPHIs’ public health functions. These themes included contribution to improving country surveillance systems, role in providing leadership in outbreak responses, and strengthening countries’ and the NPHIs’ surveillance workforce capacity. Participants also shared challenges around FETPs’ implementation and suggestions for improvement. Conclusion: The results demonstrate the value of FETPs in strengthening public health systems through building workforce capacity and improving surveillance systems. By identifying the successes of FETPs in contributing to essential public health functions, our findings might inform current and future FETP implementation and its integration into NPHIs.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

A deterministic gradient-based approach to avoid saddle points

Abstract Loss functions with a large number of saddle points are one of the major obstacles for training modern machine learning (ML) models efficiently. First-order methods such as gradient descent (GD) are usually the methods of choice for training ML models. However, these methods converge to saddle points for certain choices of initial guesses. In this paper, we propose a modification of the recently proposed Laplacian smoothing gradient descent (LSGD) [Osher et al., arXiv:1806.06317 ], called modified LSGD (mLSGD), and demonstrate its potential to avoid saddle points without sacrificing the convergence rate. Our analysis is based on the attraction region, formed by all starting points for which the considered numerical scheme converges to a saddle point. We investigate the attraction region’s dimension both analytically and numerically. For a canonical class of quadratic functions, we show that the dimension of the attraction region for mLSGD is $\lfloor (n-1)/2\rfloor$ , and hence it is significantly smaller than that of GD whose dimension is $n-1$ .

Mathematics↗
DOE OSTI2022

Monitoring Fracture Saturation With Internal Seismic Sources and Twin Neural Networks

Seismic coda-wave analysis is a well-developed method for detecting subtle physical changes in complex media by measuring arrival times in the late-arriving energy from multiply scattered or reflected waves. However, a challenge arises when multiply scattered waves are not sufficiently separated in time from the direct arrivals to provide a clear coda wave train. Additional complications for monitoring changes in fracture systems arise when the signals originate from unsynchronized internal sources, such as natural or induced seismicity, from acoustic emission, or from transportable intra-fracture sources (chattering dust), that generate uncontrolled signals that vary in arrival time, amplitude and frequency content. Here, we use a twin neural network (TNN also known as a Siamese neural network) for dimensionality reduction to analyze signals from chattering dust to classify the fluid saturation state of a synthetic fracture system. The TNN with shared weights generates a low-dimensional representation of the data input by minimizing contrastive loss, serving as the input to a multiclass classifier that accurately classifies whether multiple fractures in a fracture system are fully saturated or partially saturated, or whether a change in saturation has occurred in different fractures in the system. Furthermore, these results show that information buried in unresolved codas from uncontrolled sources can be extracted using machine learning to monitor the evolution of fracture systems caused by physical and chemical processes even when the scattered and direct wave fields overlap.

58 GEOSCIENCES↗
DOE OSTI2024

Probing the evolution of fault properties during the seismic cycle with deep learning

We use seismic waves that pass through the hypocentral region of the 2016 M6.5 Norcia earthquake together with Deep Learning (DL) to distinguish between foreshocks, aftershocks and time-to-failure (TTF). Binary and N-class models defined by TTF correctly identify seismograms in test with > 90% accuracy. We use raw seismic records as input to a 7 layer CNN model to perform the classification. Here we show that DL models successfully distinguish seismic waves pre/post mainshock in accord with lab and theoretical expectations of progressive changes in crack density prior to abrupt change at failure and gradual postseismic recovery. Performance is lower for band-pass filtered seismograms (below 10 Hz) suggesting that DL models learn from the evolution of subtle changes in elastic wave attenuation. Tests to verify that our results indeed provide a proxy for fault properties included DL models trained with the wrong mainshock time and those using seismic waves far from the Norcia mainshock; both show degraded performance. Our results demonstrate that DL models have the potential to track the evolution of fault zone properties during the seismic cycle. If this result is generalizable it could improve earthquake early warning and seismic hazard analysis.

58 GEOSCIENCES↗
DOE OSTI2022

Stringent σ8 constraints from small-scale galaxy clustering using a hybrid MCMC + emulator framework

ABSTRACT We present a novel simulation-based hybrid emulator approach that maximally derives cosmological and Halo Occupation Distribution (HOD) information from non-linear galaxy clustering, with sufficient precision for DESI Year 1 (Y1) analysis. Our hybrid approach first samples the HOD space on a fixed cosmological simulation grid to constrain the high-likelihood region of cosmology + HOD parameter space, and then constructs the emulator within this constrained region. This approach significantly reduces the parameter volume emulated over, thus achieving much smaller emulator errors with fixed number of training points. We demonstrate that this combined with state-of-the-art simulations result in tight emulator errors comparable to expected DESI Y1 LRG sample variance. We leverage the new abacussummit simulations and apply our hybrid approach to CMASS non-linear galaxy clustering data. We infer constraints on σ8 = 0.762 ± 0.024 and fσ8(zeff = 0.52) = 0.444 ± 0.016, the tightest among contemporary galaxy clustering studies. We also demonstrate that our fσ8 constraint is robust against secondary biases and other HOD model choices, a critical first step towards showcasing the robust cosmology information accessible in non-linear scales. We speculate that the additional statistical power of DESI Y1 should tighten the growth rate constraints by at least another 50–60 ${{\ \rm per\ cent}}$, significantly elucidating any potential tension with Planck. We also address the ‘lensing is low’ tension, which we find to be in the same direction as a potential tension in fσ8. We show that the combined effect of a lower fσ8 and environment-based bias accounts for approximately $50{{\ \rm per\ cent}}$ of the discrepancy.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Synthetic Infrasound Data for Machine Learning Detectors

Synthetic data is a powerful tool to generate large amounts of training data for machine learning models. The methods outlined in this report will be used to retrain the deep learning classifier for increased accuracy. Synthetic data will be useful to address the natural class imbalance between the different categories in the original ML work. Additionally, these tools will be applied for a variety of signal analysis methods that would use signals with a known signal-to-noise ratio for validation and testing.

58 GEOSCIENCES↗
DOE OSTI2021

X-Ray Spectra and Multiwavelength Machine Learning Classification for Likely Counterparts to Fermi 3FGL Unassociated Sources

We conduct X-ray spectral fits on 184 likely counterparts to Fermi-LAT 3FGL unassociated sources. Characterization and classification of these sources allows for more complete population studies of the high-energy sky. Most of these X-ray spectra are well fit by an absorbed power-law model, as expected for a population dominated by blazars and pulsars. A small subset of seven X-ray sources have spectra unlike the power law expected from a blazar or pulsar and may be linked to coincident stars or background emission. We develop a multiwavelength machine learning classifier to categorize unassociated sources into pulsars and blazars using gamma-ray and X-ray observations. Training a random forest (RF) procedure with known pulsars and blazars, we achieve a cross-validated classification accuracy of 98.6%. Applying the RF routine to the unassociated sources returned 126 likely blazar candidates (defined as P {sub bzr} ≥ 90%) and five likely pulsar candidates (P {sub bzr} ≤ 10%). Our new X-ray spectral analysis does not drastically alter the RF classifications of these sources compared to previous works, but it builds a more robust classification scheme and highlights the importance of X-ray spectral fitting. Our procedure can be further expanded with UV, visual, or radio spectral parameters or by measuring flux variability.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Blackout Diffusion: Generative Diffusion Models in Discrete-State Spaces

Typical generative diffusion models rely on a Gaussian diffusion process for training the backward transformations, which can then be used to generate samples from Gaussian noise. However, real world data often takes place in discrete-state spaces, including many scientific applications. Here, we develop a theoretical formulation for arbitrary discrete-state Markov processes in the forward diffusion process using exact (as opposed to variational) analysis. We relate the theory to the existing continuous-state Gaussian diffusion as well as other approaches to discrete diffusion, and identify the corresponding reverse-time stochastic process and score function in the continuous-time setting, and the reverse-time mapping in the discrete-time setting. As an example of this framework, we introduce “Blackout Diffusion”, which learns to produce samples from an empty image instead of from noise. Numerical experiments on the CIFAR-10, Binarized MNIST, and CelebA datasets confirm the feasibility of our approach. Generalizing from specific (Gaussian) forward processes to discrete-state processes without a variational approximation sheds light on how to interpret diffusion models, which we discuss.

Santos, Javier E.↗
DOE OSTI2025

Fluidized Bed Gasification For Conversion of Biomass and Waste Materials to Renewable Hydrogen

This project was undertaken in order to study the potential for hydrogen production, at low cost, from mixtures of biomass and municipal solid waste (MSW). This approach allows for the production of hydrogen with a very low fossil carbon burden, while taking advantage of tipping fees (associated with MSW) to improved process economics. The team sourced three primary feedstocks (wood, MSW, and waste plastics) and characterized them comprehensively using established techniques with a long track record in the field of gasification. All three primary feedstocks were highly reactive and lost most of their mass during initial devolatilization. The production of tars, including heavy tars, was quite high, and was most problematic in the case of the MSW and Waste Plastics feedstocks. Little practical difference was identified between the MSW and Waste Plastics materials, and the addition of bed-forming materials (dolomite and brown alumina) to the feedstocks was found to reduce production of tars during devolatilization under thermogravimetric analysis and/or Fischer Assay conditions. A series of four tests in a lab-scale bubbling-fluidized-bed gasifier, at 50 psig of pressure and about 825 C, confirmed these findings. Pellet feedstocks, broken into fragments, were used for these tests, and pellets comprised of 50% MSW and 50% biomass were found to be the best option in terms of fossil carbon burden, economic potential, and gasification characteristics. Tests were then undertaken in a pilot-scale gasifier facility based on the GTI U-Gas fluidized-bed gasification technology. The feedstock handling train of the 20 TPD U-Gas pilot-scale gasifier located in Des Plaines, IL, was operated under simulated gasification conditions, and the 50/50 pellets were found to be very robust and unproblematic. An improved design for the forward end of the feedstock injection screw of the gasifier was developed and installed. The design approach was based on improved passive cooling of the front-most shroud at the end of the screw, since this approach was found in comprehensive modeling studies to be more than sufficient to accomplish the project objectives associated with this phase of the work, while also avoiding thermal gradients that could have caused heat-stress-induced damage to the refractory around the feedstock inlet if an active cooling approach had been applied. Careful technoeconomic analysis (TEA) of two possible 1000 TPD facilities was carried out. The TEA of conversion of MSW with corn stover in one case, and MSW with woody feedstock in the other case, showed that both had the potential to provide hydrogen at about $1/kg (minimum selling price, 2018 dollar basis). Of the two TEA cases, the one that was based on the conversion of wood in the southeastern USA was found to have slightly better economic potential. The other TEA case was based on a real location in Nebraska and called for corn stover feedstock conversion along with MSW. An underserved communities outreach program plan was developed in cooperation with personnel from the Nebraska Public Power District.

08 HYDROGEN↗
DOE OSTI2025

Prototype-Wise Sensitivity Analysis of Urban Building Energy Simulation Surrogate Modeling Accuracy

Urban Building Energy Modeling (UBEM) is an important reference for urban energy-related policymaking. Because of the significant impact of urban microclimates on the energy simulation, UBEM requires simulations of many microclimate-prototype pairs. Surrogate modeling is commonly used to reduce the cost of simulation computations. In UBEM surrogate modeling, it is important to determine the percentage of microclimates related to a prototype used for generating surrogate model training data. This study analyzes the prototype-wise variations and sensitivities of surrogate model estimation accuracy to the microclimate sampling ratios. The results of the study can help determine the number of simulations used for generating surrogate modeling data, avoid redundant simulations, and reduce the computational cost for UBEM surrogate modeling and its time.

Pan, Xiyu↗
DOE OSTI2024

Comparison of point cloud and image-based models for calorimeter fast simulation

Score based generative models are a new class of generative models that have been shown to accurately generate high dimensional calorimeter datasets. Recent advances in generative models have used images with 3D voxels to represent and model complex calorimeter showers. Point clouds, however, are likely a more natural representation of calorimeter showers, particularly in calorimeters with high granularity. Furthermore, point clouds preserve all of the information of the original simulation, more naturally deal with sparse datasets, and can be implemented with more compact models and data files. In this work, two state-of-the-art score based models are trained on the same set of calorimeter simulation and directly compared.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

Automated Programmable Logic Controller Memory Forensics Using RGB Image Analysis and Deep Learning

The introduction of Industry 4.0 and Internet-based technologies has enhanced industrial control system operations but have inadvertently increased their vulnerabilities to cyber attacks. When an industrial control system is compromised, security analysts need to identify the root cause quickly to start the recovery process and develop mitigation strategies. Memory forensics is critical in the incident analysis process to ascertain what occurred. Approaches for analyzing the persistent memory in industrial control devices are limited and almost nonexistent for volatile memory. This chapter proposes an automated methodology for programmable logic controller memory dump analysis using computer vision and deep learning techniques. The methodology converts the sequences of bytes in a programmable logic controller memory dump to red-green-blue pixels and employs a deep learning model that learns the underlying patterns and features of pre-labeled forensic artifacts in images and segments them into distinct regions. The trained model is employed to automatically segment new memory images and identify forensic artifacts. Evaluation of the methodology on a Schneider Electric Modicon M221 programmable logic controller under code injection and code modification attacks demonstrates its ability to detect attack artifacts in memory dumps.

Asmar Awad, Rima [ORNL] (ORCID:0000000233407742)↗
DOE OSTI2024

Adaptive Data-Driven Deep-Learning Surrogate Model for Frontal Polymerization in Dicyclopentadiene

Frontal polymerization (FP) is a self-sustaining curing process that enables rapid and energy-efficient manufacturing of thermoset polymers and composites. Computational methods conventionally used to simulate the FP process are time-consuming, and repeating simulations are required for sensitivity analysis, uncertainty quantification, or optimization of the manufacturing process. Here, in this work, we develop an adaptive surrogate deep-learning model for FP of dicyclopentadiene (DCPD), which predicts the evolution of temperature and degree of cure orders of magnitude faster than the finite-element method (FEM). The adaptive algorithm provides a strategy to select training samples efficiently and save computational costs by reducing the redundancy of FEM-based training samples. The adaptive algorithm calculates the residual error of the FP governing equations using automatic differentiation of the deep neural network. A probability density function expressed in terms of the residual error is used to select training samples from the Sobol sequence space. The temperature and degree of cure evolution of each training sample are obtained by a 2D FEM simulation. The adaptive method is more efficient and has a better prediction accuracy than the random sampling method. With the well-trained surrogate neural network, the FP characteristics (front speed, shape, and temperature) can be extracted quickly from the predicted temperature and degree-of-cure fields.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

High-throughput screening of tribological properties of monolayer films using molecular dynamics and machine learning

Monolayer films have shown promise as a lubricating layer to reduce friction and wear of mechanical devices with separations on the nanoscale. These films have a vast design space with many tunable properties that can affect their tribological effectiveness. For example, terminal group chemistry, film composition, and backbone chemistry can all lead to films with significantly different tribological properties. This design space, however, is very difficult to explore without a combinatorial approach and an automatable, reproducible, and extensible workflow to screen for promising candidate films. Here, using the Molecular Simulation Design Framework (MoSDeF), a combinatorial screening study was performed to explore 9747 unique monolayer films (116 964 total simulations) and a machine learning (ML) model using a random forest regressor, an ensemble learning technique, to explore the role of terminal group chemistry and its effect on tribological effectiveness. The most promising films were found to contain small terminal groups such as cyano and ethylene. The ML model was subsequently applied to screen terminal group candidates identified from the ChEMBL small molecule library. Approximately 193 131 unique film candidates were screened with approximately a five order of magnitude speed-up in analysis compared to simulation alone. The ML model was thus able to be used as a predictive tool to greatly speed up the initial screening of promising candidate films for future simulation studies, suggesting that computational screening in combination with ML can greatly increase the throughput in combinatorial approaches to generate in silico data and then train ML models in a controlled, self-consistent fashion.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗