Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “generative machine learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,783 records · Page 99

Feedback Control Approaches for Restoration of Power Grids from Blackouts

The automated restoration of power systems with variable energy resources is a timely problem to tackle. Automated restoration advice can support operators in deciding on strategic actions to restore power grids from a blackout with a mix of conventional and renewable generation resources. To this end, this paper frames the restoration process of power grids with solar resources as a nonlinear dynamic model with algebraic constraints in discrete time which is steered by feedback control loops. We discuss two feedback-control strategies based on greedy and reinforcement learning algorithms, and contrast their performance with restoration plans generated by a mixed-integer linear program. We found that the reinforcement learning algorithm infers restoration actions faster than the greedy one. However, the tuning process of the reinforcement learning parameters is slower than for the greedy one.

machine learning↗

Polymers for Extreme Conditions Designed Using Syntax-Directed Variational Autoencoders

We report the design/discovery of new materials is highly nontrivial owing to the near-infinite possibilities of material candidates and multiple required property/performance objectives. Thus, machine learning tools are now commonly employed to virtually screen material candidates with desired properties by learning a theoretical mapping from material-to-property space, referred to as the forward problem. However, this approach is inefficient and severely constrained by the candidates that the human imagination can conceive. Thus, in this work on polymers, we tackle the materials discovery challenge by solving the inverse problem: directly generating candidates that satisfy desired property/performance objectives. We utilize syntax-directed variational autoencoders (VAE) in tandem with Gaussian process regression (GPR) models to discover polymers expected to be robust under three extreme conditions: (1) high temperatures, (2) high electric field, and (3) high temperature and high electric field, useful for critical structural, electrical, and energy storage applications. This approach to learn from and augment) human ingenuity is general and can be extended to discover polymers with other targeted properties and performance measures.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Improving E3SM Land Model Photosynthesis Parameterization via Satellite SIF, Machine Learning, and Surrogate Modeling

The parameterization of key photosynthesis parameters is one of the key uncertain sources in modeling ecosystem gross primary productivity (GPP). Solar-induced chlorophyll fluorescence (SIF) offers a good proxy for GPP since it marks the actual process of photosynthesis; while machine learning (ML) provides a robust approach to model the GPP-SIF relationship. Here, we trained the boosted regressing tree (BRT) and the Random Forest ML models with Greenhouse Gases Observing Satellite SIF data and in situ GPP observations from 49 eddy covariance towers. These trained ML GPP-SIF models were fed into the Energy Exascale Earth System Model (E3SM) Land Model (ELM) to generate ELM-simulated global SIF estimates, which were then benchmarked against satellite SIF observations with a surrogate modeling approach. Our results indicated good modeling performance of the ML-based GPP-SIF relationship. The ELM model when fed with the ML GPP-SIF models also can well predict the spatial-temporal variations in SIF. We also found high model accuracy for the surrogate modeling. Model parameter sensitivity analysis suggested that the fraction of leaf nitrogen in RuBisCO (flnr) is the most sensitive parameter to the SIF; other sensitive parameters include the Ball-Berry stomatal conductance slope (mbbopt) and the vcmax entropy (vcmaxse). The posterior uncertainty in simulated GPP was greatly reduced after benchmarking, and the model produced improved spatial patterns of mean GPP relative to FLUXCOM GPP. Our integrated approach provides a new avenue for improving land models and using remote-sensing SIF, which can be further improved in the future with more ground- and satellite-based observations.

54 ENVIRONMENTAL SCIENCES↗

Developing Scenario‐Based Strategies for Health, Climate, and Environmental Preparedness: The One Health, One Earth Approach

Climate change amplifies many threats to human health. Despite advances in understanding climate change dynamics and impacts, there remains a critical gap in translating scientific knowledge into equitable, and community-driven health interventions. The inaugural One Earth, One Health workshop sought to explore this gap through human-centered design exercises involving interdisciplinary researchers from climate and Earth sciences, engineering, epidemiology, microbiology, and environmental health. Although participants did not co-develop solutions with affected communities, they used stakeholder role-playing to guide ideation and lay groundwork for actionable plans. Through these methods, participants identified community needs and proposed prototype solutions to alleviate health threats exacerbated by global environmental change. Prototypes were organized around infectious diseases, extreme weather, and air quality, as illustrative themes rather than an exhaustive set of risks. Key solutions included strategies for anticipatory systems and early warning (e.g., integrating environmental signals with health data), inclusive communication and infrastructure needs for responding to extreme weather events, and integrated platforms visualizing air quality trends to support tailored, context-aware guidance beyond one-size-fits-all alerts. The workshop highlighted opportunities such as leveraging machine learning, Earth observation, and real-time surveillance to protect communities, but also noted barriers including data quality, technological redundancy, privacy, and governance challenges. Additionally, participants emphasized the need for interdisciplinary teams capable of collaborating across sectors, breaking down silos and addressing gaps in training and education. Overall, the workshop illustrates how process-driven, human-centered approaches can help surface user needs and generate testable prototype concepts, while underscoring the importance of direct community partnership for implementation.

Abadi, Azar M. [University of Alabama, Birmingham,↗

Training data selection for accuracy and transferability of interatomic potentials

Abstract Advances in machine learning (ML) have enabled the development of interatomic potentials that promise the accuracy of first principles methods and the low-cost, parallel efficiency of empirical potentials. However, ML-based potentials struggle to achieve transferability, i.e., provide consistent accuracy across configurations that differ from those used during training. In order to realize the promise of ML-based potentials, systematic and scalable approaches to generate diverse training sets need to be developed. This work creates a diverse training set for tungsten in an automated manner using an entropy optimization approach. Subsequently, multiple polynomial and neural network potentials are trained on the entropy-optimized dataset. A corresponding set of potentials are trained on an expert-curated dataset for tungsten for comparison. The models trained to the entropy-optimized data exhibited superior transferability compared to the expert-curated models. Furthermore, the models trained to the expert-curated set exhibited a significant decrease in performance when evaluated on out-of-sample configurations.

36 MATERIALS SCIENCE↗

Particle hit clustering and identification using point set transformers in liquid argon time projection chambers

Liquid argon time projection chambers are often used in neutrino physics and dark-matter searches because of their high spatial resolution. The images generated by these detectors are extremely sparse, as the energy values detected by most of the detector are equal to 0, meaning that despite their high resolution, most of the detector is unused in a particular interaction. Instead of representing all of the empty detections, the interaction is usually stored as a sparse matrix, a list of detection locations paired with their energy values. Traditional machine learning methods that have been applied to particle reconstruction such as convolutional neural networks (CNNs), however, cannot operate over data stored in this way and therefore must have the matrix fully instantiated as a dense matrix. Operating on dense matrices requires a lot of memory and computation time, in contrast to directly operating on the sparse matrix. We propose a machine learning model using a point set neural network that operates over a sparse matrix, greatly improving both processing speed and accuracy over methods that instantiate the dense matrix, as well as over other methods that operate over sparse matrices. Compared to competing state-of-the-art methods, our method improves classification performance by 14%, segmentation performance by more than 22%, while taking 80% less time and using 66% less memory. Compared to state-of-the-art CNN methods, our method improves classification performance by more than 86%, segmentation performance by more than 71%, while reducing runtime by 91% and reducing memory usage by 61%.

calibration and fitting methods↗

Nonparametric Bayesian Modeling for Automated Database Schema Matching

The problem of merging databases arises in many government and commercial applications. Schema matching, a common first step, identifies equivalent fields between databases. We introduce a schema matching framework that builds nonparametric Bayesian models for each field and compares them by computing the probability that a single model could have generated both fields. Our experiments show that our method is more accurate and faster than the existing instance-based matching algorithms in part because of the use of nonparametric Bayesian models.

Bayesian↗

Data-Driven Day-Ahead PV Estimation Using Autoencoder-LSTM and Persistence Model

Inherent variability in photovoltaic (PV) and associated impacts on power systems is a challenging problem for both the PV owners and the grid operators. Existing statistical and machine learning algorithms typically work well for weather conditions similar to historical data. Furthermore, uncertain weather conditions pose a great challenge to the estimation accuracy of the estimation models. With the enhanced integration of intelligent electronic devices and the realization of associated automation in the power grid, renewable energy data is becoming more accessible, which can be utilized by deep learning models and improve the PV power generation estimation accuracy. In this paper, a hybrid deep learning model driven by external weather data is proposed to do day-ahead PV output forecasting at 15-minute-interval. The proposed model is motivated by the recent advancement of Long-Short-Term-Memory (LSTM) networks and AutoEncoder (AE), which estimates uncertainties in sequence while making the prediction for complex weather conditions. Meanwhile, the persistence model (PM) is used to predict continuous sunny weather conditions. The forecasting result is validated with data from multiple locations

42 ENGINEERING↗

Bulk Handling Facility Modeling and Simulation for Safeguards Analysis

The Separation and Safeguards Performance Model (SSPM) uses MATLAB/Simulink to provide a tool for safeguards analysis of bulk handling nuclear processing facilities. Models of aqueous and electrochemical reprocessing, enrichment, fuel fabrication, and molten salt reactor facilities have been developed to date. These models are used for designing the overall safeguards system, examining new safeguards approaches, virtually testing new measurement instrumentation, and analyzing diversion scenarios. The key metrics generated by the models include overall measurement uncertainty and detection probability for various material diversion or facility misuse scenarios. Safeguards modeling allows for rapid and cost-effective analysis for Safeguards by Design. The models are currently being used to explore alternative safeguards approaches, including more reliance on process monitoring data to reduce the need for destructive analysis that adds considerable burden to international safeguards. Machine learning techniques are being applied, but these techniques need large amounts of data for training and testing the algorithms. The SSPM can provide that training data. This paper will describe the SSPM and its use for applying both traditional nuclear material accountancy and newer machine learning options.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

Electric Grid Security (EGS) FY24 Annual Report

Sandia’s Electric Grid Security program advances a national vision of a secure, resilient, and affordable electric system for all users. Our achievements reflect a strategic approach combining technology development; modeling, simulation, and data analytics; and partnered demonstrations and outreach to further the adoption of advanced grid and storage technologies. Our FY24 efforts leverage the strengths of our partnerships—spanning Sandia’s core science and technology competencies as well as external technology leaders—to develop the solutions today which enable the grid of tomorrow. Key accomplishments in this report that support our strategy span our technical program areas and include: • The advancement of energy storage technologies, including creation of a national Long Duration Energy Storage Consortium; • Applications of artificial intelligence and machine learning to enhanced grid operations and planning; • Development of solid-state power conversion technologies and a new medium-voltage research lab; • New technologies to assess wildfire vulnerabilities and mitigate potential impacts; • Advanced applications of new cybersecurity technologies with industry partners; • Contributions to understanding the impacts of electromagnetic pulses and geomagnetic disturbances on grid components; and • Digital twin development for hybrid microgrids with multiple generators, storage, and loads.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Improving Text Classification with Large Language Model-Based Data Augmentation

Large Language Models (LLMs) such as ChatGPT possess advanced capabilities in understanding and generating text. These capabilities enable ChatGPT to create text based on specific instructions, which can serve as augmented data for text classification tasks. Previous studies have approached data augmentation (DA) by either rewriting the existing dataset with ChatGPT or generating entirely new data from scratch. However, it is unclear which method is better without comparing their effectiveness. This study investigates the application of both methods to two datasets: a general-topic dataset (Reuters news data) and a domain-specific dataset (Mitigation dataset). Our findings indicate that: 1. ChatGPT generated new data consistently enhanced model’s classification results for both datasets. 2. Generating new data generally outperforms rewriting existing data, though crafting the prompts carefully is crucial to extract the most valuable information from ChatGPT, particularly for domain-specific data. 3. The augmentation data size affects the effectiveness of DA; however, we observed a plateau after incorporating 10 samples. 4. Combining the rewritten sample with new generated sample can potentially further improve the model’s performance.

97 MATHEMATICS AND COMPUTING↗

Application of machine learning to estimate fireball characteristics and their uncertainty from infrared spectral data

Experiments or events involving high explosives (HE) can be monitored remotely by infrared (IR) sensors to gather information about the configuration or materials involved in the device. Researchers at the Air Force Institute of Technology (AFIT) developed a phenomenological model for HE fireball spectra in the IR range that allows for parameters to be extracted from Fourier transform infrared (FTIR) data. This model includes parameters tied to physical characteristics of the fireball: temperature, size, soot, and gas concentrations. Previous works have sought to recover these parameters by the fitting of either whole spectra or select wavenumber bands to this phenomenological model. Difficulties arise due to the complex relationships between the parameters to be fit. Uncertainty quantification of the estimated fireball parameters is also problematic since HE experiments do not have any ground truth information on the parameters. It is suggested that artificial neural network (ANN) based approaches may be well suited to this problem, because of their ability to capture complex and highly nonlinear relationships. As such, this work seeks to explore the efficacy of deep artificial neural networks (DNNs) for this problem of parameter recovery from spectra and to also investigate the uncertainty of recovering the fireball parameters from FTIR data. Networks are designed using the hyperparameter optimization tool Hyperopt and trained/tested on artificial data generated using the phenomenological model developed by AFIT. The results of applying the network to the artificial data set are compared to a physics-based band approach that uses a selected number of bands based on their physical properties. Information on the uncertainty of estimating parameters from remotely sensed experimental data is obtained by treating the accuracy of the DNN model on artificial data as an upper bound and by examining the impact of emissivity due to soot on parameter estimation error; the results for artificial data are likely to be optimistic as compared to recovering parameters from experimental data.

42 ENGINEERING↗

PDF Entity Annotation Tool (PEAT)

While different text mining approaches – including the use of Artificial Intelligence (AI) and other machine based methods - continue to expand at a rapid pace, the tools used by researchers to create the labeled datasets required for training, modeling, and evaluation remain rudimentary. Labeled datasets contain the target attributes the machine is going to learn; for example, training an algorithm to delineate between images of a car or truck would generally require a set of images with a quantitative description of the underlying features of each vehicle type. Development of labeled textual data that can be used to build natural language machine learning models for scientific literature is not currently integrated into existing manual workflows used by domain experts. Published literature is rich with important information, such as different types of embedded text, plots, and tables that can all be used as inputs to train ML/natural language processing (NLP) models, when extracted and prepared in machine readable formats. Currently, both normalized data extraction of use to domain experts and extraction to support development of ML/NLP models are labor intensive and cumbersome manual processes. Automatic extraction of data and information from formats such as PDFs that are optimized for layout and human readability, not machine readability. The PDF (Portable Document Format) Entity Annotation Tool (PEAT) was developed with the goal of allowing users to annotate publications within their current print format, while also allowing those annotations to be captured in a machine-readable format. One of the main issues with traditional annotation tools is that they require transforming the PDF into plain text to facilitate the annotation process. While doing so lessens the technical challenges of annotating data, the user loses all structure and provenance that was inherent in the underlying PDF. Also, textual data extraction from PDFs can be an error prone process. Challenges include identifying sequential blocks of text and a multitude of document formats (multiple columns, font encodings, etc.). As a result of these challenges, using existing tools for development of NLP/ML models directly from PDFs is difficult because the generated outputs are not interoperable. We created a system that allows annotations to be completed on the original PDF document structure, with no plain text extraction. The result is an application that allows for easier and more accurate annotations. In addition, by including a feature that grants the user the ability to easily create a schema, we have developed a system that can be used to annotate text for different domain-centric schemas of relevance to subject matter experts. Different knowledge domains require distinct schemas and annotation tags to support machine learning.

97 MATHEMATICS AND COMPUTING↗

Adaptive enhanced sampling by force-biasing using neural networks

A machine learning assisted method is presented for molecular simulation of systems with rugged free energy landscapes. The method is general and can be combined with other advanced sampling techniques. In the particular implementation proposed here, it is illustrated in the context of an adaptive biasing force approach where, rather than relying on discrete force estimates, one can resort to a self-regularizing artificial neural network to generate continuous, estimated generalized forces. By doing so, the proposed approach addresses several shortcomings common to adaptive biasing force and other algorithms. Specifically, the neural network enables smooth estimates of generalized forces in sparsely sampled regions, force estimates in previously unexplored regions, and continuous force estimates with which to bias the simulation, as opposed to biases generated at specific points of a discrete grid. The usefulness of the method is illustrated with three different examples, chosen to highlight the wide range of applicability of the underlying concepts. In all three cases, the new method is found to enhance considerably the underlying traditional adaptive biasing force approach. Furthermore, the method is also found to provide improvements over previous implementations of neural network assisted algorithms.

97 MATHEMATICS AND COMPUTING↗

Predicting Atomistic Transitions with Transformers

Accurate knowledge of the atomistic transition pathways in materials and material surfaces is crucial for many material science problems. However, conventional simulation techniques used to find these transitions are extremely computationally intensive. Even with large-scale, accelerated material simulations, the computational cost constrains the applicable domain in practice. Machine learning models, with the potential to learn the complex emergent behaviors governing atomistic transitions as a fast surrogate model, have great promise to predict transitions with a vastly reduced computational cost. Here, we demonstrate how transformers can be trained to predict atomistic transitions in nano-clusters. We show how we evaluate physical validity of the predictions and how a multitude of additional, different microstates can be generated by slightly varying the data provided to the model.

36 MATERIALS SCIENCE↗

THE SYNTHETIC-OVERSAMPLING METHOD: USING PHOTOMETRIC COLORS TO DISCOVER EXTREMELY METAL-POOR STARS

Extremely metal-poor (EMP) stars ([Fe/H] ≤ −3.0 dex) provide a unique window into understanding the first generation of stars and early chemical enrichment of the universe. EMP stars are exceptionally rare, however, and the relatively small number of confirmed discoveries limits our ability to exploit these near-field probes of the first ∼500 Myr after the Big Bang. Here, a new method to photometrically estimate [Fe/H] from only broadband photometric colors is presented. I show that the method, which utilizes machine-learning algorithms and a training set of ∼170,000 stars with spectroscopically measured [Fe/H], produces a typical scatter of ∼0.29 dex. This performance is similar to what is achievable via low-resolution spectroscopy, and outperforms other photometric techniques, while also being more general. I further show that a slight alteration to the model, wherein synthetic EMP stars are added to the training set, yields the robust identification of EMP candidates. In particular, this synthetic-oversampling method recovers ∼20% of the EMP stars in the training set, at a precision of ∼0.05. Furthermore, ∼65% of the false positives from the model are very metal-poor stars ([Fe/H] ≤ −2.0 dex). The synthetic-oversampling method is biased toward the discovery of warm (∼F-type) stars, a consequence of the targeting bias from the Sloan Digital Sky Survey/Sloan Extension for Galactic Understanding survey. This EMP selection method represents a significant improvement over alternative broadband optical selection techniques. The models are applied to >12 million stars, with an expected yield of ∼600 new EMP stars, which promises to open new avenues for exploring the early universe.

79 ASTRONOMY AND ASTROPHYSICS↗

Computational and empirical studies predict Mycobacterium tuberculosis -specific T cells as a biomarker for infection outcome

Identifying biomarkers for tuberculosis (TB) is an ongoing challenge in developing immunological correlates of infection outcome and protection. Biomarker discovery is also necessary for aiding design and testing of new treatments and vaccines. To effectively predict biomarkers for infection progression in any disease, including TB, large amounts of experimental data are required to reach statistical power and make accurate predictions. We took a two-pronged approach using both experimental and computational modeling to address this problem. We first collected 200 blood samples over a 2-year period from 28 non-human primates (NHP) infected with a low dose of Mycobacterium tuberculosis . We identified T cells and the cytokines that they were producing (single and multiple) from each sample along with monkey status and infection progression data. Machine learning techniques were used to interrogate the experimental NHP datasets without identifying any potential TB biomarker. In parallel, we used our extensive novel NHP datasets to build and calibrate a multi-organ computational model that combines what is occurring at the site of infection (e.g., lung) at a single granuloma scale with blood level readouts that can be tracked in monkeys and humans. We then generated a large in silico repository of in silico granulomas coupled to lymph node and blood dynamics and developed an in silico tool to scale granuloma level results to a full host scale to identify what best predicts Mycobacterium tuberculosis (Mtb) infection outcomes. The analysis of in silico blood measures identifies Mtb -specific frequencies of effector T cell phenotypes at various time points post infection as promising indicators of infection outcome. As a result, we emphasize that pairing wetlab and computational approaches holds great promise to accelerate TB biomarker discovery.

59 BASIC BIOLOGICAL SCIENCES↗

Representing Small Commercial Building Faults in EnergyPlus, Part II: Model Validation

Automated fault detection and diagnosis (AFDD) tools based on machine-learning algorithms hold promise for lowering cost barriers for AFDD in small commercial buildings; however, access to high-quality training data for such algorithms is often difficult to obtain. To fill the gap in this research area, this study covers the development (Part I) and validation (Part II) of fault models that can be used with the building energy modeling software EnergyPlus® and OpenStudio® to generate a cost-effective training data set for developing AFDD algorithms. Part II (this paper) first presents a methodology of validating fault models with OpenStudio and then presents validation results, which are compared against measurements from a reference building. We discuss the results of our experiments with eight different faults in the reference building (a total of 39 different baseline and faulted scenarios), including our methodology for using fault models along with the reference building model to simulate the same faulted scenarios. Then, we present validation of the fault models by comparing results of simulations and experiments either quantitatively or qualitatively.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗