Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “machine learning optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,261 records · Page 70
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Transmission-distribution long-term volt-var planning considering reactive power support capability of distributed PV

High penetration of grid-edge, inverter-based photovoltaic (PV) can cause significant voltage fluctuations not only at the distribution but also at the sub-transmission levels due to PV output intermittency. Traditional reactive power planning approaches do not consider intermittency, nor the possibility of coordinating the control of existing and future volt-ampere reactive resources. This paper proposes a reactive power planning tool for sub-transmission systems to mitigate voltage violations and fluctuations caused by high PV penetration and intermittency with a minimum investment cost. The planning tool coordinates with an optimization-based volt-var operational tool for: a) modeling the coordination of all existing var assets in both sub-transmission and distribution systems to reduce the need of new equipment, and b)selecting a set of scenarios with voltage violations, derived from PV intermittency c) testing the final investment decision. The tool obtains an investment need for each intermittency scenario with a proposed optimal power-flow framework with efficient techniques to handle a high number of discrete variables. Two options are provided for final planning decision: i) a conservative direct combination of investment need solutions and ii) a machine learning-based selection of representative investment needs at most time steps. The final investment decision options are verified using a realistic large-scale sub-transmission system and 5-minute PV and load data. The results show a significant voltage performance improvement with a lower investment cost for additional var equipment compared to conventional approaches.

14 SOLAR ENERGY↗
DOE OSTI2020

TensorFlow Quantum: A Software Framework for Quantum Machine Learning

We introduce TensorFlow Quantum (TFQ), an open source library for the rapid prototyping of hybrid quantum-classical models for classical or quantum data. This framework offers high-level abstractions for the design and training of both discriminative and generative quantum models under TensorFlow and supports high-performance quantum circuit simulators. We provide an overview of the software architecture and building blocks through several examples and review the theory of hybrid quantum-classical neural networks. We illustrate TFQ functionalities via several basic applications including supervised learning for quantum classification, quantum control, simulating noisy quantum circuits, and quantum approximate optimization. Moreover, we demonstrate how one can apply TFQ to tackle advanced quantum learning tasks including meta-learning, layerwise learning, Hamiltonian learning, sampling thermal states, variational quantum eigensolvers, classification of quantum phase transitions, generative adversarial networks, and reinforcement learning. We hope this framework provides the necessary tools for the quantum computing and machine learning research communities to explore models of both natural and artificial quantum systems, and ultimately discover new quantum algorithms which could potentially yield a quantum advantage.

Broughton, Michael↗
DOE OSTI2023

Machine Learning-Based Model Predictive Control of Two-Time-Scale Systems

In this study, we present a general form of nonlinear two-time-scale systems, where singular perturbation analysis is used to separate the dynamics of the slow and fast subsystems. Machine learning techniques are utilized to approximate the dynamics of both subsystems. Specifically, a recurrent neural network (RNN) and a feedforward neural network (FNN) are used to predict the slow and fast state vectors, respectively. Moreover, we investigate the generalization error bounds for these machine learning models approximating the dynamics of two-time-scale systems. Next, under the assumption that the fast states are asymptotically stable, our focus shifts toward designing a Lyapunov-based model predictive control (LMPC) scheme that exclusively employs the RNN to predict the dynamics of the slow states. Additionally, we derive sufficient conditions to guarantee the closed-loop stability of the system under the sample-and-hold implementation of the controller. A nonlinear chemical process example is used to demonstrate the theory. In particular, two RNN models are constructed: one to model the full two-time-scale system and the other to predict solely the slow state vector. Both models are integrated within the LMPC scheme, and we compare their closed-loop performance while assessing the computational time required to execute the LMPC optimization problem.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

A systematic comparison of machine learning methods for modeling of dynamic processes applied to combustion emission rate modeling

Ten established, data-driven dynamic algorithms are surveyed and a practical guide for understanding these methods generated. Existing Python programming packages for implementing each algorithm are acknowledged, and the model equations necessary for prediction are presented. A case study on a coal-fired power plant’s NO x emission rates is performed, directly comparing each modeling method’s performance on a mutual system. Each model is evaluated by its root mean squared error (RMSE) on out-of-sample future horizon predictions. Optimal hyperparameters are identified using either an exhaustive search or genetic algorithm. The top five model structures of each method are used to recursively predict future NO x emission rates over a 60-step time horizon. The RMSE at each future timestep is determined, and the recursive output prediction trends compared against measurements in time. The GRU neural network is identified as the best candidate for representing the system, demonstrating accurate and stable predictions across the future horizon by all considered models, while satisfactory performance was observed in several of the ARX/NARX formulations. Finally, these efforts have contributed 1) a concise resource of multiple proven dynamic machine learning methods, 2) a practical guide explaining the use of these methods, effectively lowering the “barrier-to-entry” of deploying such models in control systems, 3) a comparison study evaluating each method’s performance on a mutual system, 4) demonstration of accurate multi-timestep emissions modeling suitable for systems-level control, and 5) generalizable results demonstrating the suitability of each method for prediction over a multi-step future horizon to other complex dynamic systems.

42 ENGINEERING↗
DOE OSTI2025

Accessible, uniform protein property prediction with a scikit-learn based toolset AIDE

Summary Protein property prediction via machine learning with and without labeled data is becoming increasingly powerful, yet methods are disparate and capabilities vary widely over applications. The software presented here, “Artificial Intelligence Driven protein Estimation (AIDE)”, enables instantiating, optimizing, and testing many zero-shot and supervised property prediction methods for variants and variable length homologs in a single, reproducible notebook or script by defining a modular, standardized application programming interface (API), i.e. drop-in compatible with scikit-learn transformers and pipelines. Availability and implementation AIDE is an installable, importable python package inheriting from scikit-learn classes and API and is installable on Windows, Mac, and Linux. Many of the wrapped models internal to AIDE will be effectively inaccessible without a GPU, and some assume CUDA. The newest stable, tested version can be found at https://github.com/beckham-lab/aide_predict and a full user guide and API reference can be found at https://beckham-lab.github.io/aide_predict/. Static versions of both at the time of writing can be found on Zenodo.

36 MATERIALS SCIENCE↗
DOE OSTI2023

An Update on the Geothermal Data Repository's Data Standards and Pipelines: Geospatial Data and Distributed Acoustic Sensing Data: Preprint

The Department of Energy's (DOE) Geothermal Data Repository (GDR) team has implemented data standards and automated data pipelines for the following data types: 1) drilling data, 2) geospatial datasets, and 3) DAS data. An additional data pipeline is proposed for stimulation data. These data standards and pipelines are intended to improve the real-world applicability of geothermal machine learning outputs through improving the quality of data. More specifically, through standardizing high-value datasets, the GDR is reducing project-specific data curation requirements, allowing more time to be spent on actual research. By automating this process, the burden of standardization is taken off of the user, overall increasing the availability of standardized data. This paper provides an update on the GDR's transition toward data standardization through automated data pipelines and calls for feedback from the community on how we can improve this process.

cloud-optimized↗
DOE OSTI2022

DIII-D research advancing the physics basis for optimizing the tokamak approach to fusion energy

DIII-D physics research addresses critical challenges for the operation of ITER and the next generation of fusion energy devices. This is done through a focus on innovations to provide solutions for high performance long pulse operation, coupled with fundamental plasma physics understanding and model validation, to drive scenario development by integrating high performance core and boundary plasmas. Substantial increases in off-axis current drive efficiency from an innovative top launch system for EC power, and in pressure broadening for Alfven eigenmode control from a co-/counter-I p steerable off-axis neutral beam, all improve the prospects for optimization of future long pulse/steady state high performance tokamak operation. Fundamental studies into the modes that drive the evolution of the pedestal pressure profile and electron vs ion heat flux validate predictive models of pedestal recovery after ELMs. Understanding the physics mechanisms of ELM control and density pumpout by 3D magnetic perturbation fields leads to confident predictions for ITER and future devices. Validated modeling of high-Z shattered pellet injection for disruption mitigation, runaway electron dissipation, and techniques for disruption prediction and avoidance including machine learning, give confidence in handling disruptivity for future devices. For the non-nuclear phase of ITER, two actuators are identified to lower the L–H threshold power in hydrogen plasmas. With this physics understanding and suite of capabilities, a high poloidal beta optimized-core scenario with an internal transport barrier that projects nearly to Q = 10 in ITER at ~8 MA was coupled to a detached divertor, and a near super H-mode optimized-pedestal scenario with co-I p beam injection was coupled to a radiative divertor. The hybrid core scenario was achieved directly, without the need for anomalous current diffusion, using off-axis current drive actuators. Also, a controller to assess proximity to stability limits and regulate β N in the ITER baseline scenario, based on plasma response to probing 3D fields, was demonstrated. Finally, innovative tokamak operation using a negative triangularity shape showed many attractive features for future pilot plant operation.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Paw-Net: Stacking ensemble deep learning for segmenting scanning electron microscopy images of fine-grained shale samples

Segmentation of scanning electron microscopy (SEM) images is critical yet time-consuming for geological analyses, as it needs to differentiate the boundaries for different mineral objects to facilitate subsequent analyses, such as porosity calculation. Recently, various machine learning methods, especially convolutional neural networks (CNNs), have been explored to segment SEM images of fine-grained shale samples. However, we found that general CNNs do not yield optimal performance due to insufficient training data and imbalanced objects in SEM images. This work has revised the U-Net architecture, a popular approach for biomedical image analyses, by incorporating a loss function that addresses the imbalance issue. Furthermore, we used the ensemble learning method to train multiple models and combined the results to improve the overall performance of segmentation. We prepared 2162 sub-images from raw SEM images in our experiments and divided them into training, validation, and testing datasets. The overall results show that our method improves the average Intersection over Union (IOU) of mineral objects from 0.49 to 0.58, compared to the original U-Net model. Our method can clearly distinguish each object from others with boundaries, even in highly imbalanced images. Training our models takes less than three minutes using a single GPU, while manual labeling can take up to three hours for each image. Furthermore, the method helps geoscientists gain insights quickly and effectively by building neural network models from a small dataset of SEM images.

58 GEOSCIENCES↗
DOE OSTI2022

Pose Classification Using Three-Dimensional Atomic Structure-Based Neural Networks Applied to Ion Channel–Ligand Docking

The identification of promising lead compounds showing pharmacological activities toward a biological target is essential in early stage drug discovery. With the recent increase in available small-molecule databases, virtual high-throughput screening using physics-based molecular docking has emerged as an essential tool in assisting fast and cost-efficient lead discovery and optimization. However, the best scored docking poses are often suboptimal, resulting in incorrect screening and chemical property calculation. We address the pose classification problem by leveraging data-driven machine learning approaches to identify correct docking poses from AutoDock Vina and Glide screens. To enable effective classification of docking poses, we present two convolutional neural network approaches: a three-dimensional convolutional neural network (3D-CNN) and an attention-based point cloud network (PCN) trained on the PDBbind refined set. We demonstrate the effectiveness of our proposed classifiers on multiple evaluation data sets including the standard PDBbind CASF-2016 benchmark data set and various compound libraries with structurally different protein targets including an ion channel data set extracted from Protein Data Bank (PDB) and an in-house KCa3.1 inhibitor data set. Our experiments show that excluding false positive docking poses using the proposed classifiers improves virtual high-throughput screening to identify novel molecules against each target protein compared to the initial screen based on the docking scores.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Machine Learning Models for Predicting Molecular UV–Vis Spectra with Quantum Mechanical Properties

Accurate understanding of Ultraviolet–visible (UV–Vis) spectra is critical for highthroughput design of compounds for drug discovery. Experimentally determining UV–Vis spectra can become expensive when dealing with a large quantity of novel molecules. This provides us an opportunity to drive computational advances in molecular property predictions using quantum mechanics and machine learning. In this work, we use both Quantum Mechanically (QM) predicted and measured UV–Vis spectra as input to modify four different machine learning architectures: UVvis-SchNet, UVvis- DTNN, UVvis-Transformer, and UVvis-MPNN. Here we find that the UVvis-MPNN model outperforms the other models when using optimized 3D coordinates and QM predicted spectra as input features. This model has the highest performance for predicting UVVisible spectra with a training RMSE of 0.06 and validation RMSE of 0.08.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Machine Learning for Searching the Dark Energy Survey for Trans-Neptunian Objects

In this paper we investigate how implementing machine learning could improve the efficiency of the search for Trans-Neptunian Objects (TNOs) within Dark Energy Survey (DES) data when used alongside orbit fitting. The discovery of multiple TNOs that appear to show a similarity in their orbital parameters has led to the suggestion that one or more undetected planets, an as yet undiscovered “Planet 9”, may be present in the outer solar system. DES is well placed to detect such a planet and has already been used to discover many other TNOs. Here, we perform tests on eight different supervised machine learning algorithms, using a data set consisting of simulated TNOs buried within real DES noise data. We found that the best performing classifier was the Random Forest which, when optimized, performed well at detecting the rare objects. We achieve an area under the receiver operating characteristic (ROC) curve, (AUC) = 0.996 ± 0.001. After optimizing the decision threshold of the Random Forest, we achieve a recall of 0.96 while maintaining a precision of 0.80. Finally, by using the optimized classifier to pre-select objects, we are able to run the orbit-fitting stage of our detection pipeline five times faster.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Divertor Plasma Detachment Control Neural Network

DivControlNN is a state-of-the-art software tool that leverages advanced machine learning techniques to predict and control divertor plasma behavior in fusion reactors. Plasma, a highly energetic and electrically charged gas, requires meticulous management to protect reactor components and maintain optimal energy production. Conventional simulation methods, although extremely detailed, typically demand extensive computational time-making them unsuitable for real-time control scenarios. DivControlNN addresses this challenge by learning from tens of thousands of high-fidelity simulations, thereby creating a rapid surrogate model that can deliver near-instantaneous predictions. At the core of its functionality is a sophisticated technique known as latent space mapping, which condenses complex, high-dimensional plasma data into a compact, lower-dimensional representation. This streamlined representation enables the system to quickly forecast essential plasma properties and determine the precise conditions required for effective detachment. Detachment is a crucial process in which the plasma is cooled before reaching the divertor plates, thereby reducing heat loads and mitigating material erosion. In recent experiments conducted on the KSTAR tokamak in South Korea, DivControlNN successfully guided the detachment process without any fine-tuning-even when applied to a new tungsten divertor configuration. By achieving a computational speed-up of over one hundred million times compared to traditional simulation methods while maintaining low prediction errors, DivControlNN stands to significantly enhance real-time control and diagnostic capabilities in future fusion reactors. This breakthrough paves the way for safer, more reliable reactor operation and represents a major advancement toward realizing fusion energy as a practical, sustainable, and clean power source.

Xu, Xueqiao [Lawrence Livermore National Laborator↗
DOE OSTI2023

An Update on the Geothermal Data Repository's Data Standards and Pipelines: Geospatial Data and Distributed Acoustic Sensing Data

The Department of Energy's (DOE) Geothermal Data Repository (GDR) team has implemented or is currently implementing data standards and automated data pipelines for the following geothermal data types: 1) drilling data, 2) geospatial datasets, and 3) Distributed Acoustic Sensing (DAS) data. These data standards and pipelines are intended to improve the real-world applicability of geothermal machine learning outputs through improving the quality of data. More specifically, through standardizing high-value datasets, the GDR is reducing project-specific data curation requirements, allowing more time to be spent on actual research. By automating this process, the burden of standardization is taken off of the user, overall increasing the availability of standardized data. This paper provides an update on the GDR's transition toward data standardization through automated data pipelines and calls for feedback from the community on how the GDR team can improve this process.

cloud-optimized↗
DOE OSTI2022

Breeding Realistic D‐Brane Models

Abstract Intersecting branes provide a useful mechanism to construct particle physics models from string theory with a wide variety of desirable characteristics. The landscape of such models can be enormous, and navigating towards regions which are most phenomenologically interesting is potentially challenging. Machine learning techniques can be used to efficiently construct large numbers of consistent and phenomenologically desirable models. In this work we phrase the problem of finding consistent intersecting D‐brane models in terms of genetic algorithms, which mimic natural selection to evolve a population collectively towards optimal solutions. For a four‐dimensional supersymmetric type IIA orientifold with intersecting D6‐branes, we demonstrate that unique, fully consistent models can be easily constructed, and, by a judicious choice of search environment and hyper‐parameters, of the found models contain the desired Standard Model gauge group factor. Having a sizable sample allows us to draw some preliminary landscape statistics of intersecting brane models both with and without the restriction of having the Standard Model gauge factor.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

PANNA: Properties from Artificial Neural Network Architectures

We report prediction of material properties from first principles is often a computationally expensive task. Recently, artificial neural networks and other machine learning approaches have been successfully employed to obtain accurate models at a low computational cost by leveraging existing example data. Here, we present a software package “Properties from Artificial Neural Network Architectures” (PANNA) that provides a comprehensive toolkit for creating neural network models for atomistic systems following the Behler–Parrinello topology. Besides the core routines for neural network training, it includes data parser, descriptor builder for Behler–Parrinello class of symmetry functions and force-field generator suitable for integration within molecular dynamics packages. PANNA offers a variety of activation and cost functions, regularization methods, as well as the possibility of using fully-connected networks with custom size for each atomic species. PANNA benefits from the optimization and hardware-flexibility of the underlying TensorFlow engine which allows it to be used on multiple CPU/GPU/TPU systems, making it possible to develop and optimize neural network models based on large datasets.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Chapter 7: Learning Stable Local Volt/Var Controllers in Distribution Grids

This chapter describes a framework to synthesize provably stable local Volt/Var controllers for distributed energy resources (DERs) in power distribution grids (DGs). The goal is to control the reactive power injections of DERs to improve the system performance as quantified by a generic optimal reactive power flow (ORPF) problem. To achieve this, we jointly design for each DER the control function, which prescribes the reactive power update rule, and the equilibrium function, which approximates the ORPF solutions from local measurements of voltages and powers. We provide conditions on the equilibrium functions and the control parameters ensuring the stability of the closed-loop system. In particular, we discuss the trade-offs between each set of conditions accounting for practical considerations, like fully exploiting the DERs' generation capabilities and reducing the optimality gap. These conditions are then translated into learning constraints on the neural networks' parameters that are enforced in the training phase. We validate our framework with numerical simulations on the IEEE 37-bus network and through a comparison with an optimized version of standard piece wise linear control rules.

closed-loop asymptotic stability↗
DOE OSTI2024

An Efficient Checkpointing System for Large Machine Learning Model Training

As machine learning models increase in size and complexity rapidly, the cost of checkpointing in ML training became a bottleneck in storage and performance (time). For example, the latest GPT-4 model has massive parameters at the scale of 1.76 trillion. It is highly time and storage consuming to frequently writes the model to checkpoints with more than 1 trillion floating point values to storage. This work aims to understand and attempt to mitigate this problem. First, we characterize the checkpointing interface in a collection of representative large machine learning/language models with respect to storage consumption and performance overhead. Second, we propose the two optimizations: i) A periodic cleaning strategy that periodically cleans up outdated checkpoints to reduce the storage burden; ii) A data staging optimization that coordinates checkpoints between local and shared file systems for performance improvement.

machine learning, artificial intelligence↗
DOE OSTI2024

Optimizing Prediction Error for Time-dependent Solar Radiation Modeling

Numerical weather forecasting models and statistical methods have found wide use to help power companies estimate renewable output, but better methods are needed, particularly for extended forecasts. Machine learning approaches have been used here as well, but so far a major limitation is the ability to also predict the corresponding uncertainty in a forecast. Here we show that both can be done and demonstrate this using a long-term-short memory neural network where the difference between predicted and ground truth data are used to train a model for the corresponding forecast uncertainties.

97 MATHEMATICS AND COMPUTING↗