Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “reinforcement Learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Machine Learning an Ab-Initio Based Bond-Order Potential for Bismuthene

Bismuthene is a heavy 2D material whose strong spin–orbit coupling and recently observed single-element ferroelectricity have intensified interest in its structural, vibrational, and transport properties. Accurate modeling of these behaviors requires a short-range interatomic potential that can reproduce the underlying bonding physics at a fraction of the computational cost of first-principles methods. However, such a potential is currently unavailable. Here, in this work, we construct a Tersoff bond-order potential for β-bismuthene using a reinforcement-learning framework that integrates a continuous Monte Carlo Tree Search with a simplex-based local optimizer. The optimized parameter sets reproduce first-principles lattice constants, cohesive energy, the equation of state, elastic constants, and phonon dispersion. We validate the models by performing thermal-conductivity calculations and uniaxial fracture simulations our findings confirm the reliability of the resulting models across multiple thermomechanical regimes. Comparison of the three best solutions reveals how differences in pairwise interactions, angular terms, and bond-order behavior govern phonon features and mechanical responses. We demonstrate an interpretable and computationally efficient potential for bismuthene and demonstrate a general reinforcement-learning strategy for developing bond-order models in emerging 2D materials.

deformation↗
DOE OSTI2026

Towards AI-assisted neutrino flavor theory design

Particle physics theories, such as those which explain neutrino flavor mixing, arise from a vast landscape of model-building possibilities. A model’s construction typically relies on the intuition of theorists. It also requires considerable effort to identify appropriate symmetry groups, assign field representations, and extract predictions for comparison with experimental data. We develop Autonomous Model Builder (AMBer), a framework in which a reinforcement learning agent interacts with a streamlined physics software pipeline to search these spaces efficiently. AMBer selects symmetry groups, particle content, and group representation assignments to construct models while minimizing the number of free parameters introduced. We validate our approach in well-studied regions of theory space and extend the exploration to a previously unexamined symmetry group. While demonstrated in the context of neutrino flavor theories, this approach of reinforcement learning with physics software feedback may be extended to other theoretical model-building problems in the future.

Baretz, Jason Benjamin↗
DOE OSTI2023

Quantum logic gate synthesis as a Markov decision process

Reinforcement learning has witnessed recent applications to a variety of tasks in quantum programming. The underlying assumption is that those tasks could be modeled as Markov decision processes (MDPs). Here, we investigate the feasibility of this assumption by exploring its consequences for single-qubit quantum state preparation and gate compilation. By forming discrete MDPs, we solve for the optimal policy exactly through policy iteration. We find optimal paths that correspond to the shortest possible sequence of gates to prepare a state or compile a gate, up to some target accuracy. Our method works in both the absence and presence of noise and compares favorably to other quantum compilation methods, such as the Ross–Selinger algorithm. This work provides theoretical insight into why reinforcement learning may be successfully used to find optimally short gate sequences in quantum programming.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2026

Surrogate model for third-integer resonance extraction at the Fermilab Delivery Ring

We present an ongoing work in which a surrogate model is being developed to reproduce the response dynamics of the third-integer resonant extraction process in the Delivery Ring (DR) at Fermilab. This is in pursuit of smoothly extracting circulating beam to the Mu2e Experiment’s production target, whereby the goal is to extract a uniform slice of the circulating 1e12 protons in the DR over 25,000 turns (43 ms). The DR contains 3 harmonic sextupoles that excite a third-integer resonance and three fast, tune-ramping quadrupole magnets that drive the horizontal tune towards the 29/3 resonance. In our initial work, the surrogate model trains on a semi-analytical simulation provided in the same format as live data. Using Reinforcement Learning (and other potential ML methods), the trained surrogate acts as the “environment” in which a simple ML control agent could learn to dynamically adjust the quadrupole ramp at 430 break points within the 43 microsecond spill window. The controller will be hosted on a dedicated Arria 10 FPGA. In this work, we report the accuracy and fidelity of the surrogate model in comparison to the response dynamics of the physics simulator.

Narayanan, Aakaash [Fermilab]↗
DOE OSTI2025

Optimizing Traffic Signal Control to Enhance Transportation Efficiency and Maximize Pedestrian Benefits in the Road Network

Increasing urban mobility requirements demand efficient transportation system strategies for both vehicular and pedestrian movement. This study enhances the Decentralized Graph-based Multi-Agent Reinforcement Learning (DGMARL) approach, originally tailored for vehicular traffic signal timing, to incorporate pedestrian traffic dynamics. The improved algorithm considers crucial metrics such as Eco_PI, assesses vehicle fuel consumption by factoring in stops and delays, and addresses pedestrian waiting time, crucial for system efficiency while acknowledging driver waiting time impact. Utilizing Digital Twin simulation along the MLK Smart Corridor in Chattanooga, Tennessee, the algorithm's performance is compared for various pedestrian control scenarios. To evaluate the effectiveness of DGMARL, this study compared DGMARL-enabled signal management with automated pedestrian traffic detection and an actuated signal management system (real-word baseline) with pedestrian recall, which predetermingly enforces a pedestrian phase every cycle. Findings indicate substantial improvements with DGMARL, showing a 28.29% enhancement in vehicle Eco_PI, a 60.55 % reduction in pedestrian waiting time, and a 55.74% decrease in driver stop delay, on average, compared to the baseline actuated signal timing plan.

Kumarasamy, Vijayalakshmi K [The University of Ten↗
DOE OSTI2022

AI-Enabled Discovery and Physics-Based Optimization of Energy Efficient Processing Strategies for Advanced Turbine Alloys (Final Technical Report)

In this project, the multi-organizational team of academic and industrial researchers from the University of Kentucky an aerospace and energy generation OEM partner has leveraged novel Digital Process Twin (DPT) models of process/structure interactions (i.e., process-induced surface integrity) to advance a paradigm of fully integrated computational materials engineering (ICME). Using efficient process models as the core of a digital process simulator for a reinforcement learning algorithm, the team has integrated industrial data and metrics of structure/performance/energy relationships and manufacturing-related energy metrics to optimize dynamic processing parameters for significantly improved life-cycle energy efficiency of advanced γ-TiAl low-pressure turbine (LPT) alloys, as indicated by a set of design relevant parameters (e.g., residual stresses and scrap rate). The key objective and anticipated outcome of the project was at least a 10% reduction in life-cycle embodied energy for a recently developed, γ-TiAl low-pressure turbine (LPT) alloy and nickel-based superalloy Inconel 718, through the adoption of the proposed AI-enabled process optimization approach. The final project outcomes significantly exceeded this original target, realizing manufacturing-related energy efficiency improvements of more than 130% for TiAl and up to 80% for Inconel 718. Rather than following the prevailing and highly inefficient empirical paradigm, the proposed study demonstrated the feasibility of adopting a digital, physics-based process design and optimization paradigm. The recurring need for manual intervention, rework, reinspection causes significant production bottlenecks and unnecessary expense associated with delivering the requisite component quality. The OEM partner, and turbine industry in general, expect to reap significant cost and resource savings if an AI-optimized set of parameters can be applied to specific machining operations. The technical scope of the proposed project involved the paving of a realistic path towards model-based and AI-enabled Integrated Computational Materials Engineering (ICME), and away from inefficient empirical process optimization and legacy manufacturing practices, which are no longer able to efficiently process novel high-performance turbine alloy materials. The project team will address the fundamental knowledge gap that currently exists within the ICME paradigm with respect to the process/structure/performance/energy impacts of finishing processes. While significant resources have been devoted to the ‘early stages’ of manufacturing, such as alloy design, primary and secondary processing, finishing processes have not been adequately integrated within ICME. To provide an actionable path towards model-based finishing process design (e.g., machining, burnishing, grinding, polishing), we will employ a novel AI-enabled process optimization paradigm, based on a computationally efficient, physics-based process simulator. Through limited experimental work to calibrate and validate our process simulator model via an advanced in-situ characterization technique and process optimization via reinforcement learning, the project will seek to demonstrate a viable alternative to the inefficient ‘legacy’ processing strategies, empirical testing and broad scope machining learning approaches, all of which fail to adequately consider complex process physics. The project team has identified an intermetallic γ-TiAl LPT alloy, which is currently being used as part of the OEM partner’s advanced gas turbine designs. This particular alloy poses significant manufacturing challenges during finishing operations, which limit the degree to which the current turbine design can be manufactured in an energy- and cost-efficient manner. Empirical testing and numerical modeling efforts to optimize processing parameters for γ-TiAl have not been able to resolve these manufacturing challenges, so the proposed physics-based AI-enabled optimization technology would offer a truly novel and transformative capability. The multi-organizational team of academic and industry experts from the UKY and the OEM partner will work together closely to demonstrate the analytical and experimental critical function and characteristic proof of concept of this novel approach.

20 FOSSIL-FUELED POWER PLANTS↗
DOE OSTI2022

A high-fidelity building performance simulation test bed for the development and evaluation of advanced controls

We present an open-source building performance simulation test bed, the Advanced Controls Test Bed (ACTB), that interfaces high-fidelity Spawn of EnergyPlus building models, with advanced controllers implemented in Python. Additionally, the ACTB leverages the Building Optimization Testing and Alfalfa platforms for managing simulations, providing an external clock, a representational state transfer (REST) application programming interface (API), and key performance indicators for evaluating the effectiveness of control strategies. The REST API allows the development of external controllers programmed in languages such as Python, which provides flexibility and a rich choice of scientific libraries for designing control sequences. We present three test cases based on the U.S. Department of Energy's Reference Small Office Building to demonstrate the ACTB's capabilities: (a) rule-based controls compliant with ASHRAE Guideline 36 control sequences; (b) an economic model predictive control implemented using do-mpc; and (c) a deep Q-network reinforcement learning agent implemented using OpenAI Gym.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2024

Physics-Reinforced Machine Learning Algorithms for Multiscale Closure Model Discovery

The central objective of this project was to address the challenge of modeling and simulating complex multiscale turbulence phenomena by leveraging physics-guided machine learning (PGML) and hybrid modeling approaches. By integrating physics-based methods with data-driven models, the research focused on achieving robust and scalable solutions for geophysical turbulence, enhancing numerical weather prediction and climate research tools. The project resulted in significant advancements in computational modeling paradigms, predictive tools for reduced-order modeling, and innovative algorithms for fluid dynamics.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Reinforcement Learning-based Output Structured Feedback for Distributed Multi-Area Power System Frequency Control

Load frequency control (LFC) is a key factor to maintain the stable frequency in multi-area power systems. As the modern power systems evolve from centralized to decentralized paradigm, LFC needs to consider the decentralized scheme that considers limited information from the information-exchange graph for the generator control of each interconnected area. This paper aims to solve a data-driven constrained LQR problem with mean-variance risk constraints and output structured feedback, and applies this framework to solve the LFC problem in multi-area power systems. By reformulating the constrained optimization problem into a minimax problem, the stochastic gradient descent max-oracle (SGDmax) algorithm with zero-order policy gradient (ZOPG) is adopted to find the optimal feedback gain from the learning, while guaranteeing the convergence. In addition, to improve the adaptation of the proposed learning method to new or varying models, we construct an emulator grid that approximates the dynamics of a physical grid and performs training based on this model. Once the feedback gain is obtained from the emulator grid, it is applied to the physical grid with a robustness test to check whether the controller from the approximated emulator applies to the actual system. Numerical tests show that the obtained feedback controller can successfully control the frequency of each area, while mitigating the uncertainty from the loads, with reliable robustness that ensures the adaptability of the obtained feedback gain to the actual physical grid.

Kwon, Kyung-bin↗
DOE OSTI2023

Correlated Trajectory Uncertainty for Adaptive Sequential Decision Making

One of the great challenges with decision making tasks on real world systems is the fact that data is sparse and acquiring additional data is expensive. In these cases, it is often crucial to make a model of the environment to assist in making decisions. At the same time, limited data means that learned models are erroneous, making it just as important to equip the model with good predictive uncertainties. In the context of learning sequential decision making policies, these uncertainties can prove useful for informing which data to collect for the greatest improvement in policy performance \citep{mehta2021experimental, mehta2022exploration} or informing the policy about unsure regions of state and action space to avoid during test time \citep{yu2020mopo}. Additionally, assuming that realistic samples of the environment can be drawn, an adaptable policy can be trained that attempts to make optimal decisions for any given possible instance of the environment \citep{ghosh2022offline, chen2021offline}. In this work, we examine the so-called ``probabilistic neural network'' (PNN) model that is ubiquitous in model-based reinforcement learning (MBRL) works. We argue that while PNN models may have good marginal uncertainties, they form a distribution of non-smooth transition functions. Not only are these samples unrealistic and may hamper adaptability, but we also assert that this leads to poor uncertainty estimates when predicting multiple step trajectory estimates. To address this issue, we propose a simple sampling method that can be implemented on top of pre-existing models.We evaluate our sampling technique on a number of environments, including a realistic nuclear fusion task, and find that, not only do smooth transition function samples produce more calibrated uncertainties, but they also lead to better downstream performance for an adaptive policy.

Offline Reinforcement Learning↗
DOE OSTI2022

A machine learning pipeline for membrane segmentation of cryo-electron tomograms

We describe how to use several machine learning techniques organized in a learning pipeline to segment and identify cell membrane structures from cryo electron tomograms. These tomograms are difficult to analyze with traditional segmentation tools. The learning pipeline in our approach starts from supervised learning via a special convolutional neural network trained with simulated data. It continues with semi-supervised reinforcement learning and/or a region merging technique that tries to piece together disconnected components belonging to the same membrane structure. A parametric or non-parametric fitting procedure is then used to enhance the segmentation results and quantify uncertainties in the fitting. Domain knowledge is used in generating the training data for the neural network and in guiding the fitting procedure through the use of appropriately chosen priors and constraints. We demonstrate that the approach proposed here works well for extracting membrane surfaces in two real tomogram datasets.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

A transfer learning approach to energy-efficient control of small and medium-sized commercial buildings

Model-free reinforcement learning (RL) provides a data-driven and adaptive approach to optimize building energy use while satisfying occupant comfort. This powerful tool does not need any prior knowledge about the environment and system it is optimizing and can adapt its policy based on the changes in captures. Like any other data-driven tool, it faces high training costs due to the extensive agent-environment interactions required to capture long-term building dynamics and user comfort. Transfer learning, particularly policy distillation, offers a promising way to accelerate training by leveraging pretrained RL agents in different building and system types. Here, this study investigates online student distillation, in which the student model updates its neural network weights using outputs from teacher models. The work introduces a student distillation strategy designed for efficient knowledge transfer, along with a teacher selection method that ensures high-quality guidance. The approach is validated using a highly calibrated whole building energy model for a small/medium commercial building test facility. Results show substantial reductions in training time and data requirements while surpassing the performance of ASHRAE Guideline 36, an advanced rule-based control strategy. The distilled RL model required 45% less data and achieved 20% higher cumulative rewards than a state-of-the-art RL model, with faster convergence and lower energy consumption. These outcomes demonstrate that effective transfer learning enables a scalable and data-efficient energy management solution for commercial buildings.

ASHRAE guideline 36↗