Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “machine learning and control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Controlling optical-cavity locking using reinforcement learning

Abstract This study applies an effective methodology based on Reinforcement Learning to a control system. Using the Pound–Drever–Hall locking scheme, we match the wavelength of a controlled laser to the length of a Fabry-Pérot cavity such that the cavity length is an exact integer multiple of the laser wavelength. Typically, long-term drift of the cavity length and laser wavelength exceeds the dynamic range of this control if only the laser’s piezoelectric transducer is actuated, so the same error signal also controls the temperature of the laser crystal. In this work, we instead implement this feedback control grounded on Q-Learning. Our system learns in real-time, eschewing reliance on historical data, and exhibits adaptability to system variations post-training. This adaptive quality ensures continuous updates to the learning agent. This innovative approach maintains lock for eight days on average.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

Quantum optimal control of superconducting qubits based on machine-learning characterization

Implementing fast and high-fidelity quantum operations using open-loop quantum optimal control relies on having an accurate model of the quantum dynamics. Any deviations between this model and the complete dynamics of the device, such as the presence of spurious modes or pulse distortions, can degrade the performance of optimal controls in practice. Here, we propose an experimentally simple approach to realize optimal quantum controls tailored to the device parameters and environment while specifically characterizing this quantum system. Concretely, we use physics-inspired machine learning to infer an accurate model of the dynamics from experimentally available data and then optimize our experimental controls on this trained model. We show the power and feasibility of this approach by optimizing arbitrary single-qubit operations in detailed numerical simulations of a superconducting transmon qubit. Furthermore, we demonstrate that this framework produces an accurate description of the device dynamics under arbitrary controls, together with the precise pulses achieving arbitrary single-qubit gates with a high fidelity of ∼99.99%.

Artificial neural networks↗
DOE OSTI2024

Chapter 7: Learning Stable Local Volt/Var Controllers in Distribution Grids

This chapter describes a framework to synthesize provably stable local Volt/Var controllers for distributed energy resources (DERs) in power distribution grids (DGs). The goal is to control the reactive power injections of DERs to improve the system performance as quantified by a generic optimal reactive power flow (ORPF) problem. To achieve this, we jointly design for each DER the control function, which prescribes the reactive power update rule, and the equilibrium function, which approximates the ORPF solutions from local measurements of voltages and powers. We provide conditions on the equilibrium functions and the control parameters ensuring the stability of the closed-loop system. In particular, we discuss the trade-offs between each set of conditions accounting for practical considerations, like fully exploiting the DERs' generation capabilities and reducing the optimality gap. These conditions are then translated into learning constraints on the neural networks' parameters that are enforced in the training phase. We validate our framework with numerical simulations on the IEEE 37-bus network and through a comparison with an optimized version of standard piece wise linear control rules.

closed-loop asymptotic stability↗
NASA NTRS2022

Run Time Assurance for Electric Vertical Takeoff and Landing Aircraft

NASA is conducting research to demonstrate and evaluate the application of Run Time Assurance (RTA) as a means to assure safety in Electric Vertical Takeoff and Landing (eVTOL) aircraft with highly automated or autonomous flight capability supervised by a single onboard pilot. The work described in this report demonstrates an application of RTA and examines the implications for design and analysis of aircraft functions and systems; aircraft safety hazards; safety assurance; development assurance; and pilot tasks and performance. This research effort also seeks to assess the efficacy of the combined application of traditional Functional Hazard Analysis (FHA) and the more modern System Theoretic Process Analysis (STPA) techniques to perform hazard analyses on aircraft with complex automated and autonomous systems and an onboard pilot. During the research effort we developed architectural designs of two alternate eVTOL aircraft, generally following the process characterized in the SAE standards ARP4754 and ARP4761. The design has focused on the control architectures of these aircraft, which are identical except that one incorporates RTA techniques to reduce the criticality of some key software components. Artifacts of this process include a taxonomy of aircraft-level functions, aircraft-level architecture diagrams, aircraft-level functional hazard assessments (AFHA), function allocations onto aircraft systems and subsystems, functional block diagrams for a select set of control-related functions, and system-level functional hazard assessments (SFHA) for those functions. This project has highlighted the notion that DAL D is something of a sweet spot for low-confidence controllers in an RTA-based design. Among the many activities described in DO-178C, the activities related to requirement verifiability, algorithmic accuracy, and test coverage can be the most challenging for the kinds of advanced control techniques that may be desirable in novel UAM designs, such as adaptive control, machine-learning, artificial intelligence, numerical search, and Monte Carlo based algorithms. Moreover, the standard requires that development teams demonstrate that errors leading to unacceptable failure conditions have been removed from the software. The RTA architecture, which cordons off the low-confidence function, makes it much easier to show this for these kinds of algorithms. With regard to the use of STPA and FHA as complementary hazard analysis techniques, our research effort led us to the conclusion that STPA should be used to derive requirements for hardware and software systems and/or components. Also, STPA is a natural complement to other processes in ARP4754A involving design studies and iteration.

Run-time assurance↗
DOE OSTI2025

Development of Predictive Models for Advanced Reactor Autonomous Control

Advanced reactor designs including microreactors and small modular reactors will contribute to the clean production of cheap energy, and autonomous control for advanced reactors is an appealing option for reducing cost. However, there is a lack of industry experience applying autonomous control for advanced nuclear reactors. To accelerate the development and industry acceptance of autonomous control software for nuclear reactors, we aim to demonstrate autonomous control of the Purdue University research reactor (PUR-1) using INL-developed model predictive control (MPC) methods. To prepare for this demonstration, data-driven predictive models based on process data collected from PUR-1 have been developed and integrated with MPC and used to control a physics-based model of PUR-1. A data-driven dynamics model and a gated recurrent unit (GRU) network were both trained on process data from PUR-1. The dynamics model was shown to effectively control the reactor model with MPC when provided reactivity as a control variable but failed to control the model through the control rod positions. The GRU network produced more accurate predictions than the dynamics model when evaluated on operational data, and future work will include the evaluation of the GRU network in the controller.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2026

Continental-Scale Controls on Hyporheic Respiration Revealed by Knowledge-Guided Machine Learning

Hyporheic zone sediments regulate organic matter turnover and in-stream respiration, yet controls on sediment respiration remain poorly constrained across heterogeneous river networks, limiting prediction of stream metabolism and carbon processing at continental scales. Here, we integrate observations from ~90 river corridors across the United States in the WHONDRS consortium with a knowledge-guided machine learning (KGML) framework that couples thermodynamic rate theory with machine learning to identify dominant controls on hyporheic respiration. Diagnostic analyses show that organic matter concentration and thermodynamic favorability define an upper bound on respiration potential, whereas biological catalytic capacity and physical accessibility jointly govern realized respiration rates through interaction effects. To represent unmeasurable accessibility constraints, we use the mechanistic model as a scaffold for KGML, allowing machine learning to target residual structure not explained by process theory. This hybrid framework improves predictive skill relative to both the mechanistic model alone and fully data-driven models while preserving interpretability. These results indicate that variability in hyporheic respiration is largely mechanistically structured and demonstrate how integrating process theory with explainable AI enhances predictive performance while enabling scalable synthesis of river corridor observations.

Zheng, Jianqiu↗
DOE OSTI2025

Optimizing Facility Operations by Applying Machine Learning to the Army Reserve Enterprise Building Control System (Final Report)

Thousands of U.S. Department of Defense (DoD) buildings have building automation systems (BASs) and/or advanced meters. Although these systems have a wealth of data, performance optimization requires time and expertise to review and act on that information. Machine learning (ML) can provide automated and actionable insights to controls operators. This demonstration implemented proven ML methods on the Army Reserve Enterprise Building Control System. ML refers to algorithms that “learn” from data and improve their performance on a given task over time. In the buildings domain these tasks range from predicting future energy consumption, to identifying operational issues before faults occur, to optimizing control decisions. To learn, ML requires input data, which – for buildings – typically consists of instrument data such as energy consumption data and subsystem controls information such as set-point temperatures, and context data consisting of information such as the physical location of the building, the area of the building, and the weather. ML models use the relationships learned from the input data to make predictions with new, previously unseen, data. The team was able to investigate and successfully implement the following ML use cases: labeling consumption data as anomalous or non-anomalous; baseline whole-building load prediction (unknown fault status); fault detection (validation not possible); and site prioritization for energy-related projects. Due to the constraints of the project, interventions were not able to be implemented during the demonstration; therefore, assessments of operational cost savings and maintenance avoided could not be performed. The project has been presented at two leading national building conferences and two additional publications to peer-reviewed journals are currently in preparation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2025

Machine Learning for Automated Weld Quality Monitoring and Control

Resistance Spot Welding (RSW) is a critical process in the automotive industry, valued for its cost-effectiveness, short cycle time, and robustness. However, achieving consistent high-quality joints remains challenging due to the complex interplay of various factors, like materials, processes, and manufacturing uncertainties, etc. Under the collaborative project between Oak Ridge National Laboratory (ORNL) and General Motors (GM), we have developed a robust and expansible machine learning (ML) framework aimed at enhancing quality control in RSW. By harnessing the power of machine learning, we have developed the ability to ensure every aspect of the welding process, from the initial process design stage to the final weld joint quality. The framework operates by analyzing a variety of data streams, including in-line process signals, process parameters, materials, and postprocessed weld joint data. Through this analysis, the models have been trained to detect deviations from optimal quality standards, leveraging their ability to identify signature data patterns and anomalies within in-line signals and construct complex correlations between these signals and weld quality parameters. Meanwhile, the machine learning framework is designed to adapt to a variety of materials, including high strength steels and aluminum alloys, etc. Its flexible architecture facilitates the incorporation of diverse data sources and features, enabling precise modeling and prediction across a broad range of material properties and weld quality variables. The expansible ML frameworks represent a promising transformation in weld quality monitoring and control, empowering industry to achieve high levels of efficiency, consistency, and reliability in manufacturing.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2025

Machine Learning for Automated Weld Quality Monitoring and Control

Resistance Spot Welding (RSW) is a critical process in the automotive industry, valued for its cost-effectiveness, short cycle time, and robustness. However, achieving consistent high-quality joints remains challenging due to the complex interplay of various factors, like materials, processes, and manufacturing uncertainties, etc. Under the collaborative project between Oak Ridge National Laboratory (ORNL) and General Motors (GM), we have developed a robust and expansible machine learning (ML) framework aimed at enhancing quality control in RSW. By harnessing the power of machine learning, we have developed the ability to ensure every aspect of the welding process, from the initial process design stage to the final weld joint quality. The framework operates by analyzing a variety of data streams, including in-line process signals, process parameters, materials, and postprocessed weld joint data. Through this analysis, the models have been trained to detect deviations from optimal quality standards, leveraging their ability to identify signature data patterns and anomalies within in-line signals and construct complex correlations between these signals and weld quality parameters. Meanwhile, the machine learning framework is designed to adapt to a variety of materials, including high strength steels and aluminum alloys, etc. Its flexible architecture facilitates the incorporation of diverse data sources and features, enabling precise modeling and prediction across a broad range of material properties and weld quality variables. The expansible ML frameworks represent a promising transformation in weld quality monitoring and control, empowering industry to achieve high levels of efficiency, consistency, and reliability in manufacturing.

42 ENGINEERING↗
DOE OSTI2025

Safety Assurance of Software and Machine Learning Development for Nuclear Instrumentation and Controls

Digital instrumentation and control (DI&C) systems monitor and control parameters in nuclear power plants. Ensuring their safety is a critical part of ensuring overall plant safety. Nuclear power plant licensing generates thousands of safety documents that could be organized more effectively using a safety assurance case (SAC). We conducted a literature survey of SACs and created a SAC framework for DI&C software using Goal Structuring Notation (GSN). This framework focuses on four software development processes: management & assurance, pre-developed software (PDS) qualification, the Software Development Life Cycle (SDLC), and the Machine Learning Development Life Cycle (MLDLC). We organized our framework using a novel level structure that can be applied to other SACs to improve their clarity. Finally, we demonstrate how our framework can be incorporated as part of a SAC for a larger reactor system.

46 - INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AN↗
NASA NTRSDate not supplied

Autonomous Spacecraft Attitude Control Using Deep Reinforcement Learning

While machine learning and spacecraft autonomy continue to gain research interest, significant work remains to be done in efficiently applying modern machine learning techniques to problems in space ight. This study presents a framework for deriving a discrete neural spacecraft attitude controller using reinforcement learning, a paradigm of machine learning, without the need for high-performance computing. The developed attitude controller is an approximately time-optimal solution to a highly constrained control problem, able to achieve well above industry-standard pointing accuracies. Control examples are also presented of the agent performing large-angle spacecraft slews in the developed simulation environment and future extensions of this work are discussed.

ATAP↗
DOE OSTI2025

A CHIL Validation of Machine Learning-Assisted Methods for Real-Time Controls of Solar PV for Grid Services

Recent research has highlighted the potential for solar to act as a zero-marginal-cost and zero-emission flexibility resource on the bulk power system when operated with advanced control systems. To increase the performance of these systems, leading technologies, including machine learning (ML) and hierarchical inverter set point allocation, have been proposed; however, these technologies lack comprehensive validation under real-world application scenarios. This paper addresses this gap by designing and developing a controller-hardware-in-the-loop framework to evaluate the performance of different flexible solar technologies in responding to automatic generation control signals in a closed-loop fashion. Simulation results indicate the superior performance of an ML-based approach compared to the conventional reference-control grouping-based approach, showcasing its potential to support grid stability and operational efficiency.

14 SOLAR ENERGY↗
DOE OSTI2024

A CHIL Validation of Machine Learning-Assisted Methods for Real-Time Controls of Solar PV for Grid Services

Recent research has highlighted the potential for solar to act as a zero-marginal-cost and zero-emission flexibility resource on the bulk power system when operated with advanced control systems. To increase the performance of these systems, leading technologies, including machine learning (ML) and hierarchical inverter set point allocation, have been proposed; however, these technologies lack comprehensive validation under real-world application scenarios. This paper addresses this gap by designing and developing a controller-hardware-in-the-loop framework to evaluate the performance of different flexible solar technologies in responding to automatic generation control signals in a closed-loop fashion. Simulation results indicate the superior performance of an ML-based approach compared to the conventional reference-control grouping-based approach, showcasing its potential to support grid stability and operational efficiency.

closed-loop validation↗
DOE OSTI2024

A CHIL Validation of Machine Learning-Assisted Methods for Real-Time Controls of Solar PV for Grid Services: Preprint

Recent research has highlighted the potential for solar to act as a zero-marginal-cost and zero-emission flexibility resource on the bulk power system when operated with advanced control systems. To increase the performance of these systems, leading technologies, including machine learning (ML) and hierarchical inverter set point allocation, have been proposed; however, these technologies lack comprehensive validation under real-world application scenarios. This paper addresses this gap by designing and developing a controller-hardware-in-the-loop framework to evaluate the performance of different flexible solar technologies in responding to automatic generation control signals in a closed-loop fashion. Simulation results indicate the superior performance of an ML-based approach compared to the conventional reference-control grouping-based approach, showcasing its potential to support grid stability and operational efficiency.

closed-loop validation↗
NASA NTRS2004

Learning to Control Advanced Life Support Systems

Advanced life support systems have many interacting processes and limited resources. Controlling and optimizing advanced life support systems presents unique challenges. In particular, advanced life support systems are nonlinear coupled dynamical systems and it is difficult for humans to take all interactions into account to design an effective control strategy. In this project. we developed several reinforcement learning controllers that actively explore the space of possible control strategies, guided by rewards from a user specified long term objective function. We evaluated these controllers using a discrete event simulation of an advanced life support system. This simulation, called BioSim, designed by Nasa scientists David Kortenkamp and Scott Bell has multiple, interacting life support modules including crew, food production, air revitalization, water recovery, solid waste incineration and power. They are implemented in a consumer/producer relationship in which certain modules produce resources that are consumed by other modules. Stores hold resources between modules. Control of this simulation is via adjusting flows of resources between modules and into/out of stores. We developed adaptive algorithms that control the flow of resources in BioSim. Our learning algorithms discovered several ingenious strategies for maximizing mission length by controlling the air and water recycling systems as well as crop planting schedules. By exploiting non-linearities in the overall system dynamics, the learned controllers easily out- performed controllers written by human experts. In sum, we accomplished three goals. We (1) developed foundations for learning models of coupled dynamical systems by active exploration of the state space, (2) developed and tested algorithms that learn to efficiently control air and water recycling processes as well as crop scheduling in Biosim, and (3) developed an understanding of the role machine learning in designing control systems for advanced life support.

Subramanian, Devika↗
NASA NTRS1991

Application of machine learning and expert systems to Statistical Process Control (SPC) chart interpretation

Statistical Process Control (SPC) charts are one of several tools used in quality control. Other tools include flow charts, histograms, cause and effect diagrams, check sheets, Pareto diagrams, graphs, and scatter diagrams. A control chart is simply a graph which indicates process variation over time. The purpose of drawing a control chart is to detect any changes in the process signalled by abnormal points or patterns on the graph. The Artificial Intelligence Support Center (AISC) of the Acquisition Logistics Division has developed a hybrid machine learning expert system prototype which automates the process of constructing and interpreting control charts.

Shewhart, Mark↗