SEARCH · Engineering Papers

Results for “Operator learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,585 records · Page 88

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Grid-Interactive Multi-Zone Building Control Using Reinforcement Learning with Global-Local Policy Search: Preprint

In this paper, we develop a grid-interactive multi-zone building controller based on a deep reinforcement learning (RL) approach. The controller is designed to facilitate building operation during normal conditions and demand response events, while ensuring occupants comfort and energy efficiency. We leverage a continuous action space RL formulation, and devise a two-stage global-local RL training framework. In the first stage, a global fast policy search is performed using a gradient-free RL algorithm. In the second stage, a local fine-tuning is conducted using a policy gradient method. In contrast to the state-of-the-art model predictive control (MPC) approach, the proposed RL controller does not require complex computation during real-time operation and can adapt to non-linear building models. We illustrate the controller performance numerically using a five-zone commercial building.

30 DIRECT ENERGY CONVERSION↗

DOE OSTI2025

Prediction of Silicon Content in a Blast Furnace via Machine Learning: A Comprehensive Processing and Modeling Pipeline

Silicon content plays an important role in determining the operational efficiency of blast furnaces (BFs) and their downstream processes in integrated steelmaking; however, existing sampling methods and first-principles models are somewhat limited in their capability and flexibility. Current data-based prediction models primarily rely on a limited set of manually selected furnace parameters. Additionally, different BFs present a diverse set of operating parameters and state variables that are known to directly influence the hot metal’s silicon content, such as fuel injection, blast temperature, and raw material charge composition, among other process variables that have their own impacts. The expansiveness of the parameter set adds complexity to parameter selection and processing. This highlights the need for a comprehensive methodology to integrate and select from all relevant parameters for accurate silicon content prediction. Providing accurate silicon content predictions would enable operators to adjust furnace conditions dynamically, improving safety and reducing economic risk. To address these issues, a two-stage approach is proposed. First, a generalized data processing scheme is proposed to accommodate diverse furnace parameters. Second, a robust modeling pipeline is used to establish a machine learning (ML) model capable of predicting hot metal silicon content with reasonable accuracy. The method employed herein predicted the average Si content of the upcoming furnace cast with an accuracy of 91% among 200 target predictions for a specific furnace provisioned by the XGBoost model. This prediction is achieved using only the past shift’s operating conditions, which should be available in real time. This performance provides a strong baseline for the modeling approach with potential for further improvement through provision of real-time features.

Chemistry↗

NASA NTRS2020

Adaptive Stress Testing: Finding Likely Failure Events with Reinforcement Learning

Finding the most likely path to a set of failure states is important to the analysis of safety-critical systems that operate over a sequence of time steps, such as aircraft collision avoidance systems and autonomous cars. In many applications such as autonomous driving, failures cannot be completely eliminated due to the complex stochastic environment in which the system operates.As a result, safety validation is not only concerned about whether a failure can occur, but also discovering which failures are most likely to occur. This article presents adaptive stress testing (AST), a framework for finding the most likely path to a failure event in simulation. We consider a general black box setting for partially observable and continuous-valued systems operating in an environment with stochastic disturbances. We formulate the problem as a Markov decision process and use reinforcement learning to optimize it. The approach is simulation-based and does not require internal knowledge of the system, making it suitable for black-box testing of large systems. We present different formulations depending on whether the state is fully observable or partially observable. In the latter case, we present a modified Monte Carlo tree search algorithm that only requires access to the pseudorandom number generator of the simulator to overcome partial observability. We also present an extension of the framework, called differential adaptive stress testing (DAST), that can find failures that occur in one system but not in another. This type of differential analysis is useful in applications such as regression testing, where we are concerned with finding areas of relative weakness compared to a baseline. We demonstrate the effectiveness of the approach on an aircraft collision avoidance application, where a prototype aircraft collision avoidance system is stress tested to find the most likely scenarios of near mid-air collision.

Verification and Validation↗

NASA NTRS1996

Early Warning: Development of Confidential Incident Reporting Systems

Accidents hardly ever happen without warning. The combination, or sequence, of failures and mistakes that cause an accident may indeed be unique but the individual failures and mistakes rarely are. In the USA in 1974 the crews on two different aircraft misunderstood the same aeronautical chart and descended towards their destination dangerously early towards a mountain. The first crew were in good weather conditions and could see the mountain and resolved their misinterpretation of the chart. The second crew six weeks later were not so lucky. In cloud they had no clues to point out their mistake nor the presence of the mountain. The resulting crash and the ensuing inquiry, which brought to light the previous incident, shocked the country but gave it the impetus to instigate a safety reporting system. This system eventually became the NASA's Aviation Safety Reporting System (ASRS). The programme collects incident reports from pilots, controllers, mechanics, cabin attendants and many others involved in aviation operations. By disseminating this safety information the ASRS has helped enormously to give US airlines and airspace the highest safety standards. Accident prevention is a goal sought by everyone in the aviation industry and establishing effective incident reporting programmes can go a long way toward achieving that goal. This article will describe the steps and issues required to establish an incident reporting system. The authors summarize the lessons learned from the ASRS, now in its twentieth year of operation and from the Confidential Human Factors Reporting (HER) Programme run by British Airways, an airline that is a recognized world leader in safety reporting and analysis. The differences between government and airline operation of confidential safety reporting systems will be addressed.

OLeary, Mike J.↗

NASA NTRSDate not supplied

An Intern Experience with NASA’s Artemis Program

“3...2...1...Go For Launch”. Nasa’s Kennedy Space Center is projected to launch the first mission of theArtemis Program by the end of this year, 2021. I have had the opportunity to contribute directly to this mission through my internship with their Engineering organization from January to August of 2021.Within their Engineering organization, I am an intern for the Avionics branch. Throughout this experience I have completed two projects with one project still in progress. These projects allowed me to work directly with the Orion vehicle’s software and hardware power ups, checkouts, and tests. I became familiarized with the engineering concepts that allowed this advanced technology to perform as intended.In addition, I became acquainted with the operations and tasks that are completed during a LaunchCountdown, both at a detailed level as well as a higher managerial level. Lastly, I worked directly with the software algorithm that detects and responds to onboard faults and failures. I have analyzed and made conclusions based on data and documentation, allowing my team to benefit from a condensed, centralized informational file. In addition to my technical experience, I have learned a significant amount about how the Kennedy Space Center operates, it’s history, and the history of the agency. The education I received from the Engineering department at Texas A&M University not only allowed me to obtain this internship, but provided me with the required background in computer and electrical engineering principles to be successful in this organization.

Sarah Bianco↗

DOE OSTI2023

A Transductive Graph Neural Network learning for Grid Resilience Analysis

Power grids are critical infrastructures that require robust resilience analysis to ensure reliable and uninterrupted electricity supply. Traditional simulation-based methods for grid resilience analysis suffer from computational complexity and limited ability to capture the full spectrum of potential disruptions. This paper presents a novel approach to enhance grid resilience by leveraging transductive graph neural network (GNN) learning to identify critical nodes and links. By leveraging the graph structure and system features, GNNs effectively learn resilience metrics and accurately identify critical nodes based on actual grid operational behavior. The efficacy of the proposed approach is demonstrated through case studies on node criticality scoring and critical node/line identification in cascading outage scenarios. The results highlight the advantages of learning-based methods over traditional simulation-based approaches and their potential to revolutionize grid resilience analysis. The contributions of this paper include a graph-based scalable approach for fast cascading analysis, an inductive formulation for training GNN models, and a transfer learning-based approach to scale the model to largescale power systems.

grid resilience, graph neural networks, transducti↗

NASA NTRS1975

Study to evaluate the effect of EVA on payload systems. Volume 1: Executive summary

Programmatic benefits to payloads are examined which can result from the routine use of extravehicular activity (EVA) during space missions. Design and operations costs were compared for 13 representative baseline payloads to the costs of those payloads adapted for EVA operations. The EVA-oriented concepts developed in the study were derived from these baseline concepts and maintained mission and program objectives as well as basic configurations. This permitted isolation of cost saving factors associated specifically with incorporation of EVA in a variety of payload designs and operations. The study results were extrapolated to a total of 74 payload programs. Using appropriate complexity and learning factors, net EVA savings were extrapolated to over $551M for NASA and U.S. civil payloads for routine operations. Adding DOD and ESRO payloads increases the net estimated savings of $776M. Planned maintenance by EVA indicated an estimated $168M savings due to elimination of automated service equipment. Contingency problems of payloads were also analyzed to establish expected failure rates for shuttle payloads. The failure information resulted in an estimated potential for EVA savings of $1.9 B.

Patrick, J. W.↗

DOE OSTI2021

A dimensionality reduction algorithm for mapping tokamak operational regimes using a variational autoencoder (VAE) neural network

Abstract A variational autoencoder (VAE) is a type of unsupervised neural network which is able to learn meaningful data representations in a reduced dimensional space. We present an application of VAE in identifying the operational stability boundary of tokamak plasma discharges. This model was implemented using a dataset of over 3000 discharges from the high beta tokamak-extended pulse (HBT-EP) device. We found the VAE model to be capable of forming a continuous low-dimensional operational space map and identifying the operational boundaries using a specified warning time window. By projecting the operational parameters onto the same reduced space, this provides an intuitive way for the machine operator or an automated control system to perform disruption avoidance using a relevant control actuator as a discharge approaches a boundary. Pre-programmed GPU control experiments were conducted to demonstrate this control technique using HBT-EP’s saddle control coils as a horizontal position actuator, showing the ability to avoid the oncoming disruptive event and extend the duration of the discharge.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

NASA NTRS2023

Science Objectives for Human Exploration of Mars Workshop Report

The Science Objectives for Human Exploration of Mars Workshop was held in Denver, Colorado on May 4–6, 2022. The workshop was co-sponsored by NASA’s Science Mission Directorate and the Exploration Systems Development Mission Directorate to actively engage the planetary science community to determine what planetary science should be done by human crews on the martian surface and how those science objectives can be achieved. Sessions at the Science Objectives for Human Exploration of Mars Workshop were organized around specific planetary science objectives and mission architecture concepts that were identified during the workshop as the highest priority for human exploration. The intent of this workshop was to synthesize a notional, integrated concept of operations for each scenario to aid in planning and refining the mission architecture for the first human mission to Mars. Results from the Planetary Decadal Survey Report were released xx days before the workshop and were briefed to workshop participants who incorporated the findings in the discussions. With the Artemis missions, humans will return to the Moon using innovative technologies to explore the lunar surface. We will apply what we learn about exploration architecture, surface infrastructure, and science and exploration operations on and around the Moon to conduct successful missions with the first astronauts to Mars. A human mission to Mars will be a landmark achievement and a golden opportunity to make groundbreaking scientific discoveries on Mars. The potential scope of the science activities and impact of achieving those objectives are extraordinary.

Mars↗

DOE OSTI2022

RADAMS: Resilient and adaptive alert and attention management strategy against Informational Denial-of-Service (IDoS) attacks

Attacks exploiting human attentional vulnerability have posed severe threats to cybersecurity. In this work, we identify and formally define a new type of proactive attentional attacks called Informational Denial-of-Service (IDoS) attacks that generate a large volume of feint attacks to overload human operators and hide real attacks among feints. Here, we incorporate human factors (e.g., levels of expertise, stress, and efficiency) and empirical psychological results (e.g., the Yerkes-Dodson law and the sunk cost fallacy) to model the operators’ attention dynamics and their decision-making processes along with the real-time alert monitoring and inspection. To assist human operators in dismissing the feints and escalating the real attacks timely and accurately, we develop a Resilient and Adaptive Data-driven alert and Attention Management Strategy (RADAMS) that de-emphasizes alerts selectively based on the abstracted category labels of the alerts. RADAMS uses reinforcement learning to achieve a customized and transferable design for various human operators and evolving IDoS attacks. The integrated modeling and theoretical analysis lead to the Product Principle of Attention (PPoA), fundamental limits, and the tradeoff among crucial human and economic factors. Experimental results corroborate that the proposed strategy outperforms the default strategy and can reduce the IDoS risk by as much as 20%. Besides, the strategy is resilient to large variations of costs, attack frequencies, and human attention capacities. We have recognized interesting phenomena such as attentional risk equivalency, attacker’s dilemma, and the half-truth optimal attack strategy.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2016

FatMan vs. LittleBoy: Scaling Up Linear Algebraic Operations in Scale-Out Data Platforms

Linear algebraic operations such as matrix manipulations form the kernel of many machine learning and other crucial algorithms. Scaling up as well as scaling out such algorithms are highly desirable to enable efficient processing over millions of data points. To this end, we present a matrix manipulation approach to effectively scale-up each node in a scale-out data parallel platform such as Apache Spark. Specifically, we enable hardware acceleration for matrix multiplications in a distributed Spark setup without user intervention. Our approach supports both dense and sparse distributed matrices, and provides flexible control of acceleration by matrix density. We demonstrate the benefit of our approach for generalized matrix multiplication operations over large matrices with up to four billion elements. To connect the effectiveness of our approach with machine learning applications, we performed Gramian matrix computation via generalized matrix multiplications. Our experiments show that our approach achieves more than 2× performance speed-up, and up to 96.1% computation improvement, compared to a state of the art Spark MLlib for dense matrices.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2016

FatMan vs. LittleBoy: Scaling up Linear Algebraic Operations in Scale-out Data Platforms

Linear algebraic operations such as matrix manipulations form the kernel of many machine learning and other crucial algorithms. Scaling up as well as scaling out such algorithms are highly desirable to enable efficient processing over millions of data points. To this end, we present a matrix manipulation approach to effectively scale-up each node in a scale-out data parallel platform such as Apache Spark. Specifically, we enable hardware acceleration for matrix multiplications in a distributed Spark setup without user intervention. Our approach supports both dense and sparse distributed matrices, and provides flexible control of acceleration by matrix density. We demonstrate the benefit of our approach for generalized matrix multiplication operations over large matrices with up to four billion elements. To connect the effectiveness of our approach with machine learning applications, we performed Gramian matrix computation via generalized matrix multiplications. Our experiments show that our approach achieves more than 2x performance speed-up, and up to 96.1% computation improvement, compared to a state of the art Spark MLlib for dense matrices.

Lim, Seung-Hwan↗

DOE OSTI2025

Data-Enabled Fusion Technology (Final Scientific/Technical Report)

Advancing Scientific Understanding in Fusion Energy and Machine Learning This research represented a significant step forward in machine learning (ML) applications for fusion energy experiments. The project integrated advanced data-driven modeling, optimization techniques, and artificial intelligence to enhance the predictive capabilities and operational efficiency of plasma-based fusion systems. Specifically, tasks focused on ML-enhanced diagnostics, operator guidance tools, and predictive modeling helped improve the ability to interpret complex fusion experiments. Key areas of advancement included: 1) data-driven plasma control, i.e., using ML algorithms to optimize experimental conditions and classify plasma behaviors based on historical data; 2) spectroscopy and diagnostics, i.e., applying AI models to extract previously inaccessible insights from experimental spectroscopy data; and 3) configuration mapping and operator guidance, i.e., developing a predictive framework to assist scientists in identifying the most effective experimental parameters, reducing reliance on manual adjustments. By refining these ML-driven techniques, the project contributed to the broader scientific community’s understanding of plasma dynamics and fusion energy viability. Technical Effectiveness and Economic Feasibility The methods investigated demonstrated high technical effectiveness, as reflected in milestones assessing the predictive accuracy, performance, and optimization of fusion configurations. The development of an Operator Guidance Tool (OGT), for example, led to more precise control of plasma conditions by learning from experimental data and offering real-time adjustments. From an economic standpoint, DeFT provided: 1) the ability to reduce trial-and-error experimentation, which lowered operational costs; 2) improved data interpretation methods, which enabled more efficient resource allocation in large-scale fusion research projects; and 3) the automation of key diagnostic tasks, which reduced manual labor and human error, increasing overall efficiency. 13 The final assessments of predictive models and optimization strategies demonstrated that these approaches were scalable and could be implemented across multiple fusion energy research programs. Public Benefit and Societal Impact This project contributed directly to the broader goal of achieving sustainable and commercially viable fusion energy, which had profound implications for clean energy production and climate change mitigation. The integration of AI-driven solutions into fusion research: 1) sped up scientific discovery, accelerating progress towards achieving energy breakthroughs; 2) reduced the cost of experimentation, making fusion research more accessible; and 3) provided a framework for future AI applications in high-energy physics, benefiting adjacent fields like space exploration, material science, and renewable energy. Additionally, by fostering collaborations between AI researchers and plasma physicists, this project promoted interdisciplinary innovation that could lead to broader applications beyond fusion research.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2021

Coordinated Modeling of Electric Grid and Natural Gas Network Operations

Presentation based upon full report of a Colorado case study and coordination framework, which is available at https://www.nrel.gov/docs/fy20osti/77096.pdf. JISEA analysts Brian Sergi, Omar Guerra, and Bri-Mathias Hodge will present on recent JISEA work on coordination between the natural gas and electricity sectors. Power and gas are becoming increasingly interdependent but weren't designed to function together. How does greater coordination impact system operations with different levels of solar and wind penetrations? This free JISEA presentation will take place in a Webinar on Tuesday, May 11, 2021 at 12 p.m.: Learn how greater coordination between the natural gas and electricity sectors impacts system operations with different levels of solar and wind penetrations. Presentation information includes: Historical and projected data for natural gas consumption and power generation in the United States as demonstrated by 2014 East Coast Polar Vortex, 2021 Texas Winter Storm Uri. Showing coupling points, FERC identified need for better coordination, different levels of coordination (decision making and optimization control), coordination framework. Case study on Colorado Front Range, ramping requirements and gas nominations, real-time dispatch (June and December scenarios), results of analysis of impacts of coordination on unserved load. Total real-time gas offtakes by node, impacts on unserved gas, effect on CO2 emissions. Discussions on clusions drawn from Colorado Case study expanded co-simulation via HELICS, objectives of the HELICS+ natural gas use case, modeling of hydrogen blending, techno economic assessment of blending, and blending impacts on energy content and pressure.

analysis↗

DOE OSTI2021

Gaming the beamlines—employing reinforcement learning to maximize scientific outcomes at large-scale user facilities

Abstract Beamline experiments at central facilities are increasingly demanding of remote, high-throughput, and adaptive operation conditions. To accommodate such needs, new approaches must be developed that enable on-the-fly decision making for data intensive challenges. Reinforcement learning (RL) is a domain of AI that holds the potential to enable autonomous operations in a feedback loop between beamline experiments and trained agents. Here, we outline the advanced data acquisition and control software of the Bluesky suite, and demonstrate its functionality with a canonical RL problem: cartpole. We then extend these methods to efficient use of beamline resources by using RL to develop an optimal measurement strategy for samples with different scattering characteristics. The RL agents converge on the empirically optimal policy when under-constrained with time. When resource limited, the agents outperform a naive or sequential measurement strategy, often by a factor of 100%. We interface these methods directly with the data storage and provenance technologies at the National Synchrotron Light Source II, thus demonstrating the potential for RL to increase the scientific output of beamlines, and layout the framework for how to achieve this impact.

36 MATERIALS SCIENCE↗

DOE OSTI2017

Applications of data mining technology to enhance O and M: automatic planning of electrical isolation with deep learning - 255

Data mining technology is now readily available, thanks to high powered, low cost computing resource and useful coding tools. We study several applications of data mining to enhance operation and maintenance (O and M), such as abnormality prediction, efficiency diagnosis, document traceability, image recognition, etc. In field support, we try automatic planning of electrical isolation with deep learning. Currently, a skilled engineer plans electrical isolation procedure with hundreds of the circuit diagrams and the related documents, taking man-hours. If this task becomes automatic, it is very efficient. The automatic planning has two issues. One is much calculation time of electrical circuit simulator, searching billions of electrical conducting paths. The other is interpretation of human fuzzy information, such as ease of work. Deep learning is helpful to resolve them. We performed a model case study of the electrical isolation. We applied a deep neural network (DNN) for dropping in the calculation time. We trained the circuit diagrams and the responses of the circuit simulator to the DNN, constructing an efficient path search algorithm in the DNN. The calculation time of the DNN was shorter by a factor of 32 for the model case, compared with that of the electrical circuit simulator. We applied a deep Q-network (DQN) for the interpretation of fuzzy information. The DQN optimized isolation procedure from the viewpoint of the work load with the equipment layout, related doses, and values of wet bulb globe temperature (WBGT) to prevent heatstroke. (authors)

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

NASA NTRSDate not supplied

Reports of Resilient Performance: Investigating Operators' Descriptions of Safety-producing Behaviors in the Aviation Safety Reporting System

While many existing taxonomies and frameworks provide a common vocabulary for describing how human operators fail in the context of sociotechnical systems, at present, there is no common vocabulary to describe how humans succeed. Such a framework would facilitate systematically collecting and analyzing data on how human performance can produce safety, not just how it can reduce safety. One potentially rich source of currently available information for exploring desired performance is the reports submitted to NASA’s Aviation Safety Reporting System (ASRS). These de-identified, confidential, and voluntary narrative reports are submitted by pilots, controllers, ground operators, and others within aviation operations. While these reports are primarily submitted to describe safety risks, incidents, and problems, they also often describe how those risks were mitigated, and provide a window into aspects of everyday work in aviation. These reports can be searched in a variety of ways. This paper describes methods for systematically examining ASRS narratives to understand how operators talk about their own resilient behaviors during adverse safety conditions and events. Guided by Erik Hollnagel’s Resilience Assessment Grid framework (i.e., anticipate, monitor, respond, learn), various approaches and tools for such inquiries are described. The approach, process, challenges, and suggestions to building an operator-based description of resilient performance are discussed, and tools to facilitate data analysis to maximize learning from these reports are described.

ASRS narratives↗

NASA NTRS1996

NASA Mir program: Mission operations concept

The joint NASA/Russian Space Agency mission program is discussed, considering the lessons learned. The initial Shuttle Mir science program and the NASA Mir program are described. The NASA Mir program is organized into ten distinct working groups which are co-chaired by representatives from the two cooperating nations. The NASA component is managed from the Johnson Space Center (TX). The support provided by NASA for long-duration missions and Mir expeditions is described. The scope of the scientific research carried out within the framework of the joint program is considered. The NASA Mir training approach is discussed and the mission operations are reviewed with emphasis on the Mir 21/NASA 2 mission.

Cardenas, Jeffrey A.↗