Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Operator learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Reinforcement Learning-Based Oscillation Dampening: Scaling Up Single-Agent Reinforcement Learning Algorithms to a 100-Autonomous-Vehicle Highway Field Operational Test

In this article, we explore the technical details of the reinforcement learning (RL) algorithms that were deployed in the largest field test of automated vehicles designed to smooth traffic flow in history as of 2023, uncovering the challenges and breakthroughs that come with developing RL controllers for automated vehicles. We delve into the fundamental concepts behind RL algorithms and their application in the context of self-driving cars, discussing the developmental process from simulation to deployment in detail, from designing simulators to reward function shaping. We present the results in both simulation and deployment, discussing the flow-smoothing benefits of the RL controller. From understanding the basics of Markov decision processes to exploring advanced techniques such as deep RL, our article offers a comprehensive overview and deep dive of the theoretical foundations and practical implementations driving this rapidly evolving field. We also showcase real-world case studies and alternative research projects that highlight the impact of RL controllers in revolutionizing autonomous driving. From tackling complex urban environments to dealing with unpredictable traffic scenarios, these intelligent controllers are pushing the boundaries of what automated vehicles can achieve. Furthermore, we examine the safety considerations and hardware-focused technical details surrounding deployment of RL controllers into automated vehicles. As these algorithms learn and evolve through interactions with the environment, ensuring their behavior aligns with safety standards becomes crucial. Here, we explore the methodologies and frameworks being developed to address these challenges, emphasizing the importance of building reliable control systems for automated vehicles.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
NASA NTRS2011

International Space Station (ISS) Risk Reduction Activities

As the assembly of the ISS nears completion, it is worthwhile to step back and review some of the actions pursued by the Program in recent years to reduce risk and enhance the safety and health of ISS crewmembers, visitors, and space flight participants. While the ISS requirements and initial design were intended to provide the best practicable levels of safety, it is always possible to reduce risk -- given the determination and commitment to do so. The following is a summary of some of the steps taken by the ISS Program Manager, by our International Partners, by hardware and software designers, by operational specialists, and by safety personnel to continuously enhance the safety of the ISS. While decades of work went into developing the ISS requirements, there are many things in a Program like the ISS that can only be learned through actual operational experience. These risk reduction activities can be divided into roughly three categories: (1) Areas that were initially noncompliant which have subsequently been brought into compliance or near compliance (i.e., Micrometeoroid and Orbital Debris [MMOD] protection, acoustics) (2) Areas where initial design requirements were eventually considered inadequate and were subsequently augmented (i.e., Toxicity Level 4 materials, emergency hardware and procedures) (3) Areas where risks were initially underestimated, and have subsequently been addressed through additional mitigation (i.e., Extravehicular Activity [EVA] sharp edges, plasma shock hazards) Due to the hard work and cooperation of many parties working together across the span of nearly a decade, the ISS is now a safer and healthier environment for our crew, in many cases exceeding the risk reduction targets inherent in the intent of the original design. It will provide a safe and stable platform for utilization and discovery.

Fodroci, Michael↗
NASA NTRSDate not supplied

Lessons Learned: Using UTM paradigm for Urban Air Mobility Operations

Urban Air Mobility (UAM) aims to reduce congestion on the roads and highways by offering air taxi as an alternative to driving on surface roads. Integration of UAM operations in the National Airspace System (NAS) has been the focus of the research conducted at NASA Ames Research Center. A simulation was performed in collaboration with Uber Technologies Inc to investigate if NASA’s UTM architecture and its implementation as demonstrated in the 2019 UTM field tests were extensible for UAM operations, and if the data exchange between multiple operators as planned under UTM were adequate for UAM operations in the shared airspace. In order to explore these research questions, three Use Cases were defined to investigate different airspace management challenges. This paper will describe the lessons learned from exercising the uses cases and the airspace management services including scheduling and separation developed to facilitate initial UAM operations.

Urban Air Mobility↗
DOE OSTI2024

Entropy-Infused Deep Learning Loss Function for Capturing Extreme Values in Wind Power Forecasting

Extreme scenarios in wind power generation occur with higher frequency and larger magnitude in the recent years due to the ever-increasing extreme meteorological factors. Accurate forecasting of the occurrence of extreme values in wind power generation is of great concern to ensure reliable power system operation. Recently, deep learning models have surged in popularity for wind power forecasting, with the mean squared error (MSE) loss function being commonly used. However, the MSE loss function, being sensitive to extreme values, disproportionately penalizes larger errors, cannot adequately capture the extreme values present in wind energy data, and novel loss functions have seldom been tailored for wind power forecasting. To this end, in this paper, we introduce a novel loss function specifically crafted to capture extreme values in wind power forecasting. The experimental results with four fundamental deep learning methods on open source wind power dataset validate that the new loss function is efficient and superior in all cases compared to MSE in capturing extreme values while maintaining forecasting performance.

17 WIND ENERGY↗
NASA NTRS2010

Human Spaceflight Trajectory Operations: A Case Study from STS-132/ULF4

Prior to the launch of the STS-132/ULF4 mission, the International Space Station (ISS) Flight Control Team was alerted to the approach of a space debris with a predicted time of closest approach approximately 1 hr after Space Shuttle docking. As a result, the ISS team considered performing a collision avoidance maneuver prior to the Space Shuttle s approach. The purpose of this presentation will be to use this case study to illustrate best practices of the human spaceflight trajectory operations team. The presentation will begin with an overview of the roles of the real-time ISS and Space Shuttle trajectory flight control teams. Then, the STS-132/ULF4 case study will be used to illustrate the issues and lessons learned involved in performing Space Shuttle rendezvous maneuver planning and execution. Specific elements of the STS-132 Space Shuttle rendezvous maneuver profile that can be used to provide mission assurance in cases of trajectory uncertainty will be described and discussed. Additionally, this case study will be used to provide context for identifying lessons learned in an operational environment that requires technical and decision-making coordination between two control center teams with two potentially maneuvering space vehicles.

Cutri-Kohart, Rebecca↗
DOE OSTI2021

Practical CO2—WAG Field Operational Designs Using Hybrid Numerical-Machine-Learning Approaches

Machine-learning technologies have exhibited robust competences in solving many petroleum engineering problems. The accurate predictivity and fast computational speed enable a large volume of time-consuming engineering processes such as history-matching and field development optimization. The Southwest Regional Partnership on Carbon Sequestration (SWP) project desires rigorous history-matching and multi-objective optimization processes, which fits the superiorities of the machine-learning approaches. Although the machine-learning proxy models are trained and validated before imposing to solve practical problems, the error margin would essentially introduce uncertainties to the results. In this paper, a hybrid numerical machine-learning workflow solving various optimization problems is presented. By coupling the expert machine-learning proxies with a global optimizer, the workflow successfully solves the history-matching and CO2 water alternative gas (WAG) design problem with low computational overheads. The history-matching work considers the heterogeneities of multiphase relative characteristics, and the CO2-WAG injection design takes multiple techno-economic objective functions into accounts. This work trained an expert response surface, a support vector machine, and a multi-layer neural network as proxy models to effectively learn the high-dimensional nonlinear data structure. The proposed workflow suggests revisiting the high-fidelity numerical simulator for validation purposes. The experience gained from this work would provide valuable guiding insights to similar CO2 enhanced oil recovery (EOR) projects.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
NASA NTRS2011

Quadrocopter Control Design and Flight Operation

A limiting factor in control system design and analysis for spacecraft is the inability to physically test new algorithms quickly and cheaply. Test flights of space vehicles are costly and take much preparation. As such, EV41 recently acquired a small research quadrocopter that has the ability to be a test bed for new control systems. This project focused on learning how to operate, fly, and maintain the quadrocopter, as well as developing and testing protocols for its use. In parallel to this effort, developing a model in Simulink facilitated the design and analysis of simple control systems for the quadrocopter. Software provided by the manufacturer enabled testing of the Simulink control system on the vehicle.

Karwoski, Katherine↗
DOE OSTI2019

Data-Driven Day-Ahead PV Estimation Using Hybrid Deep Learning

Ongoing smart grid activities and associated automation resulted in rich set of data. Here, these data can be utilized for monitoring and estimation of real time photovoltaic (PV) generation. Inherent variability in PV and related impact on power systems is a challenging problem. Improving the accuracy of PV generation estimation is beneficial for both the PV owners and the grid operators. Recently, deep learning algorithms possible by the availability of data have shown its advantages for time series estimation; however, its application on PV generation estimation is still in the early stage. In this paper, a hybrid estimation model with a combination of long-short-term-memory network (LSTM) and persistence model (PM) is developed to provide day-ahead PV estimation at 15-minute time interval with high accuracy and robustness. Simulation results show the superior performance of the proposed method over existing methods for most of the test cases.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2019

Data-based Aggregate Model of Refrigerators for Electric Power Grid Services

Electrical power system stability under high penetration of intermittent renewable energy generation, such as wind and solar energy, has become a high concern. It is unpredictable and hard to control due to little inertial behavior characteristic to these generation resources. To help mitigate possibly undesirable frequency excursions, demand-side frequency control presents an emerging solution. By observing the frequency deviation, devices could switch operating modes for a short time to rebalance generation. To analyze device level control algorithms, aggregate models for their behavior are needed. Refrigerators are household devices that could help with load modulation. This paper uses real measured data to analyze the refrigerator behavior, learn about its operating mode and power consumption ranges. A Markov chain process is proposed to model the transition of a refrigerator population from one operating mode to another, such that the overall load is modulated according to the control input. The Markov-chain model can be used to design control so that the overall load is modulated according to the control input.

Markov chain model, demand response, refrigerator,↗
NASA NTRS2009

A Methodology for Training International Space Station Crews to Respond to On-Orbit Emergencies

Most spaceflight crewmembers agree that emergency training is among the most important training they receive. If an emergency event occurs on-orbit crewmembers want to be able to rely on a thorough and proficient knowledge of emergency operations and procedures. The inherent complexity of ISS and the international nature of the onboard operations have resulted in emergency procedures that are complex by any measure; as a result, a very robust apparatus has been developed to give crewmembers initial training on emergency procedures and ensure proficiency up to (and even after) launch. One of the most important aspects of complex onboard operations in general, and emergency operations specifically, is learning how to coordinate roles and responsibilities with fellow crewmembers. A primary goal of NASA s emergency training program is to allow the crewmembers who will actually be together on-orbit to practice executing the emergency responses together before they fly. As with any operation that includes the use of software and hardware, the fidelity of the simulation environment is a critical element to successful training. The NASA training division has spent considerable time and effort to develop a simulator that addresses the most important aspects of emergency response, working within very difficult space and budgetary constraints.

Balmain, Clinton↗
DOE OSTI2025

A Reinforcement Learning Approach to Augment Conventional PID Control in Nuclear Power Plant Transient Operation

The ability of nuclear reactors to operate their power conversion cycles more flexibly will enhance their value to energy grids with variable pricing. Current nuclear control systems are typically classical controllers that are often based on proportional-integral-derivative (PID) control. This paper presents a method of augmenting the existing PID control for difficult transient operations in nuclear power plants using a reinforcement learning–derived feedforward signal applied in real time. The agents, which are trained on a test thermal load-following problem, are designed to improve steam generator outlet temperature control for a range of fast load-following scenarios covering ramp rates from 9%/min to 15%/min. Several reinforcement learning algorithms were initially investigated for the training of the feedforward agents with deep Q-learning (DQN) and proximal policy optimization (PPO) networks, which were found to be the most promising. The DQN controllers utilize discrete actions, giving them a better disturbance rejection at steady state but inconsistent response to initial temperature deviations. In contrast, PPO-trained agents, which take continuous actions except for a dead zone around zero, were shown to have the best combination of high disturbance rejection at steady state and good tracking of the desired temperature value. The ability of the PPO agent was also examined, with the average time of decision making found to be on the order of 1 ms. The fault properties of the controller under the loss of the reinforcement learning agent feedforward signal were also examined. The controller showed strong performance in situations of “no-signal” faults. but was less good at handling “stuck-at” faults, where the feedforward signal remains at a set value. In both cases, however, the PID was able to successfully maintain stability, eventually returning the system to a steady state. It is hoped that this work will allow for the proposed control architecture to be examined for more difficult control problems such that it may eventually be used to adapt existing nuclear plants for more aggressive load-following on grids of the future.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS2008

Preparing Cassini Uplink Operations for Extended Mission

The Cassini-Huygens Mission to Saturn and Titan, a joint venture between the National Aeronautics and Space Administration, the European Space Agency, and the Italian Space Agency, is conducting a four-year, prime mission exploring the Saturnian system, including its atmosphere, rings, magnetosphere, moons and icy satellites. Launched in 1997, Cassini began its prime mission in 2004. Cassini is now preparing for a new era, a two-year extended mission to revisit many of the highlights and new discoveries made during the prime mission. Because of the light time delay from Earth to Saturn, and the time needed to coordinate the complicated science and engineering activities that take place on the spacecraft, commanding on Cassini is done in approximately 40-day intervals known as sequences. The Cassini Uplink Operations team is responsible for the final development and validation of the pointing profile and instrument and spacecraft commands that are contained in a sequence. During this final analysis prior to uplink to the spacecraft, thorough and exact evaluation is necessary to ensure there are no mistakes during commanding. In order to perform this evaluation, complete and refined processes and procedures are fundamental. The Uplink Operations team is also responsible for anomaly response during sequence execution, a process in which critical decisions often are made in real-time. Recent anomalies on other spacecraft missions have highlighted two major risks in the operations process: (1) personnel turnover and the retirement of critical knowledge and (2) aging, outdated operations procedures. If other missions are a good barometer, the Cassini extended mission will be presented with a high personnel turnover of the Cassini flight team, which could lead to a loss of expertise that has been essential to the success of the prime mission. In order to prepare the Cassini Uplink Operations Team for this possibility and to continue to develop and operate safe science and engineering sequences, a review and major update of the current documentation and operations procedures was needed. This paper will address the changes made to extended mission sequence generation processes primarily due to new restrictions in spacecraft operating capability and lessons learned from prime mission. In addition, it will address the state of the prime mission operations procedures, the philosophy changes and updates that were made to those procedures in response to process improvement, and the validation of those new procedures through the training of current and new personnel. And lastly, it will address the lessons learned throughout prime mission and how the Uplink Operations team chose to incorporate those lessons into the working documentation and team knowledge. This incorporation was necessary to facilitate the success of the extended mission with potentially all new personnel at some point prior to the end of the mission.

Maxwell, Jennifer L.↗
NASA NTRS2016

On the Maneuvers Operational Response for NASA’s Soil Moisture Active-Passive (SMAP) Mission

The Soil-Moisture Active-Passive (SMAP) spacecraft requires various kinds of in-orbit maneuvers over the course of its three-year mission. The types of maneuvers include pre-planned commissioning maneuvers to reach its science orbit, regularly executed orbit maintenance maneuvers to overcome drag and other nominally occurring phenomena, as well as (the possibility of) collision avoidance maneuvers. The architecture of the spacecraft – in terms of availability of commanding via ground assets, the inherited avionics' ability to sequence and execute commands, and the capability of available subsystems able to carry out maneuvers – was well defined early in the development of the spacecraft and mission, well before the operational plan for responding to maneuver requests was cemented. The systems engineering challenge became: how to accommodate all three types of maneuvers in the confines of this well-defined architecture. This paper will describe how the operations team on SMAP successfully met this challenge. Specifically, it will dive into the three pronged approach that SMAP developed to handle each type of maneuver described above – to meet the timeliness requirements leveraged on the operations team to execute said maneuvers, while continuing to fit within the allotted staffing profile during nominal operations. Defining this paradigm to fit the mission’s architecture meant re-defining the original paradigm, (planned maneuvers being thought of separately than collision avoidance maneuvers), and re-classifying all responses to maneuver requests as variations and permutations of a singular operational response to a maneuver request. This paper will also describe the tools that were created to simplify the human interface and automate as much of the response as was possible. Finally, this paper will describe, at a very high level, some of the problems encountered and lessons learned by the operations team when this process was executed the first four times during the first ninety days of operations. Though the architecture of the operations team's response to maneuver requests will never be repeated exactly, the flexibility that was inserted via redefining the scope of the problem and by redefining the human interfaces should influence future projects' architectures earlier in their development – in the hopes that said influence will save time and money in the future.

Tirona, Joseph↗
NASA NTRS2002

Utilizing the US Lab Nadir Research Window for Remote Sensing Operations with The Window Observational Research Facility (WORF)

The Window Observational Research Facility (WORF) is an ISPR-based rack facility designed to take advantage of the high optical quality US Lab Nadir research window. The WORF is based on the ISS Expedite the Processing of Experiments to Space Station (EXPRESS) rack mechanical structure and electronic systems. The WORF has a unique payload volume located at the center of the rack that provides access to the window. The interior dimensions of the payload volume are 34-in. (86.36 cm) wide by 33-in. (83.82 cm) high by 23-in. (58.42 cm) deep. This facility supports the deployment of payloads such as 9 in. aerial photography cameras and 12 in. diameter optical equipment. The WORF coupled with the optical quality of the United States Lab window support the deployment of various payload disciplines. The WORF provides payloads with power, data command and control, air cooling, water cooling, and video processing. The WORF's payload mounting surfaces and interfaces include the interior payload mounting shelf and the interior and exterior aircraft-like seat tracks. The payload mounting shelf is limited to a maximum mass of 136 kg (299 pounds). The WORF can accommodate large payloads such as the commonly used Leica-Heerbrug RC-30 aerial photography camera (whose dimensions are 53.3 cm (21-in.) wide by 50.8 cm (20-in.) deep by 76.2 cm (30-in.) long). The performance characteristics of the WORF allow it to support an array of payload disciplines. The WORF provides a maximum of 3 Kw at 28 Vdc and has a maximum data rate of 10 Mbps. The WORF's unique payload volume is designed to be light-tight, down to 2.8 x 10(exp -11) Watts/cm2/steradian, and have low-reflective surfaces. This specially designed WORF interior supports payload investigations that observe low-light-level phenomenon such as aurora. Although the WORF rack does not employ any active rack isolation (i.e., vibration dampening) technology, the rack provides a very stable environment for payload operations (on the order of X microradians). The facility's software is capable of being updated during its period of deployment. The WORF project also includes a Suitcase Simulator to allow for a payload developer to verify data interfaces at his development site, a trainer rack for astronauts to learn how to operate the WORF prior to flight, and the use of the EXPRESS Functional Checkout Units to allow for payload checkout at the KSC prior to launch.

Turner, Richard↗
NASA NTRS2011

Long-Term International Space Station (ISS) Risk Reduction Activities

As the assembly of the ISS nears completion, it is worthwhile to step back and review some of the actions pursued by the Program in recent years to reduce risk and enhance the safety and health of ISS crewmembers, visitors, and space flight participants. While the initial ISS requirements and design were intended to provide the best practicable levels of safety, it is always possible to further reduce risk given the determination, commitment, and resources to do so. The following is a summary of some of the steps taken by the ISS Program Manager, by our International Partners, by hardware and software designers, by operational specialists, and by safety personnel to continuously enhance the safety of the ISS, and to reduce risk to all crewmembers. While years of work went into the development of ISS requirements, there are many things associated with risk reduction in a Program like the ISS that can only be learned through actual operational experience. These risk reduction activities can be divided into roughly three categories: Areas that were initially noncompliant which have subsequently been brought into compliance or near compliance (i.e., Micrometeoroid and Orbital Debris [MMOD] protection, acoustics) Areas where initial design requirements were eventually considered inadequate and were subsequently augmented (i.e., Toxicity hazard level-4 materials, emergency procedures, emergency equipment, control of drag-throughs) Areas where risks were initially underestimated, and have subsequently been addressed through additional mitigation (i.e., Extravehicular Activity [EVA] sharp edges, plasma shock hazards). Due to the hard work and cooperation of many parties working together across the span of more than a decade, the ISS is now a safer and healthier environment for our crew, in many cases exceeding the risk reduction targets inherent in the intent of the original design. It will provide a safe and stable platform for utilization and discovery for years to come.

Forroci, Michael P.↗