SEARCH · Engineering Papers

Results for “Operator learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,513 records · Page 84

All sources NASA NTRS DOE OSTI

DOE OSTI2026

Short-term electricity load forecasting: Application-driven evaluation of machine learning models across spatial and temporal scales

As we transition towards a decarbonized economy, the integration of variable renewable energy resources and new demands (e.g., electric vehicles, heat pumps) into the electricity grid places unprecedented pressure on grid operators to effectively anticipate and manage peak load. In this context, machine learning algorithms are proving to be indispensable for accurate short-term load forecasting, a crucial task to address these challenges. This study benchmarks 6 machine learning algorithms, including three neural networks and three tree-based algorithms, across various levels of spatial aggregation and time horizons (1, 4, 8, 24, and 48 h). The central contribution of this work is the comparison and analysis of load forecasting models not only based on statistical metrics, but also based on a novel error metric, which evaluates the cost implications of forecast errors for power system stakeholders. Results show that tree-based models outperform neural networks, based on statistical metrics, and yield less skewed error distributions for most spatial scales. However, through the lens of the novel error metric, neural networks are the more competitive choice, especially for forecast horizons that exceed 8 h. The study concludes with actionable recommendations to grid operators and highlights the need for the development of error metrics that link forecasting accuracy to operational costs. To promote transparency and open science, the datasets and Python code are open-sourced via a supplementary repository.

Houben, Nikolaus↗

DOE OSTI2016

Innovative Approach for Technetium-99 Disposal Reduces Operational Impacts - 16460

The Environmental Management Waste Management Facility (EMWMF) on the Oak Ridge Reservation began receiving and disposing waste with elevated technetium-99 (Tc-99) activities on November 11, 2013. Prior to receipt, the Tennessee Department of Environment and Conservation expressed concerns about potential releases of this highly mobile radionuclide into the environment and requested additional safeguards to minimize potential releases. Several additional requirements were negotiated to address these concerns and allow this waste lot to be safely disposed at EMWMF. Because of the 130-cm (51-inch) annual average precipitation at the landfill, the potential release of Tc-99 (above the release criterion) to surface water during the EMWMF water management activities was of the greatest concern. The EMWMF Operations staff developed an innovative approach to resolve regulator concerns regarding disposal of low-level waste containing elevated levels of Tc-99 at EMWMF. This approach minimized the spread of contamination and the generation of water that contacted the Tc-99 waste. Lessons learned and recommendations are being implemented for the next, more challenging elevated Tc-99 waste from the K-27 Building, currently scheduled for early 2016. These lessons learned are also applicable to other sites looking to minimize operational impacts from disposal of waste with elevated activities of Tc-99 or other highly mobile contaminants. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2023

Enhancing Cyber Resilience of Networked Microgrids using Vertical Federated Reinforcement Learning

This paper presents a novel federated reinforcement learning (Fed-RL) methodology to inject sufficient resiliency into the operations of the network of microgrids. We consider adversarial actions to the voltage and power control loop reference signals at the grid forming (GFM) inverters in the microgrids which are essential to integrate renewable resources. Therefore, we formulate a resilient reinforcement learning training setup that uses these adversarial injections to generate episodic trajectories and train the RL agents to alleviate their impact on performance. To circumvent the concerns about data-sharing and privacy for different owners of the microgrids in the networked setting, we bring in the aspects of the federated operation to propose novel Fed-RL algorithms. As the dynamics of each microgrid are coupled due to electrical interlinks, the conventional federated RL approaches using decoupled independent environments are not applicable, which leads us to propose a multi-agent vertically federated variation of actor-critic algorithms, namely federated soft actor-critic (FedSAC). We have performed numerical simulations on an IEEE 123-bus benchmark test feeder with three microgrids by creating a customized simulation setup by encapsulating the microgrid dynamic simulations in GridLAB-D/HELICS co-simulation platform with the OpenAI Gym environment and validated the proposed resilient and secured learning methodology.

Artificial Intelligence (AI), reinforcement learni↗

DOE OSTI2022

Deep Reinforcement Learning From Demonstrations to Assist Service Restoration in Islanded Microgrids

Microgrids can be operated in island mode during utility grid outages to support service restoration and improve system resilience. To schedule and dispatch distributed energy resources (DERs) in an islanded microgrid, conventional model-based methods rely on accurate distribution network models and lack generalization and adaptability. Data-driven methods are promising for DER coordination but face practical challenges such as potential hazards to microgrids during online training and insufficient online training opportunities due to low outage rates. This paper presents a novel two-stage learning framework that builds on the deep deterministic policy gradient from demonstrations to identify an optimal restoration strategy. At the pre-training stage, imitation learning is applied to equip the control agent with expert experiences to guarantee acceptable initial performance. At the online training stage, action clipping, reward shaping, and expert demonstrations are leveraged to ensure safe exploration while accelerating the training process. In conclusion, the proposed method is illustrated using the IEEE 123-node system and compared with a representative model-based method and the standard deep deterministic policy gradient method to prove solution accuracy and demonstrate increased computational efficiency.

Du, Yan↗

DOE OSTI2016

MO-A-BRB-00: TG191: Clinical Use of Luminescent Dosimeters

This presentation will highlight the upcoming TG-191 report: Clinical Use of Luminescent Dosimeters. Luminescent dosimetry based on TLD and OSLD is a practical, accurate, and precise technique for point dosimetry in medical physics applications. The charges of Task Group 191 were to detail the methodologies for practical and optimal luminescent dosimetry in a clinical setting. This includes (1) To review the variety of TLD/OSL materials available, including features and limitations of each. (2) To outline the optimal steps to achieve accurate and precise dosimetry with luminescent detectors and to evaluate the uncertainty induced when less rigorous procedures are used. (3) To develop consensus guidelines on the optimal use of luminescent dosimeters for clinical practice. (4) To develop guidelines for special medically relevant uses of TLDs/OSLs (e.g., mixed field i.e. photon/neutron dosimetry, particle beam dosimetry, skin dosimetry). While this report provides general guidelines for arbitrary TLD and OSLD processes, the report, and therefore this presentation, provide specific guidance for TLD-100 (LiF:Ti,Mg) and nanoDot (Al2O3:C) dosimeters because of their prevalence in clinical practice. Learning Objectives: Understand the available dosimetry systems, and basic theory of their operation Understand the range of dose determination methodologies and the uncertainties associated with them Become familiar with special considerations for TLD/OSLD relevant for special clinical situations Learn recommended commissioning and QA procedures for these dosimetry systems.

61 RADIATION PROTECTION AND DOSIMETRY↗

DOE OSTI2016

MO-A-BRB-01: TG191: Clinical Use of Luminescent Dosimeters

61 RADIATION PROTECTION AND DOSIMETRY↗

DOE OSTI2023

Learning macroscopic internal variables and history dependence from microscopic models

This paper concerns the study of history dependent phenomena in heterogeneous materials in a two-scale setting where the material is specified at a fine microscopic scale of heterogeneities that is much smaller than the coarse macroscopic scale of application. Here, we specifically study a polycrystalline medium where each grain is governed by crystal plasticity while the solid is subjected to macroscopic dynamic loads. The theory of homogenization allows us to solve the macroscale problem directly with a constitutive relation that is defined implicitly by the solution of the microscale problem. However, the homogenization leads to a highly complex history dependence at the macroscale, one that can be quite different from that at the microscale. In this paper, we examine the use of machine-learning, and especially deep neural networks, to harness data generated by repeatedly solving the finer scale model to: (i) gain insights into the history dependence and the macroscopic internal variables that govern the overall response; and (ii) to create a computationally efficient surrogate of its solution operator, that can directly be used at the coarser scale with no further modeling. We do so by introducing a recurrent neural operator (RNO), and show that: (i) the architecture and the learned internal variables can provide insight into the physics of the macroscopic problem; and (ii) that the RNO can provide multiscale, specifically FE 2 , accuracy at a cost comparable to a conventional empirical constitutive relation.

36 MATERIALS SCIENCE↗

DOE OSTI2025

Simultaneous prediction of structural properties in epitaxially–grown GaN with quantum and conventional multi–output learning algorithms

Hundreds of GaN thin film crystal plasma–assisted molecular beam epitaxy synthesis experiment records spanning two decades were organized into a dataset correlating the growth experiment design parameters with discrete, binary determinations of crystallinity and surface morphology. Conventional data science techniques as well as both quantum and classical multi–output supervised machine learning algorithms were implemented to investigate the relationships between the operating parameter data and the structural figures of merit. Correlation coefficients, decision tree nodes, p–values, and SHAP values all support substrate temperature and gallium effusion cell conditions as being statistically significant for simultaneously influencing GaN crystallinity and surface morphology. Here, a conventional deep neural network learned best from the data, followed by a quantum–classical hybrid gradient boosting algorithm. When combined with calculations of uncertainty intervals based on VennAbers predictors, machine learning predictions of both structural properties show good agreement with results reported in published experimental literature.

36 MATERIALS SCIENCE↗

DOE OSTI2021

Lessons from the COVID Era and Visions for the Future

In December 2020, the U.S. Department of Energy Office of Science convened a virtual Roundtable of its 27 operating scientific user facilities to discuss facility challenges and lessons learned during the COVID-19 pandemic as well as facility responses, best practices, and innovations that could be adopted going forward. Roundtable participants included facility staff, users, and user executive committee chairs. This report summarizes their discussions, which encompassed topics such as user research and facility operations in virtual and physically distanced contexts; user training and engagement; computation, data, and network resources; and crosscutting issues.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

NASA NTRS1994

An operations and command systems for the extreme ultraviolet explorer

About 40% of the budget of a scientific spacecraft mission is usually consumed by Mission Operations & Data Analysis (MO&DA) with MO driving these costs. In the current practice, MO is separated from spacecraft design and comes in focus relatively late in the mission life cycle. As a result, spacecraft may be designed that are very difficult to operate. NASA centers have extensive MO expertise but often lessons learned in one mission are not exploited for other parallel or future missions. A significant reduction of MO costs is essential to ensure a continuing and growing access to space for the scientific community. We are addressing some of these issues with a highly automated payload operations and command system for an existing mission, the Extreme Ultraviolet Explorer (EUVE). EUVE is currently operated jointly by the Goddard Space Flight Center (GSFC), responsible for spacecraft operations, and the Center for Extreme Ultraviolet Astrophysics (CEA) of the University of California, Berkeley, which controls the telescopes and scientific instruments aboard the satellite. The new automated system is being developed by a team including personnel from the NASA Ames Research Center (ARC), the Jet Propulsion Laboratory (JPL) and the Center for EUV Astrophysics (CEA). An important goal of the project is to provide AI-based technology that can be easily operated by nonspecialists in AI. Another important goal is the reusability of the techniques for other missions. Models of the EUVE spacecraft need to be built both for planning/scheduling and for monitoring. In both cases, our modeling tools allow the assembly of a spacecraft model from separate sub-models of the various spacecraft subsystems. These sub-models are reusable; therefore, building mission operations systems for another small satellite mission will require choosing pre-existing modules, reparametrizing them with respect to the actual satellite telemetry information, and reassembling them in a new model. We briefly describe the EUVE mission and indicate why it is particularly suitable for the task. Then we briefly outline our current work in mission planning/scheduling and spacecraft and instrument health monitoring.

Muscettola, Nicola↗

DOE OSTI2023

Autonomous System Inference, Trojan, and Adversarial Reprogramming Attack and Defense (Final)

In the world of ever-advancing technology, Autonomous Systems (AS) find extensive application, bolstering functionalities of critical infrastructures such as nuclear power plants. These systems, however, are increasingly becoming a target for nefarious activities, namely through inference attacks, trojan attacks, and adversarial reprogramming. This paper delves into a comprehensive exploration of machine learning (ML)-driven autonomous control systems within advanced nuclear reactor designs, revealing the vulnerabilities and proposing strategies for defense against potential cyber-attacks. Advanced cyber-attacks against critical infrastructure and the energy sector are becoming more common. With the invention of autonomous control systems (ACS) within advanced nuclear reactor designs, system designers, reactor operators, and regulators must consider cybersecurity during the design and operational phases. This article provides a cyber threat assessment of machine learning (ML)- based digital twinning (DT) technologies in the context of advanced reactor ACS. A cyber-physical testbed was created to emulate nuclear reactor digital instrumentation and controls (I&C) and act as a basis for the ACS. The ACS was designed as two plant-level DTs predicting reactor malfunctions and determining control actions and two component-level DTs responsible for classifying component states and forecasting component inputs and outputs (I/O). Two duplicate ACS designs– one using a traditional ML framework and one using an automated ML (AutoML) framework– were created and tested against cyber-attacks on training data, real-time process data, and ML model architectures to determine their respective qualitative cyber-risk in terms of likelihood and impact. Both frameworks showed similar cyber-resilience against training, real-time, and ML architecture attacks, proving that neither is inherently more secure. Recommended safeguard and security measures are posed to system designers, reactor operators, and regulators to maintain the cybersecurity of ML-based DT technologies such as ACS, prompting a holistic view of shared responsibility for maintaining cyber-secure ML-based systems. As global reliance on generation III reactors begins to be critically assessed, the evolution towards advanced reactor systems utilizing digital instrumentation and controls (I&C) becomes not merely preferable, but essential. The integration of semi and fully autonomous control systems (ACS), powered by digital I&C and machine learning (ML)-based digital twinning (DT) technologies, emerges as a potent strategy to mitigate operations and maintenance costs, thereby enhancing the economic feasibility of novel reactor designs. However, with a staggering 500% and 380% increase in cyber-attacks reported against the energy sector by the United States Department of Energy (DoE) and the European Union respectively, a surge in cyber vulnerabilities specifically targeting the nuclear industry has been 2 markedly observed. Notable incidents, such as the W32.Ramnit spyware infiltration at the Gundremmingen nuclear power plant in Germany and the Dtrack spyware intrusion at the Kudankulam nuclear power plant in India, while not directly compromising core industrial control systems (ICS), underscore a compelling necessity to fortify cybersecurity protocols in safeguarding reactor systems against increasingly adept digital adversaries. In light of this, our investigation extends beyond conventional cybersecurity parameters, diving into the intricate web of potential vulnerabilities woven into ML-based DTs and ACS in advanced reactor systems. A crafted cyber-physical testbed and preliminary ACS were devised to act as a mirror, reflecting potential configurations of advanced reactor control designs. Moreover, this study is intertwined with a scrutinization of ML models, developed either through conventional, manually tuned methodologies or via automated means through AutoML, probing into their cyber-risk profiles within operational technology (OT) environments. Expanding on this, two distinct ACS blueprints were forged – one navigating through the corridors of traditional ML and the other traversing the path of AutoML – in an effort to holistically encapsulate the considerations pivotal to ML-based DT control system design. Employing the SANS Institute Industrial Control System (ICS) Kill Chain and the MITRE ATT&CK Tactics, Techniques, and Procedures (TTP) framework, a structured analysis was conducted, launching three targeted attacks against the training dataset, real-time dataset, and ML models, therein dissecting the potential cyber-attack implications against both ML frameworks within an ACS milieu. It is essential to note that three distinct categories of attacks were conducted against both ACS configurations, each encompassing three distinct ML-based DTs, cumulating in a total of 18 varied attacks. This exploration extends into the realms of Autonomous System Inference, Trojan, and Adversarial Reprogramming Attack and Defense, unraveling vulnerabilities, and opportunities for fortified defenses against such intrusions, particularly where ML-driven technologies, and by extension, ACS, are deployed. Final recommendations, articulated through a lens of security, safeguard, and implementation considerations, are presented for both traditional and AutoML models, anchoring upon the existing knowledge landscape and ML-based DT modeling for ACS, and are offered as a beacon to guide the nuclear industry through the intricate cybersecurity challenges that lie ahead.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2020

Machine Learning Algorithms for Automated NIF Capsule Mandrel Selection

Small shells, approximately 2 mm in diameter, made from Poly(α-methylstyrene) (PAMS) are used as mandrels in the production of glow discharge polymer capsules located at the center of inertial confinement fusion experiments. The visual inspection process of microscope images of these shell mandrels, including detection of micron-sized defects on the shell surface as well as the handling and sorting, is a very labor-intensive, repetitive, and highly subjective process that stands to benefit greatly from automation. As part of an effort to decrease the number of labor hours spent in capsule handling, inspection, and metrology, the development of robotic systems was presented in a paper by Carlson et al., “Automation in Target Fabrication” [Fusion Sci. Technol., Vol. 70, p. 274 (2016)]. The current work expands the automated image acquisition systems developed previously and adds the use of convolutional neural networks to select capsules best suited for use in the downstream production process. Through the use of these machine learning algorithms, the selection process becomes robust, repeatable, and operator independent. Finally, as an added benefit the system developed as part of this work is able to provide defect statistics on entire shell batches and feed this information upstream to the production team.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2018

Glovebox Handling of High-Consequence Materials with Super Baxter and Gesture-Based Programming - 18598

The handling of high-consequence materials is a difficult task requiring safe object manipulation while avoiding the risk of contamination or spillage. Specific operations including opening containers, segregating the wastes, and repackaging are required to be safely executed inside contained spaces such as gloveboxes. However, the workers' ability and dexterity to manipulate objects through thick protective gloves are compromised in many aspects. The fixed position of the glovebox arm ports restricts the movements of the workers' arms, which also makes it hard to lift heavy objects. Further, the operational workspace inside the glovebox is restricted by the gloves' reachability. Safety of workers is the paramount concern in glovebox operations and the very reason for their existence. Sharp edges and tools increases the risk of glove punctures, which may expose the operator to chemicals and radiation and risks contamination in the vicinity outside the glovebox. The operators are also affected by ergonomic stressors due to prolonged and repetitive operations. To achieve a high degree of human safety, robotic solutions to handle high-consequence materials inside the glovebox are desirable, as they remove the operators from the hazards listed above. However, robots, in general, lack the degree of adaptability necessary for most high-consequence material handling tasks as these tasks are often unstructured or highly variable. Likewise, most human operators, while highly skilled in the tasks at hand (task expertise), lack the robot programming skills necessary to adapt the robot's motions to the variations (programming expertise). A way to bridge this divide is to make robot programming more intuitive to human operators. Since humans naturally teach one another through demonstration and learning, robotic 'programming by demonstration' paradigms have begun to appear to reduce the burden of robot reprogramming. Gesture-Based Programming attempts to achieve that by enabling a robot to observe the normal actions and affordances of a human performing a task to learn to map those onto the skills of the robot. In the end, this approach permits benefits from both the human and the robot: the human's adaptive decision making and the robot's resilience to high-consequence material. Robotic solutions assuringly provide protection for the operators, however their autonomy brings about other potential risks. Artificial Intelligence is not perfect; it is a black box, which can produce results, but the full extent of those results may not be known precisely. In self-driving cars, a hypothetical AI problem is the 'green man detector'. Since self-driving cars have presumably never seen a green man, how can we know it won't veer toward him the first time it sees a green man? Likewise, even though GbP provides an open method for safe task execution replicating a human, incorrect task inferences can lead to potential collisions or unintended operations. Thus, a hardware safety measure is proposed to provide reliable operation inside the glovebox, and true safe operations are achieved by mechanically limiting the operational range of each arm link. The operational range is computed and validated through exhaustive offline simulations, which can be streamlined by affordable computing power. Super Baxter is one such robot which incorporates these capabilities. It is a human-like, bimanual robot, being developed at the Collaborative Robotics Lab (CRL) in conjunction with the Dept. of Energy/Environmental Management, Rethink Robotics, Barrett Technology, and the NSF Center for Robots and Sensors for the Human Well-Being (RoSe-HUB). Super Baxter is envisioned to represent the next generation of collaborative robots that will be intrinsically human-safe, as well as exceptionally human-intuitive. In summary, Super Baxter will enable safe glovebox operations through low-shot Gesture based Programming incorporated with hardware safety measures. Two aspects of safety will be discussed in the paper. First, the safety of the operator achieved by allowing him to program Super Baxter remotely through Gesture based Programming. Second, the safety of the glovebox, achieved by incorporating hardware safety measures using joint limits. Further, the capability of Super Baxter on Gesture based Programming is demonstrated through typical object manipulation tasks in glovebox operations. (authors)

42 ENGINEERING↗

DOE OSTI2019

Optimizing Dynamic Timing Analysis with Reinforcement Learning

There are multiple factors involved in successfully manufacturing ASICIVLSI chips, and ensuring operational specifications are maintained throughout the design and manufacturing process is often challenging. Dynamic timing analysis (DTA) is the principal method used to validate that a manufactured chip complies to its design specifications. In DTA functionality of both synchronous and asynchronous designs are verified by applying input signals and checking for correct output signals. In complex designs where the number of input signal permutations is extremely large, the computing resources required to properly verify the functionality of a chip is prohibitive. In this paper, a strategy using reinforcement learning (RL) for reducing DTA time and resources in such cases is discussed. RL assisted DTA holds much promise in ensuring that VLSI chip design and functionality are fully and optimally verified.

42 ENGINEERING↗

NASA NTRSDate not supplied

Imagery Analysis for Space Operations

The purpose of this project is to build a prototype camera system that utilizes artificial intelligence/machine learning and computer vision to track an object in 2D and recreate it in 3D.The goal is to improve image analysis and photogrammetry function for Pad 39B operations

Robotics↗

NASA NTRSDate not supplied

Imagery Analysis for Space Operations

Computer Programming↗

NASA NTRSDate not supplied

Machine Learning Application for Improving Cloud Detection and Phase Determination Over Sunglint Regions for Geostationary Satellites

Cloud detection and phase determination over sunglint regions has been a challenge, especially for geostationary (GEO) satellites. Sunglint is observed when the sunlight specular reflection is at the same viewing angle of the satellite sensor. This intense reflection in the visible channels (VIS) is often comparable to that from optically thick clouds. It also contaminates the shortwave infrared channels (SWIR). Consequently, VIS and SWIR channels become less useful - or not useful- when they are saturated, hampering the detection of cloudy and clear-sky pixels. Sunglint contamination happens frequently and exists nearly in every daytime GEO full disk satellite images. However, sunglint intensity and region are difficult to model due to variable viewing geometry and ocean surface conditions. Moreover, existing physical models do not meet the accuracy required for operational GEO satellite cloud detection. We developed a machine learning algorithm to improve cloud detection in sunglint conditions for the NASA Langley’s Satellite ClOud and radiation Property retrieval System (SatCORPS). This poster presents our recent progress in the algorithm development, validation and applications. The algorithm is validated using collocated SatCORPS GOES-East and GOES-West cloud products. We demonstrate that the machine learning cloud detection in sunglint regions is superior to the traditional approach by improving temporal consistency between sunglint and non-sunglint conditions.

Machine Learning, Cloud detection, Sunglint, SatCO↗

DOE OSTI2025

Explainable discrepancy checker and diagnosis for digital Twin-based supervisory control system

By virtually representing a physical object and process, a digital twin (DT) enables optimal autonomous operations by combining classical and novel frameworks in sensors, state predictions, and multi-input/multi-output systems. A DT’s values depend on how well models estimate quantities of interest and on how uncertainty is handled. Moreover, DTs often combine physics-based and data-driven models with mixed fidelities, where classical uncertainty quantification (UQ) struggles with many sources of uncertainty and real-time constraints. Here, this work presents a UQ-based discrepancy checking and diagnosis tool for a DT-based supervisory control system. The tool is developed using metadata from an automated DT development process to learn correlations between sources of uncertainties and outcomes. During operation, it compares predictions with measurements, attributes discrepancies to dominant sources, and recommends parameter and configuration updates. We verify the workflow on a synthetic temperature-control problem and deploy it on a virtual Thermal Energy Delivery System, reducing mismatch and improving control robustness.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗