Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “sensor augmentation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Improving an Acoustic Vehicle Detector Using an Iterative Self-Supervision Procedure

In many non-canonical data science scenarios, obtaining, detecting, attributing, and annotating enough high-quality training data is the primary barrier to developing highly effective models. Moreover, in many problems that are not sufficiently defined or constrained, manually developing a training dataset can often overlook interesting phenomena that should be included. To this end, we have developed and demonstrated an iterative self-supervised learning procedure, whereby models are successfully trained and applied to new data to extract new training examples that are added to the corpus of training data. Successive generations of classifiers are then trained on this augmented corpus. Using low-frequency acoustic data collected by a network of infrasound sensors deployed around the High Flux Isotope Reactor and Radiochemical Engineering Development Center at Oak Ridge National Laboratory, we test the viability of our proposed approach to develop a powerful classifier with the goal of identifying vehicles from continuously streamed data and differentiating these from other sources of noise such as tools, people, airplanes, and wind. Using a small collection of exhaustively manually labeled data, we test several implementation details of the procedure and demonstrate its success regardless of the fidelity of the initial model used to seed the iterative procedure. Finally, we demonstrate the method’s ability to update a model to accommodate changes in the data-generating distribution encountered during long-term persistent data collection.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Spin-squeezed vector atomic magnetometry

Atomic magnetometers based on Zeeman shift measurement have the potential for high sensitivity and long-term stability. Like other atomic sensors, including atomic clocks and atom interferometers, the atomic magnetometer could, in principle, be augmented with spin squeezing for further sensitivity enhancement. However, existing atomic magnetometers are not compatible with spin squeezing because the atoms can hardly be in a pure quantum state during operation. Moreover, a natural challenge faced by an atomic magnetometer is the arbitrary direction of the magnetic field. In this paper, we propose a cold-atom-based magnetometer with spin squeezing that can measure both the magnitude and the direction of an arbitrary magnetic field. For experimentally accessible parameters, we show that the technique described above could potentially achieve a sensitivity nearly two orders of magnitude higher than that of the best existing magnetometers.

Li, Jinyang [Northwestern U.] (ORCID:0000000194328↗

Molecularly Imprinted Polymer Sensor Empowered by Bound States in the Continuum for Selective Trace-Detection of TGF-beta

The integration of advanced materials and photonic nanostructures can lead to enhanced biodetection capabilities, crucial in clinical scenarios and point-of-care diagnostics, where simplified strategies are essential. Herein, a molecularly imprinted polymer (MIP) photonic nanostructure is demonstrated, which selectively binding to transforming growth factor-beta (TGF-β), in which the sensing transduction is enhanced by bound states in the continuum (BICs). The MIP operating as a synthetic antibody matrix and coupled with BIC resonance, enhances the optical response to TGF-β at imprinted sites, leading to an augmented detection capability, thoroughly evaluated through spectral shift and optical lever analogue readout. The validation underscores the MIP-BIC sensor capability to detect TGF-β in spiked saliva, achieving a limit of detection of 10 fM and a resolution of 0.5 pM at physiological concentrations, with a precision of two orders of magnitude above discrimination threshold in patients. The MIP tailored selectivity is highlighted by an imprinting factor of 52, showcasing the sensor resistance to interference from other analytes. The MIP-BIC sensor architecture streamlines the detection process eliminating the need for complex sandwich immunoassays and demonstrates the potential for high-precision quantification. This positions the system as a robust tool for biomarker detection, especially in real-world diagnostic scenarios.

77 NANOSCIENCE AND NANOTECHNOLOGY↗

Searching for metastable particles using graph computing

The reconstruction of charged particle trajectories at the Large Hadron Collider and future colliders relies on energy depositions in sensors placed at distances ranging from a centimeter to a meter from the colliding beams. We propose a method of detecting charged particles that decay invisibly after traversing a short distance of about 25 cm inside the experimental apparatus. One of the decay products may constitute the dark matter known to be 84% of all matter at galactic and cosmological distance scales. Our method uses graph computing to cluster spacepoints recorded by two-dimensional silicon pixel sensors into mathematically-defined patterns. The algorithm may be implemented on silicon-based integrated circuits using field-programmable gate array technology to augment or replace traditional computing platforms.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Design of Digital Twin Sensing Strategies Via Predictive Modeling and Interpretable Machine Learning

This work develops a methodology for sensor placement and dynamic sensor scheduling decisions for digital twins. The digital twin data assimilation is posed as a classification problem, and predictive models are used to train optimal classification trees that represent the map from observed data to estimated digital twin states. In addition to providing a rapid digital twin updating capability, the resulting classification trees yield an interpretable mathematical representation that can be queried to inform sensor placement and sensor scheduling decisions. The proposed approach is demonstrated for a structural digital twin of a 12 ft wingspan unmanned aerial vehicle. Offline, training data are generated by simulating scenarios using predictive reduced-order models of the vehicle in a range of structural states. Furthermore, these training data can be further augmented using experimental or other historical data. In operation, the trained classifier is applied to observational data from the physical vehicle, enabling rapid adaptation of the digital twin in response to changes in structural health. Within this context, we study the performance of the optimal tree classifiers and demonstrate how they enable explainable structural assessments from sparse sensor measurements and also inform optimal sensor placement.

47 OTHER INSTRUMENTATION↗

Combining In-situ Diagnostics and Data Analytics for Discovery of Process-Structure-Property Relationships in AM parts – A Step Toward Digital Twins

In-situ additive manufacturing (AM) diagnostic tools (e.g., optical/infrared imaging, acoustic, etc.) already exist to correlate process anomalies to printed part defects. This current work aimed to augment existing capabilities by: 1) Incorporating in-situ imaging w/ machine learning (ML) image processing software (ORNL- developed "Peregrine") for AM process anomaly detection 2) Synchronizing multiple in-situ sensors for simultaneous analysis of AM build events 3) Correlating in-situ AM process data, generated part defects and part mechanical properties The key R&D question investigated was to determine if these new combined hardware/software tools could be used to successfully quantify defect distributions for parts build via SNL laser powder bed fusion (LPBF) machines, aiming to better understand data-driven process-structure-property- performance relationships. High resolution optical cameras and acoustic microphones were successfully integrated in two LPBF machines and linked to the Peregrine ML software. The software was successfully calibrated on both machines and used to image hundreds of layers of multiple builds to train the ML software in identifying printed part vs powder. The software's validation accuracy to identify this aspect increased from 56% to 98.8% over three builds. Lighting conditions inside the chamber were found to significantly impact ML algorithm predictions from in-situ sensors, so these were tailored to each machine's internal framework. Finally, 3D part reconstructions were successfully generated for a build from the compressed stack of layer-wise images. Resolution differences nearest and furthest from the optical camera were discussed. Future work aims to improve optical resolution, increase process anomalies identified, and integrate more sensor modalities.

36 MATERIALS SCIENCE↗

Using Fiducial Markers for Pose Estimation of an OSWEC in a Wave Tank: Preprint

In this study, we consider a novel method of sensing the motion of a wave energy converter during testing in a wave flume under the influence of incoming waves. The wave energy converter considered in our research is an oscillating surge wave energy converter, which is a hinged paddle that responds to incoming waves. Motion sensing is normally done with inertial sensors, which can hinder the motion due to suspended cables that carry power and transmit signals. Our proposed method is contactless and can be implemented economically. A camera is used to record different marker patterns affixed to the moving paddle and the motion deduced by pose estimation algorithms. Fiducial markers are commonly used for robot localization and in augmented reality. There are many types of fiducial markers, including ArUco-type markers which are accurate, fast and robust. The system consists of markers attached to the paddle element and recorded using a machine vision camera. A pose estimation algorithm is then applied to the detected markers to estimate the tilt of the paddle. In this work, we examine the challenges of image acquisition and calibration for underwater targets, compare the motion obtained by this new system with a calibrated tilt sensor and identify areas where the new system may be superior.

computer vision↗

Power generation forecasting for solar plants based on Dynamic Bayesian networks by fusing multi-source information

A Dynamic Bayesian network (DBN) model for solar power generation forecasting in solar plants is proposed in this paper. The key idea is to fuse sensor data, operational indicators, meteorological data, lagged output power information, and model errors for more accurate short-term (e.g., hours) and mid-term (e.g., days to weeks) power generation forecasting. The proposed DBN augments automated data-driven structure learning with expert knowledge encoding using continuous and categorical data given constraints to represent causal relationships within a solar inverter system. Additionally, an error compensation mechanism is proposed to capture temporal fluctuation. The effectiveness of the DBN on solar power generation forecasting was evaluated by rolling window analysis with one-year testing data collected from a local solar plant. The proposed DBN is compared with four state-of-art methods including support-vector regression (SVR), k-nearest neighbors (kNN), artificial neural network (ANN), and long short-term memory (LSTM) models. The result show that the proposed DBN achieves better accuracy in general, and it is not as data-hungry as some neural network-based models. The proposed DBN is also shown to have robust and consistent forecasting power with different forecasting horizons. The accuracy is 92% - 95% from one hour to one week ahead forecasting.

14 SOLAR ENERGY↗

Control Algorithms for Dual-wavefront Sensor Single-conjugate Adaptive Optics

High-contrast imaging systems using active control with adaptive optics (AO) are often limited by non-common path (NCP) aberrations that are seen only at the final science image. AO systems employing focal-plane wavefront sensors (FP-WFSs) are able to simultaneously correct NCP aberrations and measure science images, but they typically require a second stage of control that adds system cost and complexity. We present control algorithms to augment AO systems with FP-WFSs within their existing control setup. We demonstrate inter-arm NCP aberration transfer can be mitigated through temporal filtering, present frequency- and time-domain validation of controller stability and performance, and discuss the optimality of the chosen controllers. This work will enable the development, testing, and installation of FP-WFS technologies for direct imaging of exoplanets.

79 ASTRONOMY AND ASTROPHYSICS↗

Implementing a Terrestrial Timing Solution: Best Practices

This document is an overview and guide on the concept of precision time and how an alternative terrestrial timing solution to the Global Positioning System (GPS) can be implemented. What follows is a set of recommendations and best practices, derived from research at Oak Ridge National Laboratory (ORNL), across industry, and within the U.S. government, to implement a system of precision timing synchronization to support resilient operations for the U.S. grid. Precision time is a fundamental necessity for operating the grid today, and it becomes even more important as the grid modernizes with clean energy sources, microgrids, and precision sensors placed throughout the grid system to ensure failureresistant operations. The Global Navigation Satellite System (GNSS), which includes (GPS), is currently the primary provider of precision timing today. A terrestrial-based system for time delivery and synchronization to augment GPS is outlined below and it would provide secure time, synchronized with Coordinated Universal Time (UTC), in the event of outages or other interruptions associated with time delivery.

47 OTHER INSTRUMENTATION↗

Spatially Adaptive Tunable Lighting Control System with Expanded Wellness and Energy Saving Benefits

Lighting design is becoming increasingly complex, including active dimming for energy savings and spectral tuning for human wellbeing. Modern commercial lighting control systems are already difficult to use, and maintain, and even with so-called Smart Lighting, optimizing light settings is rapidly exceeding the capabilities of direct human control, limiting the adoption of lighting control systems. The growing importance of Occupancy Centric Controls (OCC) for advanced building management systems can be applied to the development of autonomous lighting system controls needed to drive the adoption of advanced lighting controls for improved adaptive “sculpted illumination” for both greater energy savings and broader human wellbeing perspectives. This project, led by Rensselaer Polytechnic Institute, entitled “Spatially adaptive tunable lighting control system with expanded wellness and energy saving benefits” will develop and test an entirely new platform for automated optimized lighting design and control that requires little or no human engagement, yet contours lighting profiles automatically to minimize lighting energy use. We call this approach to lighting control “dynamic light sculpting” since the right amount of illumination is automatically delivered to occupants in real time. Since the system uses new, privacy-preserving occupant position and pose detection technologies developed by Rensselaer for broad OCC building applications, the control system will analyze how to deliver the right amount of the right type of illumination only where and when it is needed based these OCC platforms. To create these powerful autonomous lighting control platforms, the project will integrate evolving augmented reality (AR) and virtual reality (VR) tools with sophisticated lighting and interior design toolkits to create interactive lighting design and control simulators. Using the quickly growing paradigm of digital twins, these tools will integrate light fixture properties, occupancy sensor data, interior design data, and lighting specifications to accurately visualize how various design concepts interact with simulated yet realistic human activities that occur in commercial office buildings of various types. These advanced digital twin design and simulation tools will be combined with a new class of digitally-programmable LED lighting fixtures that can dynamically change the spectral content and direction of light emission. When fully integrated, the new autonomous lighting control system will take all of the guesswork out of optimizing light quality while minimizing energy consumption. Digital twin tools will simplify the design, installation, commissioning, operation, and maintenance of future energy-efficient lighting systems. It should be possible to reduce lighting energy costs from 40% to 70% with OCC based dynamic light sculpting systems. The work will be led by the Center for Lighting Enabled Systems and Applications (LESA) and the Center for Architectural Science and Ecology (CASE), both at Renssealer Polytechnic Institute; Lumileds, a global leader in the development of advanced LED systems; and HKS, a leading global architecture design firm. This interdisciplinary project team’s experience includes all of the design simulation tools, VR/AR digital twin visualization technology, advanced occupancy sensing technology, and complex control system design capabilities that will revolutionize lighting design and control technology to autonomously deliver high-quality lighting that improves human health and wellbeing while simultaneously maximizing lighting energy savings.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Sensor discriminators and methods for detecting electrical property changes in a metal organic framework

A sensor discriminator for detecting a gaseous substance includes a power source, a discrimination module, a sensor simulator that simulates a metal organic framework under at least one simulation condition, a simulation circuitry electrically coupling the sensor simulator to the power source and the discrimination module, and a discriminator circuitry that electrically couples the power source and the discrimination module to a gas capture probe. The discrimination module compares a discrimination pulse and a simulation pulse from the power source after the discrimination pulse passes through a metal organic framework of the gas capture sensor and the simulation pulse passes through a simulation component of the sensor simulator. The discrimination module causes a discriminator output that includes the comparison of the discrimination pulse to the simulation pulse. An electrical property of the discrimination pulse depends on an electrical parameter of the metal organic framework that is augmented by the gaseous substance.

Christensen, Daniel A.↗

Controllable oxygen vacancy defect engineering of BiVO 4 porous structures for room temperature NH 3 detection

Controlling structural features of sensing material while judiciously introducing vacancy defect states for revamping the electronic properties of the sample to obtain its superior gas sensing performance, is quite rare. Herein, we report for the first time, the room temperature (RT) ammonia (NH 3 ) detection of peanut-like porous bismuth vanadate (BiVO 4 ) with a stable monoclinic phase. The developed BiVO 4 possess abundant oxygen vacancies and porosity by virtue of calcinations (400–800 ℃). BiVO 4 calcined at 400 ℃ exhibits high selectivity towards NH 3 with a maximum response of 1421 @ 270 ppm at RT, which is 2.5 fold enhanced compared to without calcined BiVO 4 (response of 547 @ 270 ppm NH 3 ). The oxygen defective BiVO 4 appeared highly durable and stable even under high humid conditions (∼60 %). Besides, the porosity of BiVO 4 not only enhances the specific surface area (19.8 m 2 /g) but also results in fast diffusion of NH 3 molecules, leading to a reduction in the decay time (34 s for 90 ppm NH 3 ). The density functional theory (DFT) uncovers that the oxygen vacancy formation in BiVO 4 augments the NH 3 sensing capabilities by enhancing the adsorption energy of NH 3 . This work provides insight into the sensing mechanism of increased response caused by defect engineering and porosity, which will be favourable for fabricating high-performance NH 3 sensors at RT.

DFT analysis↗

Coupled Aerodynamic and Hydrodynamic Hybrid Simulation of Floating Offshore Wind Turbines

The development and innovation of floating offshore wind energy in the U.S. requires detailed high-fidelity observations and measurements of turbine and platform loading due to wind, waves, and currents. However, full-scale and quasi-full-scale experiments require significant financial and temporal investments for construction, experimental testing, and long-term field campaigns. To support the commercial advancement of the offshore wind energy industry, specialized wind tunnel and wave basin experimental facilities are critical to be able to test FOWT designs at small scale under controlled conditions prior to full-scale deployment. Oregon State University (OSU) is internationally known as a leader in water and energy research, development, and testing. The O.H. Hinsdale Wave Research Laboratory (HWRL) and the Wallace Energy Systems and Renewables Facility (WESRF) at OSU have extensive experience building, modeling, monitoring, controlling, and actuating scaled systems. Experiments on wave-structure interaction have been performed at the HWRL since its establishment in 1972. Studies have included the interaction of waves with coastal structures (breakwaters, seawalls, buildings, cylinders, bridges, fixed foundations of offshore wind turbines, etc.) and with floating structures (e.g., wave energy converters, maneuvering of vessels, etc.). Hinsdale is actively used by marine energy technology developers, both for private testing and OSU-collaborative research projects. However, despite the availability of several large-scale facilities for hydrodynamic testing (at OSU and elsewhere in the U.S.), existing experimental laboratories are generally limited in their ability to accurately generate combined wind and wave conditions. The simulation of both wind and waves in experimental testing is complicated due to a number of constraints, including: [i] incompatible similitude laws governing the wind and waves for scaled experiments, [ii] producing accurate wind over a large enough control volume via fans, and [iii] generating wind that reasonably represents the atmospheric boundary layer in existing wave basins/flumes. Hence, physical test data providing insight into the simultaneous wave- and wind-structure response of floating offshore wind components can be difficult to generate. Given the aforementioned challenges in classic hydrodynamic experiments, the motivation of this project is to establish a real-time hybrid simulation (RTHS) approach that can apply aero- and hydro-dynamic loading by augmenting wave-only experimental facilities with virtual aerodynamic forces through numerical models representing the remaining dynamic forces. RTHS is a physical-numerical approach that partitions a prototype system into physical and numerical sub-assemblies that interact with each other through actuators and sensors in real time. In coupling physical and numerical models, the hybrid simulation approach applied herein is ideal for problems with: (1) structures subjected to different scaling laws, such as floating offshore wind turbines subjected to combined aero/hydro-dynamic loading, (2) structures that are too large or complex to be tested entirely in a laboratory setting, such as deep-water mooring applications, and (3) component testing, where the behavior of a portion of the assembly is uncertain but still interacts with other portions of the structure, such as testing the fatigue life of turbine blades. Few U.S. experimental facilities are able to test simultaneous aero- and hydro-dynamic loading and none can accurately produce aero/hydro-dynamic response on scaled FOWT models due to conflicting similitude laws between the wind (commonly Reynolds) and the waves (commonly Froude). To aid in accelerating the development of the U.S. floating offshore industry, there is a significant need to develop a flexible, modular framework that can expand the capacities of existing wave-only laboratories. The project goal is to demonstrate a hydrodynamic real-time hybrid simulation (hydro-RTHS) framework that couples numerical wind and physical waves acting on a FOWT, thus representing simultaneous aero/hydro-dynamic loading. The FOWT is partitioned into a full-scale numerical sub-assembly associated with the aerodynamics and a model-scale physical sub-assembly associated with the hydrodynamics. The numerical-physical partition associated with hydro-RTHS mitigates scaling constraints by supplying different scaling laws to the physical and numerical sub-assemblies. Herein, length, force, and time are scaled and exchanged between the sub-assemblies using Froude scaling to represent the open-channel flow in the physical sub-assembly. Other similitude laws could also be utilized depending on the problem definition. It is envisioned that the ability to model FOWTs under waves and wind, with mitigation of similitude distortions, would result in reduced development costs (currently, FOWT concept development is performed with full-size pro- totypes at enormous expense and risk) and increase the reliability of the FOWT industry (since extreme wave and wind conditions and contingency events can be tested safely in a controlled environment).

16 TIDAL AND WAVE POWER↗

Emerging Threats and Technology Investigation: Industrial Internet of Things - Risk and Mitigation for Nuclear Infrastructure

Industries supporting the global nuclear infrastructure striving for cost savings, expansions in efficiency, and convenience are likely to adopt components (e.g., hardware, software) that comprise the Internet of Things (IoT) and Industrial Internet of Things (IIoT). These devices offer potential improvements along with security challenges. Modern conveniences achieved through application of technology have propagated through society in the form of interconnected devices, from doorbells to microwave ovens, commonly referred to as IoT. IoT devices are often Internet-connected devices that are designed to send data back to a cloud-based server, where a smart phone application then presents device status and control options. Home-based IoT applications carry a different set of risks when compared to a business or security environment, where there is also a history of convenience and interconnection. Industrial settings have long relied on specifically designed Supervisory Control and Data Acquisition (SCADA) systems for process control where IIoT devices are intended to inform business decisions and augment traditional processes. A recent National Institute of Standards and Technology (NIST) report provides a distinction between process control and IIoT in that traditional process control is not replaced by IIoT, but rather IIoT devices are intended to enhance industrial processes through additional monitoring of various sensors and application of data analytics models using artificial intelligence (AI) and machine learning (ML) (Fagan, Marron, et al. 2021) (Ross, et al. 2021).

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

A comparative study on deep learning models for condition monitoring of advanced reactor piping systems

Advanced nuclear reactors offer innovative applications due to their portability, reliability, resiliency, and high capacity factors. To operate them on a wider scale, reducing maintenance life-cycle costs while ensuring their integrity is essential. Autonomous operations in advanced nuclear reactors using augmented Digital Twin (DT) technology can serve as a cost-effective solution by increasing awareness about the system’s health. A key component of nuclear DT frameworks is the condition monitoring of safety systems, such as piping-equipment systems, which involves acquiring and monitoring the plant’s sensor data. Here, this research proposes a condition monitoring methodology utilizing deep learning algorithms, such as multilayer perceptions (MLP) and convolutional neural networks (CNNs), to detect degradation and its severity in nuclear piping-equipment systems. Sensor signals are processed to obtain the power spectral density and the Short-Time Fourier transform, and feature extraction methodologies are proposed to develop degradation-sensitive data repositories. The performance of MLP, one-dimensional (1D) CNN, and 2D CNN within the proposed condition monitoring framework is compared using a finite element model of a 3D piping system subjected to seismic loads as the application case study. Various approaches, such as dropout, k-Fold validation, regularization, and early stopping of training the network, are investigated to avoid overfitting the models to the input sensor data. The predictive capability and computational capacity of the deep learning algorithms are also compared to detect degradation in the Z-pipe system of the Experimental Breeder Reactor II (EBRII). The Z-pipe system is subjected to harmonic excitations that represent normal operating loads, such as pump-induced vibrations. The findings of the study indicate that the proposed artificial intelligence (AI)-driven condition monitoring framework demonstrates superior prediction accuracies with a 2D CNN, whereas the MLP exhibits higher computational efficiency.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Data Visualization: Augmented Reality

In recent years, there has been an increasing interest in developing new technologies for automated characterization and visualization of condition monitoring data. Augmented Reality (AR) is a technology that is being developed to improve such data visualization. Augmented reality has been defined as a technology that merges virtual and physical components in real-time, and in three dimensions. Wearable, commercially-available AR devices allow onsite engineers and technicians to perform inspection tasks with significantly more available information such as comparisons of past and present sensor and imager data, onsite data analysis and result displays, and various forms of metadata including technical drawings, previous inspection reports and maintenance histories, operation manuals, codes and standards, and holograms representing data analysis results superimposed onto the in situ monitored system.

42 ENGINEERING↗

Intelligent Process Visualization through Nuclear Operation Process Modeling, Reasoning, and Object Detection from Field Videos (Final Report)

This report is a deliverable for the “Final Report” task of DOE NEET Project 19-16790, "Context-Aware Safety Information Display for Nuclear Field Workers." This project's overall goal is to test the hypothesis that integrating computer vision and process reasoning methods will enable proactive visualization of the safe operation and maintenance processes of Nuclear Power Plants (NPP) for field workers. Augmented Reality (AR) glasses adopting such proactive safety information visualization techniques can significantly increase personnel safety and reduce the NPP’s operating costs. The current practice of monitoring NPPs requires workers to switch between digital models, data, and physical workspaces in identifying relevant but potentially occluded objects and in assessing the risks of operation and maintenance processes. On the other hand, frequently changed field conditions require field workers to report to supervisors for real-time guidance. Such guidance is essential to ensure that changing conditions will not invalidate or endanger the work order and other ongoing processes that may jeopardize NPP operations. Additionally, incorrect recognition of equipment objects can result in communication errors and safety problems. AR techniques can assist engineers in viewing the physical workspaces with objects labeled with detailed operation procedures and safety reminders during field operations. The project team developed an “Intelligent Context-Aware Safety Information Display” (ICAD) for supporting Nuclear Power Plant (NPP) field workers in achieving safe and efficient execution of a series of operational tasks in uncertain and changing workspaces of an NPP. Before designing the ICAD-AR prototype, the project team synthesized NPP operational knowledge models through literature review studies, surveys, interviews with domain experts, and knowledge modeling. The project team conducted an extensive study of the operational procedures of various NPPs, and digital technologies that can support the safe and efficient execution of those procedures in different NPP operational contexts. This literature review helped the project team conduct surveys and interviews with nuclear engineers and field workers to identify three categories of information. The NPP knowledge modeling efforts reveal that the three categories of information identified have different levels of importance in a typical procedure of carrying out a series of tasks to achieve a specific NPP operation goal (e.g., shutdown, mode changes). These three categories of information include 1) Workspace dynamics – the changing spatial arrangements of workspaces, tools, protection equipment, and supporting materials, 2) Workflow prognostics – the dynamic dependencies between different parts of an NPP that functionally support and influence each other in terms of safety and efficiency, and 3) Hazards – objects and spaces that contain hazardous materials or physical conditions that can pose risks to workers or mechanical systems. The project team has profiled the importance levels of these categories of information into a knowledge model. This knowledge model specifies what types of information are more critical for a given task in a given workspace so that computers can automatically identify critical objects and sensors in a scene for delivering context-ware safety information to field workers through AR devices. Significant research development of this project results in technical research outcomes and a prototyping system that illustrates the technical feasibility of establishing an ICAD-AR system supporting the proactive safety information display for nuclear field workers. This final report summarizes the project team’s technological achievements in the past three years. Overall, the project team completed the development and integration of five techniques into a prototype ICAD Augmented Reality (ICAD-AR) system and demonstrated the developed system’s real-time execution in a mechanical room. The project team completed the analysis of using this prototype in other types of workspaces based on 3D image data and digital design models collected from two additional workspaces (a water treatment plant and a flow loop training facility). The integrated techniques include 1) Natural Language Processing (NLP) algorithms supporting the generation and updates of nuclear fieldwork process models based on text analysis of work packages and operation manuals; 2) sensor log analysis for predicting control actions in given sensor reading contexts; 3) computer vision algorithms for automatic localization and navigation of workers; 4) object detection algorithms for identifying task-related objects and correlated sensors for safety checking; 5) AR technique as a platform for supporting the integration. The testing results of these five techniques have shown that 1) the sensor log analysis model can predict the next control action with an accuracy of 0.883; 2) the trained natural language processing model can extract more than 80% of the critical information from paper-based procedures (PBPs); 3) the navigation algorithm with the integration of Visual Inertial Odometry (VIO) and Non-Recursive Bayesian Filter methods make operator’s trajectory estimation resilient to drift error; 4) the computer vision algorithm can detect task-specific and safety-critical objects with an average accuracy of 95.3%. The project team used work procedures collected from a flow loop training facility and two datasets collected from two mechanical rooms simulating the workspaces of NPPs to demonstrate the technical capabilities of the developed ICAD-AR prototype. The demonstration validated the technical feasibility of establishing the ICAD-AR system for nuclear field workers and identified the challenges in 1) automatic text analysis of work packages; 2) use of limited samples of sensor logs for predicting the proper timings of control actions; 3) reliably tracking workers and their task progress in mechanical rooms with many similar objects.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗