Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Visual control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Terahertz‐Nanoscale Visualization of the Microscopic Spin‐Charge Architecture of Colossal Magnetoresistive Switching

Resolving sub-10 nm spin switching and the associated terahertz (THz) electrodynamics during the colossal magnetoresistance (CMR) transition is a definitive frontier in reaching the fundamental spatial, temporal, and energy-dissipation limits of spin-electronics. Yet, simultaneous control of high magnetic field, cryogenic environment, and nanometer resolution has remained an elusive benchmark for THz nanoscopy, leaving the local THz dynamics of these transitions largely unexplored. Here, we overcome these limitations by utilizing a custom-built cryogenic magneto-THz scattering-type scanning near-field optical microscopy (cm-THz-sSNOM) to resolve the near-field THz spectroscopic evolution of the magnetic field-driven CMR transition in a manganite single crystal. Our measurements provide a nanoscale visualization of the THz conductivity, capturing the moment that magnetic-field-induced spin switching triggers the transition from an antiferromagnetic insulator to a ferromagnetic metal. An ellipsoidal near-field model reveals a multi-scale transition initiated by 1–2 nm isolated spin-flip sites at low magnetic fields, which coalesce into ∼15 nm conducting regions as the threshold field is approached. These results provide an nano-THz view of CMR switching, establishing an analysis framework for mapping spin–charge–lattice–orbit–coupled dynamics at spatial scales that transcend the nominal sSNOM resolution.

Haeuser, Samuel [Iowa State University, Ames, IA (↗

Accelerating discoveries at DIII-D with the Integrated Research Infrastructure

DIII-D research is being accelerated by leveraging high performance computing (HPC) and data resources available through the National Energy Research Scientific Computing Center (NERSC) Superfacility initiative. As part of this initiative, a high-resolution, fully automated, whole discharge kinetic equilibrium reconstruction workflow was developed that runs at the NERSC for most DIII-D shots in under 20 min. This has eliminated a long-standing research barrier and opened the door to more sophisticated analyses, including plasma transport and stability. These capabilities would benefit from being automated and executed within the larger Department of Energy Advanced Scientific Computing Research program’s Integrated Research Infrastructure (IRI) framework. The goal of IRI is to empower researchers to meld DOE’s world-class research tools, infrastructure, and user facilities seamlessly and securely in novel ways to radically accelerate discovery and innovation. For transport, we are looking at producing flux matched profiles and also using particle tracing to predict fast ion heat deposition from neutral beam injection before a shot takes place. Our starting point for evaluating plasma stability focuses on the pedestal limits that must be navigated to achieve better confinement. This information is meant to help operators run more effective experiments, so it needs to be available rapidly inside the DIII-D control room. So far this has been achieved by ensuring the data is available with existing tools, but as more novel results are produced new visualization tools must be developed. In addition, all of the high-quality data we have generated has been collected into databases that can unlock even deeper insights. This has already been leveraged for model and code validation studies as well as for developing AI/ML surrogates. The workflows developed for this project are intended to serve as prototypes that can be replicated on other experiments and can be run to provide timely and essential information for ITER, as well as next stage fusion power plants.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Employing Eye Trackers to Reduce Nuisance Alarms

When process operators anticipate an alarm prior to its annunciation, that alarm loses information value and becomes a nuisance. This study investigated using eye trackers to measure and adjust the salience of alarms with three methods of gaze-based acknowledgement (GBA) of alarms that estimate operator anticipation. When these methods detected possible alarm anticipation, the alarm’s audio and visual salience was reduced. A total of 24 engineering students (male = 14, female = 10) aged between 18 and 45 were recruited to predict alarms and control a process parameter in three scenario types (parameter near threshold, trending, or fluctuating). The study evaluated whether behaviors of the monitored parameter affected how frequently the three GBA methods were utilized and whether reducing alarm salience improved control task performance. The results did not show significant task improvement with any GBA methods (F(3,69) = 1.357, p = 0.263, partial η 2 = 0.056). However, the scenario type affected which GBA method was more utilized (X 2 (2, N = 432) = 30.147, p < 0.001). Alarm prediction hits with gaze-based acknowledgements coincided more frequently than alarm prediction hits without gaze-based acknowledgements (X 2 (1, N = 432) = 23.802, p < 0.001, OR = 3.877, 95% CI 2.25–6.68, p < 0.05). Participant ratings indicated an overall preference for the three GBA methods over a standard alarm design (F(3,63) = 3.745, p = 0.015, partial η 2 = 0.151). This study provides empirical evidence for the potential of eye tracking in alarm management but highlights the need for additional research to increase validity for inferring alarm anticipation.

99 - GENERAL AND MISCELLANEOUS↗

Observation of polarization density waves in SrTiO 3

The nature of the incipient ferroelectric transition in SrTiO 3 has been a long-standing puzzle in condensed matter physics. One explanation involves the competition between ferroelectricity and an instability characterized by the mesoscopic modulation of the polarization. These polarization density waves, which should intensify near the quantum critical point, break local inversion symmetry and are difficult to characterize with conventional X-ray scattering methods. Here, in this study, we probe inversion symmetry breaking at finite momenta and visualize the instability of the polarization at the nanometre scale in SrTiO3 by combining a femtosecond X-ray free-electron laser with terahertz coherent control methods. We found polar-acoustic collective modes that are soft, particularly at the tens of nanometre scale. These precursor collective excitations provide evidence for the conjectured mesoscopic-modulated phase in SrTiO 3 .

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Visualizing Stereodynamics in Cold Collisions through Shape Resonance Wavefunctions

Shape resonances in cold collisions are often strongly affected by stereodynamics. These resonances can sometimes be assigned by single-channel quantization along the scattering coordinate. However, sensitive steric control of collision implies strong anisotropy of the interaction potential energy surface, which usually leads to coupling among multiple scattering channels in the strongly interacting region. Hence, the resonances might be the result of quantization in a multidimensional space. Using the cold rotationally inelastic collision between para-H 2 (v 1 = 1, j 1 = 2) and HF (v 2 = 0, j 2 = 0) as an example, we analyze four low-lying shape resonances via diagonalization of the full-dimensional Hamiltonian with a stabilization method. While some resonances can indeed be assigned with a single partial wave, others apparently involve more than one scattering channel. Furthermore, a new model based on these resonance wavefunctions is developed to better understand the stereodynamics through shape resonances in cold collisions.

Collisions↗

Optical Flow Diagnostics of Counter Fluidization of Gravity-Driven Moving Packed Bed for a CSP Receiver Section Featuring Staggered Array of Cylindrical Pins

Particle fluidized beds have the potential to improve the efficiency of heat transfer in concentrated solar receiver furnace for use in next generation concentrated solar power (CSP) plants. This study presents an experimental investigation on the flow characterization of vertically downward moving packed bed with counter fluidization through an array of jets. To control the bubble size and its distribution in the bubbling fluidized bed, an array of cylindrical pin fins was arranged uniformly across the test article. The flow visualization was performed on the surface of transparent glass coated with electrically conductive materials for electrostatic dissipation purposes. The image acquisition was carried out via high-speed camera at a frequency of ~ 1kHz. The acquired images were analyzed in pairs with the help of a modern optical flow algorithm capable of calculating the movement of dense particle flow in the fluidized bed by tracking the light intensity of each predefined window of the frames. A comparison of fluidized beds with plane and pin-finned channels revealed distinct bubble behavior. Pin-finned channels were found to produce a larger number of small-sized bubbles, while plane channels generated fewer but larger bubbles at any given instant. The presence of pin fins was observed to reduce bubble size by preventing bubble merging and splitting larger bubbles when they encountered a pin.

concentrating solar power↗

Remote sensing images, DEM, and point clouds associated with “Accuracy evaluation of cost-effective 3D reconstruction approaches for hydrobiogeochemical processes in non-perennial stream riverbeds”

This data package is associated with the publication “Accuracy evaluation of cost-effective 3D reconstruction approaches for hydrobiogeochemical processes in non-perennial stream riverbeds” published in Frontiers in Environmental Science, Environmental Informatics and Remote Sensing (Bao et al., 2026; doi: 10.3389/fenvs.2026.1725258). This data package includes the drone photos for a section of Umtanum Creek in Washington, Unted States. The photos were used to reconstruct the 3-dimensional (3D) digital elevation model (DEM) of the riverbed for the investigated stream section. The reconstruction results from four approaches are provided: (1) unoccupied aerial vehicle (UAV, colloquially known as drone) imagery-based Structure-from-Motion (SfM), (2) a machine learning-based 3D reconstruction model, Visual Geometry Grounded Deep Structure from Motion (VGGSfM), (3) Visual Geometry Grounded Transformer for long sequence of images (VGGT-Long), and (4) handheld smartphone LiDAR scanning. The ground truth measurements by tripod-mounted optical level kit and ground control points GPS locations for evaluating the accuracy of the four reconstruction approaches are also provided in this data package. A preliminary version of this data package was published in October 2025 at the time of manuscript submission. It was updated in March 2026, at the time of manuscript acceptance, to include additional metadata (this readme, data dictionary, and file level metadata). The data did not change. For details on how to navigate data packages generated by this project, see https://data.ess-dive.lbl.gov/portals/PNNLRiverCorridorSFA/About. In addition to a readme, this data package also includes a file-level metadata (FLMD) file that describes each file and a data dictionary (DD) that describes all column/row headers and variable definitions. This dataset is comprised of (1) 8 folders; (2) the detailed flight configuration html files; (3) field metadata; (4) a readme; (5) a data dictionary; and (6) file-level metadata. The folders “2024_10_18_d01” and “2024_10_18_d02” contain the original drone photos for the two drone flights (d01 and d02) on October 18, 2024. The reconstruction results from each of the approaches are in the folders called “ODM_SfM”, “VGGSfM”, “VGGTLong”, and “LiDAR”. The ground truth measurements are in the folder called “optical_level_kit”. Lastly, results comparing the different approaches are in the folder called “comparisons”. All files are .csv, .html, .jpg, .obj, .txt, and .npy. For information on using the .obj and .npy files, see the readme files within the same folder as the files.

54 ENVIRONMENTAL SCIENCES↗

Meeting Global Health Needs via Infectious Disease Forecasting: Development of a Reliable Data-Driven Framework

Infectious diseases (IDs) have a significant detrimental impact on global health. Timely and accurate ID forecasting can result in more informed implementation of control measures and prevention policies. To meet the operational decision-making needs of real-world circumstances, we aimed to build a standardized, reliable, and trustworthy ID forecasting pipeline and visualization dashboard that is generalizable across a wide range of modeling techniques, IDs, and global locations. We forecasted 6 diverse, zoonotic diseases (brucellosis, campylobacteriosis, Middle East respiratory syndrome, Q fever, tick-borne encephalitis, and tularemia) across 4 continents and 8 countries. We included a wide range of statistical, machine learning, and deep learning models (n=9) and trained them on a multitude of features (average n=2326) within the One Health landscape, including demography, landscape, climate, and socioeconomic factors. The pipeline and dashboard were created in consideration of crucial operational metrics—prediction accuracy, computational efficiency, spatiotemporal generalizability, uncertainty quantification, and interpretability—which are essential to strategic data-driven decisions. While no single best model was suitable for all disease, region, and country combinations, our ensemble technique selects the best-performing model for each given scenario to achieve the closest prediction. For new or emerging diseases in a region, the ensemble model can predict how the disease may behave in the new region using a pretrained model from a similar region with a history of that disease. The data visualization dashboard provides a clean interface of important analytical metrics, such as ID temporal patterns, forecasts, prediction uncertainties, and model feature importance across all geographic locations and disease combinations. As the need for real-time, operational ID forecasting capabilities increases, this standardized and automated platform for data collection, analysis, and reporting is a major step forward in enabling evidence-based public health decisions and policies for the prevention and mitigation of future ID outbreaks.

60 APPLIED LIFE SCIENCES↗

Toward Unified Autonomous Scattering Experiments: A Cross-Facility Case Study at ALS and PETRA III

Autonomous experiments rely on the integration of control, data acquisition, analysis, and decision-making frameworks. While such systems have been demonstrated at individual facilities, adapting them to additional instruments remains challenging due to differences in local infrastructure. We present a modular workflow that connects existing open-source tools for data access (Tiled), workflow orchestration (Prefect), analysis and visualization (pyFAI, Plotly Dash), and Gaussian-process-based adaptive sampling (gpCAM) into a unified framework for autonomous scattering experiments. The same configuration operates across two synchrotron beamlines (ALS 7.3.3 and PETRA III P03) with only minimal facility-specific adjustments, as shown in proof-of-concept demonstrations. This validates that a consistent design emphasizing modularity and shared interfaces can ease deployment across diverse experimental environments. The resulting framework provides a flexible foundation for extending autonomous control and analysis capabilities beyond a single beamline or instrument.

47 OTHER INSTRUMENTATION↗

Carbon Management Projects (CONNECT) Database and Explorer

Overview The Carbon Management Projects (CONNECT) Toolkit is an online exploratory visualization tool developed by the U.S. Department of Energy's (DOE) Office of Fossil Energy and Carbon Management (FECM) with support from other federal agencies such as the U.S. Environmental Protection Agency (EPA) and the U.S. Department of Transportation (DOT). It provides a single point of access to authoritative information on federal agency investment in a portfolio of research, development, and demonstration (RD&D) projects that have been publicly announced to advance technologies for point source carbon capture, carbon dioxide removal, transport, storage, and conversion, collectively referred to as carbon management. The RD&D programs covered in this tool are authorized by annual congressional appropriations ("Base Program") and the 2021 Infrastructure Investment and Jobs Act (IIJA). The tool also incorporates public information on other federal initiatives, such as the Regional Clean Hydrogen Hubs, and public information released by other government agencies, such as the Environmental Protection Agency's (EPA) and Primacy States’ Underground Injection Control Class VI permits and EPA’s facility level greenhouse gas (GHG) emissions. Developed in a geographic information system, the tool organizes carbon management projects into five groups based on the primary technology that a project aims to advance, each visually represented as a digital layer ("carbon management project layer"). Only federally funded projects are included, which can be awarded projects that are completed or ongoing, or projects that have been selected but are currently under negotiation. Project information can be viewed in the map or in the attribute table below it when turned on. In the map view, each project is displayed at either its host site (for field work), where available, or its performer site (project lead's location, further explained in the table below). Host sites and performer sites are represented in distinct icons. Several reference layers offer additional public information on infrastructural and natural resource environment for carbon management. These reference layers, combined with multiple geographical basemaps, enable users to visualize the carbon management project layers in context. Carbon management project information will be updated monthly based on feedback and information availability. Carbon management project layers Point Source Carbon Capture (PSC) This layer contains DOE-funded projects focused on capturing carbon dioxide (CO2) from power plants or industrial facilities. Carbon Dioxide Removal (CDR) This layer contains DOE-funded projects focused on capturing CO2 from the atmosphere, including direct air capture (DAC) and DAC hubs, direct ocean capture, enhanced mineralization, and biomass carbon removal and storage. For projects with multiple host sites, each of the sites are displayed individually with the project cost and cost sharing information representing the total for the entire project. Carbon Transport This layer contains DOE- and DOT-funded projects focused on CO2 transport. The Transport Research and Development sublayer contains projects that do not involve physical infrastructure; the Proposed Transport Corridor sublayer contains projects for which either a route for the transport infrastructure has been proposed or a general area for the transport infrastructure has been identified. Carbon Storage This layer contains DOE-funded key projects focused on CO2 storage. For projects with multiple field-work sites, each of the sites are displayed individually on the map with the project cost and cost sharing information representing the overall total for the entire project. Carbon Conversion This layer contains DOE-funded projects focused on converting CO2 into economically valuable products. Reference layers The following layers provide additional information in the geographic proximity of carbon management projects. Users should reference the original sources for more details (weblinks provided below and in pop-up windows on the map). Regional Clean Hydrogen Hub and Facility These layers illustrate the approximate areas of the Regional Clean Hydrogen Hubs announced by DOE's Office of Clean Energy Demonstrations (OCED) and the approximate locations of individual facilities that constitute the hubs (see "Where are the H2Hubs located?" on the webpage linked above). EPA Facility Level GHG Emissions (direct emitter) This layer shows direct CO2 emissions from stationary sources in 2022, using data extracted from EPA's Facility Level Information on GreenHouse gases Tool (FLIGHT). Captured and injected CO2 are not deducted from direct emitters’ total emissions. Contact EPA for additional details. Underground Injection Control Class VI permit/permit application This layer shows the locations of CO2 injection wells that are granted or in the process of applying for an Underground Injection Control Class VI permit by EPA or a Primacy State (currently Louisiana, North Dakota, and Wyoming). The URLs for the permits or permit applications are provided in the pop-up windows associated with the well locations. Contact EPA for additional details. Carbon Storage Resource This layer contains information on prospective CO2 storage resources in saline formations and oil and gas reservoirs provided by the National Carbon Sequestration Database and Geographic Information System (NATCARB) spatial database. Contact NETL for additional details. Existing CO2 pipeline This layer shows active CO2 pipelines based on information digitized from the map issued by the Pipeline and Hazardous Materials Safety Administration (PHMSA). Contact PHMSA for additional details.

Carbon Conversion↗

Safety Assurance Framework for Nuclear Digital Instrumentation and Control Software

Software in digital instrumentation and control (D&C) systems poses unique challenges for the safety assurance of nuclear power plants. Through a literature survey of 21 sources, we analyzed various claims, arguments, and evidence used in safety assurance cases across industries. These were organized into a Goal Structuring Notation (GSN) safety assurance case framework to organize and visualize the various arguments for DI&C safety. We developed a classification for the levels of the framework, which can be used in other safety assurance cases to improve clarity.

46 - INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AN↗

FLARE: field line analysis and reconstruction for 3D boundary plasma modeling

The FLARE code is a magnetic mesh generator that is integrated within a suite of tools for the analysis of the magnetic geometry in toroidal fusion devices. A magnetic mesh is constructed from field line segments and permits fast reconstruction of field lines in 3D boundary plasma codes such as EMC3-EIRENE. Both intrinsically non-axisymmetric configurations (stellarators) and those with symmetry breaking perturbations of an axisymmetric equilibrium (tokamaks) are supported. The code itself is written in Modern Fortran with MPI support for parallel computing, and it incorporates object-oriented programming for the definition of the magnetic field and the material surface geometry. Extended derived types for a number of different magnetohydrodynamic equilibrium and plasma response models are implemented. The core element of FLARE is a field line tracer with adaptive step-size control, and this is integrated into tools for the construction of Poincaré maps and invariant manifolds of X-points. A collection of high-level procedures that generate output files for visualization is build on top of that. The analysis modules are build with Python frontends that facilitate customization of tasks and/or scripting of parameter scans.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Exploring the Use of Large Immersive Display Systems in the Nuclear Industry

Large immersive display systems play a critical role in the nuclear industry by enabling advanced training, design reviews, scientific visualization, and safety simulations. These systems, such as CAVE and powerwalls, allow engineers and operators to interact with virtual nuclear environments in real-time, providing a deeper understanding of complex systems. Their ability to simulate real-world scenarios for testing and optimization in a safe, controlled environment ensures that operators and engineers can refine process, enhance safety protocols, and troubleshoot complex challenges. This article outlines key lessons learned from deploying these systems, practical insights into their applications, and considerations for future improvements, aiming to guide their broader adoption and effective use in the nuclear industry.

99 - GENERAL AND MISCELLANEOUS↗

Novel thermal energy storage component: Development, performance, and phase transition diagnosis

Thermal energy storage (TES) using phase change materials (PCMs) is a promising technology for capturing and storing excess thermal energy for later use. However, challenges such as poor heat transfer efficiency and a lack of modular, scalable designs have limited widespread adoption of TES in real-world applications. This study developed and evaluated modular brick-type and blade-type TES prototypes featuring an aluminum housing, an embedded serpentine coil for active or passive thermal exchange, and a cost-effective metal mesh to enhance PCM thermal conductivity. The blade-type TES achieved notable geometric efficiency, with a thickness-to-length ratio of 0.03 and a thickness-to-width ratio of 0.08, enabling highly compact and modular thermal storage suitable for space-constrained applications. The paper presents a detailed evaluation of the TES prototypes’ performance. The comparative analysis indicated that the TES prototypes provide a highly cost-effective, thermally optimized alternative for compact energy storage and load shifting. A novel diagnostic technique was also introduced: using a portable endoscope to capture real-time visualizations of PCM phase transitions inside the TES. This method provides critical insights into internal heat transfer mechanisms, identifies potential issues, and offers valuable support for optimizing the TES design and developing the control algorithm. Overall, the modular brick-type and blade-type TES designs demonstrated in this work provide a scalable, efficient, and economically viable solution for advancing TES across residential, commercial, and industrial sectors. The designs’ compact structure, enhanced thermal performance, and integrated diagnostic capabilities make them strong candidates for future deployment in energy-efficient systems.

Gao, Zhiming [ORNL] (ORCID:0000000271397995)↗

PyARC Status Report: New Integrations and Upgrades to the Fast Reactor Analysis Workflow Management Tool

PyARC was initially developed as an open source tool to support fast reactor analyses using the Argonne Reactor Computation (ARC) code suite as a part of the Nuclear Energy Advanced Modeling and Simulation (NEAMS) Workbench initiative in FY17. The goal of this initiative is to provide a common user interface for model generation, real-time validation, execution, output processing, and visualization for all integrated codes. This is accomplished through the reliance on tools available in the Workbench framework and runtime environment. While initially developed to support the ARC codes, PyARC was extended in FY22 to wrap other NEAMS and non-ARC codes, including Griffin and OpenMC, in the supported other neutronics workflows, and support users in the adoption of NEAMS-supported high fidelity analysis codes. Most recently, NUBOW-3D, a recently adopted ARC code, was integrated to support reactor bowing calculations as well. Integration of these codes into the NEAMS Workbench directly benefits the advanced reactor modeling community by: • Providing a set of controlled, maintained, documented and validated scripts to generate inputs, which promotes best practices, reduces the learning curve, and facilitates project collaboration. • Improving the user experience: the Workbench interface provides assistance for building an input through auto-completion, real-time validation, document navigation, and geometry and results visualization. • Automating complex calculations and workflows for reactor analysis. • Helping users transition to using high-fidelity NEAMS codes along-side the ARC codes. In FY22, a progress report was published that described the state of each of the tools integrated into PyARC. Since then, there have been many enhancements and upgrades to the existing integrations as well as entirely new code integrations as well. This report details all new integrations and major developments in PyARC since the version 2.0.0 release highlighted in the FY22 report.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DeepDiagnostics: A Software Package for Streamlined Posterior Evaluation

Automated prediction techniques like simulation-based inference (SBI) are important tasks for science experiments that produce large amounts of complex, raw data. However, their development remains in its early stages because the uncertainties of these techniques lack sufficient trustworthiness and interpretability. Packages for SBI provide a growing set of diagnostics; however, the software requirements are substantial, as they are tied to the inference technology itself, and the APIs lack adaptability. We introduce the DeepDiagnostics package for diagnosing posteriors from analytic likelihood-based methods and SBI methods, such as neural posterior estimation. DeepDiagnostics produces a comprehensive set of high-quality visualizations and metrics in a highly accessible, easy-to-use, and flexible package. We address all of these goals by providing a command-line inference tool and a Python API that is controlled through a configuration file. The package includes common diagnostics, such as parity plots, corner (covariance) plots, simulation-based calibration (SBC) diagnostics (including posterior coverage and rank histograms), Lemos et al. s PQMass and TARP, Masserano et al. s WALDO, Linhart et al. s LC2ST, as well as credible region diagnostics developed by our group.

Voetberg, Maggie [Fermilab]↗

INS Nuclear Material Accounting and Control Instructional Video Materials- FY 25 Guides

This training video provides an essential guide to understanding and utilizing Tamper Indicating Devices (TIDs), with a specific focus on a common Mylar (Adhesive) variety during the application process. TIDs are a critical part of nuclear security, used to detect unauthorized access to sensitive materials or facilities. The Mylar (adhesive) TID employes a tamper-evident film integrated into the TID that visually indicates any tampering attempts. This video will explain how an example Mylar (adhesive) TID works, demonstrate proper installation techniques, and highlight how to interpret its indicators to ensure the security of nuclear materials.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

Unsupervised multimodal fusion of in-process sensor data for advanced manufacturing process monitoring

Effective monitoring of manufacturing processes is crucial for maintaining product quality and operational efficiency. Modern manufacturing environments often generate vast amounts of complementary multimodal data, including visual imagery from various perspectives and resolutions, hyperspectral data, and machine health monitoring information such as actuator positions, accelerometer readings, and temperature measurements. However, fusing and interpreting this complex, high-dimensional data presents significant challenges, particularly when labeled datasets are unavailable or impractical to obtain. This paper presents a novel approach to multimodal sensor data fusion in manufacturing processes, inspired by the Contrastive Language-Image Pre-training (CLIP) model. We leverage contrastive learning techniques to correlate different data modalities without the need for labeled data, overcoming limitations of traditional supervised machine learning methods in manufacturing contexts. Our proposed method demonstrates the ability to handle and learn encoders for five distinct modalities: visual imagery, audio signals, laser position (x and y coordinates), and laser power measurements. By compressing these high-dimensional datasets into low-dimensional representational spaces, our approach facilitates downstream tasks such as process control, anomaly detection, and quality assurance. The unsupervised nature of our method makes it broadly applicable across various manufacturing domains, where large volumes of unlabeled sensor data are common. We evaluate the effectiveness of our approach through a series of experiments, demonstrating its potential to enhance process monitoring capabilities in advanced manufacturing systems. This research contributes to the field of smart manufacturing by providing a flexible, scalable framework for multimodal data fusion that can adapt to diverse manufacturing environments and sensor configurations. The proposed method paves the way for more robust, data-driven decision-making in complex manufacturing processes.

Contrastive Learning↗