Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “machine vision”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Dynamic STEM-EELS for single-atom and defect measurement during electron beam transformations

This study introduces the integration of dynamic computer vision–enabled imaging with electron energy loss spectroscopy (EELS) in scanning transmission electron microscopy (STEM). This approach involves real-time discovery and analysis of atomic structures as they form, allowing us to observe the evolution of material properties at the atomic level, capturing transient states traditional techniques often miss. Rapid object detection and action system enhances the efficiency and accuracy of STEM-EELS by autonomously identifying and targeting only areas of interest. This machine learning (ML)–based approach differs from classical ML in that it must be executed on the fly, not using static data. We apply this technology to V-doped MoS 2 , uncovering insights into defect formation and evolution under electron beam exposure. This approach opens uncharted avenues for exploring and characterizing materials in dynamic states, offering a pathway to increase our understanding of dynamic phenomena in materials under thermal, chemical, and beam stimuli.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

Automating Testing of DUNE Electronics via a Finite State Machine

The Deep Underground Neutrino Experiment (DUNE) is a flagship international collaboration designed to study neutrinos tiny, nearly massless particles that may hold answers to fundamental questions about the Universe. Fermilab s Robotic Test Stand (RTS) plays a critical role in ensuring the quality of approximately 50,000 Application-Specific Integrated Circuit (ASIC) chips that will be used in DUNE s massive liquid argon detectors. These electronics will be inside the cryostat; therefore, they will need to have a high yield of working chips and low noise. To improve the automation and reliability of the RTS, this project focused on designing and implementing a Python-based finite state machine (FSM) to manage chip handling workflows. The FSM was developed as a modular software framework to coordinate robotic arm movements, manage chip tray positions, and monitor system states during testing. Key features include robust error handling routines, a pause/resume system for safe mid-cycle interruptions, and a simulation mode for iterative testing without hardware dependencies. The system was designed to prepare for seamless integration with RTS hardware components such as the robotic arm and vision system. This integration will streamline collaboration and enable efficient deployment of updates across the six total institutions performing testing. The outcomes of this internship contribute to Fermilab s mission to advance high-energy physics and support the DOE s national goals by directly improving the testing of equipment to be used in DUNE. The project also provided valuable experience in software design and contributing to the success of DUNE.

Kang, Caleb [William Rainey Harper Coll.]↗
DOE OSTI2025

Automating Testing of DUNE Electronics via a Finite State Machine

The Deep Underground Neutrino Experiment (DUNE) is a flagship international collaboration designed to study neutrinos—tiny, nearly massless particles that may hold answers to fundamental questions about the Universe. Fermilab’s Robotic Test Stand (RTS) plays a critical role in ensuring the quality of approximately 50,000 Application-Specific Integrated Circuit (ASIC) chips that will be used in DUNE’s massive liquid argon detectors. These electronics will be inside the cryostat; therefore, they will need to have a high yield of working chips and low noise. To improve the automation and reliability of the RTS, this project focused on designing and implementing a Python-based finite state machine (FSM) to manage chip handling workflows. The FSM was developed as a modular software framework to coordinate robotic arm movements, manage chip tray positions, and monitor system states during testing. Key features include robust error handling routines, a pause/resume system for safe mid-cycle interruptions, and a simulation mode for iterative testing without hardware dependencies. The system was designed to prepare for seamless integration with RTS hardware components such as the robotic arm and vision system. This integration will streamline collaboration and enable efficient deployment of updates across the six institutions performing testing. The outcomes of this internship contribute to Fermilab’s mission to advance high-energy physics and support the DOE’s national goals by directly improving the testing of equipment to be used in DUNE. The project also provided valuable experience in software design and contributing to the success of DUNE.

Kang, Caleb [Fermilab]↗
DOE OSTI2025

Exocortex Network for AI-Augmented Human-Led Scientific Expedition

AI advances in science can be viewed along two main directions with a fluid boundary: enhancing efficiency through automation and smart tools to accelerate tasks that humans can already perform; and enabling exploration into uncharted territories and potentially toward AGI. These advances manifest in the AI cognitive core through the development and explainability of foundation models; in the physical embodiment of instruments and facilities; and in the integrated agency of AI workflows exemplified by the science exocortex. To address the role of humans in this evolving landscape, in this Perspective, we suggest a third direction: the development of personalized agents that form human-centered networks, supporting both efficiency and exploration while ensuring that AI remains aligned with human vision.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Deep learning approaches for neural decoding across architectures and recording modalities

Decoding behavior, perception or cognitive state directly from neural signals is critical for brain–computer interface research and an important tool for systems neuroscience. In the last decade, deep learning has become the state-of-the-art method in many machine learning tasks ranging from speech recognition to image segmentation. The success of deep networks in other domains has led to a new wave of applications in neuroscience. In this article, we review deep learning approaches to neural decoding. Here, we describe the architectures used for extracting useful features from neural recording modalities ranging from spikes to functional magnetic resonance imaging. Furthermore, we explore how deep learning has been leveraged to predict common outputs including movement, speech and vision, with a focus on how pretrained deep networks can be incorporated as priors for complex decoding targets like acoustic speech or images. Deep learning has been shown to be a useful tool for improving the accuracy and flexibility of neural decoding across a wide range of tasks, and we point out areas for future scientific development.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Transformer Masked Autoencoders for RF Device Fingerprinting

Machine learning methods for RF device fingerprinting typically rely on CNN-based models. Transformer-based models have outperformed CNNs for modulation classification tasks, but there are few implementations for device fingerprinting. We train a transformer for device fingerprinting with the largest device count to date and explore several variations of the architecture. Additionally, we demonstrate that pre-training an RF transformer as a Masked Autoencoder improves classification accuracy, as has been observed for CNN fingerprinting models and vision transformers.

artificial intelligence↗
DOE OSTI2025

Intercomparison of Deep Learning Model Architectures for Atmospheric River Prediction

With a rapid surge in the application of machine learning (ML) for a diverse range of tasks in climate science, the present study addresses a challenge for climate scientists when selecting the optimal ML or deep learning (DL) architecture for a given application. In particular, a DL intercomparison study was performed with a focus on forecasting the position of atmospheric rivers (ARs) on short-range time scales (up to 5-day lead times). AR predictions from multiple DL architectures, including various types of convolutional autoencoders and a vision transformer (ViT), were compared against ECMWF ERA5 reanalysis and hindcasts from a global climate model. DL models with similar trainable parameters were trained on ERA5 reanalysis data and AR positions derived from a thresholding algorithm to ensure a fair comparison among the DL models. Each model’s performance and accuracy in forecasting AR location and key input fields within a 5-day window were assessed using metrics of root-mean-square error, anomaly correlation, and mean intersection over union. The ViT architecture outperformed other autoencoder models in most of the metrics. Incorporating additional meteorological fields only yielded slight improvements in forecasting certain fields at longer lead times. The results also suggest that a smaller number of input time steps or smaller number of autoregressive steps can achieve better prediction skills, while also improving the overall computational efficiency. This research offers valuable insights into the strengths and weaknesses of different DL techniques for AR forecasting, hopefully guiding the development of improved models for forecasting this phenomenon.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

A Heterogeneous System for Eagle Detection, Deterrent, and Wildlife Collision Detection for Wind Turbines (Final Technical Report)

This report summarizes the design, implementation, and test of an integrated system for automated detection and deterrence of eagles, with included wind turbine blade strike detection and imaging functionality. A machine learning approach was used in conjunction with a 360° camera system for automated detection and classification of golden eagles. This was developed using footage obtained from trained golden eagles and other raptors, in collaboration with wildlife biologists and professional bird handlers. Oregon State University developed a visual deterrent system, which uses inflatable anthropomorphic sculptures with random, kinetic motion to deter eagles, and conducted limited field testing on live eagles; the deterrent can be triggered by the visual detection of eagles using the vision system. Finally, a multi-sensor module was developed that is mounted at the turbine blade root. This module measures vibration and other motions to detect blade strikes, and an integrated on-blade camera captures an image of any impacting objects. Long-term, this blade strike detection system is intended to support an automatic monitoring and certification system for the eagle detection and deterent system. Independent field testing of each system component is described. Testing of the integrated system on an operational wind turbine was conducted across three separate field tests. This includes multi-day fields tests on a General Electric 1.5MW wind turbine at the National Renewable Energy Laboratory (NREL) National Wind Technology Center (NWTC) in Boulder, CO in October 2018 and July 2019; installation procedures, test procedures, and a summary of collected data are presented. A third multi-day on-turbine field test is also presented, which was performed using a General Electric 1.5MW wind turbine at the North American Wind Research and Training Center (NAWRTC) at Mesalands Community College, Tucumcari, NM in April 2019. Across these field tests, the vision system was demonstrated using unmanned aerial vehicles (UAV), and the eagle classification algorithm was not tested; the visual deterrent system was demonstrated, including automatic, remote deployment following surrogate visual detections; and, multi-sensor on-blade data was recorded across multiple wind turbine operational conditions and through more than 100 surrogate blade strikes using soft projectiles, including the successful demonstration of automatic image capture of striking objects. This data set was also used for offline development and validation of enhanced collision detection algorithms. As summarized in this report, the development and field validation of an integrated detection, deterrent, and blade collision detection system represents a critical proof of concept for future technology development of related detection and deterrent technologies, where both deterrent as well as collision detection recording devices are needed for future siting, monitoring, and operation of wind turbine installations, both onshore and offshore.

17 WIND ENERGY↗
DOE OSTI2024

Building Stock Models for Embodied Carbon Emissions—A Review of a Nascent Field

Building stock modeling emerges as a critical tool in the strategic reduction of embodied carbon emissions, which is pivotal in reshaping the evolving construction sector. This review provides an overall view of modern methodologies in building stock modeling, homing in on the nuances of embodied carbon analysis in construction. Examining 23 seminal papers, our study delineates two primary modeling paradigms—top-down and bottom-up—each further compartmentalized into five innovative methods. This study points out the challenges of data scarcity and computational demands, advocating for methodological advancements that promise to refine the precision of building stock models. A groundbreaking trend in recent research is the incorporation of machine learning algorithms, which have demonstrated remarkable capacity, improving stock classification accuracy by 25% and urban material quantification by 40%. Furthermore, the application of remote sensing has revolutionized data acquisition, enhancing data richness by a factor of five. This review offers a critical examination of current practices and charts a course toward an environmentally prudent future. It underscores the transformative impact of building stock modeling in driving ecological stewardship in the construction industry, positioning it as a cornerstone in the quest for sustainability and its significant contribution toward the grand vision of an eco-efficient built environment.

Hu, Ming (ORCID:0000000325831161)↗
DOE OSTI2026

Intern Poster Session 08/13: Autonomous Nuclear Robotics: Applications in nuclear waste inspection and hot cell experiments

The nuclear industry is experiencing renewed interest in autonomous robotics, yet most deployed systems remain teleoperated with limited autonomy. This work presents two contributions toward fully autonomous nuclear robotic systems: autonomous waste inspection at the Hanford Site and an autonomous hot cell laboratory framework. Inspections of Hanford's underground waste storage tanks are performed manually at significant cost and personnel exposure. We developed a reinforcement-learning (RL) training pipeline for a custom-built inspection arm. In parallel, we are designing an autonomous laboratory framework for post-irradiation examination in hot cells at the Specimen Preparation Laboratory (SPL) that integrates computer vision, task and motion planning, hardware execution, and operator-in-the-loop control. These systems demonstrate a path toward safer, more efficient nuclear operations by reducing human exposure while maintaining rigorous human oversight at critical decision points.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2026

Locating Undocumented Wells Using Historical Oil and Gas Exploration Maps: A Case Study in Osage County, Oklahoma

Undocumented oil and gas wells lack reliable information about their locations and characteristics, making them difficult to identify. These wells can result in unanticipated delays and costs in the development of nearby surface and subsurface resources, and, if improperly plugged, can cause contamination. This study leverages historical petroleum exploration maps to locate such wells, focusing on Osage County, Oklahoma. Two sets of early 20th century oil and gas exploration maps by the United States Geological Survey were georeferenced and analyzed using a computer vision model to detect well symbols. The locations of detected wells were compared to the location of known wells in the database from the Bureau of Indian Affairs Osage Agency to identify potential undocumented wells. The analysis yielded over 500 potential undocumented wells, with dry holes constituting the largest fraction. Field verification confirmed the presence of some undocumented wells. Comparison with prior work revealed limited overlap, underscoring the complementary value of historical oil and gas maps for locating undocumented wells. This approach demonstrates the utility of integrating historical cartographic resources with modern geospatial and machine learning techniques to improve the identification and management of undocumented wells.

Energy - Petroleum↗
DOE OSTI2023

Estimation of Arrivals on Green at Signalized Intersections Using Stop-Bar Video Detection

Across the world, traffic congestion is increasing with alarming rapidity. Traffic signal control effectiveness, in coordinated networks, is often investigated in relation to the type of vehicle arrivals at the signalized intersections. Recently, several transportation agencies have switched from traditional loop detectors to video detection. When video cameras are accompanied by computer vision, one can extract more information about traffic “dynamics” than by using traditional inductive loop detectors. Collecting arrival times of multiple vehicles after the first arrival at the stop-bar detector might be challenging when using inductive loop detectors (since after the first arrival, detector status is always occupied). However, emerging video detection systems allow tracking of each vehicle’s entrance time in the detection zone, departure time from the detection zone, and the type of vehicle. This information can be used to estimate vehicular arrival and departure times, which then can be fed into machine learning algorithms to estimate arrivals on green (AOG). However, such research ideas have not been documented so far. Thus, this paper presents an estimation model for AOG, which was developed using multigene genetic programming. A robust experimental dataset was collected from a highly calibrated and validated microsimulation model of an 11-intersection corridor in Chattanooga, TN. The results of the model’s performance analysis showed the high accuracy of the training-, testing-, and validation datasets. The practical benefit of this model is that it can be applied to estimate arrival types at intersections where only stop-bar video detection exists.

Engineering↗
DOE OSTI2021

Scoreboard

Emerging HPC machines have given rise to enhanced compute power that far outstrips the machine's ability to save large scale results for post-processing. To combat this, in situ data analysis techniques are slowly being adopted. With in situ data management favoring workflows composed of multiple simulations and analyses connected in transit on heterogeneous machines, scientists and engineers need a tool that enables them to create data extracts, visualizations, and interactively monitor and steer their simulations. Scoreboard Phase II is a next generation analysis software that supports composite in transit workflows on heterogeneous architectures and restores interactivity to in situ data analysis through simulation monitoring and computational steering. Scoreboard provides a simulation dashboard with graphs of metrics over time, controls for setting custom simulation steering parameters, controls for managing the set of data extracts being produced in the simulation, as well as the ability to explore data extracts, all from a web browser. Realizing the vision outlined in this project required research into making a system that integrates end to end from simulations all the way to the user. In situ tools generally suffer from complexity and excessive software dependencies. Scoreboard, by contrast, is easy to build and integrate into simulation codes and it provides first class FORTRAN support. The Scoreboard library is capable of in situ and in transit data analysis that can produce data extracts commonly needed for Computational Fluid Dynamics (CFD) analysis. Simulations can transparently stage data in transit to a Scoreboard Endpoint program, which can accept their data and produce the requested data extracts. This lets simulations return to their work while the Endpoint works on the analysis. Efficiently staging the data at scale was a topic of this research. Scoreboard provides the means to let the user manage data extracts and monitor/steer many simulations from a web browser. This area of the research focused on discovery of in transit network components to expose and control their steering parameters within an interactive browser-based user interface that includes: system topology, gathered metrics, notifications, dynamically-generated steering controls, and exploration of visualization data products.

Whitlock, BradJoseph [Intelligent Light] (00000001↗
DOE OSTI2024

Effects of input gradient regularization on neural networks time-series forecasting of thermal power systems

This study proposes using neural networks, specifically gated recurrent unit (GRU), long-short-term memory (LSTM), and transformer networks, to improve control strategies in a 450 MW coal-fired power plant. However, neural networks face issues of becoming overly dependent on just a few variables to make predictions, which negatively impacts control decisions that rely on the model to determine the value of all manipulated variables. The paper introduces regularization techniques, including noise injection and input gradient regularization, during the training phase. Here, the work presents novel contributions in adapting neural networks to control industrial systems and applying regularization techniques from computer vision to industrial process control. Results demonstrate the effectiveness of input gradient regularization in reducing model dependence on subsets of variables, emphasizing the balance between fidelity and controllability. Further exploration is recommended, including the development of recurrent transformers, closed-loop control testing, and a sensitivity analysis on computer models to provide further insight.

20 FOSSIL-FUELED POWER PLANTS↗
DOE OSTI2026

Intelligent Experiments Through Real-time AI: Fast Data Processing and Autonomous Detector Control for sPHENIX and Future EIC Detectors (Final Report)

The overall vision of this project was to integrate real-time artificial intelligence (AI) directly into the data acquisition and detector-control systems of nuclear physics experiments, including both fast online event selection and an autonomous detector-control feedback loop. The work carried out under the award focused on the fast online event-selection half of that vision: the efficient recording of low-momentum heavy-flavor (HF) hadron decays in proton-proton collisions at the sPHENIX experiment at the Relativistic Heavy Ion Collider (RHIC)—an observable that requires fast tracking and topological trigger selection not previously demonstrated at RHIC, and that is essential for QCD studies at future facilities such as the Electron-Ion Collider (EIC). The autonomous detector-control (GPU-based feedback) component named in the project title remained a design concept and was not implemented under this award. The Massachusetts Institute of Technology (MIT) group led the offline simulation and data processing needed to train the machine-learning (ML) models, the translation of trained models to Field-Programmable Gate Array (FPGA) firmware using the hls4ml framework, and the physics validation of heavy-flavor reconstruction. Over the award period, the team developed and hardware-tested the principal components of an AI-based heavy-flavor trigger on simulated and recorded sPHENIX tracker data: a software Bipartite Graph Attention Network (BiGAT) trigger model reaching > 95% signal efficiency at 99% background rejection; an FPGA-native hit clusterizer matching the offline clustering; smaller networks synthesized to FPGA within the required sub-10 µs latency; and an assembled decoder–clusterizer–inference firmware chain exercised on the FELIX readout board. A complete, fully integrated hardware demonstrator was not finished within the award period. This report documents the project goals, the MIT group’s contributions, the technical accomplishments, and the outlook toward applications at the future EIC ePIC detector.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2025

Electric Grid Security (EGS) FY24 Annual Report

Sandia’s Electric Grid Security program advances a national vision of a secure, resilient, and affordable electric system for all users. Our achievements reflect a strategic approach combining technology development; modeling, simulation, and data analytics; and partnered demonstrations and outreach to further the adoption of advanced grid and storage technologies. Our FY24 efforts leverage the strengths of our partnerships—spanning Sandia’s core science and technology competencies as well as external technology leaders—to develop the solutions today which enable the grid of tomorrow. Key accomplishments in this report that support our strategy span our technical program areas and include: • The advancement of energy storage technologies, including creation of a national Long Duration Energy Storage Consortium; • Applications of artificial intelligence and machine learning to enhanced grid operations and planning; • Development of solid-state power conversion technologies and a new medium-voltage research lab; • New technologies to assess wildfire vulnerabilities and mitigate potential impacts; • Advanced applications of new cybersecurity technologies with industry partners; • Contributions to understanding the impacts of electromagnetic pulses and geomagnetic disturbances on grid components; and • Digital twin development for hybrid microgrids with multiple generators, storage, and loads.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Accelerating the rate of discovery: toward high-repetition-rate HED science

As high-intensity short-pulse lasers that can operate at high-repetition-rate (HRR) (>10 H z ) come online around the world, the high energy density (HED) science they enable will experience a radical paradigm shift. The >10 3 increase in shot rate over today's shot-per-hour drivers translates into dramatically faster data acquisition, more experiments, and the ability to exploit machine learning, and thus the potential to significantly accelerate the advancement of HED science. A wide range of HED experiments, from opacity investigations to secondary source generation to plasma nuclear physics, will benefit from the increased statistics, precision, and exploration of phase space. Besides increasing the rate at which scientific experiments can be performed, HRR also allows for the rapid delivery of optimal experiments supported by simulations and modeling augmented by close coupling to empirical data. To fully realize such an HRR framework, numerous subsystems must be developed and brought together, including feedback laser control loops, high-throughput targetry and diagnostics, cognitive simulation, enhanced HED codes, and advanced data analytics. This paper describes the vision for an integrated HRR laser experimental HED system and outlines some of the major considerations and challenges for realizing it.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗