SEARCH · Engineering Papers
Results for “VISION”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Provable Repair of Vision Transformers
Vision Transformers have emerged as state-of-the-art image recognition tools, but may still exhibit incorrect behavior. Incorrect image recognition can have disastrous consequences in safety-critical real-world applications such as self-driving automobiles. In this paper, we present Provable Repair of Vision Transformers (PRoViT), a provable repair approach that guarantees the correct classification of images in a repair set for a given Vision Transformer without modifying its architecture. PRoViT avoids negatively affecting correctly classified images (drawdown) by minimizing the changes made to the Vision Transformer’s parameters and original output. Here, we observe that for Vision Transformers, unlike for other architectures such as ResNet or VGG, editing just the parameters in the last layer achieves correctness guarantees and very low drawdown. We introduce a novel method for editing these last-layer parameters that enables PRoViT to efficiently repair state-of-the-art Vision Transformers for thousands of images, far exceeding the capabilities of prior provable repair approaches.
ORBIT-2 Dataset for Scaling Exascale Vision Foundation Models for Weather and Climate Downscaling
This dataset release corresponds to the work conducted in ORBIT-2: Scaling Exascale Vision Foundation Models for Weather and Climate Downscaling, where large-scale AI methods were applied to improve climate and weather resolution. The collection integrates four widely used, publicly available datasets: ERA5, PRISM, DAYMET, and IMERG. To prepare the data for ORBIT-2 model training and evaluation, we applied a preprocessing pipeline that generates paired low-resolution and high-resolution samples, enabling supervised downscaling experiments. The transformation from coarse to fine scales was performed using bilinear regridding, consistent with the procedures described in WeatherBench2, a community benchmark for weather and climate AI models. This dataset supports the development and evaluation of foundation models designed for weather and climate downscaling at exascale. Additional details on methodology and applications can be found in Wang et al., ORBIT-2 (arXiv:2505.04802, 2025).
A Vision for Coupling Operation of US Fusion Facilities with HPC Systems and the Implications for Workflows and Data Management
The operation of large US Department of Energy (DOE) research facilities, like the DIII-D National Fusion Facility, results in the collection of complex multi-dimensional scientific datasets, both experimental and model-generated. In the future, it is envisioned that integrated data analysis coupled with large-scale high performance computing (HPC) simulations will be used to improve experimental planning and operation. Practically, massive data sets from these simulations provide the physics basis for generation of both reduced semi-analytic and machine-learning-based models. Storage of both HPC simulation datasets (generated from US DOE leadership computing facilities) and experimental datasets presents significant challenges. In this paper, we present a vision for a DOE-wide data management workflow that integrates US DOE fusion facilities with leadership computing facilities. Data persistence and long-term availability beyond the length of allocated projects is essential, particularly for verification and recalibration of artificial intelligence and machine learning (AI/ML) models. Because these data sets are often generated and shared among hundreds of users across multiple leadership computing facility centers, they would benefit from cross-platform accessibility, persistent identifiers (e.g. DOI, or digital object identifier), and provenance tracking. Here, the ability to handle different data access patterns suggests that a combination of low cost, high latency (e.g. for storing ML training sets) and high cost, low latency systems (e.g. for real-time, integrated machine control feedback) may be needed.
A Vision for Co-optimized T&D System Interaction with Renewables and Demand Response
The evolution of the power system to the reliable, effi- cient and sustainable system of the future will involve development of both demand- and supply-side technology and operations. The use of demand response to counterbalance the intermittency of re- newable generation brings the consumer into the spotlight. Though individual consumers are interconnected at the low-voltage distri- bution system, these resources are typically modeled as variables at the transmission network level. In this paper, a vision for co- optimized interaction of distribution systems, or microgrids, with the high-voltage transmission system is described. In this frame- work, microgrids encompass consumers, distributed renewables and storage. The energy management system of the microgrid can also sell (buy) excess (necessary) energy from the transmission system. Preliminary work explores price mechanisms to manage the microgrid and its interactions with the transmission system. Wholesale market operations are addressed through the devel- opment of scalable stochastic optimization methods that provide the ability to co-optimize interactions between the transmission and distribution systems. Modeling challenges of the co-optimization are addressed via solution methods for large-scale stochastic op- timization, including decomposition and stochastic dual dynamic programming.
Individual Wave Detection and Tracking within a Rotating Detonation Engine through Computer Vision Object Detection applied to High-Speed Images
Known for their simplistic design and continuous detonation, rotating detonation engines (RDEs) constitute a majority of current pressure gain combustion (PGC) research efforts. Experimental RDE operation times have been continuously extended through the use of rig cooling techniques. As the window of observable behavior is expanded, and as the technology matures toward eventual integration within gas turbines, monitoring techniques must evolve to better match industrial diagnostics. High-speed image analysis techniques prove useful to capture and evaluate the unsteady detonation behavior within the RDE. Traditional image analysis techniques, however, require extensive processing times which prohibit simultaneous monitoring. To better address this problem, a computer vision object detection methodology is proposed to quickly detect individual detonation waves within a single down-axis image. Detonation waves are detected in individual images by the implemented computer vision method You Only Look Once (YOLO) object detection network. In order to detect detonation waves, the network must first be trained using RDE images of interest, for which each required phase of network development is outlined. Detection of waves is improved through proper treatment of the collected image set, variation of Intersection over Union (IoU) and confidence thresholding, and through a parametric study of annotation dimensions. Each detected wave is described by its location and rotational direction, and locations are tracked to calculate wave velocity across each frame, leading to a timestep resolution of 20 µs. Wave velocities are also calculated through a series of frames, leading to a suitable average velocity estimation using as few as 10 frames. Uncertainty analysis accounting for variation in camera framerate, pixel width and annotation centroid locations estimates a total uncertainty of ±4.3% for velocity calculations, using the smallest annotation boxes. This method offers great reductions in processing times, as a step toward real-time monitoring of detonation waves within an RDE. Improving on previous studies, this technique is impartial to wave modes not included in the original training set and calculates wave velocities independent of high-speed pressure data. The ability to isolate waves within predicted bounding boxes will likely facilitate analysis of pixel intensity variation as an estimation of wave strength in future work.
Catalytic mechanism of a retinoid isomerase essential for vertebrate vision
Abstract not provided
Traffic Signal Control for Large-Scale Urban Traffic Networks: Real-World Experiments using Vision-Based Sensors
Effective control of traffic signals plays a critical role in ensuring smooth vehicle flow in urban areas. Expertly engineered traffic signal controllers can considerably minimize travel delays and enhance sustainability. In this paper, the team proposes the Model Predictive Control (MPC) traffic signal control strategy using real-time traffic flow data from a vision-based camera as feedback information. Also, a realistic signal timing plan that considers National Electrical Manufacturers Association (NEMA) constraints has been developed to be applied to real-world scenarios. The primary aim is to reduce the number of vehicles across all links in the controlled area, thereby optimizing traffic flow and reducing energy consumption. To validate the proposed method, several real-life experiments were conducted at 24 intersections in Chattanooga, Tennessee, by collaborating with traffic field engineers. These experiments demonstrated significant performance improvements in comparison to the existing method.
Variable structure Human Intention Estimator with mobility and vision constraints as model selection criteria
Not provided.
Predictive nuclear power plant outage control through computer vision and data-driven simulation
Not provided.
A Vision for Managing Extreme-Scale Data Hoards
Scientific data collections grow ever larger, both in terms of the size of individual data items and of the number and complexity of items. To use and manage them, it is important to directly address issues of robust and actionable provenance. We identify three key drivers as our focus: managing the size and complexity of metadata, lack of a priori information to match usage intents between publishers and consumers of data, and support for campaigns over collections of data driven by multi-disciplinary, collaborating teams. We introduce the Hoarde abstraction as an attempt to formalize a way of looking at collections of data to make them more tractable for later use. Hoarde leverages middleware and systems infrastructures for scientific and technical data management. Through the lens of a select group of challenging data usage scenarios, we discuss some of the aspects of implementation, usage, and forward portability of this new view on data management.
Change in Brooklyn and Queens: How New York?s Reforming the Energy Vision Program and Con Edison Are Reshaping Electric Distribution Planning
The New York City boroughs of Brooklyn and Queens are undergoing a period of gentrification, infrastructure rebuilding, new construction, and load growth not experienced in decades. Significant numbers of residents are moving in, and structures that had been abandoned or were in disrepair are being refurbished and modernized to accommodate the burgeoning population. Homes, businesses, and industries are reviving areas long in decline, and Brooklyn's growth has made it the nation's fourth most populous city, according to the U.S. Census Bureau.
Vision - Vision 1.0
This package contains modules that model a visual sensor in Umbra. It is typically used to represent eyesight of characters in Umbra. This library also includes the sensor property, seeable, and an Active Denial sensor.
Plant science decadal vision 2020–2030: Reimagining the potential of plants for a healthy and sustainable future
Abstract Plants, and the biological systems around them, are key to the future health of the planet and its inhabitants. The Plant Science Decadal Vision 2020–2030 frames our ability to perform vital and far‐reaching research in plant systems sciences, essential to how we value participants and apply emerging technologies. We outline a comprehensive vision for addressing some of our most pressing global problems through discovery, practical applications, and education. The Decadal Vision was developed by the participants at the Plant Summit 2019, a community event organized by the Plant Science Research Network. The Decadal Vision describes a holistic vision for the next decade of plant science that blends recommendations for research, people, and technology. Going beyond discoveries and applications, we, the plant science community, must implement bold, innovative changes to research cultures and training paradigms in this era of automation, virtualization, and the looming shadow of climate change. Our vision and hopes for the next decade are encapsulated in the phrase reimagining the potential of plants for a healthy and sustainable future. The Decadal Vision recognizes the vital intersection of human and scientific elements and demands an integrated implementation of strategies for research (Goals 1–4), people (Goals 5 and 6), and technology (Goals 7 and 8). This report is intended to help inspire and guide the research community, scientific societies, federal funding agencies, private philanthropies, corporations, educators, entrepreneurs, and early career researchers over the next 10 years. The research encompass experimental and computational approaches to understanding and predicting ecosystem behavior; novel production systems for food, feed, and fiber with greater crop diversity, efficiency, productivity, and resilience that improve ecosystem health; approaches to realize the potential for advances in nutrition, discovery and engineering of plant‐based medicines, and "green infrastructure." Launching the Transparent Plant will use experimental and computational approaches to break down the phytobiome into a "parts store" that supports tinkering and supports query, prediction, and rapid‐response problem solving. Equity, diversity, and inclusion are indispensable cornerstones of realizing our vision. We make recommendations around funding and systems that support customized professional development. Plant systems are frequently taken for granted therefore we make recommendations to improve plant awareness and community science programs to increase understanding of scientific research. We prioritize emerging technologies, focusing on non‐invasive imaging, sensors, and plug‐and‐play portable lab technologies, coupled with enabling computational advances. Plant systems science will benefit from data management and future advances in automation, machine learning, natural language processing, and artificial intelligence‐assisted data integration, pattern identification, and decision making. Implementation of this vision will transform plant systems science and ripple outwards through society and across the globe. Beyond deepening our biological understanding, we envision entirely new applications. We further anticipate a wave of diversification of plant systems practitioners while stimulating community engagement, underpinning increasing entrepreneurship. This surge of engagement and knowledge will help satisfy and stoke people's natural curiosity about the future, and their desire to prepare for it, as they seek fuller information about food, health, climate and ecological systems.
Buildings of the Future Scoping Study: A Framework for Vision Development
The Buildings of the Future Scoping Study, funded by the U.S. Department of Energy (DOE) Building Technologies Office, seeks to develop a vision for what U.S. mainstream commercial and residential buildings could become in 100 years. This effort is not intended to predict the future or develop a specific building design solution. Rather, it will explore future building attributes and offer possible pathways of future development. Whether we achieve a more sustainable built environment depends not just on technologies themselves, but on how effectively we envision the future and integrate these technologies in a balanced way that generates economic, social, and environmental value. A clear, compelling vision of future buildings will attract the right strategies, inspire innovation, and motivate action. This project will create a cross-disciplinary forum of thought leaders to share their views. The collective views will be integrated into a future building vision and published in September 2015. This report presents a research framework for the vision development effort based on a literature survey and gap analysis. This document has four objectives. First, it defines the project scope. Next, it identifies gaps in the existing visions and goals for buildings and discusses the possible reasons why some visions did not work out as hoped. Third, it proposes a framework to address those gaps in the vision development. Finally, it presents a plan for a series of panel discussions and interviews to explore a vision that mitigates problems with past building paradigms while addressing key areas that will affect buildings going forward.
Data Fusion for a Vision-Radiological System for Source Tracking and Discovery
A multidisciplinary approach to allow the tracking of the movement of radioactive sources by fusing data from multiple radiological and visual sensors is under development. The goal is to improve the ability to detect, locate, track and identify nuclear/radiological threats. The key concept is that such widely available visual and depth sensors can impact radiological detection, since the intensity fall-off in the count rate can be correlated to movement in three dimensions. To enable this, we pose an important question; what is the right combination of sensing modalities and vision algorithms that can best compliment a radiological sensor, for the purpose of detection and tracking of radioactive material? Similarly what is the best radiation detection methods and unfolding algorithms suited for data fusion with tracking data? Data fusion of multi-sensor data for radiation detection have seen some interesting developments lately. Significant examples include intelligent radiation sensor systems (IRSS), which are based on larger numbers of distributed similar or identical radiation sensors coupled with position data for network capable to detect and locate radiation source. Other developments are gamma-ray imaging systems based on Compton scatter in segmented detector arrays. Similar developments using coded apertures or scatter cameras for neutrons have recently occurred. The main limitation of such systems is not so much in their capability but rather in their complexity and cost which is prohibitive for large scale deployment. Presented here is a fusion system based on simple, low-cost computer vision and radiological sensors for tracking of multiple objects and identifying potential radiological materials being transported or shipped. The main focus of this work is the development on two separate calibration algorithms for characterizing the fused sensor system. The deviation from a simple inverse square-root fall-off of radiation intensity is explored and accounted for. In particular, the computer vision system enables a map of distance-dependence of the sources being tracked. Infrared, laser or stereoscopic vision sensors are all options for computer-vision implementation depending on interior vs exterior deployment, resolution desired and other factors. Similarly the radiation sensors will be focused on gamma-ray or neutron detection due to the long travel length and ability to penetrate even moderate shielding. There is a significant difference between the vision sensors and radiation sensors in the way the 'source' or signals are generated. A vision sensor needs an external light-source to illuminate the object and then detects the re-emitted illumination (or lack thereof). However, for a radiation detector, the radioactive material is the source itself. The only exception to this is the field of active interrogations where radiation is beamed into a material to entice new/additional radiation emission beyond what the material would emit spontaneously. The aspect of the nuclear material being the source itself means that all other objects in the environment are 'illuminated' or irradiated by the source. Most radiation will readily penetrate regular material, scatter in new directions or be absorbed. Thus if a radiation source is located near a larger object that object will in turn scatter some radiation that was initially emitted in a direction other than the direction of the radiation detector, this can add to the count rate that is observed. The effect of these scatter is a deviation from the traditional distance dependence of the radiation signal and is a key challenge that needs a combined system calibration solution and algorithms. Thus both an algebraic approach as well as a statistical approach have been developed and independently evaluated to investigate the sensitivity to this deviation from the simplified radiation fall-off as a function of distance. The resulting calibrated system algorithms are used and demonstrated in various laboratory scenarios, and later in realistic tracking scenarios. The selection and testing of radiological and computer-vision sensors for the additional specific scenarios will be the subject of ongoing and future work. (authors)
Vision-Guided Automation and Assistance
Robotic automation and assistance is an excellent way to increase efficiency and safety when working in a potentially hazardous workspace. However, many traditional automation approaches rely on the precise placement of items to be manipulated. The scope of such automated tasks is also limited to the manipulation of one type of object since there is no way to distinguish between different objects. The addition of a vision pipeline, where robot vision is used to localize and recognize objects, can allow the robot to perform more complex tasks in a variable workspace. In my vision pipeline, a depth camera is used to generate a 3D point cloud of the workspace. Objects to be manipulated are isolated in the point cloud by filtering out shared surfaces and using Euclidean cluster extraction to group clusters of points. The locations of the objects are then estimated from the point cloud of the isolated objects. The localization of objects with robot vision allows for variable workspaces, reducing the overhead associated with requiring all objects to be precisely placed in the workspace. For object recognition, deep learning is used to train a neural network that classifies objects from the raw point clouds. More complex tasks that require multiple objects to be manipulated differently can be achieved through the vision pipeline. Through its ability to visualize the workspace, robot vision greatly improves efficiency and safety of automated and assistive tasks.