SEARCH · Engineering Papers

Results for “Vision transformers”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

All sources NASA NTRS DOE OSTI

DOE OSTI2021

A METHODOLOGY TO SUPPORT THE DEVELOPMENT OF A NEW STATE VISION FOR THE UNITED STATES NUCLEAR INDUSTRY

A new strategy in the way in which the United States nuclear power plants (NPPs) are operated, maintained, and supported is needed. One such strategy is to transform the NPP operating model through a business-driven approach that leverages technology to enable new capabilities that improve performance and reduce cost. This paper presents a methodology for developing an achievable yet transformative new state vision that ensures continued safe and efficient operations of the United States NPP fleet. This work builds on existing guidance and leverages previous research to comprehensively address both bottom-up (i.e., utility needs) and top-down (i.e., first principles) considerations important for developing a new state vision. The proposed methodology is intended to provide industry-wide guidance for developing a new state vision that leverages both the selected vendor’s capabilities in a way that meets the utility’s modernization goals while ensuring state-of-the-art systems engineering and human factors engineering principles are applied that promote overall plant safety, performance, and efficiency.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2022

A Methodology to Support the Development of a New State Vision for the U.S. Nuclear Industry

Recent changes in natural gas prices combined with reduced capital costs for solar and wind systems has created challenges for the continued operation of existing nuclear power plants (NPPs) in the United States. A new strategy in the way in which U.S. NPPs are operated, maintained, and supported is needed. One such strategy is to transform the NPP operating model through a business-driven approach that leverages technology to enable new capabilities that improve performance and reduce costs. This paper presents a methodology for developing an achievable yet transformative new state vision that ensures the continued safe and efficient operations of the U.S. NPP fleet. Here this work builds on existing guidance and leverages previous research to comprehensively address both utility needs and high-level human factors engineering design principles when developing a new state vision. The proposed methodology is intended to provide industry-wide guidance for developing a new state vision that leverages both the selected vendor’s capabilities in a way that meets the utility’s modernization goals while ensuring state-of-the-art systems engineering and human factors engineering principles are applied that promote overall plant safety, performance, and efficiency.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2021

USE OF COGNITIVE WORK ANALYSIS IN DEVELOPING A NUCLEAR POWER PLANT NEW STATE VISION

The United States nuclear industry is in need of identifying and implementing a new strategy that will lower operating and maintenance costs while maintaining safety for existing plants. The industry must also have a clear and strategic vision of their transformative new state that focuses on ways in which technologies can be integrated to maximize the benefits of technology and people. While there are ongoing efforts in this area, this work discusses how the use of cognitive work analysis may further support these current efforts. This paper provides an overview of cognitive work analysis, as well as the state of current new state development efforts. The use of cognitive work analysis, as a tool to enhance existing practices, is presented in ensuring that the development of a new state vision is systematic and complete.

99 GENERAL AND MISCELLANEOUS↗

NASA NTRS2018

An “EPIIC” Vision to Evolve Project Integration, Innovation, and Collaboration with Broad Impact for How NASA Executes Complex Projects

Evolving Project Integration, Innovation, and Collaboration (EPIIC) is a vision defined to transform the way projects manage information to support real-time decisions, capture best practices and lessons learned, perform assessments, and manage risk across a portfolio of projects. The foundational project management needs for data and information will be revolutionized through innovations on how we manage and access that data, implement configuration control, and certify compliance. The embedded intelligence of new interactive data interfaces integrate technical and programmatic data such that near real time analytics can be accomplished to more efficiently and accurately complete systems engineering and project management tasks. The system-wide data analytics that are integrated into customized data interfaces allows the growing team of engineers and managers required to develop and implement major NASA missions the ability to access authoritative source(s) of system information while greatly reducing the labor required to complete system assessments. This would allow, for example, much of what is accomplished in a scheduled design review to take place as needed, between any team members, at any time. An intelligent data interface that rigorously integrates systems engineering and project management information in near real time can provide substantially greater insight for systems engineers, project managers, and the large diverse teams required to complete a complex project. System engineers, programmatic personnel (those who focus on cost, schedule, and risk), the technical engineering disciplines, and project management can realize immediate benefit from the shared vision described herein. Implementation of the vision also enables significant improvements in the performance of the engineered system being developed.

Moreland, Robert J.↗

DOE OSTI2025

Contaminant Investigation and Pre‐Processing Opportunities for Textile‐To‐Textile Recycling

Millions of metric tons of textiles are landfilled or incinerated each year in the United States, with less than 1% of textiles recycled into new clothing or fabrics. To counter this trend, a growing number of companies and researchers are exploring how a circular economy can be applied to support textile‐to‐textile recycling. A significant barrier they face comes down to quickly and efficiently extracting pure feedstock material from post‐consumer garments that feature a mix of natural and synthetic fibers. Textile recyclers prefer pure feedstocks, as working with mixed sources typically means lower throughput, higher risk of equipment failure, and diminished business margins. To facilitate a circular economy for textiles, methods, and technologies are needed that can efficiently separate out materials and contaminants from end‐of‐life textiles to increase the flow of pure feedstocks to recyclers. This paper summarizes findings from interviews with a cross section of textile recyclers and from a review of literature to define basic feedstock requirements. In addition to our qualitative research, we deconstruct a bale of post‐consumer textiles and analyze them using computer‐vision imaging, Fourier transform infrared spectroscopy (FTIR), and machine learning. The resulting data are used to set system‐level design inputs for an automated contaminant removal system to process post‐consumer clothing into appropriate feedstocks for recycling. To set the system's levels for automated real‐time near‐infrared analysis, we identify the minimum percentage of primary material that any single garment in a load of used clothing must contain for the average of the full output stream to meet the target purity levels of recyclers. Here, the envisioned automated system can also address undesirable trace materials that might contaminate the processed stream by using imaging cameras coupled with artificial intelligence to identify sections of clothing for de‐trimming. Proof‐of‐concept machine learning algorithms are evaluated to locate and identify trims or garment areas with hidden contaminant materials. Integrating these methods into automated textile cutting systems can provide a cost‐effective means for increasing feedstock purity from used clothing, which can advance circularity for textiles by helping recyclers to reach production volumes and quality targets that were not possible solely with manual dismantling operations.

Parsons, Ryan [Rochester Institute of Technology, ↗

DOE OSTI2021

ThermalTracker 3D

ThermalTracker-3D is a stereo-vision solution for evaluating flight tracks of birds and bats around offshore wind turbines. Using a pair of thermal video cameras, the technology remotely senses movement of animals and objects, day and night, near critical assets. It generates motion tracks by collapsing a sequence of video frames from each camera into a single image that contains an entire flight track and then applies stereo-vision processing to transform the flight track into three dimensions. The approach allows tracking in near real time and automatically identifies moving objects based on features from the motion track and object size.

Matzner, Shari↗

NASA NTRS1987

The cortex transform - Rapid computation of simulated neural images

With a goal of providing means for accelerating the image processing, machine vision, and testing of human vision models, an image transform was designed, which makes it possible to map an image into a set of images that vary in resolution and orientation. Each pixel in the output may be regarded as the simulated response of a neuron in human visual cortex. The transform is amenable to a number of shortcuts that greatly reduce the amount of computation.

Watson, Andrew B.↗

NASA NTRS2004

Hand-Eye Calibration of Robonaut

NASA's Human Space Flight program depends heavily on Extra-Vehicular Activities (EVA's) performed by human astronauts. EVA is a high risk environment that requires extensive training and ground support. In collaboration with the Defense Advanced Research Projects Agency (DARPA), NASA is conducting a ground development project to produce a robotic astronaut's assistant, called Robonaut, that could help reduce human EVA time and workload. The project described in this paper designed and implemented a hand-eye calibration scheme for Robonaut, Unit A. The intent of this calibration scheme is to improve hand-eye coordination of the robot. The basic approach is to use kinematic and stereo vision measurements, namely the joint angles self-reported by the right arm and 3-D positions of a calibration fixture as measured by vision, to estimate the transformation from Robonaut's base coordinate system to its hand coordinate system and to its vision coordinate system. Two methods of gathering data sets have been developed, along with software to support each. In the first, the system observes the robotic arm and neck angles as the robot is operated under external control, and measures the 3-D position of a calibration fixture using Robonaut's stereo cameras, and logs these data. In the second, the system drives the arm and neck through a set of pre-recorded configurations, and data are again logged. Two variants of the calibration scheme have been developed. The full calibration scheme is a batch procedure that estimates all relevant kinematic parameters of the arm and neck of the robot The daily calibration scheme estimates only joint offsets for each rotational joint on the arm and neck, which are assumed to change from day to day. The schemes have been designed to be automatic and easy to use so that the robot can be fully recalibrated when needed such as after repair, upgrade, etc, and can be partially recalibrated after each power cycle. The scheme has been implemented on Robonaut Unit A and has been shown to reduce mismatch between kinematically derived positions and visually derived positions from a mean of 13.75cm using the previous calibration to means of 1.85cm using a full calibration and 2.02cm using a suboptimal but faster daily calibration. This improved calibration has already enabled the robot to more accurately reach for and grasp objects that it sees within its workspace. The system has been used to support an autonomous wrench-grasping experiment and significantly improved the workspace positioning of the hand based on visually derived wrench position. estimates.

Nickels, Kevin↗

DOE OSTI2025

Distributed Cross-Channel Hierarchical Aggregation for Foundation Models

Vision-based scientific foundation models hold significant promise for advancing scientific discovery and innovation. This potential stems from their ability to aggregate images from diverse sources—such as varying physical groundings or data acquisition systems—and to learn spatio-temporal correlations using transformer architectures. However, tokenizing and aggregating images can be compute-intensive, a challenge not fully addressed by current distributed methods. In this work, we introduce the Distributed Cross-Channel Hierarchical Aggregation (D-CHAG) approach designed for datasets with a large number of channels across image modalities. Our method is compatible with any model-parallel strategy and any type of vision transformer architecture, significantly improving computational efficiency. We evaluated D-CHAG on hyperspectral imaging and weather forecasting tasks. When integrated with tensor parallelism and model sharding, our approach achieved up to a 75% reduction in memory usage and more than doubled sustained throughput on up to 1,024 AMD GPUs on the Frontier Supercomputer.

Tsaris, Aristeidis (aris) [ORNL] (ORCID:0000000277↗

NASA NTRS1993

Visual information processing II; Proceedings of the Meeting, Orlando, FL, Apr. 14-16, 1993

Various papers on visual information processing are presented. Individual topics addressed include: aliasing as noise, satellite image processing using a hammering neural network, edge-detetion method using visual perception, adaptive vector median filters, design of a reading test for low-vision image warping, spatial transformation architectures, automatic image-enhancement method, redundancy reduction in image coding, lossless gray-scale image compression by predictive GDF, information efficiency in visual communication, optimizing JPEG quantization matrices for different applications, use of forward error correction to maintain image fidelity, effect of peanoscanning on image compression. Also discussed are: computer vision for autonomous robotics in space, optical processor for zero-crossing edge detection, fractal-based image edge detection, simulation of the neon spreading effect by bandpass filtering, wavelet transform (WT) on parallel SIMD architectures, nonseparable 2D wavelet image representation, adaptive image halftoning based on WT, wavelet analysis of global warming, use of the WT for signal detection, perfect reconstruction two-channel rational filter banks, N-wavelet coding for pattern classification, simulation of image of natural objects, number-theoretic coding for iconic systems.

Huck, Friedrich O.↗

NASA NTRS1990

Hybrid vision activities at NASA Johnson Space Center

NASA's Johnson Space Center in Houston, Texas, is active in several aspects of hybrid image processing. (The term hybrid image processing refers to a system that combines digital and photonic processing). The major thrusts are autonomous space operations such as planetary landing, servicing, and rendezvous and docking. By processing images in non-Cartesian geometries to achieve shift invariance to canonical distortions, researchers use certain aspects of the human visual system for machine vision. That technology flow is bidirectional; researchers are investigating the possible utility of video-rate coordinate transformations for human low-vision patients. Man-in-the-loop teleoperations are also supported by the use of video-rate image-coordinate transformations, as researchers plan to use bandwidth compression tailored to the varying spatial acuity of the human operator. Technological elements being developed in the program include upgraded spatial light modulators, real-time coordinate transformations in video imagery, synthetic filters that robustly allow estimation of object pose parameters, convolutionally blurred filters that have continuously selectable invariance to such image changes as magnification and rotation, and optimization of optical correlation done with spatial light modulators that have limited range and couple both phase and amplitude in their response.

Juday, Richard D.↗

DOE OSTI2022

Controlling Selectivity in Plasmonic Catalysis: Switching Reaction Pathway from Hydrogenation to Homocoupling Under Visible‐Light Irradiation

Abstract Plasmonic catalysis enables the use of light to accelerate molecular transformations. Its application to the control reaction selectivity is highly attractive but remains challenging. Here, we have found that the plasmonic properties in AgPd nanoparticles allowed different reaction pathways for tunable product formation under visible‐light irradiation. By employing the hydrogenation of phenylacetylene as a model transformation, we demonstrate that visible‐light irradiation can be employed to steer the reaction pathway from hydrogenation to homocoupling. Our data showed that the decrease in the concentration of H species at the surface due to plasmon‐enhanced H 2 desorption led to the control in selectivity. These results provide important insights into the understanding of reaction selectivity with light, paving the way for the application of plasmonic catalysis to the synthesis of 1,3‐diynes, and bringing the vision of light‐driven transformations with target selectivity one step closer to reality.

Peiris, Erandi↗

NASA NTRSDate not supplied

NASA R&M Efficiency through Findable, Accessible, Interoperable, and Reusable (FAIR) Digital Assets

At the intersection of mission, technology, and place is NASA’s need to modernize for a digital-forward future. Digitalization, the process of moving toward digital business, is occurring everywhere and remains an ongoing process across the federal government.”[1] Whereas, Digital Transformation is “employing digitization/digital technologies (e.g., Artificial Intelligence (AI), mobile, cloud, data) to change a process, product, or capability so dramatically (e.g., real-time, intelligent, personalized, anywhere, anytime) that it is unrecognizable compared to its traditional form.” [2] In order to facilitate a digital transformation it is essential for NASA to understand and identify where data exists today and which data are value-needed in the future, understand where there are unfulfilled data needs that limit the advancement of NASA work, and ensure NASA efficiency through Findable, Accessible, Interoperable, and Reusable (FAIR) digital assets in the future. Therefore, NASA’s Reliability & Maintainability (R&M) Enterprise Data Sharing team is working to leverage both Digitization and Digital Transformation to achieve their vision of developing an R&M data discovery framework that enables our community, our partners, and our stakeholders with the ability to efficiently, robustly, and seamlessly access information that enables real-time knowledge and model-based, analytics driven, decision-making impacting R&M. As a result the R&M Enterprise Data Sharing team has conducted a survey of its Reliability, Maintainability, and Availability (RMA) community members to identify data existence (created or used) and where there are corresponding barriers to data acquisition and/or R&M or other issues as shown within this presentation.

Digital Transformation↗

NASA NTRSDate not supplied

Trilateral Task Force – Reliability Analysis Supporting Mission Extension/Post Mission Disposal

Digital Transformation, Reliability Engineering↗

DOE OSTI2023

Chip-integrated metasurface full-Stokes polarimetric imaging sensor

Abstract Polarimetric imaging has a wide range of applications for uncovering features invisible to human eyes and conventional imaging sensors. Chip-integrated, fast, cost-effective, and accurate full-Stokes polarimetric imaging sensors are highly desirable in many applications, which, however, remain elusive due to fundamental material limitations. Here we present a chip-integrated Meta surface-based Full-Stokes Polar imetric Im aging sensor (MetaPolarIm) realized by integrating an ultrathin (~600 nm) metasurface polarization filter array (MPFA) onto a visible imaging sensor with CMOS compatible fabrication processes. The MPFA is featured with broadband dielectric-metal hybrid chiral metasurfaces and double-layer nanograting polarizers. This chip-integrated polarimetric imaging sensor enables single-shot full-Stokes imaging (speed limited by the CMOS imager) with the most compact form factor, records high measurement accuracy, dual-color operation (green and red) and a field of view up to 40 degrees. MetaPolarIm holds great promise to enable transformative applications in autonomous vision, industry inspection, space exploration, medical imaging and diagnosis.

47 OTHER INSTRUMENTATION↗

DOE OSTI2022

Non-Powered Dam Retrofit Exemplary Design for Hydropower Applications

Non-powered dams (NPDs) represent complex systems, situated at the intersection of natural stream environments and the built environment. The presence and operations of NPDs affect stream constituents (e.g., fish, recreational craft, sediment, water) and serve one or more engineered purposes (e.g., recreation, flood control, water supply, irrigation, tailings and debris control, navigation). By definition, NPDs do not provide hydropower generation. However, every NPD contains some untapped hydropower resource potential, represented by the pre-existing hydraulic head created by the dam and presence of flowing water. Adding power generation to an NPD requires retrofitting the NPD, which could involve the addition of any new component or function beyond that currently installed. Because this research is funded by the US Department of Energy Water Power Technologies Office, adding hydropower to NPDs is among the primary objectives for research investment. However, NPD retrofitting extends beyond the purview of hydropower additions. This report provides background information regarding NPD development in the United States, including an overview of the US NPD population, development potential, and recent development. It also summarizes challenges and opportunities facing NPD retrofit development, highlights the importance of maintaining or improving stream and dam functionality, notes key NPD characteristics, describes NPD retrofit methods, and identifies innovation areas for spurring future NPD retrofit development. Section 4 describes retrofit exemplary design principles and concepts that could apply to a wide range of NPD projects. These principles and concepts inform an NPD Retrofit Exemplary Design Specification (REDS), as included in Appendix A. The NPD REDS is heavily influenced by prior work under the standard modular hydropower technology acceleration research initiative, especially the work documented in the Exemplary Design Envelope Specification for Standard Modular Hydropower Technology (Witt et al., 2017). Whereas this prior work focuses on a particular technology class that applies most intentionally for new stream-reach development (i.e., greenfield sites), the NPD REDS contained herein applies more broadly to the NPD resource class and is technology-agnostic (i.e., the specific technologies intended to be described are not explicitly prescribed). This report serves as a part of early-phase research on NPD retrofit development, aimed toward spurring additional, follow-on efforts to specifically address development challenges, capitalize on development opportunities, and inspire innovation. As documented in the landmark “Hydropower Vision” report (DOE 2016), “transformative technical innovations able to meet the co-objectives of environmental sustainability and low-carbon energy will be critical to enabling additional hydropower growth.” This report serves as an important step in steering transformation of US non-powered dam infrastructure.

13 HYDRO ENERGY↗

NASA NTRS2014

Robonaut 2 - IVA Experiments On-Board ISS and Development Towards EVA Capability

Robonaut 2 (R2) has completed its fixed base activities on-board the ISS and is scheduled to receive its climbing legs in early 2014. In its continuing line of firsts, the R2 torso finished up its on-orbit activities on its stanchion with the manipulation of space blanket materials and performed multiple tasks under teleoperation control by IVA astronauts. The successful completion of these two IVA experiments is a key step in Robonaut's progression towards an EVA capability. Integration with the legs and climbing inside the ISS will provide another important part of the experience that R2 will need prior to performing tasks on the outside of ISS. In support of these on-orbit activities, R2 has been traversing across handrails in simulated zero-g environments and working with EVA tools and equipment on the ground to determine manipulation strategies for an EVA Robonaut. R2 made significant advances in robotic manipulation of deformable materials in space while working with its softgoods task panel. This panel features quarter turn latches that secure a space blanket to the task panel structure. The space blanket covers two cloth cubes that are attached with Velcro to the structure. R2 was able to open and close the latches, pull back the blanket, and remove the cube underneath. R2 simulated cleaning up an EVA worksite as well, by replacing the cube and reattaching the blanket. In order to interact with the softgoods panel, R2 has both autonomously and with a human in the loop identified and localized these deformable objects. Using stereo color cameras, R2 identified characteristic elements on the softgoods panel then extracted the location and orientation of the object in its field of view using stereo disparity and kinematic transforms. R2 used both vision processing and supervisory control to successfully accomplish this important task. Teleoperation is a key capability for Robonaut's effectiveness as an EVA system. To build proficiency, crewmembers have attempted increasingly difficult tasks using R2 inside the Station. After donning motion capture equipment and a virtual reality visor, Expedition 34/35 flight engineer Tom Marshburn began operations with simple hand movements. Having gained confidence, Marshburn guided R2's arms in a leader-follower exercise with crewmate Chris Cassidy. He was also able to use the hand to grab a tumbling roll of tape, a task only demonstrable in microgravity. Later efforts saw Cassidy handle softgoods through shared control with ground operators, mimicking an activity previously achieved using only autonomy. Robotic climbing through the ISS on handrails requires both precision motion and compliant grasps in order to both position grippers on handrails/seat track and prevent large internal forces. R2 climbs using actively controlled compliance and torque limiting to meet both the precision and softness requirements. During a step, the attached leg is controlled to be strong and stiff in order to maintain precision trajectory tracking. The swing leg is controlled to be stiff but weak to minimize unintentional impact forces while maintaining precision. During a simulated dual limb grasp (as shown in Figure 1), the R2 controller maintains one limb rigid and one limb soft to prevent large internal forces from building up. R2's grippers also use a form of force control to limit grip force while not fully closed on either a handrail or seat track thus limiting unintentional forces on cables/objects that may be present in R2's translational path. The on-board torso R2 safety system relies on a single end-effector velocity limit to prevent potential impact forces from exceeding Station maximum load requirements. R2's mobile configuration required modifications to the velocity limiting safety function due to its large, dynamic inertia. R2's legs maneuver the robot's mass creating configuration dependent, joint-relative inertias. A single all-encompassing velocity limit to cover worst case inertia is prohibitively low. The upgraded R2 control and safety systems solve this problem using momentum limiting, momentum control, and kinetic energy minimization. Momentum and kinetic energy take the robot mass into account relieving low velocity restrictions on low inertia end-effectors while ensuring that the overall mass of R2 is limited from hazardous velocities. The momentum of R2's five safety nodes (each of the four end-effectors and the body) is monitored and compared to a single momentum limit. If any of the five nodes exceeds the safety limit, the motor power is removed and the robot comes to a stop. Momentum control/limiting also provides a simple, reliable method to integrate hand held tools into the safety system by providing the tool mass to the control system thus automatically reducing the allowable velocity of the end-effector with the tool. Work on the ground continues to build the skill set for an EVA Robonaut. Recent experiments (Figure 2) demonstrate how a teleoperator can use R2 to manipulate a tether hook, an important safety precaution on spacewalks. Another task displayed Robonaut's ability to pull back a protective jacket over a hose and search for damage, as well as inspect a quick-disconnect fitting for debris. Demonstrations such as these are indicative of EVA work done on ISS, specifically seen during a series of spacewalks over 2012 and 2013 where astronauts searched for an ammonia leak in one of the external cooling loops. Through experiments both on ISS and on the ground, R2 is evolving and providing the information needed to plan out the upgrades that will make an EVA Robonaut an effective tool. With the addition of legs, R2 will start climbing inside the space station and supply invaluable information on how the climbing strategies and task stabilization techniques must be refined. Ground R2 systems will continue to work with additional EVA tools and equipment in preparation for onboard IVA testing and future EVA applications.

Diftler, Myron↗

NASA NTRS2005

Camera Image Transformation and Registration for Safe Spacecraft Landing and Hazard Avoidance

Inherent geographical hazards of Martian terrain may impede a safe landing for science exploration spacecraft. Surface visualization software for hazard detection and avoidance may accordingly be applied in vehicles such as the Mars Exploration Rover (MER) to induce an autonomous and intelligent descent upon entering the planetary atmosphere. The focus of this project is to develop an image transformation algorithm for coordinate system matching between consecutive frames of terrain imagery taken throughout descent. The methodology involves integrating computer vision and graphics techniques, including affine transformation and projective geometry of an object, with the intrinsic parameters governing spacecraft dynamic motion and camera calibration.

Jones, Brandon M.↗