Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Vision transformers”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

National Smart Manufacturing Strategic Plan: To Facilitate More Rapid Development, Deployment and Adoption of Smart Manufacturing Technologies

Smart manufacturing technologies provide real-time data and insight to improve the productivity, efficiency, and competitiveness of U.S. manufacturing, creating the potential for new jobs in the manufacturing sector. These technologies can support U.S. manufacturers’ ability to increase throughput and energy efficiency, and decrease waste, defects, and costs. A wide range of manufacturing and industrial subsectors, particularly energy-intensive and energy-dependent industries, have the potential to benefit from smart manufacturing technologies. There are technical improvements still needed to reduce the costs and barriers (trained workforce and upskilling, software-hardware integration, cost, and technical barriers to deployment of advanced sensors, computing, and communication technologies for existing manufacturing assets) to the adoption of smart manufacturing technologies, especially to small and medium-sized manufacturers, and subsequently, increase the overall adoption rate of smart manufacturing technologies. This report outlines the Department of Energy’s (DOE) strategic plan to accelerate the development and implementation of smart manufacturing technologies in the United States and the actions DOE has taken to use these technologies in smart manufacturing for the United States. DOE’s plan to facilitate more rapid development, deployment and widespread adoption of smart manufacturing technologies derive from the 2018 Strategy for American Leadership in Advanced Manufacturing. The strategy outlines a vision for America’s leadership in advanced manufacturing including the development of intelligent manufacturing systems to optimize manufacturing facilities and support the manufacturing of transformative materials. DOE’s Clean Energy Smart Manufacturing Innovation Institute (CESMII), a Manufacturing USA Institute, focuses on accelerating the development and adoption of advanced sensors, controls, platforms, and models needed for smart manufacturing. The objective is to enhance U.S. manufacturing productivity and global competitiveness through the research and development of these technologies. Smart manufacturing has the potential to improve the overall performance, energy productivity, and efficiency of manufacturing, fostering the economic competitiveness of the manufacturing sector.

99 GENERAL AND MISCELLANEOUS↗
NASA NTRS2005

In Situ Resource Utilization Technology Research and Facilities Supporting the NASA's Human Systems Research and Technology Life Support Program

The NASA Microgravity Science program has transitioned research required in support of NASA s Vision for Space Exploration. Research disciplines including the Materials Science, Fluid Physics and Combustion Science are now being applied toward projects with application in the planetary utilization and transformation of space resources. The scientific and engineering competencies and infrastructure in these traditional fields developed at multiple NASA Centers and by external research partners provide essential capabilities to support the agency s new exploration thrusts including In-Situ Resource Utilization (ISRU). Among the technologies essential to human space exploration, the production of life support consumables, especially oxygen and; radiation shielding; and the harvesting of potentially available water are realistically achieved for long-duration crewed missions only through the use of ISRU. Ongoing research in the physical sciences have produced a body of knowledge relevant to the extraction of oxygen from lunar and planetary regolith and associated reduction of metals and silicon for use meeting manufacturing and repair requirements. Activities being conducted and facilities used in support of various ISRU projects at the Glenn Research Center and Marshall Space Flight Center will be described. The presentation will inform the community of these new research capabilities, opportunities, and challenges to utilize their materials, fluids and combustion science expertise and capabilities to support the vision for space exploration.

Schlagheck, Ronald A.↗
DOE OSTI2023

Universal Polarization Transformations: Spatial Programming of Polarization Scattering Matrices Using a Deep Learning‐Designed Diffractive Polarization Transformer

Abstract Controlled synthesis of optical fields having nonuniform polarization distributions presents a challenging task. Here, a universal polarization transformer is demonstrated that can synthesize a large set of arbitrarily‐selected, complex‐valued polarization scattering matrices between the polarization states at different positions within its input and output field‐of‐views (FOVs). This framework comprises 2D arrays of linear polarizers positioned between isotropic diffractive layers, each containing tens of thousands of diffractive features with optimizable transmission coefficients. After its deep learning‐based training, this diffractive polarization transformer can successfully implement N i N o = 10 000 different spatially‐encoded polarization scattering matrices with negligible error, where N i and N o represent the number of pixels in the input and output FOVs, respectively. This universal polarization transformation framework is experimentally validated in the terahertz spectrum by fabricating wire‐grid polarizers and integrating them with 3D‐printed diffractive layers to form a physical polarization transformer. Through this set‐up, an all‐optical polarization permutation operation of spatially‐varying polarization fields is demonstrated, and distinct spatially‐encoded polarization scattering matrices are simultaneously implemented between the input and output FOVs of a compact diffractive processor. This framework opens up new avenues for developing novel devices for universal polarization control and may find applications in, e.g., remote sensing, medical imaging, security, material inspection, and machine vision.

Optical neural networks↗
NASA NTRS2016

Visions of the Future: Hybrid Electric Aircraft Propulsion

The National Aeronautics and Space Administration (NASA) is investing continually in improving civil aviation. Hybridization of aircraft propulsion is one aspect of a technology suite which will transform future aircraft. In this context, hybrid propulsion is considered a combination of traditional gas turbine propulsion and electric drive enabled propulsion. This technology suite includes elements of propulsion and airframe integration, parallel hybrid shaft power, turbo-electric generation, electric drive systems, component development, materials development and system integration at multiple levels.

electric motor vehicles↗
DOE OSTI2026

Project 1: COLLECTS Project and Project 2: M&V for Saputo Cheese Project: Cooperative Research and Development (Final Report)

Project 1: Combined Power's Hyperlight Energy and NLR will collaborate on the DOE Concentrating Optics for Lower Levelized Energy CosTS (COLLECTS) project, DE-FOA-0001268. COLLECTS funding opportunity announcement (FOA) seeks to further concentrated solar power (CSP) system technologies by soliciting disruptive, transformative projects for the concentrating solar collectors in the CSP plant. These innovative projects will seek to surpass the targets set out in the SunShot Vision Study, enabling CSP to be cost-competitive with conventional forms of electric power generation. Projects will target the design and manufacturing of novel solar collectors with the ability to significantly reduce the solar field contribution to the overall levelized cost of energy (LCOE) of the CSP plant, and which have not been previously specifically targeted by CSP FOAs. Project 2: Hyperlight will perform measurement and verification (M&V) of Green House Gases and NLR will evaluate energy consumption reduction.

14 SOLAR ENERGY↗
NASA NTRS1990

Programmable Remapper

Input image remapped rapidly and accurately onto different coordinate grid. Analog/digital electronic image-processing system developed to warp input images onto arbitrary coordinate grids at video rates. Advantages of system include antialiasing effect of many-to-one data path and speed of lookup-table operation. Lookup tables reprogrammed easily with help of computer that generates table values from mathematical description of desired transformation. Applications include real-time corrections of distortions in input optics of image sensors, corrections for repeatable nonlinear scanning, and aiding persons of impaired vision by deliberately distorting images onto remaining functional portions of retinas.

Juday, Richard D.↗
NASA NTRS1997

Head-Mounted and Head-Up Display Glossary

One of the problems in head-up and helmet-mounted display (HMD) literature has been a lack of standardization of words and abbreviations. Several different words have been used for the same concept; for example, flight path angle, flight path marker, velocity vector, and total velocity vector all refer to the same thing. In other cases, the same term has been used with two different meanings, such as binocular field-of-view which means the field-of-view visible to both left and right eyes according to some or the field-of-view visible to either the left or right eye or both according to others. Many of the terms used in HMD studies have not been well-defined. We need to have a common language to ensure that system descriptions are communicated. As an example, the term 'stabilized' has been widely used with two meanings. 'Roll-stabilized' has been used to mean a symbol which rotates to indicate the roll or bank of the aircraft. 'World-stabilized' and 'head-stabilized' have both been used to indicate symbols which move to remain fixed with respect to external objects. HMDs present unique symbology problems not found in HUDs. Foremost among these is the issue of maintaining spatial orientation of the symbols. All previous flight displays, round dial instruments, HDDs, and HUDs have been fixed in the cockpit. With the HMD, the flight display can move through a large angle. The coordinates use in transforming from the real-world to the aircraft to the HMD have not been consistently defined. This glossary contains terms relating to optics and vision, displays, and flight information, weapons and aircraft systems. Some definitions, such as Navigation Display, have been added to clarify the definitions for Primary Flight Display and Primary Flight Reference. A list of HUD/HMD related abbreviations is also included.

Newman, Richard L.↗
NASA NTRS2025

Distributed Sensing Node Configuration

A vision and radar sensor integration strategy for surveillance of Advance Air Mobility Mission concept of airspace is contained in the CAD drawing in this publication developed as part of NASA's Transformational Tools and Technology project. High-density airspace operations with multiple aircraft type and potentially noncooperative aircraft require distributed sensor detection and tracking systems to monitor airspace for safe, autonomous flight operations for advanced air mobility, urban air mobility, and high density small uncrewed air systems (SUAS) flight concepts. Contained herein are drawings that are available in the stp and creo file formats of the distributed sensing nodes. Additional dovetail integration is included to enable rapid changing of embedded system. A day in the life node deployment is included in the 2023_last_flight_day.mp4. Please cite this publication if you use these drawings. This sensor configuration methodology is presented in detail in the publication reference below. Chester Dolph, Thomas Lombaerts, Corey A. Ippolito, Vahram Stepanyan, Evan Kawamura, Keerthana Kannan, George Szatkowski, Todd Ferante, Christopher Morris, Federica Vitiello, Flavia Causa, Roberto Opromolla and Giancarmine Fasano. "Distributed Sensor Fusion of Ground and Air Nodes using Vision and Radar Modalities for Tracking Multirotor Small Uncrewed Air Systems and Birds," AIAA 2024-1781. AIAA SCITECH 2024 Forum. January 2024.

Chester V Dolph↗
DOE OSTI2024

Effects of input gradient regularization on neural networks time-series forecasting of thermal power systems

This study proposes using neural networks, specifically gated recurrent unit (GRU), long-short-term memory (LSTM), and transformer networks, to improve control strategies in a 450 MW coal-fired power plant. However, neural networks face issues of becoming overly dependent on just a few variables to make predictions, which negatively impacts control decisions that rely on the model to determine the value of all manipulated variables. The paper introduces regularization techniques, including noise injection and input gradient regularization, during the training phase. Here, the work presents novel contributions in adapting neural networks to control industrial systems and applying regularization techniques from computer vision to industrial process control. Results demonstrate the effectiveness of input gradient regularization in reducing model dependence on subsets of variables, emphasizing the balance between fidelity and controllability. Further exploration is recommended, including the development of recurrent transformers, closed-loop control testing, and a sensitivity analysis on computer models to provide further insight.

20 FOSSIL-FUELED POWER PLANTS↗
DOE OSTI2026

3D reconstruction and neural rendering for adversarial machine learning

While evasion attacks on computer vision systems have been widely studied, creating attacks that remain effective under significant changes in viewpoint continues to be challenging. Traditional approaches often rely on affine transformations of images, but these approaches degrade at larger perspective shifts and often produce unrealistic or ineffective perturbations. Recent methods use differentiable renderers to improve viewpoint robustness, but they typically depend on manually constructed 3D models. We introduce a semi-automated pipeline that generates physically printable and perspective-invariant adversarial patches using only a small set of 2D images. Our method integrates 3D reconstruction, neural rendering, adversarial patch optimization, and an object detection victim model into a unified workflow. We use 2D Gaussian Splatting for high fidelity mesh reconstruction and FlexPara for surface parameterization that produces texture maps suitable for patch editing. Together, these components form a fully differentiable pipeline in PyTorch3D that links texture modification to model outputs, enabling efficient optimization of patches that remain effective across many viewpoints. The complete process, from image capture to patch printing and physical evaluation, can be completed within a few hours. We demonstrate the effectiveness of the resulting patches through attacks on the YOLOv8 object detection model and discuss remaining challenges and opportunities for improving robustness and scalability.

Singhvi, Vivaan [ORNL] (ORCID:0009000586288221)↗
DOE OSTI2026

Plant Design for a Developing Bioeconomy Workshop Report: Frontier Science for the Bioeconomy Workshop Series

Recent advances in fundamental plant biology research, synthetic biology, and artificial intelligence (AI) are unlocking powerful new capabilities in plant biodesign, offering unprecedented potential to reimagine plants as programmable platforms for resource-efficient production of bioenergy, biomaterials, chemicals, and more. The U.S. Department of Energy (DOE) convened the Plant Design for a Developing Bioeconomy virtual workshop on March 12 through 14, 2025, to bring together leaders across plant science, engineering, and computation to assess the current landscape and define a bold vision for future research. Discussions during the workshop built upon findings included in DOE’s Biological and Environmental Research (BER) workshop report Overcoming Barriers in Plant Transformation: A Focus on Bioenergy Crops (U.S. DOE 2024; genomicscience. energy.gov/plant-transformation). Participants identified critical knowledge gaps, technical barriers, and emerging opportunities in the design and engineering of plant systems to support a robust, resilient domestic bioeconomy aligned with DOE’s mission.

09 BIOMASS FUELS↗
DOE OSTI2026

Future foundries: A convergent manufacturing platform

This article introduces the Future Foundries platform developed at Oak Ridge National Laboratory, a first-generation research system designed to demonstrate convergent manufacturing. Convergent manufacturing brings together additive, subtractive, and transformative processes in a digitally interconnected environment to enable end-to-end production workflows. By linking traditionally discrete steps, convergent platforms accelerate production, improve repeatability, and support high-mix, low-volume manufacturing. The Future Foundries platform exemplifies this vision in practice by combining four modular, vendor-agnostic process cells that include robotic WAAM, induction heating, optical metrology, and machining, coordinated through an automated pallet handler and a ROS 2-based digital thread. This architecture provides the flexibility and scalability needed for agile production in small and medium-sized manufacturing enterprises and for field deployable manufacturing. Two case studies illustrate the platform’s capabilities. The first presents an integrated workflow for fabricating, transforming, and repairing critical replacement components, showing how consolidated thermal, additive, inspection, and machining operations reduce manual part handling and streamline process flow. The second case study highlights coordinated multi-part production enabled by automated pallet logistics and multi-cell scheduling. Together, these examples showcase convergent manufacturing as a practical and scalable strategy for strengthening domestic casting and forging capacity, improving supply-chain resilience, and enabling rapid, adaptable production of mission-critical components.

Convergent manufacturing↗
NASA NTRS1994

Technical guidance for the development of a solid state image sensor for human low vision image warping

This report surveys different technologies and approaches to realize sensors for image warping. The goal is to study the feasibility, technical aspects, and limitations of making an electronic camera with special geometries which implements certain transformations for image warping. This work was inspired by the research done by Dr. Juday at NASA Johnson Space Center on image warping. The study has looked into different solid-state technologies to fabricate image sensors. It is found that among the available technologies, CMOS is preferred over CCD technology. CMOS provides more flexibility to design different functions into the sensor, is more widely available, and is a lower cost solution. By using an architecture with row and column decoders one has the added flexibility of addressing the pixels at random, or read out only part of the image.

Vanderspiegel, Jan↗
NASA NTRS1988

Discrete analysis of spatial-sensitivity models

Procedures for reducing the computational burden of current models of spatial vision are described, the simplifications being consistent with the prediction of the complete model. A method for using pattern-sensitivity measurements to estimate the initial linear transformation is also proposed which is based on the assumption that detection performance is monotonic with the vector length of the sensor responses. It is shown how contrast-threshold data can be used to estimate the linear transformation needed to characterize threshold performance.

Nielsen, Kenneth R. K.↗
DOE OSTI2023

Advanced Research Directions on AI for Science, Energy, and Security: Report on Summer 2022 Workshops

Over the past decade, fundamental changes in artificial intelligence (AI)—from foundational to applied—have delivered dramatic insights across a wide breadth of U.S. Department of Energy (DOE) mission space. AI is helping to augment and improve scientific and engineering workflows (e.g., for control, design, and dramatic performance gains through surrogate models) in national security, the Office of Science, and DOE’s applied energy programs. The progress and potential for AI in DOE science was captured in the 2020 “AI for Science” report from the DOE laboratory community in collaboration with academia and industry. Specific scientific areas ready to further leverage the power of AI ranged from the scale and performance of computational models to data analysis to creating new classes of observations using computer vision. Since that report, the scale and scope of scientific AI have accelerated, revealing new, emergent properties that yield insights that go beyond enabling opportunities to being potentially transformative in the way that scientific problems are posed and solved. Thus, under the guidance of both the Office of Science (SC) and the National Nuclear Security Administration (NNSA), the DOE national laboratories organized a series of workshops in 2022 to gather input on new and rapidly emerging opportunities and challenges of scientific AI. This 2023 report is a synthesis of those workshops. The scientific community believes AI can have a foundational impact on a broad range of DOE missions, including science, energy, and national security. Further, DOE has unique capabilities that enable the community to drive progress in scientific use of AI, building on long-standing DOE strengths and investments in computation, data, and communications infrastructure, spanning the Energy Sciences Network (ESnet), the Exascale Computing Project (ECP), and integrative programs such as the NNSA Office of Defense Programs Advanced Simulation and Computing (ASC) and the SC Scientific Discovery through Advanced Computing (SciDAC) programs.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2007

Sensing Super-Position: Human Sensing Beyond the Visual Spectrum

The coming decade of fast, cheap and miniaturized electronics and sensory devices opens new pathways for the development of sophisticated equipment to overcome limitations of the human senses. This paper addresses the technical feasibility of augmenting human vision through Sensing Super-position by mixing natural Human sensing. The current implementation of the device translates visual and other passive or active sensory instruments into sounds, which become relevant when the visual resolution is insufficient for very difficult and particular sensing tasks. A successful Sensing Super-position meets many human and pilot vehicle system requirements. The system can be further developed into cheap, portable, and low power taking into account the limited capabilities of the human user as well as the typical characteristics of his dynamic environment. The system operates in real time, giving the desired information for the particular augmented sensing tasks. The Sensing Super-position device increases the image resolution perception and is obtained via an auditory representation as well as the visual representation. Auditory mapping is performed to distribute an image in time. The three-dimensional spatial brightness and multi-spectral maps of a sensed image are processed using real-time image processing techniques (e.g. histogram normalization) and transformed into a two-dimensional map of an audio signal as a function of frequency and time. This paper details the approach of developing Sensing Super-position systems as a way to augment the human vision system by exploiting the capabilities of Lie human hearing system as an additional neural input. The human hearing system is capable of learning to process and interpret extremely complicated and rapidly changing auditory patterns. The known capabilities of the human hearing system to learn and understand complicated auditory patterns provided the basic motivation for developing an image-to-sound mapping system. The human brain is superior to most existing computer systems in rapidly extracting relevant information from blurred, noisy, and redundant images. From a theoretical viewpoint, this means that the available bandwidth is not exploited in an optimal way. While image-processing techniques can manipulate, condense and focus the information (e.g., Fourier Transforms), keeping the mapping as direct and simple as possible might also reduce the risk of accidentally filtering out important clues. After all, especially a perfect non-redundant sound representation is prone to loss of relevant information in the non-perfect human hearing system. Also, a complicated non-redundant image-to-sound mapping may well be far more difficult to learn and comprehend than a straightforward mapping, while the mapping system would increase in complexity and cost. This work will demonstrate some basic information processing for optimal information capture for headmounted systems.

Maluf, David A.↗
DOE OSTI2023

Bottleneck Detection in Modular Construction Factories Using Computer Vision

The construction industry is increasingly adopting off-site and modular construction methods due to the advantages offered in terms of safety, quality, and productivity for construction projects. Despite the advantages promised by this method of construction, modular construction factories still rely on manually-intensive work, which can lead to highly variable cycle times. As a result, these factories experience bottlenecks in production that can reduce productivity and cause delays to modular integrated construction projects. To remedy this effect, computer vision-based methods have been proposed to monitor the progress of work in modular construction factories. However, these methods fail to account for changes in the appearance of the modular units during production, they are difficult to adapt to other stations and factories, and they require a significant amount of annotation effort. Due to these drawbacks, this paper proposes a computer vision-based progress monitoring method that is easy to adapt to different stations and factories and relies only on two image annotations per station. In doing so, the Scale-invariant feature transform (SIFT) method is used to identify the presence of modular units at workstations, and the Mask R-CNN deep learning-based method is used to identify active workstations. This information was synthesized using a near real-time data-driven bottleneck identification method suited for assembly lines in modular construction factories. This framework was successfully validated using 420 h of surveillance videos of a production line in a modular construction factory in the U.S., providing 96% accuracy in identifying the occupancy of the workstations and an F-1 Score of 89% in identifying the state of each station on the production line. The extracted active and inactive durations were successfully used via a data-driven bottleneck detection method to detect bottleneck stations inside a modular construction factory. The implementation of this method in factories can lead to continuous and comprehensive monitoring of the production line and prevent delays by timely identification of bottlenecks.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗