Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Vision transformers”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16

Forecasting Commercial Building Electricity Consumption, Zone Airflow and Zone Temperature: Update - Development of a Generalized Machine Learning Approach

The U.S. power grid is being transformed to make it smarter, more efficient, and cleaner. This transformation is leading to the addition of a significant of energy generated by distributed, variable, and renewable resources. Because of the variable nature of renewable generation, the short- and long-term supply and demand imbalances are less predictable, and conventional approaches to mitigating the imbalances will be less efficient or cost effective. To address this challenge and to support the mission and the vision of the U.S. Department of Energy’s (DOE’s) Office of Energy Efficiency and Renewable Energy (EERE) Building Technologies Office has developed a Grid-Interactive Efficient Building Strategy. The strategy focuses on simultaneously improving building energy efficiency and supporting reliability and resilience of the electric grid more efficiently and at a lower cost. In addition, EERE and DOE’s Office of Electricity created an initiative led by DOE and supported by the national laboratories under the Grid Modernization Lab Consortium structure to enhance grid modernization. The work reported in this document is part of the first set of projects funded under the initiative to design, develop, and validate scalable transactive control technologies for the commercial buildings sector. Transactive controls requires the ability of individual end-use loads to express flexibility as a function of a transactive signal (e.g., price). Empirical grey- and black-box models have been widely used to express flexibility. Although this approach is generally easy to construct and simple to use, it does not capture non-linear behavior that some end-use loads represent. Therefore, Pacific Northwest National Laboratory (PNNL) with support from Western Washington University conducted this research to explore the use of deep machine learning (ML) techniques. The work reported in this document is limited to forecasting whole building electricity consumption, the zone airflow and the zone temperature predictions.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Ten Years of Land Cover Change on the California Coast Detected using Landsat Satellite Image Analysis

Landsat satellite imagery was analyzed to generate a detailed record of 10 years of vegetation disturbance and regrowth for Pacific coastal areas of Marin and San Francisco Counties. The Landsat Ecosystem Disturbance Adaptive Processing System (LEDAPS) methodology, a transformation of Tasseled-Cap data space, was applied to detected changes in perennial coastal shrubland, woodland, and forest cover from 1999 to 2009. Results showed several principal points of interest, within which extensive contiguous areas of similar LEDAPS vegetation change (either disturbed or restored) were detected. Regrowth areas were delineated as burned forest areas in the Point Reyes National Seashore (PRNS) from the 1995 Vision Fire. LEDAPS-detected disturbance patterns on Inverness Ridge, PRNS in areas observed with dieback of tanoak and bay laurel trees was consistent with defoliation by sudden oak death (Phytophthora ramorum). LEDAPS regrowth pixels were detected over much of the predominantly grassland/herbaceous cover of the Olema Valley ranchland near PRNS. Extensive restoration of perennial vegetation cover on Crissy Field, Baker Beach and Lobos Creek dunes in San Francisco was identified. Based on these examples, the LEDAPS methodology will be capable of fulfilling much of the need for continual, low-cost monitoring of emerging changes to coastal ecosystems.

Land Cover Change↗

Laser electro-optic system for rapid three-dimensional /3-D/ topographic mapping of surfaces

It is pointed out that the generic utility of a robot in a factory/assembly environment could be substantially enhanced by providing a vision capability to the robot. A standard videocamera for robot vision provides a two-dimensional image which contains insufficient information for a detailed three-dimensional reconstruction of an object. Approaches which supply the additional information needed for the three-dimensional mapping of objects with complex surface shapes are briefly considered and a description is presented of a laser-based system which can provide three-dimensional vision to a robot. The system consists of a laser beam array generator, an optical image recorder, and software for controlling the required operations. The projection of a laser beam array onto a surface produces a dot pattern image which is viewed from one or more suitable perspectives. Attention is given to the mathematical method employed, the space coding technique, the approaches used for obtaining the transformation parameters, the optics for laser beam array generation, the hardware for beam array coding, and aspects of image acquisition.

Altschuler, M. D.↗

Real World Use Case Evaluation of Radar Retro-reflectors for Autonomous Vehicle Lane Detection Applications

Lane detection plays a critical role in autonomous vehicles for safe and reliable navigation. Lane detection is traditionally accomplished using a camera sensor and computer vision processing. The downside of this traditional technique is that it can be computationally intensive when high quality images at a fast frame rate are used and has reliability issues from occlusion such as, glare, shadows, active road construction, and more. This study addresses these issues by exploring alternative methods for lane detection in specific scenarios caused from road construction-induced lane shift and sun glare. Specifically, a U-Net, a convolutional network used for image segmentation, camera-based lane detection method is compared with a radar-based approach using a new type of sensor previously unused in the autonomous vehicle space: radar retro-reflectors. This evaluation is performed using ground truth data, obtained by measuring the lane positions and transforming them into pixel coordinates. The performance of each method is assessed using the statistical R2 score, indicating the correlation between the detected lane lines and the ground truth. The results show that the U-Net camera-based method exhibits limitations in accurately detecting and aligning the lane lines, particularly in challenging scenarios. However, the radar-based lane detection method demonstrates a strong correlation with the ground truth which implies that the use of this sensor may improve current reliability issues from conventional camera lane detection approach. Furthermore, the study highlights the limitations of the U-Net model for camera lane detection, especially in scenarios with sun glare. This study shows that infrastructure-based radar retro-reflectors can improve autonomous vehicle lane detection reliability. The integration of different sensor modalities and the development of advanced computer vision algorithms are crucial for improving the accuracy, reliability, and energy efficiency of lane detection systems. Addressing these challenges contributes to the advancement of autonomous vehicles and the realization of safer and more efficient transportation systems.

Brown, Nicolas E.↗

Terrain Model Registration for Single Cycle Instrument Placement

This paper presents an efficient and robust method for registration of terrain models created using stereo vision on a planetary rover. Our approach projects two surface models into a virtual depth map, rendering the models as they would be seen from a single range sensor. Correspondence is established based on which points project to the same location in the virtual range sensor. A robust norm of the deviations in observed depth is used as the objective function, and the algorithm searches for the rigid transformation which minimizes the norm. An initial coarse search is done using rover pose information from odometry and orientation sensing. A fine search is done using Levenberg-Marquardt. Our method enables a planetary rover to keep track of designated science targets as it moves, and to hand off targets from one set of stereo cameras to another. These capabilities are essential for the rover to autonomously approach a science target and place an instrument in contact in a single command cycle.

Deans, Matthew↗

Extensions of algebraic image operators: An approach to model-based vision

Researchers extend their previous research on a highly structured and compact algebraic representation of grey-level images which can be viewed as fuzzy sets. Addition and multiplication are defined for the set of all grey-level images, which can then be described as polynomials of two variables. Utilizing this new algebraic structure, researchers devised an innovative, efficient edge detection scheme. An accurate method for deriving gradient component information from this edge detector is presented. Based upon this new edge detection system researchers developed a robust method for linear feature extraction by combining the techniques of a Hough transform and a line follower. The major advantage of this feature extractor is its general, object-independent nature. Target attributes, such as line segment lengths, intersections, angles of intersection, and endpoints are derived by the feature extraction algorithm and employed during model matching. The algebraic operators are global operations which are easily reconfigured to operate on any size or shape region. This provides a natural platform from which to pursue dynamic scene analysis. A method for optimizing the linear feature extractor which capitalizes on the spatially reconfiguration nature of the edge detector/gradient component operator is discussed.

Lerner, Bao-Ting↗

Stellar Imager

The Stellar Imager (SI) is one of NASA's "Vision Missions" - concepts for future, space-based, strategic missions that could enormously increase our capabilities for observing the Cosmos. SI is designed as a UV/Optical Interferometer which will enable 0.1 milli-arcsecond (mas) spectral imaging of stellar surfaces and, via asteroseismology, stellar interiors and of the Universe in general. The ultra-sharp images of the Stellar Imager will revolutionize our view of many dynamic astrophysical processes by transforming point sources into extended sources, and snapshots into evolving views. SI, with a characteristic angular resolution of 0.1 milli-arcseconds at 2000 Angstroms, represents an advance in image detail of several hundred times over that provided by the Hubble Space Telescope. The Stellar Imager will zoom in on what today-with few exceptions - we only know as point sources, revealing processes never before seen, thus providing a tool as fundamental to astrophysics as the microscope is to the study of life on Earth. SI's science focuses on the role of magnetism in the Universe, particularly on magnetic activity on the surfaces of stars like the Sun. It's prime goal is to enable long-term forecasting of solar activity and the space weather that it drives, in support of the Living With a Star program in the Exploration Era. SI will also revolutionize our understanding of the formation of planetary systems, of the habitability and climatology of distant planets, and of many magneto-hydrodynamically controlled processes in the Universe. Stellar Imager is included as a "Flagship and Landmark Discovery Mission" in the 2005 Sun Solar System Connection (SSSC) Roadmap and as a candidate for a "Pathways to Life Observatory" in the Exploration of the Universe Division (EUD) Roadmap (May, 2005) and as such is a candidate mission for the 2025-2030 timeframe. An artist's drawing of the current "baseline" concept for SI is presented.

Carpenter, Kenneth↗

Adaptive Protective Relay Settings – A Vision to the Future

Adaptive relaying utilizes the continuously changing status of the power system as the basis for online adjustment of the power system relay settings. Fundamentally they are protection schemes that adjust settings and/or logic of operations based on the prevailing conditions of the system. These adjustments can include, but are not limited to, the logging of data for post-mortem analysis, communication throughout the system, as well as changing relay parameters. Adaptive relaying considers the fact that the status of a power system can change. These include system configuration changes, load effect, cold load pickup, end-of-line protection, transformer protection, and automatic reclosing. In this research, the author focus on the need for a secure, selective, and reliable system for adaptive overcurrent protection in T&D and Distributed Energy Systems. Various types of adaptive methods are presented and explained along with some pros and cons of each.

Mukherjee, Srijib↗

VISION: a modular AI assistant for natural human-instrument interaction at scientific user facilities

Scientific user facilities, such as synchrotron beamlines, are equipped with a wide array of hardware and software tools that require a codebase for human-computer-interaction. This often necessitates developers to be involved to establish connection between users/researchers and the complex instrumentation. The advent of generative AI presents an opportunity to bridge this knowledge gap, enabling seamless communication and efficient experimental workflows. Here we present a modular architecture for the Virtual Scientific Companion by assembling multiple AI-enabled cognitive blocks that each scaffolds large language models (LLMs) for a specialized task. With VISION, we performed LLM-based operation on the beamline workstation with low latency and demonstrated the first voice-controlled experiment at an x-ray scattering beamline. The modular and scalable architecture allows for easy adaptation to new instruments and capabilities. Development on natural language-based scientific experimentation is a building block for an impending future where a science exocortex—a synthetic extension to the cognition of scientists—may radically transform scientific practice and discovery.

36 MATERIALS SCIENCE↗

Information Systems for NASA's Aeronautics and Space Enterprises

The aerospace industry is being challenged to reduce costs and development time as well as utilize new technologies to improve product performance. Information technology (IT) is the key to providing revolutionary solutions to the challenges posed by the increasing complexity of NASA's aeronautics and space missions and the sophisticated nature of the systems that enable them. The NASA Ames vision is to develop technologies enabling the information age, expanding the frontiers of knowledge for aeronautics and space, improving America's competitive position, and inspiring future generations. Ames' missions to accomplish that vision include: 1) performing research to support the American aviation community through the unique integration of computation, experimentation, simulation and flight testing, 2) studying the health of our planet, understanding living systems in space and the origins of the universe, developing technologies for space flight, and 3) to research, develop and deliver information technologies and applications. Information technology may be defined as the use of advance computing systems to generate data, analyze data, transform data into knowledge and to use as an aid in the decision-making process. The knowledge from transformed data can be displayed in visual, virtual and multimedia environments. The decision-making process can be fully autonomous or aided by a cognitive processes, i.e., computational aids designed to leverage human capacities. IT Systems can learn as they go, developing the capability to make decisions or aid the decision making process on the basis of experiences gained using limited data inputs. In the future, information systems will be used to aid space mission synthesis, virtual aerospace system design, aid damaged aircraft during landing, perform robotic surgery, and monitor the health and status of spacecraft and planetary probes. NASA Ames through the Center of Excellence for Information Technology Office is leading the effort in pursuit of revolutionary, IT-based approaches to satisfying NASA's aeronautics and space requirements. The objective of the effort is to incorporate information technologies within each of the Agency's four Enterprises, i.e., Aeronautics and Space Transportation Technology, Earth, Science, Human Exploration and Development of Space and Space Sciences. The end results of these efforts for Enterprise programs and projects should be reduced cost, enhanced mission capability and expedited mission completion.

Kutler, Paul↗

Use of multiresolution wavelet feature pyramids for automatic registration of multisensor imagery

The problem of image registration, or the alignment of two or more images representing the same scene or object, has to be addressed in various disciplines that employ digital imaging. In the area of remote sensing, just like in medical imaging or computer vision, it is necessary to design robust, fast, and widely applicable algorithms that would allow automatic registration of images generated by various imaging platforms at the same or different times and that would provide subpixel accuracy. One of the main issues that needs to be addressed when developing a registration algorithm is what type of information should be extracted from the images being registered, to be used in the search for the geometric transformation that best aligns them. The main objective of this paper is to evaluate several wavelet pyramids that may be used both for invariant feature extraction and for representing images at multiple spatial resolutions to accelerate registration. We find that the bandpass wavelets obtained from the steerable pyramid due to Simoncelli performs best in terms of accuracy and consistency, while the low-pass wavelets obtained from the same pyramid give the best results in terms of the radius of convergence. Based on these findings, we propose a modification of a gradient-based registration algorithm that has recently been developed for medical data. We test the modified algorithm on several sets of real and synthetic satellite imagery.

Evaluation Studies↗

Defining the cortical visual systems: "what", "where", and "how"

The visual system historically has been defined as consisting of at least two broad subsystems subserving object and spatial vision. These visual processing streams have been organized both structurally as two distinct pathways in the brain, and functionally for the types of tasks that they mediate. The classic definition by Ungerleider and Mishkin labeled a ventral "what" stream to process object information and a dorsal "where" stream to process spatial information. More recently, Goodale and Milner redefined the two visual systems with a focus on the different ways in which visual information is transformed for different goals. They relabeled the dorsal stream as a "how" system for transforming visual information using an egocentric frame of reference in preparation for direct action. This paper reviews recent research from psychophysics, neurophysiology, neuropsychology and neuroimaging to define the roles of the ventral and dorsal visual processing streams. We discuss a possible solution that allows for both "where" and "how" systems that are functionally and structurally organized within the posterior parietal lobe.

NASA Discipline Space Human Factors↗

New Directions for Chemical Engineering

Over the past century, the work of chemical engineers has helped transform societies and the lives of individuals, from the synthetic fertilizers that helped feed the world to the development of novel materials used in fuels, electronics, medical devices, and other products. Chemical engineers' ability to apply systems-level thinking from molecular to manufacturing scales uniquely positions them to address today’s most pressing problems, including climate change and the overuse of resources by a growing population. New Directions for Chemical Engineering details a vision to guide chemical engineering research, innovation, and education over the next few decades. This report calls for new investments in U.S. chemical engineering and the interdisciplinary, cross-sector collaborations necessary to advance the societal goals of transitioning to a low-carbon energy system, ensuring our production and use of food and water is sustainable, developing medical advances and engineering solutions to health equity, and manufacturing with less waste and pollution. The report also calls for changes in chemical engineering education to ensure the next generation of chemical engineers is more diverse and equipped with the skills necessary to address the challenges ahead.

National Academies of Sciences, Engineering, and M↗

Use of Multi-Resolution Wavelet Feature Pyramids for Automatic Registration of Multi-Sensor Imagery

The problem of image registration, or alignment of two or more images representing the same scene or object, has to be addressed in various disciplines that employ digital imaging. In the area of remote sensing, just like in medical imaging or computer vision, it is necessary to design robust, fast and widely applicable algorithms that would allow automatic registration of images generated by various imaging platforms at the same or different times, and that would provide sub-pixel accuracy. One of the main issues that needs to be addressed when developing a registration algorithm is what type of information should be extracted from the images being registered, to be used in the search for the geometric transformation that best aligns them. The main objective of this paper is to evaluate several wavelet pyramids that may be used both for invariant feature extraction and for representing images at multiple spatial resolutions to accelerate registration. We find that the band-pass wavelets obtained from the Steerable Pyramid due to Simoncelli perform better than two types of low-pass pyramids when the images being registered have relatively small amount of nonlinear radiometric variations between them. Based on these findings, we propose a modification of a gradient-based registration algorithm that has recently been developed for medical data. We test the modified algorithm on several sets of real and synthetic satellite imagery.

Zavorin, Ilya↗

Who's Got the Bridge? - Towards Safe, Robust Autonomous Operations at NASA Langley's Autonomy Incubator

NASA aeronautics research has made decades of contributions to aviation. Both aircraft and air traffic management (ATM) systems in use today contain NASA-developed and NASA sponsored technologies that improve safety and efficiency. Recent innovations in robotics and autonomy for automobiles and unmanned systems point to a future with increased personal mobility and access to transportation, including aviation. Automation and autonomous operations will transform the way we move people and goods. Achieving this mobility will require safe, robust, reliable operations for both the vehicle and the airspace and challenges to this inevitable future are being addressed now in government labs, universities, and industry. These challenges are the focus of NASA Langley Research Center's Autonomy Incubator whose R&D portfolio includes mission planning, trajectory and path planning, object detection and avoidance, object classification, sensor fusion, controls, machine learning, computer vision, human-machine teaming, geo-containment, open architecture design and development, as well as the test and evaluation environment that will be critical to prove system reliability and support certification. Safe autonomous operations will be enabled via onboard sensing and perception systems in both data-rich and data-deprived environments. Applied autonomy will enable safety, efficiency and unprecedented mobility as people and goods take to the skies tomorrow just as we do on the road today.

Allen, B. Danette↗

A vision system for a Mars rover

A Mars rover must be able to sense its local environment with sufficient resolution and accuracy to avoid local obstacles and hazards while moving a significant distance each day. Power efficiency and reliability are extremely important considerations, making stereo correlation an attractive method of range sensing compared to laser scanning, if the computational load and correspondence errors can be handled. Techniques for treatment of these problems, including the use of more than two cameras to reduce correspondence errors and possibly to limit the computational burden of stereo processing, have been tested at JPL. Once a reliable range map is obtained, it must be transformed to a plan view and compared to a stored terrain database, in order to refine the estimated position of the rover and to improve the database. The slope and roughness of each terrain region are computed, which form the basis for a traversability map allowing local path planning. Ongoing research and field testing of such a system is described.

Wilcox, Brian H.↗

Weapon Material Program: FY24 Annual Report

The Weapon Material Program (WMP) framework was started in July of Fiscal Year (FY) 2022, along with other Mission Support organizations. The first year was about building the organization with the right people, identifying needs, and developing a vision for expectations. It was also a year to move forward with new equipment and technology for weapon material processes. With numerous improvements, WMP is building world-class systems for production, warehousing, and analytical testing. FY 2023 was a year of development. The WMP was a new organization defining its identity, its focus, and its mission. At the end of FY 2023, WMP evaluated its current position and its immediate needs along with its future goals. The entire organization set out on this FY 2024 journey to success. The lack of funding, staffing, and visibility dramatically impacted weapon material operations, testing, and qualifications. This neglect required numerous areas to be addressed and improvement plans to be developed and implemented. WMP processes, procedures, and overall business operations were dramatically in need of updating, revision, and formal documentation. This included all areas: technology, equipment, facility modifications/upgrades, and testing improvements. To ensure organizational improvement and forward momentum in FY 2024, WMP executed a multi-faceted strategic vision. Action items included the following: • Filling open positions with capable personnel who would contribute to a more robust organization • Establishing more formal operations and problem-solving techniques • Improving departmental procedures to align with expanding scope • Improving training • Establishing collaboration meetings with other departments/organizations • Focusing on the importance of identifying and mitigating concerns with At-Risk Materials (@RM) • Leading modernization eff orts for Blending and Packaging (B&P) We are on the verge of a transformational improvement in all of our processes in support of the mission. The following area achievements outlined in this report reflect the hard work toward achieving the goal of “being the material Subject Matter Experts in the nuclear enterprise”.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Capsule network-based semantic segmentation model for thermal anomaly identification on building envelopes

Thermography technology is widely used to inspect thermal anomalies in building façade systems. Computer vision-based techniques provide opportunities to autonomously detect such heat anomalies to significantly improve the efficiency of decision-making for building envelope retrofitting and maintenance. Here, in this work, we propose a novel Capsule Network-based deep learning model – CapsLab – that detects and identifies thermal anomalies by semantic segmentation. CapsLab is built based on our proposed prediction-tuning capsule (PT-Capsule) layer. Different from a traditional capsule layer, which consists of part-whole transformation and capsule-routing process, the proposed layer is composed of a prediction and tuning process, which helps decreasing the number of model parameters significantly. While the applicability of traditional Capsule Networks (CapsNets) has been limited to simpler tasks and smaller datasets due to their scalability issue, we can leverage the lightweight of the proposed PT-Capsule layer, and apply it to the semantic segmentation task. In this work, we also employ our previously presented performance metric, referred to as the Anomaly Identification Metric (AIM) (Kakillioglua et al. 2021), to evaluate the segmentation outputs. Traditional performance metrics do not accurately reflect the true performance of the segmentation models in thermal anomaly identification due to the high subjectivity in the annotation process and higher overlap ratio sensitivity of the standard metrics. AIM, on the other hand, is robust to these drawbacks. Experimental results show, both qualitatively and quantitatively, that our proposed segmentation method can effectively segment the thermal anomalies. Specifically, our model provides 9.38% and 13.53% improvements over the baseline model – DeepLabV3+ – based on traditional mIoU score and the AIM score, respectively, while requiring less model parameters and less computation at the same time. In addition, the scores that the AIM metric generates better align with the scores provided by building performance experts.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗