Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Visioning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 703 records · Page 39
All sourcesNASA NTRSDOE OSTI
NASA NTRS2022

Camera Simulation for the Perseverance Rover’s Lander Vision System

On February 18, 2021, the Perseverance Rover safely landed on Mars at Jezero Crater. Part of the successful landing was due to the Lander Vision System (LVS), which takes descent images from the LVS Camera (LCAM) and IMU measurements and estimates the lander position relative to a map of the Jezero landing site. The LVS Simulation LCAM (LVSS LCAM) model is an image rendering program developed to test the LVS in a variety of scenarios to ensure performance amid uncertainty. The LVSS LCAM model includes a pointing misalignment model, an exposure timing model, shadowing, a terrain reflectance model, atmospheric attenuation from dust, and sensor effects. This model was used for performance analysis, verification, and validation of the LVS algorithms in a Mars-like simulation prior to landing. This paper describes the LVSS LCAM rendering algorithm and compares flight images from LVS operation during the Perseverance landing with their rendered counterparts.

Zheng, Jason↗
NASA NTRS2022

Mars 2020 Lander Vision System Flight Performance 1

The Mars 2020 Entry Descent and Landing (EDL) system delivered the Perseverance rover to the surface of Mars on February 18th, 2021. A large fraction of the Jezero Crater landing site was covered with landing hazards including cliffs, inescapable dune fields and rocks. These hazards were identified or inferred using orbital imagery before launch so that they could be avoided using Terrain Relative Navigation (TRN) which was composed of two parts: the Lander Vision System (LVS) and Safe Target Selection (STS). During EDL, the LVS successfully estimated map relative position by fusing landmarks matched between descent imagery and a map of the landing site with Inertial Measurement Unit (IMU) data. This position estimate was used by STS to identify the safest target for landing that was also reachable given fuel and other constraints. The EDL system then used the powered descent phase to retarget to this location and land safely. The overall error between the targeted location and actual landing location was 5m which was an order of magnitude less than the 60m touchdown error requirement. This paper will describe the final tests of the LVS before launch, the checkout of the LVS during operations and the LVS performance during EDL.

Zheng, Jason↗
NASA NTRSDate not supplied

ML/AI Applications to the Atmosphere Science Data and Simulations (Demonstration and Vision)

Artificial Intelligence has been recognized as one of the most powerful tools for scientific research. It has a wide range of applications in atmospheric science and plays a significant role in advancing our understanding of the Earth-Atmosphere system, as well as improving our ability to monitor, predict, and respond to atmospheric observations and simulations. Different AI architectures can help us handle various problems according to our research goals. In this presentation, we will introduce several use cases and models for applying computer vision to active remote sensing data from CALIOP, analyzing cloud dynamical relationships for climate model parameterizations, and building surrogate models for lidar simulations. The presentation will highlight the advantages that AI offers compared to traditional analytical methods. By assembling pieces of the multidimensional atmospheric science puzzle, we can integrate our knowledge from observations to simulations.

Shan Zeng↗
NASA NTRS2024

Effective Visualization of Radar Data for Users Impacted By Color Vision Deficiency

Color Vision Deficiency (CVD) is a decreased ability to discern between particular colors. Eight percent of genetic males and half a percent of genetic females have some form of CVD, with many in the radar community falling into this group. When presenting data on a two-dimensional plane, it is common to use colors to represent values via a colormap. Colormap choice in the radar community is influenced by the ability to highlight scientifically interesting features in data, institutional choices, and domain dominance of legacy colormaps. The problem with these current colormaps is that many do not project well for those with CVD (i.e., green next to red). In working with the CVD community to address this problem, multiple colormaps for moments such as equivalent reflectivity factor and Doppler velocity were created for users with forms of CVD such as deuteranomaly, protanomaly, protanopia, and deuteranopia using Python tools such as colorspacious and viscm. We show how these colormaps can improve interpretability for four cases: a mesoscale convective system, a pyrocumulonimbus storm, a wintertime mid-latitude cyclone, and widespread storms with a large bird migration. These new radar equivalent reflectivity factor, Doppler velocity, and polarization colormaps are designed to highlight rain, frozen precipitation, non-meteorological targets, velocity-based items, be perceptually uniform, and visually friendly for those with CVD.

Zachary Sherman↗
NASA NTRS2024

A Collaborative Vision for Deep Space Human Exploration

Artemis will be the first cooperative campaign among nations for crewed planetary exploration. This paradigm shift reflects a collaborative vision for deep space human exploration that embraces the concept that, “If you want to go fast, go alone; If you want to go far, go together.” This paper elaborates on the way the National Aeronautics and Space Administration(NASA)is striving to make its Moon to Mars effort a collaborative endeavor for long-term human exploration of deep space and for the international space community. The agency is making great strides in unifying its workforce, mission directorates, technical authorities, and centers behind shared goals and processes. Enhanced communication and a reinvigoration of systems engineering ideals have been a hallmark of that effort, guided by the Moon to Mars Architecture, which defines what NASA aims to accomplish and how it will execute those goals. This interdisciplinary approach ensures the Artemis campaign supports NASA’s engineering, science, and technology development goals. As of September 2024, over 43 nations have signed the Artemis Accords, which establish a common set of principles for peaceful cooperation in civil space exploration. Building off these efforts, NASA has been engaging in separate and distinct discussions about partner contributions to Artemis. International partnerships for the Artemis campaign include long-standing partners as well as agencies with which NASA has only more recently begun collaborating. The Gateway Program is an early example of both –benefitting from partnerships with Canada, Japan, and Europe forged though decades of cooperation on the International Space Station, as well as a new partnership with the United Arab Emirates. The agency’s Moon to Mars Objectives and Architecture unify the NASA workforce with common goals and shared systems-engineering processes. The architecture offers pathways for broader participation from industry, academia, and international partners. The agency evolves this collaborative approach through its annual Architecture Concept Review cycle, seeking input from the NASA workforce, and through workshops, from industry, academia, and the international space community to refine NASA’s roadmap for exploration. Exploring together, NASA and its partners are setting humanity on the path to long-term presence at the Moon and our eventual journey to Mars

Objective↗
NASA NTRSDate not supplied

Computer Vision Dataset for Aircraft Taxi Operations

The development and democratization of computer vision algorithms are contingent on the availability of high-quality datasets. In this paper, we introduce a database of forward-facing videos from taxiing aircraft as well as the time-correlated flight data at approximately 1 to10 Hz containing aircraft state data and environmental conditions. The video data is sourced from the National Aeronautics and Space Administration Airborne Science Program archive and includes over 33 hours of 4k, 1080p, and 720p video from twenty-two airports around the world. This paper describes the method of the database construction and a brief analysis of its contents.

Ryan Horn↗
NASA NTRSDate not supplied

Distributed Vision Sensing of Small Uncrewed Aircraft Systems in Urban Traffic Corridors

The NASA Advanced Air Mobility mission will enable widespread low altitude passenger travel, cargo delivery, and a variety of public services through the development of Uncrewed Aerial Systems (UAS) operations. Ensuring safe, autonomous operations in densely populated environments requires careful consideration towards hazards including other aircraft, infrastructure, and evolving weather. Small Uncrewed Aerial Systems (SUAS) present a unique hazard to UAS operations as they share airspace and may be readily operated in a non-cooperative fashion. This work investigates distributed sensing of SUAS traversing an air traffic corridor in an urban setting. This work develops a distributed vision detect and track strategy at NASA Langley Research Center. Three nodes, each with at least one global shutter camera, are distributed around a traffic corridor to surveil flight operations for two SUAS performing low altitude flight operations. Each node is equipped with a GPS and cellular modem to enable timestamping and remote control of acquisition. Node one faces a traffic roundabout with buildings in the background and achieves 99% surveillance coverage for two SUAS against building and tree backgrounds at ranges 50 to 130m. The second node points down Langley Boulevard with trees and buildings in the background and achieves 99% coverage at separation distances between 70 and 180m. The analysis for the second node is limited to ranges below 180m due to low contrast against dark, tree backgrounds. Finally, the third node points down Langley Boulevard from another perspective and achieves 99% coverage at ranges 60m to 200m against mostly building with a few sections of trees in the background.

Chester V Dolph↗
DOE OSTI2022

Development of an end state vision to implement digital monitoring in nuclear plants

Transitioning from an onsite Maintenance & Diagnostics Center to cloud-based services offers many new opportunities with computing power and storage, but also new challenges in terms of networking and security. This report will cover everything required for that transition including data processing and uploading to cloud services, feature selection, model creation, and result visualization for decision making. Although there are several other cloud-based services (e.g. Amazon Web Services and Google Cloud), this report explores Microsoft Azure to simplify nomenclature and maintain a consistent focus. Many of the services offered by Microsoft Azure are also available in the other cloud-based services, and their differences have been recorded in other literature. The Azure services most important to a nuclear power plant including networking & security, storage & databases, and Artificial Intelligence (AI) are reviewed here. Networking covers all aspects related to communication to Azure resources including security, privacy, and redundancy. Storage & databases includes data storage, upgrading, patching, backups, and monitoring. The AI services allows the user access to the machine learning (ML) techniques developed with Azure including automated ML, anomaly detection, computer vision, and natural language processing. This report summaries the features, capabilities, and challenges when using cloud-based services in a user-friendly manner.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

VLM4Bio: A Benchmark Dataset to Evaluate Pretrained Vision-Language Models for Trait Discovery from Biological Images

Images are increasingly becoming the currency for documenting biodiversity on the planet, providing novel opportunities for accelerating scientific discoveries in the field of organismal biology, especially with the advent of large vision-language models (VLMs). We ask if pre-trained VLMs can aid scientists in answering a range of biologically relevant questions without any additional fine-tuning. In this paper, we evaluate the effectiveness of 12 state-of-the-art (SOTA) VLMs in the field of organismal biology using a novel dataset, VLM4Bio, consisting of 469K question8 answer pairs involving 30K images from three groups of organisms: fishes, birds, and butterflies, covering five biologically relevant tasks. We also explore the effects of applying prompting techniques and tests for reasoning hallucination on the performance of VLMs, shedding new light on the capabilities of current SOTA VLMs in answering biologically relevant questions using images

Maruf, M [Virginia Tech, Blacksburg]↗
DOE OSTI2025

SHF: Symmetrical Hierarchical Forest with Pretrained Vision Transformer Encoder for High-Resolution Medical Segmentation

This paper presents a novel approach to addressing the long-sequence problem in high-resolution medical images for Vision Transformers (ViTs). Using smaller patches as tokens can enhance ViT performance, but quadratically increases computation and memory requirements. Therefore, the common practice for applying ViTs to high-resolution images is either to: (a) employ complex sub-quadratic attention schemes or (b) use large to medium-sized patches and rely on additional mechanisms within the model to capture the spatial hierarchy of details. We propose Symmetrical Hierarchical Forest (SHF), a lightweight approach that adaptively patches the input image to increase token information density and encode hierarchical spatial structures into the input embedding. We then apply a reverse depatching scheme to the output embeddings of the transformer encoder, eliminating the need for convolution-based decoders. Unlike previous methods that modify attention mechanisms or use a complex hierarchy of interacting models, SHF can be retrofitted to any ViT model to allow it to learn the hierarchical structure of details in high-resolution images without requiring architectural changes. Experimental results demonstrate significant gains in computational efficiency and performance: on the PAIP WSI dataset, we achieved a 3∼32×speedup or a 2.95%∼7.03% increase in accuracy (measured by Dice score) at a 64K2 resolution with the same computational budget, compared to state-of-the-art production models. On the 3D medical datasets BTCV and KiTS, training was 6×faster, with accuracy gains of 6.93% and 5.9%, respectively, compared to models without SHF.

Zhang, Enzhi [Hokkaido University, Japan]↗
DOE OSTI2025

MoE-Inference-Bench: Performance Evaluation of Mixture of Expert Large Language and Vision Models

Mixture of Experts (MoE) models have enabled the scaling of Large Language Models (LLMs) and Vision Language Models (VLMs) by achieving massive parameter counts while maintaining computational efficiency. However, MoEs introduce several inference-time challenges, including load imbalance across experts and the additional routing computational overhead. To address these challenges and fully harness the benefits of MoE, a systematic evaluation of hardware acceleration techniques is essential. We present MoE-Inference-Bench, a comprehensive study to evaluate MoE performance across diverse scenarios. We analyze the impact of batch size, sequence length, and critical MoE hyperparameters such as FFN dimensions and number of experts on throughput. We evaluate several optimization techniques on Nvidia H100 GPUs, including pruning, Fused MoE operations, speculative decoding, quantization, and various parallelization strategies. Our evaluation includes MoEs from the Mixtral, DeepSeek, OLMoE and Qwen families. The results reveal performance differences across configurations and provide insights for the efficient deployment of MoEs.

Chitty-Venkata, Krishna Teja↗
DOE OSTI2025

Compressing Vision Transformers in Geospatial Transfer Learning with Manifold-Constrained Optimization

Deploying geospatial foundation models on resource-constrained edge devices demands compact architectures that maintain high downstream performance. However, their large parameter counts and the accuracy loss often induced by compression limit practical adoption.In this work, we leverage manifold-constrained optimization framework DLRT to compress large vision transformer–based geospatial foundation models during transfer learning. By enforcing structured low-dimensional parameterizations aligned with downstream objectives, this approach achieves strong compression while preserving task-specific accuracy. We show that the method outperforms of-the-shelf low-rank methods as LoRA. Experiments on diverse geospatial benchmarks confirm substantial parameter reduction with minimal accuracy loss, enabling high-performing, on-device geospatial models.

Snyder, Thomas [Yale University]↗
DOE OSTI2026

QCalEval: Benchmarking Vision-Language Models for Quantum Calibration Plot Understanding

Quantum computing calibration depends on interpreting experimental data, and calibration plots provide the most universal human-readable representation for this task, yet no systematic evaluation exists of how well vision-language models (VLMs) interpret them. We introduce QCalEval, the first VLM benchmark for quantum calibration plots: 243 samples across 87 scenario types from 22 experiment families, spanning superconducting qubits and neutral atoms, evaluated on six question types in both zero-shot and in-context learning settings. The best general-purpose zero-shot model reaches a mean score of 72.3, and many open-weight models degrade under multi-image in-context learning, whereas frontier closed models improve substantially. A supervised fine-tuning ablation at the 9-billion-parameter scale shows that SFT improves zero-shot performance but cannot close the multimodal in-context learning gap. As a reference case study, we release NVIDIA Ising Calibration 1, an open-weight model based on Qwen3.5-35B-A3B that reaches 74.7 zero-shot average score.

Cao, Shuxiang↗
DOE OSTI2026

CTRL-STEER: Closed-Loop Neuron Activation Control in Vision-Language-Action Models

Vision-Language-Action (VLA) models enable test-time behavioral steering via neuron-level interventions, but existing methods use fixed strengths and operate in open loop. This static modulation fails under evolving task dynamics, leading to overcorrection, oscillations, and reduced task success—especially for temporal attributes like speed. We propose CTRL-STEER, a control-theoretic framework that casts activation steering as closed-loop feedback with adaptive, time-varying interventions. Instead of assuming neurons encode temporal concepts, we steer along motion-aligned residual directions and regulate intervention magnitude via feedback. We instantiate this with both PID and reinforcement learning controllers that jointly optimize concept adherence and task success. Experiments on fine-tuned OpenVLA policies across four LIBERO suites show improved stability and a better steering–success trade-off over fixed-coefficient baselines, without retraining the base model.

Babu, Abhijith [Florida International University, ↗
DOE OSTI2024

GWAS supported by computer vision identifies large numbers of candidate regulators of in planta regeneration in Populus trichocarpa

Plant regeneration is an important dimension of plant propagation and a key step in the production of transgenic plants. However, regeneration capacity varies widely among genotypes and species, the molecular basis of which is largely unknown. Association mapping methods such as genome-wide association studies (GWAS) have long demonstrated abilities to help uncover the genetic basis of trait variation in plants; however, the performance of these methods depends on the accuracy and scale of phenotyping. To enable a large-scale GWAS of in planta callus and shoot regeneration in the model tree Populus, we developed a phenomics workflow involving semantic segmentation to quantify regenerating plant tissues over time. We found that the resulting statistics were of highly non-normal distributions, and thus employed transformations or permutations to avoid violating assumptions of linear models used in GWAS. We report over 200 statistically supported quantitative trait loci (QTLs), with genes encompassing or near to top QTLs including regulators of cell adhesion, stress signaling, and hormone signaling pathways, as well as other diverse functions. Our results encourage models of hormonal signaling during plant regeneration to consider keystone roles of stress-related signaling (e.g. involving jasmonates and salicylic acid), in addition to the auxin and cytokinin pathways commonly considered. The putative regulatory genes and biological processes we identified provide new insights into the biological complexity of plant regeneration, and may serve as new reagents for improving regeneration and transformation of recalcitrant genotypes and species.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Measuring success for a future vision: Defining impact in science gateways/virtual research environments

Scholars worldwide leverage science gateways/virtual research environments (VREs) for a wide variety of research and education endeavors spanning diverse scientific fields. Evaluating the value of a given science gateway/VRE to its constituent community is critical in obtaining the financial and human resources necessary to sustain operations and increase adoption in the user community. In this article, we feature a variety of exemplar science gateways/VREs and detail how they define impact in terms of, for example, their purpose, operation principles, and size of user base. Further, the exemplars recognize that their science gateways/VREs will continuously evolve with technological advancements and standards in cloud computing platforms, web service architectures, data management tools and cybersecurity. We also present a number of technology advances that could be incorporated in next-generation science gateways/VREs to enhance their scope and scale of their operations for greater success/impact. The exemplars are selected from owners of science gateways in the Science Gateways Community Institute (SGCI) clientele in the United States, and from the owners of VREs in the International Virtual Research Environment Interest Group (VRE-IG) of the Research Data Alliance. Thus, community-driven best practices and technology advances are compiled from diverse expert groups with an international perspective to envisage futuristic science gateway/VRE innovations.

97 MATHEMATICS AND COMPUTING↗