Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Image representation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24
All sourcesNASA NTRSDOE OSTI
NASA NTRS2010

Modeling Regular Replacement for String Constraint Solving

Bugs in user input sanitation of software systems often lead to vulnerabilities. Among them many are caused by improper use of regular replacement. This paper presents a precise modeling of various semantics of regular substitution, such as the declarative, finite, greedy, and reluctant, using finite state transducers (FST). By projecting an FST to its input/output tapes, we are able to solve atomic string constraints, which can be applied to both the forward and backward image computation in model checking and symbolic execution of text processing programs. We report several interesting discoveries, e.g., certain fragments of the general problem can be handled using less expressive deterministic FST. A compact representation of FST is implemented in SUSHI, a string constraint solver. It is applied to detecting vulnerabilities in web applications

Fu, Xiang↗
NASA NTRSDate not supplied

Concept-based Analysis of Neural Networks via Vision-Language Models

The analysis of vision-based deep neural networks (DNNs) is highly desirable but it is very challenging due to the difficulty of expressing formal specifications for vision tasks and the lack of efficient verification procedures. In this paper, we propose to leverage emerging multimodal, vision-language, foundation models (VLMs) as a lens through which we can reason about vision models. VLMs have been trained on a large body of images accompanied by their textual description, and are thus implicitly aware of high-level, human-understandable concepts describing the images. We describe a logical specification language Con spec designed to facilitate writing specifications in terms of these concepts. To define and formally check Con spec specifications, we build a map between the internal representations of a given vision model and a VLM, leading to an efficient verification procedure of natural-language properties for vision models. We demonstrate our techniques on a ResNet-based classifier trained on the RIVAL-10 dataset using CLIP as the multimodal model.

Large Vision Language Models↗
DOE OSTI2023

Data augmentation and multimodal learning for predicting drug response in patient-derived xenografts from gene expressions and histology images

Patient-derived xenografts (PDXs) are an appealing platform for preclinical drug studies. A primary challenge in modeling drug response prediction (DRP) with PDXs and neural networks (NNs) is the limited number of drug response samples. We investigate multimodal neural network (MM-Net) and data augmentation for DRP in PDXs. The MM-Net learns to predict response using drug descriptors, gene expressions (GE), and histology whole-slide images (WSIs). We explore whether combining WSIs with GE improves predictions as compared with models that use GE alone. We propose two data augmentation methods which allow us training multimodal and unimodal NNs without changing architectures with a single larger dataset: 1) combine single-drug and drug-pair treatments by homogenizing drug representations, and 2) augment drug-pairs which doubles the sample size of all drug-pair samples. Unimodal NNs which use GE are compared to assess the contribution of data augmentation. The NN that uses the original and the augmented drug-pair treatments as well as single-drug treatments outperforms NNs that ignore either the augmented drug-pairs or the single-drug treatments. In assessing the multimodal learning based on the MCC metric, MM-Net outperforms all the baselines. Our results show that data augmentation and integration of histology images with GE can improve prediction performance of drug response in PDXs.

60 APPLIED LIFE SCIENCES↗
NASA NTRS2014

Near-Real Time Satellite-Retrieved Cloud and Surface Properties for Weather and Aviation Safety Applications

Cloud properties determined from satellite imager radiances provide a valuable source of information for nowcasting and weather forecasting. In recent years, it has been shown that assimilation of cloud top temperature, optical depth, and total water path can increase the accuracies of weather analyses and forecasts. Aircraft icing conditions can be accurately diagnosed in near-­‐real time (NRT) retrievals of cloud effective particle size, phase, and water path, providing valuable data for pilots. NRT retrievals of surface skin temperature can also be assimilated in numerical weather prediction models to provide more accurate representations of solar heating and longwave cooling at the surface, where convective initiation. These and other applications are being exploited more frequently as the value of NRT cloud data become recognized. At NASA Langley, cloud properties and surface skin temperature are being retrieved in near-­‐real time globally from both geostationary (GEO) and low-­‐earth orbiting (LEO) satellite imagers for weather model assimilation and nowcasting for hazards such as aircraft icing. Cloud data from GEO satellites over North America are disseminated through NCEP, while those data and global LEO and GEO retrievals are disseminated from a Langley website. This paper presents an overview of the various available datasets, provides examples of their application, and discusses the use of the various datasets downstream. Future challenges and areas of improvement are also presented.

Minnis, Patrick↗
NASA NTRS1977

Snow mapping and land use studies in Switzerland

The author has identified the following significant results. A system was developed for operational snow and land use mapping, based on a supervised classification method using various classification algorithms and representation of the results in maplike form on color film with a photomation system. Land use mapping, under European conditions, was achieved with a stepwise linear discriminant analysis by using additional ratio variables. On fall images, signatures of built-up areas were often not separable from wetlands. Two different methods were tested to correlate the size of settlements and the population with an accuracy for the densely populated Swiss Plateau between +2 or -12%.

Haefner, H.↗
NASA NTRS1999

Improved Algorithm to Estimate Convective and Stratiform Rain from TRMM Microwave Radiometer Data

The rain rates associated with Mesoscale Convective Systems (MCSS) deduced from the Tropical Rainfall Measuring Mission (TRMM) Precipitation Radar (PR) follow the well known log-normal type frequency distribution characteristic of conventional radars and rain gauges. On the other hand, the current microwave radiometer rain retrieval algorithms cannot give good representations of convective and stratiform precipitation. Thus they capture poorly this log-normal character. In particular, this is noticed clearly over land areas. The objective of the present study is to improve TRMM Microwave Imager (TMI) retrievals of convective and stratiform rain to a level comparable to the PR. For this purpose, we have developed a TMI rain retrieval algorithm that depends primarily on the 85 GHz brightness temperatures (Tbs) and their spatial distribution. We emphasize the 85 GHz channel because of its fine footprint size (about 5.5 km) and strong extinction property. The other spectral channels, because they are poorer in these respects, are given less importance. The brightness temperatures and spatial distribution information in the 85 GHz allows us to detect thunderstorms, or cumulonimbus clouds (Cbs), that are in different stages of development, as well as stratiform rain outside of these Cbs. Utilizing our retrieval technique, we have produced maps of convective and stratiform rain for several MCS events over land and ocean that agree well with those given by the PR. On average, the convective and stratiform rain rates given by our technique have an accuracy of about 15 % with respect to those given by the PR.

Prabhakara, C.↗
NASA NTRS1996

The Orbit of Pheobe from Earthbased and Voyager Observations

This article presents the results of a fit of a numerically integrated orbit for the Saturnian satellite Pheobe to Earthbased astrometric observations (from 1904 to 1996) and imaging data acquired by the Voyager 2 spacecraft during its encounter with Saturn. The primary results are the epoch state vector used in the integration and a set of mean elements which provide a geometrical representation of the orbit. We also assess the quality of the fit and the accuracy of the orbit.

Ephemirides planet ephemirides satellite ephemirid↗
NASA NTRSDate not supplied

Computing the effective elasticity of anisotropic porous media from X-ray computed micro-tomography images

Development and optimization of composite materials designed for thermal protection of NASA’s spacecraft requires the understanding of their physical response to high-enthalpy environments. To predict their macro-scale properties and behavior, high-fidelity 3D simulations are performed at the micro-scale on realistic representations of these composites. The digital micro-structures are generated either synthetically or through X-ray micro-tomography reconstructions. One of the main challenges in the prediction of heatshield material structural response is the computation of the effective elasticity of the fibrous composite, as well as the understanding of the deformation and stresses generated at the micro-scale. These are driven by the fiber layout within the micro-structure and the distribution of the infused matrix. In this effort, the micro-mechanical linear elastic behavior of fibrous ablators is modeled through the use of a numerical method based on the Multi-Point Stress Approximation (MPSA) finite volume scheme. The MPSA, a generalization of the more commonly used Multi-Point Flux Approximation (MPFA), was discussed in a previous presentation at the 15th USCCN and in reference. To predict the behavior of fibrous and woven architectures, algorithms that compute the local fiber orientation are used. The implementation of the MPSA was verified using analytical solutions, engineering test cases and compared against legacy Finite Element Analysis (FEA) software. The stress analysis models were then applied to real geometries used by NASA in thermal protection systems such as fibrous preforms and woven materials and the results were compared to experimental data.

Federico Semeraro↗
NASA NTRS2022

Flood Monitoring Using Enhanced Resolution Passive Microwave Data: A Test Case Over Bangladesh

Monitoring floods is a major issue in water resources management and risk mitigation, especially in the Global South. Optical and radar observations, even providing a fine spatial resolution, are still limited by cloud cover interaction or insufficient temporal resolution. On the other hand, passive microwave (PMW) sensors collect information on a daily frequency with minor cloud cover interaction, but they have been historically limited in terms of spatial resolution. Here, we evaluate the capability of an enhanced spatial resolution PMW dataset (3.125 km) in monitoring spatio-temporal evolution of flood events, focusing on a major flood event that occurred in October 2005 in Bangladesh. We apply an algorithm aimed to remove the seasonal variability of surface temperature from the PMW timeseries, exploiting the difference in emissivity between dry and water-covered pixels. We assess the capability of the algorithm in capturing flood evolution and extension through the comparison with quantities obtained from optical data collected by the Moderate Resolution Imaging Spectroradiometer (MODIS) and water level measurements. We also compare the enhanced product with the historical coarser resolution dataset by means of a variogram-based analysis to evaluate the improvements in terms of spatial representation. Finally, we evaluate the possibility to extract the water fraction within a single pixel by using an Advanced Microwave Scanning Radiometer—Earth Observing System (AMSR-E) emissivity dataset and compare the estimates with MODIS-derived water fractions. Our results show that the enhanced PMW product outperforms the coarser one when compared to flood mapped from optical data based on information content, indicating that it is possible to integrate such a product into the mapping of floods at a global scale on a daily basis.

flood detection↗
DOE OSTI2020

Evaluation of Techniques for Vegetation Removal from UAV-Based Photogrammetric DSM near WIPP Land Withdrawal Act Boundary, NM

The Waste Isolation Pilot Plant is located within New Mexico's karst landscape and is the United States' only deep geologic radioactive waste repository which isolates transuranic waste from defense activities underground in a bedded salt formation. Characteristic karst topography in this region is made up of material (such as limestone) that can be dissolved by water over a long period of time, and includes features such as springs, surface streams, sinking streams, caves, and sinkholes. Karst features can potentially impact the integrity of the waste repository in the future and compromise WIPP's performance, particularly when coupled with incompatible land-use activities within the Land Withdrawal Act (LWA) boundary as well as increased water withdrawals outside the LWA. In order to better capture the ground surface topography to support hydrology modeling efforts, the U.S. DOE has enlisted FIU to develop a high-resolution digital elevation model (DEM). In a previous field study, a high-resolution digital surface model (DSM) of an adjacent representative site, Basin 6, was developed by photogrammetry using aerial images captured with unmanned aerial vehicles. The DSM generated was based on point clouds representing the entire landscapes, including points of terrain, vegetation and infrastructure. These above-ground features need to be removed to create a DSM/DEM with good representation of the bare ground. In recent years, many techniques including both pre- and post-processing of DSMs, were developed for vegetation removal with varied success. This research focuses on applying and comparing various vegetation removal methods using point clouds generated of the Basin 6 pilot study area. The 3D RGB-based point cloud classification using Python scripts was a modification of the procedure by Themistocleous (2019). The method using Liblas classifies the point cloud three-dimensionally, whereas the method by Themistocleous (2019) is two-dimensional. The preliminary result of the method using Liblas, however, still had a large amount of vegetation remaining compared to the Pix4D Machine Learning method. The vegetation layer remains highly visible in the DEMs processed using the vegetation indices as opposed to the Pix4D Machine Learning method, which produces a DEM where mostly bare ground is seen. Further modifications to the input parameters are required to improve the vegetation vs. bare ground classification using this approach. The next step will be to improve the method using Liblas and to also test other vegetation removal methods found in the literature review to compare them against the two described here.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
NASA NTRS2002

Bioinspired engineering of exploration systems for NASA and DoD

A new approach called bioinspired engineering of exploration systems (BEES) and its value for solving pressing NASA and DoD needs are described. Insects (for example honeybees and dragonflies) cope remarkably well with their world, despite possessing a brain containing less than 0.01% as many neurons as the human brain. Although most insects have immobile eyes with fixed focus optics and lack stereo vision, they use a number of ingenious, computationally simple strategies for perceiving their world in three dimensions and navigating successfully within it. We are distilling selected insect-inspired strategies to obtain novel solutions for navigation, hazard avoidance, altitude hold, stable flight, terrain following, and gentle deployment of payload. Such functionality provides potential solutions for future autonomous robotic space and planetary explorers. A BEES approach to developing lightweight low-power autonomous flight systems should be useful for flight control of such biomorphic flyers for both NASA and DoD needs. Recent biological studies of mammalian retinas confirm that representations of multiple features of the visual world are systematically parsed and processed in parallel. Features are mapped to a stack of cellular strata within the retina. Each of these representations can be efficiently modeled in semiconductor cellular nonlinear network (CNN) chips. We describe recent breakthroughs in exploring the feasibility of the unique blending of insect strategies of navigation with mammalian visual search, pattern recognition, and image understanding into hybrid biomorphic flyers for future planetary and terrestrial applications. We describe a few future mission scenarios for Mars exploration, uniquely enabled by these newly developed biomorphic flyers.

Space Flight↗
NASA NTRS2014

Use of MODIS Cloud Top Pressure to Improve Assimilation Yields of AIRS Radiances in GSI

Radiances from hyperspectral sounders such as the Atmospheric Infrared Sounder (AIRS) are routinely assimilated both globally and regionally in operational numerical weather prediction (NWP) systems using the Gridpoint Statistical Interpolation (GSI) data assimilation system. However, only thinned, cloud-free radiances from a 281-channel subset are used, so the overall percentage of these observations that are assimilated is somewhere on the order of 5%. Cloud checks are performed within GSI to determine which channels peak above cloud top; inaccuracies may lead to less assimilated radiances or introduction of biases from cloud-contaminated radiances.Relatively large footprint from AIRS may not optimally represent small-scale cloud features that might be better resolved by higher-resolution imagers like the Moderate Resolution Imaging Spectroradiometer (MODIS). Objective of this project is to "swap" the MODIS-derived cloud top pressure (CTP) for that designated by the AIRS-only quality control within GSI to test the hypothesis that better representation of cloud features will result in higher assimilated radiance yields and improved forecasts.

Zavodsky, Bradley↗
NASA NTRS1991

A graph theoretic approach to scene matching

The ability to match two scenes is a fundamental requirement in a variety of computer vision tasks. A graph theoretic approach to inexact scene matching is presented which is useful in dealing with problems due to imperfect image segmentation. A scene is described by a set of graphs, with nodes representing objects and arcs representing relationships between objects. Each node has a set of values representing the relations between pairs of objects, such as angle, adjacency, or distance. With this method of scene representation, the task in scene matching is to match two sets of graphs. Because of segmentation errors, variations in camera angle, illumination, and other conditions, an exact match between the sets of observed and stored graphs is usually not possible. In the developed approach, the problem is represented as an association graph, in which each node represents a possible mapping of an observed region to a stored object, and each arc represents the compatibility of two mappings. Nodes and arcs have weights indicating the merit or a region-object mapping and the degree of compatibility between two mappings. A match between the two graphs corresponds to a clique, or fully connected subgraph, in the association graph. The task is to find the clique that represents the best match. Fuzzy relaxation is used to update the node weights using the contextual information contained in the arcs and neighboring nodes. This simplifies the evaluation of cliques. A method of handling oversegmentation and undersegmentation problems is also presented. The approach is tested with a set of realistic images which exhibit many types of sementation errors.

Ranganath, Heggere S.↗
DOE OSTI2021

Disentangling Rotational Dynamics and Ordering Transitions in a System of Self-Organizing Protein Nanorods via Rotationally Invariant Latent Representations

The dynamics of complex ordering systems with active rotational degrees of freedom exemplified by protein self-assembly is explored using a machine learning workflow that combines deep learning-based semantic segmentation and rotationally invariant variational autoencoder-based analysis of orientation and shape evolution. The latter allows for disentanglement of the particle orientation from other degrees of freedom and compensates for lateral shifts. The disentangled representations in the latent space encode the rich spectrum of local transitions that can now be visualized and explored via continuous variables. The time dependence of ensemble averages allows insight into the time dynamics of the system and, in particular, illustrates the presence of the potential ordering transition. Finally, analysis of the latent variables along the single-particle trajectory allows tracing these parameters on a single-particle level. The proposed approach is expected to be universally applicable for the description of the imaging data in optical, scanning probe, and electron microscopy seeking to understand the dynamics of complex systems where rotations are a significant part of the process.

representation learning↗
DOE OSTI2026

Using Flory–Huggins-informed human-in-the-loop Bayesian optimization to map the phase diagram of polymer blends

Mapping the phase diagram of polymer blends is an essential step in controlling the structure–property relationship of polymer-based materials. However, traditional grid-based approaches are inefficient and rely on subjective judgements for terminating the experimental campaign. Artificial intelligence-guided experimentation offers a compelling alternative, especially when data-driven decision-making is interfaced with established polymer thermodynamics to improve efficiency and interpretability. Here, we introduce a physics-informed Bayesian optimization approach to guide the mapping of the phase diagram of a model blend containing poly(methyl methacrylate) and poly(styrene-ran-acrylonitrile). Physical information is derived from a Flory–Huggins representation of the spinodal curve, which is integrated into the Bayesian optimization process as a structured prior mean that acts as a soft constraint. Implemented as a human-in-the-loop workflow, the approach leverages optical imaging of film cloudiness with iterative Gaussian process surrogate modeling and a parameter selection decision policy to identify the composition-temperature conditions for sequential iterations. Convergence of kernel and Flory–Huggins-based hyperparameters provided a stopping criterion, ensuring an objective and interpretable termination of the experimental campaign. The framework recovered the known lower critical solution temperature (∼160 °C), while increasing material efficiency through targeted sampling. This work establishes a proof-of-concept for the application of Bayesian optimization workflows to study polymer blend miscibility.

36 MATERIALS SCIENCE↗
NASA NTRS2015

Subharmonic Oscillations and Chaos in Dynamic Atomic Force Microscopy

The increasing use of dynamic atomic force microscopy (d-AFM) for nanoscale materials characterization calls for a deeper understanding of the cantilever dynamics influencing scan stability, predictability, and image quality. Model development is critical to such understanding. Renormalization of the equations governing d- AFM provides a simple interpretation of cantilever dynamics as a single spring and mass system with frequency dependent cantilever stiffness and damping parameters. The renormalized model is sufficiently robust to predict the experimentally observed splitting of the free-space cantilever resonance into multiple resonances upon cantilever-sample contact. Central to the model is the representation of the cantilever sample interaction force as a polynomial expansion with coefficients F(sub ij) (i,j = 0, 1, 2) that account for the effective interaction stiffness parameter, the cantilever-to-sample energy transfer, and the amplitude of cantilever oscillation. Application of the Melnikov method to the model equation is shown to predict a homoclinic bifurcation of the Smale horseshoe type leading to a cascade of period doublings with increasing drive displacement amplitude culminating in chaos and loss of image quality. The threshold value of the drive displacement amplitude necessary to initiate subharmonic generation depends on the acoustic drive frequency, the effective damping coefficient, and the nonlinearity of the cantilever-sample interaction force. For parameter values leading to displacement amplitudes below threshold for homoclinic bifurcation other bifurcation scenarios can occur, some of which lead to chaos.

Cantrell, John H.↗
NASA NTRS2017

Electric Field Conjugation in the Presence of Model Uncertainty

The Wide-Field Infrared Survey Telescope (WFIRST) is a 2.4-meter-diameter space telescope NASA program. The payload will include a coronagraph instrument (CGI). The CGI designs under development use deformable mirrors (DM) to create a point spread function (PSF) with a dark region around the obscured star object. Electric field conjugation (EFC) is an iterative nonlinear optimization procedure that uses measurements of the electric field at the image to determine the DM displacements to modify the PSF to create the region of high contrast in the image. EFC requires a numerical model of the coronagraph to calculate the Jacobian of the system, which is used, along with regularization, to solve for the DM displacements for each iteration of the nonlinear optimization. Ideally, the coronagraph is aligned and calibrated, and the calibration data are used in the numerical model for calculating the Jacobian. However, calibration and alignment measurements always contain uncertainty resulting in calibration error. Therefore, the Jacobian calculated from the numerical model is not an exact representation of the physical coronagraph, and the resulting DM solution for an EFC iteration does not have the exact impact on the electric field of the coronagraph as predicted by the EFC. The result is slow convergence, and, as will be shown, drives the use of more restrictive regularization. Using Monte Carlo trials, we investigate the effect of calibration error on EFC convergence and regulatization. Comparison to results from the High Contrast Imaging Testbed Hybrid Lyot Coronagraph are also presented.

Marx, David↗
NASA NTRS2005

Validation of AIRS/AMSU Cloud Retrievals Using MODIS Cloud Analyses

The AIRS/AMSU (flying on the EOS-AQUA satellite) sounding retrieval methodology allows for the retrieval of key atmospheric/surface parameters under partially cloudy conditions (Susskind et al.). In addition, cloud parameters are also derived from the AIRS/AMSU observations. Within each AIRS footprint, cloud parameters at up to 2 cloud layers are determined with differing cloud top pressures and effective (product of infrared emissivity at 11 microns and physical cloud fraction) cloud fractions. However, so far the AIRS cloud product has not been rigorously evaluated/validated. Fortunately, collocated/coincident radiances measured by MODIS/AQUA (at a much lower spectral resolution but roughly an order of-magnitude higher spatial resolution than that of AIRS) are used to determine analogous cloud products from MODIS. This allows us for a rather rare and interesting possibility: the intercomparisons and mutual validation of imager vs. sounder-based cloud products obtained from the same satellite positions. First, we present results of small-scale (granules) instantaneous intercomparisons. Next, we will evaluate differences of temporally averaged (monthly) means as well as the representation of inter-annual variability of cloud parameters as presented by the two cloud data sets. In particular, we present statistical differences in the retrieved parameters of cloud fraction and cloud top pressure. We will investigate what type of cloud systems are retrieved most consistently (if any) with both retrieval schemes, and attempt to assess reasons behind statistically significant differences.

Molnar, Gyula I.↗