Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data analysis, visualization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,585 records · Page 88

Towards Autonomous Experiments by Connecting High Performance Microscopy with High Performance Computing

The digitization of controls, data, and analysis in microscopy is bringing the idea of autonomous microscopes closer to reality than ever before. Automated transmission electron microscopy (TEM) is already fairly routine for some experiments the only require simple repetitive tasks such as imaging biological macromolecules for single particle cryoEM [1], tilt series for electron tomography [2], and movies for crystallography [3]. The vast majority of TEM experiments are conducted completely by human operators who choose the regions of interest, optimize experimental parameters, and make decisions about data quality visually during an experiment. The field is still a long way from having completely autonomous TEMs that can adapt to sample difficulties and tune experimental parameters based on data quality and desired experimental outcomes. Part of the issue is the lack of capability for feeding information learned from on-line, live data analysis back into the on-going experiment [4]. Furthermore, this presentation will discuss current capabilities for large scale data reduction and analysis using high performance computing (i.e. supercomputing) and progress towards developing a true feed-back loop that places data analysis and theory in the experimental loop.

97 MATHEMATICS AND COMPUTING↗

Towards Interactive, Reproducible Analytics at Scale on HPC Systems

The growth in scientific data volumes has resulted in a need to scale up processing and analysis pipelines using High Performance Computing (HPC) systems. These workflows need interactive, reproducible analytics at scale. The Jupyter platform provides core capabilities for interactivity but was not designed for HPC systems. In this paper, we outline our efforts that bring together core technologies based on the Jupyter Platform to create interactive, reproducible analytics at scale on HPC systems. Our work is grounded in a real world science use case-applying geophysical simulations and inversions for imaging the subsurface. Our core platform addresses three key areas of the scientific analysis workflow-reproducibility, scalability, and interactivity. We describe our implemention of a system, using Binder, Science Capsule, and Dask software. We demonstrate the use of this software to run our use case and interactively visualize real-Time streams of HDF5 data.

containers↗

The Satellite Flythrough and Reconstruction Softwares at CCMC

The next-generation of model data visualization to be offered at the Community Coordinated Modeling Center (CCMC) will be based on Kamodo, an open source python package. To increase the usefulness of our services, we are also developing new capabilities based on this software, called the satellite flythrough and the reconstruction tools, to be offered both as packages for offline analysis and through an online interface (coming soon). The satellite flythrough software ‘flies’ a satellite trajectory, whether real or imaginary, through either model data hosted at CCMC or on a personal machine. This service greatly simplifies the complexity of users’ access to model data, abstracting away the time-consuming details of model data formats and interpolation. We demonstrate execution times of a few seconds to a few minutes for several example flythroughs of a trajectory stretching over a few days, depending on the parameters chosen. We also demonstrate a reconstruction tool built on top of the satellite flythrough software, for use with mission planning and model-data comparisons. This tool, based on reconstructions provided for the GDC Science and Technology Definition Team, converts multiple, simultaneous satellite flythroughs into two-dimensional reconstructions. The reconstruction tool provides a software capability for satellite constellations to determine how many satellites are needed and in what configuration to resolve the desired features in the model data. Both tools are currently available through GitHub for a selection of CCMC-hosted ITM models. Finally, we present initial results from work in progress and plans for future work, including an expansion of the reconstruction tool to provide 3D reconstruction capabilities and a line-of-sight calculation tool.

software, python↗

Splashdown Visualization of Spent Stages

The state of the art for many Earth-to-orbit trajectory analysis toolsets used at NASA is somewhat dated in terms of the languages they are written in and their user interfaces. Many of these programs are written in languages like Fortran or C that are no longer considered modern in the technology industry, and are command-line based without any means of interpreting the data outputs. This presentation is meant to demonstrate a specific use case for a newly developed web app that visualizes the output data for one of these tools. Specifically, one key usage is to investigate and validate launches for notional multi-stage vehicle concepts to various non-standard high-inclination orbits. Some NASA requirements dictate that trajectories be designed such that no surviving debris lands closer than 200 nautical miles (nm) from foreign landmasses or 27 nm from the continental United States, therefore validating re-entry locations of spent stages is an integral part of the launch planning process. In addition, at programmatic levels it can prove insightful and clarifying for the decision-making process to enhance the technical results of numerical simulations with visualizations. The intent with this tool is to provide the mission analyst and program level management with an intuitive and clear grasp of key information on possible mission scenarios either departing from or arriving at Earth, and in the future, the moon or Mars as well. The software tool was developed with an agile development approach and utilizes the latest frameworks and technologies. The front end uses the React JavaScript library for making a state of the art frontend and a backend based on the Django Python framework for handling data using Python’s powerful and free scientific libraries. In addition, the CesiumJS open-source library is key for visualizing these end-to-end trajectories on a high-resolution Earth model. The presentation will demonstrate the current capability and tested use cases.

Jack Agolli↗

Anomaly detection in Hyper Suprime-Cam galaxy images with generative adversarial networks

ABSTRACT The problem of anomaly detection in astronomical surveys is becoming increasingly important as data sets grow in size. We present the results of an unsupervised anomaly detection method using a Wasserstein generative adversarial network (WGAN) on nearly one million optical galaxy images in the Hyper Suprime-Cam (HSC) survey. The WGAN learns to generate realistic HSC-like galaxies that follow the distribution of the data set; anomalous images are defined based on a poor reconstruction by the generator and outlying features learned by the discriminator. We find that the discriminator is more attuned to potentially interesting anomalies compared to the generator, and compared to a simpler autoencoder-based anomaly detection approach, so we use the discriminator-selected images to construct a high-anomaly sample of ∼13 000 objects. We propose a new approach to further characterize these anomalous images: we use a convolutional autoencoder to reduce the dimensionality of the residual differences between the real and WGAN-reconstructed images and perform UMAP clustering on these. We report detected anomalies of interest including galaxy mergers, tidal features, and extreme star-forming galaxies. A follow-up spectroscopic analysis of one of these anomalies is detailed in the Appendix; we find that it is an unusual system most likely to be a metal-poor dwarf galaxy with an extremely blue, higher-metallicity H ii region. We have released a catalogue with the WGAN anomaly scores; the code and catalogue are available at https://github.com/kstoreyf/anomalies-GAN-HSC; and our interactive visualization tool for exploring the clustered data is at https://weirdgalaxi.es.

79 ASTRONOMY AND ASTROPHYSICS↗

User Directed Tools for Exploiting Expert Knowledge in an Immersive Segmentation and Visualization Environment

Volumetric data sets have become common in medicine and many sciences through technologies such as computed x-ray tomography (CT), magnetic resonance (MR), positron emission tomography (PET), confocal microscopy and 3D ultrasound. When presented with 2D images humans immediately and unconsciously begin a visual analysis of the scene. The viewer surveys the scene identifying significant landmarks and building an internal mental model of presented information. The identification of features is strongly influenced by the viewers expectations based upon their expert knowledge of what the image should contain. While not a conscious activity, the viewer makes a series of choices about how to interpret the scene. These choices occur in parallel with viewing the scene and effectively change the way the viewer sees the image. It is this interaction of viewing and choice which is the basis of many familiar visual illusions. This is especially important in the interpretation of medical images where it is the expert knowledge of the radiologist which interprets the image. For 3D data sets this interaction of view and choice is frustrated because choices must precede the visualization of the data set. It is not possible to visualize the data set with out making some initial choices which determine how the volume of data is presented to the eye. These choices include, view point orientation, region identification, color and opacity assignments. Further compounding the problem is the fact that these visualization choices are defined in terms of computer graphics as opposed to language of the experts knowledge. The long term goal of this project is to develop an environment where the user can interact with volumetric data sets using tools which promote the utilization of expert knowledge by incorporating visualization and choice into a tight computational loop. The tools will support activities involving the segmentation of structures, construction of surface meshes and local filtering of the data set. To conform to this environment tools should have several key attributes. First, they should be only rely on computations over a local neighborhood of the probe position. Second, they should operate iteratively over time converging towards a limit behavior. Third, they should adapt to user input modifying they operational parameters with time.

Senger, Steven O.↗

Improving the Accessibility and Use of NASA Earth Science Data

Many of the NASA Langley Atmospheric Science Data Center (ASDC) Distributed Active Archive Center (DAAC) multidimensional tropospheric and atmospheric chemistry data products are stored in HDF4, HDF5 or NetCDF format, which traditionally have been difficult to analyze and visualize with geospatial tools. With the rising demand from the diverse end‐user communities for geospatial tools to handle multidimensional products, several applications, such as ArcGIS, have refined their software. Many geospatial applications now have new functionalities that enable the end user to: Store, serve, and perform analysis on each individual variable, its time dimension, and vertical dimension. Use NetCDF, GRIB, and HDF raster data formats across applications directly. Publish output within REST image services or WMS for time and space enabled web application development. During this webinar, participants will learn how to leverage geospatial applications such as ArcGIS, OPeNDAP and ncWMS in the production of Earth science information, and in increasing data accessibility and usability.

Tisdale, Matthew↗

Cammp Team

The International Space Station (ISS) Configuration Analysis Modeling and Mass Properties (CAMMP) Team is presenting a demo of certain CAMMP capabilities at a Booz Allen Hamilton conference in San Antonio. The team will be showing pictures of low fidelity, simplified ISS models, but no dimensions or technical data. The presentation will include a brief description of the contract and task, description and picture of the Topology, description of Generic Ground Rules and Constraints (GGR&C), description of Stage Analysis with constraints applied, and wrap up with description of other tasks such as Special Studies, Cable Routing, etc. The models include conceptual Crew Exploration Vehicle (CEV) and Lunar Lander images and animations created for promotional purposes, which are based entirely on public domain conceptual images from public NASA web sites and publicly available magazine articles and are not based on any actual designs, measurements, or 3D models. Conceptual Mars rover and lander are completely conceptual and are not based on any NASA designs or data. The demonstration includes High Fidelity Computer Aided Design (CAD) models of ISS provided by the ISS 3D CAD Team which will be used in a visual display to demonstrate the capabilities of the Teamcenter Visualization software. The demonstration will include 3D views of the CAD models including random measurements that will be taken to demonstrate the measurement tool. A 3D PDF file will be demonstrated of the Blue Book fidelity assembly complete model with no vehicles attached. The 3D zoom and rotation will be displayed as well as random measurements from the measurement tool. The External Configuration Analysis and Tracking Tool (ExCATT) Microsoft Access Database will be demonstrated to show its capabilities to organize and track hardware on ISS. The data included will be part numbers, serial numbers, historical, current, and future locations, of external hardware components on station. It includes dates of all external ISS events and flights and the associated hardware changes for each event. The hardware location information does not always reveal the exact location of the hardware, only the general location. In some cases the location is a module or carrier, in other cases it is a WIF socket, handrail, or attach point. Only small portions of the data will be displayed for demonstration purposes.

Evertt, Shonn F.↗

Utah FORGE 2439: Well 16B(78)-32 Field-Test Data from Mini-Frac Tests

This submittal includes the field-test data collected during stress tests conducted in the Utah FORGE 16B(78)-32 wellbore to measure/characterize the stresses in the geothermal reservoir. The type of stress test performed is referred to as a mini-frac test or a micro-frac test. The test is a hydraulic fracture test that involves injecting a small volume of water into a short interval of the reservoir that is isolated by a straddle packer to create a fracture. The test provides information about the minimum and maximum horizontal stress at the test depths. A total of seven mini-frac tests were performed within the upper (vertical or nearly vertical) section of the 9-5/8-inch diameter uncased wellbore, between depths of 5,202 ft MD [5,202 ft TVD] and 5,980 ft MD [5966.18 ft TVD]. Two types of data were generated as part of the mini-frac tests. The first type of data is time-series data for each mini-frac test (e.g., injection-rate, test-interval pressure), and are contained in the "Mini-Frac Test Data.zip" file below. The second type is geophysical log data, attached below in the "Geophysical Logs.zip" file. Here, two types of image logs and acoustic logs were obtained before and after the mini-frac tests to examine each test depth for visual evidence of induced fracture(s) and to measure the azimuth (orientation) of any observed fractures. The attached Readme file includes the test depth information and contents of each data file included here. Analysis of the field-test data was conducted and is documented in the attached report below.

15 GEOTHERMAL ENERGY↗

Volumetric Rendering on Wavelet-Based Adaptive Grid

Numerical modeling of physical phenomena frequently involves processes across a wide range of spatial and temporal scales. In the last two decades, the advancements in wavelet-based numerical methodologies to solve partial differential equations, combined with the unique properties of wavelet analysis to resolve localized structures of the solution on dynamically adaptive computational meshes, make it feasible to perform large-scale numerical simulations of a variety of physical systems on a dynamically adaptive computational mesh that changes both in space and time. Volumetric visualization of the solution is an essential part of scientific computing, yet the existing volumetric visualization techniques do not take full advantage of multi-resolution wavelet analysis and are not fully tailored for visualization of a compressed solution on the wavelet-based adaptive computational mesh. Our objective is to explore the alternatives for the visualization of time-dependent data on space-time varying adaptive mesh using volume rendering while capitalizing on the available sparse data representation. Two alternative formulations are explored. The first one is based on volumetric ray casting of multi-scale datasets in wavelet space. Rather than working with the wavelets at the finest possible resolution, a partial inverse wavelet transform is performed as a preprocessing step to obtain scaling functions on a uniform grid at a user-prescribed resolution. As a result, a solution in physical space is represented by a superposition of scaling functions on a coarse regular grid and wavelets on an adaptive mesh. An efficient and accurate ray casting algorithm is based just on these coarse scaling functions. Additional details are added during the ray tracing by taking an appropriate number of wavelets into account based on support overlap with the interpolation point, wavelet coefficient magnitude, and other characteristics, such as opacity accumulation (front to back ordering) and deviation from frontal viewing direction. The second approach is based on complementing of wavelet-based adaptive mesh to the traditional Adaptive Mesh Refinement (AMR) mesh. Both algorithms are illustrated and compared to the existing volume visualization software for Rayleigh-Benard thermal convection and electron density data sets in terms of rendering time and visual quality for different data compression of both wavelet-based and AMR adaptive meshes.

Vezolainen, Alexei V.↗

Factorized visual representations in the primate visual system and deep neural networks

Object classification has been proposed as a principal objective of the primate ventral visual stream and has been used as an optimization target for deep neural network models (DNNs) of the visual system. However, visual brain areas represent many different types of information, and optimizing for classification of object identity alone does not constrain how other information may be encoded in visual representations. Information about different scene parameters may be discarded altogether (‘invariance’), represented in non-interfering subspaces of population activity (‘factorization’) or encoded in an entangled fashion. In this work, we provide evidence that factorization is a normative principle of biological visual representations. In the monkey ventral visual hierarchy, we found that factorization of object pose and background information from object identity increased in higher-level regions and strongly contributed to improving object identity decoding performance. We then conducted a large-scale analysis of factorization of individual scene parameters – lighting, background, camera viewpoint, and object pose – in a diverse library of DNN models of the visual system. Models which best matched neural, fMRI, and behavioral data from both monkeys and humans across 12 datasets tended to be those which factorized scene parameters most strongly. Notably, invariance to these parameters was not as consistently associated with matches to neural and behavioral data, suggesting that maintaining non-class information in factorized activity subspaces is often preferred to dropping it altogether. Thus, we propose that factorization of visual scene information is a widely used strategy in brains and DNN models thereof.

59 BASIC BIOLOGICAL SCIENCES↗

Pioneer 10 observations of the Beta Cephei stars Gamma Pegasi and Delta Ceti

The results of analyzing broad-band Pioneer 10 photometric observations of the low-amplitude pulsating Beta Cephei stars Gamma Pegasi and Delta Ceti are reported. Periods and light curve amplitudes of 3.649 + or - 0.020 hr, 0.05 + or - 0.02 mag for Gamma Peg and 3.869 + or - 0.020 hr, 0.13 + or - 0.02 mag for Delta Ceti are obtained; a power spectrum analysis of the data reveals no other periods. No evidence is found for a phase shift between the light curve maxima in the UV and visible regions. The observed amplitudes combined with published visual and near-UV data suggest a flux and temperature variability of about 200 solar luminosities and 250 K for Gamma Peg and about 600 solar luminosities and 450 K for Delta Cet. These results are compared with others obtained with satellite and ground-based instrumentation.

Peters, Geraldine J.↗

Visualizing Geospatial Data through ESRI Story Maps for Earth Science Education: Lessons Learned from My NASA Data

For 20 years My NASA Data (MND) has curated NASA Earth science data and provided the data to educators in engaging learner-centered resources. MND has recently featured story maps as an innovative way to engage students in NASA Earth data. A story map is a cloud-based lesson that engages the learner in interactive geospatial maps using NASA data, and other multimedia content, text, and tasks that can be seamlessly incorporated in classroom instruction. This immersive technology eliminates the need for the user to move among tricky interfaces to access and visualize Earth science data, and no special software is required to be downloaded. Each story map integrates data from different NASA satellite missions, retrieved from Distributed Active Archive Centers (DAACs). Story maps also employ data analysis tools, such as time series options and swipe tools that allow learners to view and analyze relationships between scientific variables. MND has produced 25 story map lesson plans on the topics of air quality, the urban heat island effect, Earth’s energy budget, phytoplankton distribution, hurricane formation, solar eclipses, ocean circulation patterns, sea ice extent, and volcanic eruptions. Nine of them are extended story maps and written in the 5E format, which is internationally recognized as best practice based on how children learn science. Each story map resource is developed by the MND team featuring a GIS programming specialist, a lead scientist, and educational specialist/s to ensure the context, content, and methods are scientifically and educationally sound. The MND story maps are written for middle and high school science teachers and students as they connect with the Earth Systems Science phenomena featured in the Next Generation Science Standards. Each story map includes supporting resources for smooth integration in the classroom. During Fiscal Year 2023, The My NASA Data website received over 100,000 story map engagements during. These metrics highlight the interest in story maps as an Earth Science educational resource.

Desiray Wilson↗

Efficient Clustering of Software Vulnerabilities using Self Organizing Map (SOM)

The common vulnerabilities and exposures (CVE) database was created with a mission to ``identify, define, and catalog publicly disclosed cybersecurity vulnerabilities''. This rich body of information can be used to enable rapid and efficient response to secure and defend cyber operations and protect critical cyber infrastructure. The main goal of this paper is to develop a visual analytics tool to enable deep analysis of CVEs using unsupervised clustering techniques. We enhance our analysis by first mapping CVEs to hierarchical-classes in Common Weakness Enumeration (CWE) using information in the National Vulnerability Database (NVD). Both the mapping and the numerical representation of CVEs are enabled by V2W-BERT, which uses natural language processing of the extensive information in NVD to generate a large tabular database of 137,226 CVE entries from 1999 to 2020, where each CVE is represented by a vector of 768 numerical features. The vectorized data is processed by Self-Organizing Maps (SOM), which is an unsupervised machine learning technique for dimensionality reduction, visual representation and clustering. Using a Torus map of 6417 units, we achieve ~10-fold data compression of ~140k CVEs using SOM. The trained map is further clustered using standard K-means clustering into 138 clusters of CVEs. We conducted a brief investigation of the rich mapping of CVEs to best-matching-units to K-means clusters, as well as CVEs to CWEs. For example, this novel mapping provided insight into the role of CWE-59 and CWE-264 in several CVEs that is otherwise hard to explore in the original data. We conclude that our this novel approach will not only enable deep analysis of the complex relationships between CVEs and CWEs, but also a mechanism to quickly respond to and design mitigation actions for rapidly evolving vulnerabilities that have not been mapped to existing CWEs.

Panchal, Khyati↗

Predicting U 3 O 8 powder processing conditions: An AI/ML approach analyzing deep learning embeddings of SEM micrographs

High-resolution SEM images of uranium-oxide powders encode micro- and nanoscale clues to their synthesis route and calcination temperature. We trained a ResNet-50 model on 11 commercial-scale U₃O₈ classes, ammonium diuranate (ADU) or uranyl peroxide (H₂O₂) precursors calcined at temperatures ranging from 400 to 750 °C and added a 256-D projection head before the classifier to analyze the learned representation. The best of eight seeds reached 92.4 % accuracy on reserved testing data, but our focus is the structure of the embedding space rather than the accuracy and labels. We quantify class relatedness in the original 256-D space using centroid similarity and distributional distances, and we use Uniform Manifold Approximation Projection (UMAP) for visualization. ‘Unknown’ images from different preparation methods, SEM operators, and from the literature localized near the expected classes under a nearest-centroid analysis without retraining, as well as clustered in similar UMAP space. In conclusion, this embedding-centered workflow complements black-box classification by providing quantitative, similarity-based comparisons of U₃O₈ morphologies and reduces storage space by up to 98 % for image data used in millisecond vector search comparisons.

36 MATERIALS SCIENCE↗

Digital preprocessing and classification of multispectral earth observation data

The development of airborne and satellite multispectral image scanning sensors has generated wide-spread interest in application of these sensors to earth resource mapping. These point scanning sensors permit scenes to be imaged in a large number of electromagnetic energy bands between .3 and 15 micrometers. The energy sensed in each band can be used as a feature in a computer based multi-dimensional pattern recognition process to aid in interpreting the nature of elements in the scene. Images from each band can also be interpreted visually. Visual interpretation of five or ten multispectral images simultaneously becomes impractical especially as area studied increases; hence, great emphasis has been placed on machine (computer) techniques for aiding in the interpretation process. This paper describes a computer software system concept called LARSYS for analysis of multivariate image data and presents some examples of its application.

Anuta, P. E.↗

Memory-Aware External Facelist Calculation: A Data-Parallel Atomic Hash Counting Approach

Unstructured volumetric meshes serve as fundamental data representations in various scientific simulations and analyses. They play a crucial role in representing complex computational domains and are essential for important numerical techniques, such as finite element analysis. Whenever such a mesh is read from a file, streamed in-situ, or generated by algorithms, scientific visualization libraries rely on calculating the external surface of a geometry, named “external facelist”, to produce a polygonal mesh for rendering. Consequently, external facelist calculation has become one of the most widely used algorithms in the scientific visualization domain, necessitating optimal performance. In this paper, we explore relevant work on external facelist calculation algorithms in two common visualization libraries, VTK and Viskores, assess their performance and memory constraints, and introduce a novel memory-aware external facelist calculation algorithm employing an atomic hash counting approach. This algorithm fully leverages Viskores' data-parallel primitive operations, facilitating its execution across diverse many-core architectures. Our algorithm features the lowest memory footprint on the GPU and the second-lowest on the CPU among all evaluated methods, and it also delivers the fastest performance on both CPU and GPU. It has been made available under an open-source license in the VTK and Viskores visualization systems.

Tsalikis, Spiros [Kitware] (ORCID:0000000151137195↗

EPIsembleVis: A geo-visual analysis and comparison of the prediction ensembles of multiple COVID-19 models

In this work, we present EPIsembleVis, a web-based comparative visual analysis tool for evaluating the consistency of multiple COVID-19 prediction models. Our approach analyzes a collection of COVID-19 predictions from different epidemiological models as an ensemble and utilizes two metrics to quantify model performance. These metrics include (a) prediction uncertainty (represented as the dispersion of predictions in each ensemble) and (b) prediction error (calculated by comparing individual model predictions with the recorded data). Through an interactive visual interface, our approach provides a data-driven workflow for (a) selecting and constructing the COVID-19 model prediction ensemble based on the spatiotemporal overlap of available predictions of multiple epidemiological models, (b) quantifying the model performance using both the uncertainty of each model prediction ensemble, and the error of each ensemble member that represents individual model predictions, and (c) visualizing the spatiotemporal variability in the projection performance of individual models using a suite of novel ensemble visualization techniques, such as the data availability map, a spatiotemporal textured-tile calendar, multivariate rose chart, and time-series leaflet glyph. We demonstrate the capability of our ensemble visual interface through a case study that investigates the performance of weekly COVID-19 predictions, which are provided through the COVID-19 Forecast Hub UMass-Amherst Influenza Forecasting Center of Excellence [47] for the United States and United States Territories. The EPIsembleVis tool is implemented using open-source web technologies and adaptive system design, rendering it interoperable with Elasticsearch and Kibana for automatically ingesting COVID-19 predictions from online repositories, and it is generalizable for analyzing worldwide projections from more epidemiological models.

60 APPLIED LIFE SCIENCES↗