Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data visualizations”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,261 records · Page 70

Streaming Data in HPC Workflows Using ADIOS

The “IO Wall” problem, in which the gap between computation rate and data access rate grows continuously, poses significant problems to scientific workflows which have traditionally relied upon using the filesystem for intermediate storage between workflow stages. One way to avoid this problem in scientific workflows is to stream data directly from producers to consumers and avoiding storage entirely. However, the manner in which this is accomplished is key to both performance and usability. This paper presents the Sustainable Staging Transport, an approach which allows direct streaming between traditional file writers and readers with few application changes. SST is an ADIOS “engine”, accessible via standard ADIOS APIs, and because ADIOS allows engines to be chosen at run-time, many existing file-oriented ADIOS workflows can utilize SST for direct application-to-application communication without any source code changes. This paper describes the design of SST and presents performance results from various applications that use SST, for feeding model training with simulation data with substantially higher bandwidth than the theoretical limits of Frontier’s file system, for strong coupling of separately developed applications for multiphysics multiscale simulation, or for in situ analysis and visualization of data to complete all data processing shortly after the simulation finishes.

Podhorszki, Norbert [ORNL] (ORCID:000000019647542X↗

Open Source Platform for Live Processing of High Data Rate Electron Microscopy (SBIR Phase I Award Topic 15, Subtopic B, Final Scientific / Technical Report)

4D scanning transmission electron microscopy (STEM) is experiencing a revolution both in terms of data size, and data veracity, as are many other areas of experimental data acquisition. This presents enormous opportunities to make new scientific discoveries, but is also hamstrung by aging software that is no longer fit for purpose. Experimental data acquisition platforms must make the same investments in software infrastructure that the simulation community made decades ago to take advantage of high-performance computing (HPC) if they hope to realize the full potential of the latest advances made in detector technology. This requires rethinking the design and deployment of hardware, moving away from small, fixed commodity computing capabilities with closed interfaces to open, modular, scalable systems designed for parallelism and throughput. The technical approach will leverage best-in-class open source technologies, and develop new approaches strategically to build an open source platform for experimental data acquisition targeting high data rate electron microscopy. The project will develop a software platform that uses the power and performance of modern HPC resources to ingest the large volume of data produced by electron microscopes using direct electron detectors. A set of configurable pipelines will be used to enable the processing to be tailored to the problem being investigated with builtin visualization of data at interactive rates. This project will leverage investments made by the DOE and other agencies to provide a powerful software platform for the experimental data community working with most powerful electron microscopes, such as 4D STEM detectors. It will create a platform that serves as a reference for the community. Permissive open source licensing will make the platform available to all, with a simple deployment strategy offering the ability to keep data close to where it is generated. This advanced software platform will require customization for each deployment, creating opportunities for software services that are typically offered to extend open source software platforms.

97 MATHEMATICS AND COMPUTING↗

Workshop on Dust on Mars

A workshop was organized to stimulate and coordinate research on the properties of Martian dust, its distribution on the planet, and the processes affecting its yearly deposition, erosion, and transport. A great deal of research was carried out to understand local, regional, and global aspects of the Martian dust transport cycle, with the goal being definition of the currently active processes and their relation to those operating over the long term. To date, however, most individual research projects have centered around interpretation of single data sets (visual images, IRTM, radar, etc.), and these many efforts were not coordinated. Many of the details of short and long term dust-related processes, as inferred from the various single data sets, are still not agreed upon. It was the intent of this workshop to encourage the informal exchange of ideas among several researchers currently investigating various aspects of the problem and to investigate cooperative efforts to synthesize the individual approaches into a unified understanding of the present seasonal and yearly Martian sediment transport cycles.

Lee, S.↗

Datacasting V3.0

Datacasting V3.0 provides an RSSbased feed mechanism for publishing the availability of Earth science data records in real time. It also provides a utility for subscribing to these feeds and sifting through all the items in an automatic manner to identify and download the data records that are required for a specific application. Datacasting is a method by which multiple data providers can publish the availability of new Earth science data and users download those files that meet a predefined need; for example, to only download data files related to a specific earthquake or region on the globe. Datacasting is a server-client architecture. The server-side software is used by data providers to create and publish the metadata about recently available data according to the Datacasting RSS (Really Simple Syndication) specification. The client software subscribes to the Datacasting RSS and other RSS-based feeds. By configuring filters associated with feeds, data consumers can use the client to identify and automatically download files that meet a specific need. On the client side, a Datacasting feed reader monitors the server for new feeds. The feed reader will be tuned by the user, via a graphical user interface (GUI), to examine the content of the feeds and initiate a data pull after some criteria are satisfied. The criteria might be, for example, to download sea surface temperature data for a particular region that has cloud cover less than 50% and during daylight hours. After the granule is downloaded to the client, the user will have the ability to visualize the data in the GUI. Based on the popular concept of podcasting, which gives listeners the capability to download only those MP3 files that match their preference, Earth science Datacasting will give users a method to download only the Earth science data files that are required for a particular application.

Bingham, Andrew W.↗

Satellite Data of Atmospheric Pollution for U.S. Air Quality Applications: Examples of Applications, Summary of Data End-user Resources, Answers to Faqs, and Common Mistakes to Avoid

Satellite data of atmospheric pollutants are becoming more widely used in the decision-making and environmental management activities of public, private sector and non-profit organizations. They are employed for estimating emissions, tracking pollutant plumes, supporting air quality forecasting activities, providing evidence for "exceptional event" declarations, monitoring regional long-term trends, and evaluating air quality model output. However, many air quality managers are not taking full advantage of the data for these applications nor has the full potential of satellite data for air quality applications been realized. A key barrier is the inherent difficulties associated with accessing, processing, and properly interpreting observational data. A degree of technical skill is required on the part of the data end-user, which is often problematic for air quality agencies with limited resources. Therefore, we 1) review the primary uses of satellite data for air quality applications, 2) provide some background information on satellite capabilities for measuring pollutants, 3) discuss the many resources available to the end-user for accessing, processing, and visualizing the data, and 4) provide answers to common questions in plain language.

remote sensing↗

Satellite Data of Atmospheric Pollution for U.S. Air Quality Applications: Examples of Applications, Summary of Data End-User Resources, Answers to FAQs, and Common Mistakes to Avoid

Satellite data of atmospheric pollutants are becoming more widely used in the decision-making and environmental management activities of public, private sector and non-profit organizations. They are employed for estimating emissions, tracking pollutant plumes, supporting air quality forecasting activities, providing evidence for "exceptional event" declarations, monitoring regional long-term trends, and evaluating air quality model output. However, many air quality managers are not taking full advantage of the data for these applications nor has the full potential of satellite data for air quality applications been realized. A key barrier is the inherent difficulties associated with accessing, processing, and properly interpreting observational data. A degree of technical skill is required on the part of the data end-user, which is often problematic for air quality agencies with limited resources. Therefore, we 1) review the primary uses of satellite data for air quality applications, 2) provide some background information on satellite capabilities for measuring pollutants, 3) discuss the many resources available to the end-user for accessing, processing, and visualizing the data, and 4) provide answers to common questions in plain language.

Duncan, Bryan↗

Exploring Innovative Methods for Earth Science Communication Through NASA’s Earth Information Center

Environmental challenges stemming from unprecedented changes in our planet have increased the need to communicate our understanding of Earth as a connected system. In June 2023, NASA unveiled the Earth Information Center (EIC), part of NASA’s Earth Action Strategy aimed at amplifying NASA’s Earth science impact by engaging decision-makers beyond the scientific community. Existing in both physical and virtual space, the EIC provides critical information on how the planet is changing and affecting livelihoods across the globe. For over fifty years, NASA has utilized the unique vantage point space offers to monitor Earth in collaboration with Federal partners. Through an expanding fleet of Earth observing satellites and sensors, NASA informs local to global decision making within nine thematic areas: agriculture, biodiversity, disasters, greenhouse gases, health and air quality, sea level rise and coastal impacts, sustainable energy, water resources, and wildfires. Visitors to the inaugural location of the EIC at NASA Headquarters in Washington D.C., can interact with Earth observation data and models through innovative exhibits and storytelling. Designed by an interdisciplinary team, the EIC is designed for multiple modality learning to reach a wide base of end-users. Data driven visualizations, near real-time information, and curated stories highlight the applications of Earth observations to address environmental challenges. Complementary exhibits and immersive storytelling allow visitors to physically step inside environmental data and connect with information on an emotional level. Taken together, the collective EIC provides scientists and non-scientists alike with the ability to see our home planet as NASA sees it, as a system of systems, while delivering easily accessible information to those who need it most.

Earth Information Center↗

Community Resilience Indicator Analysis: Commonly Used Indicators from Peer-Reviewed Research (Updated for Research Published 2003-2021)

In 2017, FEMA’s National Integration Center (NIC) Technical Assistance (TA) Branch identified a need to establish a data-driven basis for prioritizing locations for TA investment and guiding local emergency management planning. To achieve this goal, FEMA tasked Argonne National Laboratory (Argonne) with identifying commonly used indicators of community resilience across the landscape of published peer-reviewed research. FEMA and Argonne completed the first Community Resilience Indicator Analysis (CRIA) in 2018 and repeated the process in 2022. The CRIA process begins with a literature review and cataloguing of published peer-reviewed assessment methodologies on social vulnerability and community resilience. The literature review findings are then filtered by inclusion criteria established by the CRIA research team to ensure the methodologies are: (1) Quantitative, (2) Data and methodology are publicly available, (3) Calculated at the county level or lower, (4) Examine generalized hazard risk (rather than a singular hazard), and (5) Focused on pre-disaster community conditions. After this, the research team identifies the commonly used indicators across these methodologies and selects the best data source for each indicator. Finally, the research team bins the data for visual display, conducts a correlation analysis and creates a composite index, the FEMA Community Resilience Index (FEMA CRI). In 2018, the CRIA identified eight resilience and vulnerability assessment methodologies and 20 commonly used indicators (indicators used in three or more of the eight methodologies). The FEMA CRI in 2018 was created from these 20 indicators and was produced for at the county level. The 2022 CRIA updated the literature review to expand the list of methodologies examined and followed the same process, resulting in an analysis of 14 methodologies published between 2003 and 2021 and 22 indicators identified as commonly used (indicators used in five or more of the 14 methodologies). In 2022, the research team produced the FEMA CRI at the county and the census tract levels. To make the CRIA data more accessible and more actionable, each individual indicator and the FEMA CRI is binned and included in FEMA’s Resilience Analysis and Planning Tool (RAPT). RAPT enables emergency managers and community partners to quickly visualize relative differences in potential resilience by county, tribe and census tract. By reviewing the data for each of these 22 indicators individually, emergency managers can gain insights for targeted outreach strategies, planning, mitigation investments and response and recovery operations. Communities, regional governments and others can use this data to better understand potential challenges to resilience. As the social science field of examining and validating indicators of resilience evolves, FEMA will update RAPT to provide emergency managers and community partners with additional data and tools to inform planning, mitigation, response and recovery. It is important to understand that the role of the emergency manager is not to change or to “improve” the data, but to plan appropriately for the community characteristics reflected in the data. These datasets are community characteristics that researchers have identified as important considerations for resilience. For example, people with disabilities may have greater challenges to be resilient to disasters. If a community has a high population of people with disabilities, the emergency manager(s) may need to create tailored preparedness outreach programs and strategies to ensure those residents have support if evacuation is necessary. Rather than label these indicators as an absolute measure of resilience, FEMA considers “potential challenges to resilience” a better frame to understand these indicators. Everyone is vulnerable to disasters. While scholars theorize that certain characteristics may make an individual or a household more socially vulnerable, the data does not reflect measures that individuals and/or communities have taken to address potential challenges, such as emergency management planning and outreach or household preparedness measures. To aid emergency managers in understanding how to use these indicators, calling them potential challenges to resilience supports a more positive and strategic application of the data in all phases of emergency management.

99 GENERAL AND MISCELLANEOUS↗

A Deterministic Self-Organizing Map Approach and its Application on Satellite Data based Cloud Type Classification

A self-organizing map (SOM) is a type of competitive artificial neural network, which projects the high dimensional input space of the training samples into a low dimensional space with the topology relations preserved. This makes SOMs supportive of organizing and visualizing complex data sets and have been pervasively used among numerous disciplines with different applications. Notwithstanding its wide applications, the self-organizing map is perplexed by its inherent randomness, which produces dissimilar SOM patterns even when being trained on identical training samples with the same parameters every time, and thus causes usability concerns for other domain practitioners and precludes more potential users from exploring SOM based applications in a broader spectrum. Motivated by this practical concern, we propose a deterministic approach as a supplement to the standard self-organizing map. In accordance with the theoretical design, the experimental results with satellite cloud data demonstrate the effective and efficient organization as well as simplification capabilities of the proposed approach.

Initialization method↗

Refining HPCToolkit for application performance analysis at exascale

As part of the US Department of Energy’s Exascale Computing Project (ECP), Rice University has been refining its HPCToolkit performance tools to better support measurement and analysis of applications executing on exascale supercomputers. To efficiently collect performance measurements of GPU-accelerated applications, HPCToolkit employs novel non-blocking data structures to communicate performance measurements between tool threads and application threads. To attribute performance information in detail to source lines, loop nests, and inlined call chains, HPCToolkit performs parallel analysis of large CPU and GPU binaries involved in the execution of an exascale application to rapidly recover mappings between machine instructions and source code. To analyze terabytes of performance measurements gathered during executions at exascale, HPCToolkit employs distributed-memory parallelism, multithreading, sparse data structures, and out-of-core streaming analysis algorithms. To support interactive exploration of profiles up to terabytes in size, HPCToolkit’s hpcviewer graphical user interface uses out-of-core methods to visualize performance data. The result of these efforts is that HPCToolkit now supports collection, analysis, and presentation of profiles and traces of GPU-accelerated applications at exascale. These improvements have enabled HPCToolkit to efficiently measure, analyze and explore terabytes of performance data for executions using as many as 64K MPI ranks and 64K GPU tiles on ORNL’s Frontier supercomputer. HPCToolkit’s support for measurement and analysis of GPU-accelerated applications has been employed to study a collection of open-science applications developed as part of ECP. This paper reports on these experiences, which provided insight into opportunities for tuning applications, strengths and weaknesses of HPCToolkit itself, as well as unexpected behaviors in executions at exascale.

Adhianto, Laksono↗

ALPINE Overview

Data-driven sampling enables probabilistic identification of interesting regions in the data automatically, prioritizing important regions. Applied in situ to Nyx, important halo regions are preserved.

97 MATHEMATICS AND COMPUTING↗

A Graphical Model for Fusing Diverse Microbiome Data

This paper develops a Bayesian graphical model for fusing disparate types of count data. The motivating application is the study of bacterial communities from diverse high-dimensional features, in this case, transcripts, collected from different treatments. In such datasets, there are no explicit correspondences between the communities and each corresponds to different factors, making data fusion challenging. We introduce a flexible multinomial-Gaussian generative model for jointly modeling such count data. This latent variable model jointly characterizes the observed data through a common multivariate Gaussian latent space that parameterizes the set of multinomial probabilities of the transcriptome counts. The covariance matrix of the latent variables induces a covariance matrix of co-dependencies between all the transcripts, effectively fusing multiple data sources. We present a computationally scalable variational Expectation-Maximization (EM) algorithm for inferring the latent variables and the parameters of the model. Here, the inferred latent variables provide a common dimensionality reduction for visualizing the data and the inferred parameters provide a predictive posterior distribution. In addition to simulation studies that demonstrate the variational EM procedure, we apply our model to a bacterial microbiome dataset.

59 BASIC BIOLOGICAL SCIENCES↗

A Distributed Simulation Framework Applied to Artemis Analysis, Studies, Integration, and Test

The National Aeronautics and Space Administration (NASA) established the Artemis Program, a series of missions to return humans to the Moon and explore further than before. To execute the Artemis missions, NASA is collaborating with commercial and international partners to create the necessary infrastructure and logistics plan that will establish a long term presence on the Moon ahead of exploring Mars. NASA and its partners are developing a collection of space and surface systems to support crewed missions to the lunar surface that will provide the mobility, habitation, logistics, and exploration support necessary for Artemis mission successes which includes robust scientific investigations. This paper details the design, capabilities, and uses of the Artemis Distributed Simulation (ADS) being developed by the NASA Exploration Systems Simulations (NExSyS) Team to support Artemis architecture studies. ADS utilizes international interoperability standards to connect a collection of independent vehicle and service simulations; these include but are not limited to elements such as rovers, landers, and habitation elements along with services like communications, environment, visualization, and data logging. ADS’s distributed nature allows for the complex aggregation of constituent Artemis elements; this includes efficient scenario modification with the addition or removal of individual simulations representing Artemis elements or services. This capability provides support for the rapid performance of various Artemis mission trade studies exploring alternate configurations. Currently, ADS uses NASA developed simulations for development and testing; however, through the use of international simulation interoperability standards, ADS provides an integration framework to incorporate dissimilar authoritative vendor simulations as Artemis systems mature and vendor simulations become available. Vendor simulations will be able to join ADS and interact with other Artemis elements and vehicles while limiting the exposure of proprietary data. This paper describes the expansion of an existing distributed simulation infrastructure to accommodate a collaborative and dynamic framework for the Artemis Program. This work includes updated federation designs, integration into existing NASA facilities, advancements in visualizations, and advancements in human driven inputs. This paper will also outline recently completed and ongoing support and collaboration with NASA studies and testing, namely results from energetics and Human-In-The-Loop (HITL) studies. The paper concludes with a plan for future developments and facility integration to enable enhanced studies in preparation for a return of humans to the lunar surface.

Artemis↗

Effects of Simulated Microgravity and LINC-Complex on Chromosome 8 of Human Mammary Cells

Previously, human breast epithelial cells were studied under simulated microgravity to determine the role of the LINC-complex in maintaining nuclear morphology and gene expression. Data from that study was used in this project to understand how the LINC-complex influences gene expression at chromosome 8 under simulated microgravity. Differentially expressed genes found in clusters typically indicate a molecular relationship amongst them. Visual and data analyses (GO, pathways) identified 51 genes within clusters on chromosome 8 across various simulated microgravity conditions that did not share any molecular relationship. Additional results also determined that cells with a disrupted LINC-complex experienced more differential gene expression than the control, and that extended periods of simulated microgravity exacerbated differential gene expression in chromosome 8 of LINC-disrupted cells. These results suggest that the stressful simulated microgravity environment coupled with disrupted biological processes, such as in the case of the LINC-complex, can increase differential gene expression in chromosomes. Future studies where additional analyses are performed to further analyze gene expression of clusters under simulated microgravity could potentially elucidate the molecular relationship amongst affected genes.

Simulated microgravity↗

A versatile system for processing geostationary satellite data with run-time visualization capability

To better predict global climate change, scientists are developing climate models that require interdisciplinary and collaborative efforts in their building. We are currently involved in several such projects but will briefly discuss activities in support of two such complementary projects: the Atmospheric Radiation Measurement (ARM) program of the Department of Energy and Sequoia 2000, a joint venture of the University of California, the private sector, and government agencies. Our contribution to the ARM program is to investigate the role of clouds on the top of the atmosphere and on surface radiance fields through the data analysis of surface and satellite observations and complex modeling of the interaction of radiation with clouds. One of our first ARM research activities involves the computation of the broadband shortwave surface irradiance from satellite observations. Geostationary satellite images centered over the first ARM observation site are received hourly over the Internet network and processed in real time to compute hourly and daily composite shortwave irradiance fields. The images and the results are transferred via a high-speed network to the Sequoia 2000 storage facility in Berkeley, where they are archived These satellite-derived results are compared with the surface observations to evaluate the accuracy of the satellite estimate and the spatial representation of the surface observations. In developing the software involved in calculating the surface shortwave irradiance, we have produced an environment whereby we can easily modify and monitor the data processing as required. Through the principles of modular programming, we have developed software that is easily modified as new algorithms for computation are developed or input data availability changes. In addition, the software was designed so that it could be run from an interactive, icon-driven, graphical interface, TCL-TK, developed by Sequoia 2000 participants. In this way, the data flow can be interactively assessed and altered as needed. In this environment, the intermediate data processing 'images' can be viewed, enabling the investigator to easily monitor the various data processing steps as they progress. Additionally, this environment allows the rapid testing of new processing modules and allows their effects to be visually compared with previous results.

Landsfeld, M.↗

Using Science Operations Tools and Methods from NASA for Remote Presence Ocean Exploration on Earth: E/V Nautilus Cruise NA108

NASA Ames' Exploration Ground Data Systems (xGDS) supports rapid scientific decision making by synchronizing information in time and space, including video and still images, scientific instrument data, and science and operations notes in geographic and temporal context. We have deployed xGDS at multiple NASA field analog missions over the past decade.In the last two years, we have participated in SUBSEA, a multi-institution collaborative project. SUBSEA used the research ship E/V Nautilus along with its two remotely operated vehicles (ROVs), Hercules and Argus, to explore deep ocean volcanic vents as an analog for ocean worlds (e.g. Enceladus). This work allowed us to compare the existing oceanographic operations methods and technologies used for ocean exploration with corresponding tools and approaches developed and used at NASA. In the first year of SUBSEA we observed existing remote science operations from the Inner Space Center (ISC). In the second year, we deployed xGDS at ISC to complement existing capabilities with xGDS tools designed to support remote Nautilus science operations from the ISC. During operations, video, ROV position and instrument telemetry were streamed from the ship to the ISC. As the science team watched dive operations, they could annotate the data with observations that were relevant to their work domain. Later, the team members could review the data at their own pace to collaboratively develop a dive plan for the next day, which had to be delivered on a fixed daily schedule.The opportunity to compare operations under different conditions enabled us to make several key observations about conducting remote science and planning operations efficiently: (i) Reviewing data collaboratively and interactively with temporal and spatial context was critical for the remote science team's ability to plan dive operations on the Nautilus. (ii) Science team members were actively engaged with the remote dive operations because they could interact with the collected data and visualize it as they desired. (iii) Being able to replay past events at accelerated speeds, and jump to points in time and spaced based on search results, provided efficient access to critical points of interest in a massive volume of data, so the remote science team could deliver plans on time. SUBSEA (Systematic Underwater Biogeochemical Science and Exploration Analog) is a multi-institution collaboration supported by NASA, NOAA's Office of Exploration Research (OER), the Ocean Exploration Trust (OET) and the University of Rhode Island's Graduate School of Oceanography (GSO). ISC is GSO's telepresence operations facility.

Lees, David S.↗

Conducting Efficient Remote Science and Planning Operations for Ocean Exploration Using Exploration Ground Data Systems (xGDS)

NASA Ames’ Exploration Ground Data Systems (xGDS) supports rapid scientific decision making by synchronizing information in time and space, including video and still images, scientific instrument data, and science and operations notes in geographic and temporal context. We have deployed xGDS at multiple NASA field analog missions over the past decade. In the last two years, we have participated in SUBSEA, a multi-institution collaborative project*. SUBSEA used the research ship E/V Nautilus along with its two remotely operated vehicles (ROVs), Hercules and Argus, to explore deep ocean volcanic vents as an analog for ocean worlds (e.g. Enceladus). This work allowed us to compare the existing oceanographic operations methods and technologies used for ocean exploration with corresponding tools and approaches developed and used at NASA. In the first year of SUBSEA we observed existing remote science operations from the Inner Space Center (ISC)**. In the second year, we deployed xGDS at ISC to complement existing capabilities with xGDS tools designed to support remote Nautilus science operations from the ISC. During operations, video, ROV position and instrument telemetry were streamed from the ship to the ISC. As the science team watched dive operations, they could annotate the data with observations that were relevant to their work domain. Later, the team members could review the data at their own pace to collaboratively develop a dive plan for the next day, which had to be delivered on a fixed daily schedule. The opportunity to compare operations under different conditions enabled us to make several key observations about conducting remote science and planning operations efficiently: (i) Reviewing data collaboratively and interactively with temporal and spatial context was critical for the remote science team’s ability to plan dive operations on the Nautilus. (ii) Science team members were actively engaged with the remote dive operations because they could interact with the collected data and visualize it as they desired. (iii) Being able to replay past events at accelerated speeds, and jump to points in time and spaced based on search results, provided efficient access to critical points of interest in a massive volume of data, so the remote science team could deliver plans on time. * SUBSEA (Systematic Underwater Biogeochemical Science and Exploration Analog) is a multi-institution collaboration supported by NASA, NOAA’s Office of Exploration Research (OER), the Ocean Exploration Trust (OET) and the University of Rhode Island’s Graduate School of Oceanography (GSO).

Cohen, Tamar E.↗