Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Scientific Data Analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
NASA NTRS2001

SOHO Ultraviolet Coronagraph Spectrometer (UVCS) Mission Operations and Data Analysis

The scientific goal of UVCS is to obtain detailed empirical descriptions of the extended solar corona as it evolves over the solar cycle and to use these descriptions to identify and understand the physical processes responsible for coronal heating, solar wind acceleration, coronal mass ejections (CMEs), and the phenomena that establish the plasma properties of the solar wind as measured by "in situ" solar wind instruments. This report covers the period from 15 November 1998 to 14 March 2001. During that time, UVCS observations have consisted of three types: 1) standard synoptic observations comprising, primarily, the H I Lycc line profile and the O VI 103.2 and 103.7 nm intensity over a range of heights from 1.5 to about 3.0 solar radii and covering 360 degrees about the sun, 2) sit and stare watches for CMEs, and 3) special observations designed by the UVCS Lead Observer of the Week for a specific scientific purpose. The special observations are often coordinated with those of other space-based and ground based instruments and they often are part of SOHO joint observation programs and campaigns. Lead observers have included UVCS Co-Investigators, Guest Investigators, scientists from the solar physics community and several graduate and undergraduate level students. UVCS has continued to successfully meet its goal of using powerful spectroscopic diagnostic techniques to obtain a much more detailed description of coronal structures than existed before the SOHO mission. The new descriptions of coronal structures from UVCS have inspired a large number of theoretical studies aimed at identifying the physical processes responsible for solar wind acceleration in coronal holes and streamers. UVCS has proven to be a very stable instrument. Stellar observations have demonstrated its stability and the analysis of coordinated observations with Spartan 201 have verified the accuracy of the absolute calibration and spectral resolution at H I Ly (alpha) line profile. UVCS has required no flight software modifications and all mechanisms are operational. The UVCS O VI Channel with its redundant optical path for wavelengths near H I Ly (alpha) is capable of observing the entire UVCS wavelength range. Since December 1998, the O VI Channel has been used for all UVCS observations. Although the H I Ly (alpha) Channel and detector are still operational, increases in the dark count up to about 5x 10(exp-4) counts/sec/pixel and an increase in high voltage current to within a factor of two of the maximum used in the laboratory before flight led to the decision to not use that detector at the present time. There is no significant science impact since the O VI Channel can observe nearly the same wavelength range.

Kohl, John L.↗
NASA NTRS2001

SOHO Ultraviolet Coronagraph Spectrometer (UVCS) Mission Operations and Data Analysis

The scientific goal of Ultraviolet Coronagraph Spectrometer (UVCS) is to obtain detailed empirical descriptions of the extended solar corona as it evolves through the solar activity cycle and to use those descriptions to identify and understand the physical processes responsible for coronal heating, solar wind acceleration, coronal mass ejections (CMEs), and the phenomena that establish the plasma properties of the solar wind as measured by 'in situ' solar wind instruments. The UVCS on the Solar and Heliospheric Observatory (SOHO) has made major advances in identifying the physical processes responsible for solar wind acceleration, and it has made the first high resolution ultraviolet spectroscopic measurements of CMEs. The UVCS has resulted in over 220 scientific publications.

Kohl, John L.↗
NASA NTRS2002

Soho Ultraviolet Coronograph Spectrometer (UVCS) Mission Operations and Data Analysis

The scientific goal of UVCS is to obtain detailed empirical descriptions of the extended solar corona as it evolves over the solar cycle and to use these descriptions to identify and understand the physical processes responsible for coronal heating, solar wind acceleration, coronal mass ejections (CMEs), and the phenomena that establish the plasma properties of the solar wind as measured by 'in situ' solar wind instruments. This report covers the period from 01 December 2000 to 31 January 2002. During that time, UVCS observations have consisted of three types: (1) standard synoptic observations comprising, primarily, the H I Ly(alpha) line profile and the O VI 103.2 and 103.7 nm intensity over a range of heights from 1.5 to about 3.0 solar radii and covering 360 degrees about the sun; (2) sit and stare watches for CMEs; and (3) special observations designed by the UVCS Lead Observer of the Week for a specific scientific purpose. The special observations are often coordinated with those of other space-based and ground-based instruments and they often are part of SOHO joint observation programs and campaigns. Lead observers have included UVCS Co-Investigators, scientists from the solar physics community and several graduate and undergraduate level students.

Kohl, John L.↗
NASA NTRS2018

Overview of SAND-E: Semi-Autonomous Navigation for Detrital Environments

Rovers are the state of the art for the exploration and detection of past habitability and life on other worlds. One of the most basic functions of a rover is terrain navigation. Information collected by the rover is used autonomously to mitigate terrain hazards such large rocks, while humans qualitatively assess hazardous geologic terrain such as soil type and degree of rock cover. Planetary scientists use the same information to select targets such as drill sites, and for basic scientific analysis such as characterization of rock outcrops. Although the data is complementary, data from terrain analysis for navigation and terrain analysis for scientific investigations are poorly integrated. The lack of integration creates science and operation inefficiencies that limit exploration of habitable environments. As new modes of exploration come online, such as unmanned aerial systems (UAS) (e.g., the Mars Helicopter Scout and Titan Dragonfly), a need exists to integrate terrain data and science analysis to improve operational and scientific outcomes during exploration. We present an overview of a project aimed at evaluating the effectiveness and capability rover and UAS-based semi-automated terrain analysis using the Automated Soil Assessment Systems (ASAS) developed by Mission Control Space Services for navigating, selecting targets for sampling, and characterizing mafic detrital sediments along glacio-fluvial-aeolian sand transport pathways in Iceland. We describe recent advances in automated terrain analysis in sandy environments and scientific uses of terrain assessment from sandy environments. We assess fluvial and aeolian terrains in Iceland and show how terrain analysis data can inform scientific characterization of these environments.

Ewing, Ryan C.↗
DOE OSTI2022

Performance Analysis and Optimization for Scientific Data Workloads

Scientific data generated at experimental and observational facilities are increasingly being processed on large-scale compute systems. Most of the experimental data analysis workflows are not designed or implemented to run on large scale environments and take full advantage of HPC compute and storage resources. These applications are unlike the traditional tightly-coupled scientific applications and hence face significant performance and scalability challenges as the volume of data increases exponentially. In this paper, we conduct a performance and scalability analysis for experimental analysis applications and workflows operating on data from light sources. Our analysis detects and quantifies I/O performance, scalability and runtime bottlenecks for three data analysis applications that run on NERSC resources. Based on our analysis we propose and implement a set of optimizations that lead to reducing the amount of time spent on I/O operations by almost 90%.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

A machine-learning-driven data labeling pipeline for scientific analysis in MLExchange

This study introduces a novel labeling pipeline to accelerate the labeling process of scientific data sets by using artificial intelligence (AI)-guided tagging techniques. This pipeline includes a set of interconnected web-based graphical user interfaces (GUIs), where Data Clinic and MLCoach enable the preparation of machine learning (ML) models for data reduction and classification, respectively, while Label Maker is used for label assignment. Throughout this pipeline, data can be accessed through a direct connection to a file system or through Tiled for access through Hypertext Transfer Protocol (HTTP). Our experimental results present three use cases where this labeling pipeline has been instrumental for the study of large X-ray scattering data sets in the area of pattern recognition, the remote analysis of resonant soft X-ray scattering data and the fine-tuning process of foundation models. These use cases highlight the labeling capabilities of this pipeline, including the ability to label large data sets in a short period of time, to perform remote data analysis while minimizing data movement and to enhance the fine-tuning process of complex ML models with human involvement.

Chavez, Tanny (ORCID:0000000193172896)↗
DOE OSTI2021

Scalable, In-situ Data Clustering Data Analysis for Extreme Scale Scientific Computing (Final Report)

The objective of this project is to address challenges in the design and development of scalable in-situ data clustering and analytics algorithms and software. Our goal is to develop parallel software consisting of a set of spatio-temporal data clustering and anomaly detection functions, both of which are very important for large-scale analysis and have wide applicability for in-situ runs as well as post-processing analysis. Our design principles for in-situ analysis consider the following: (1) identify parts of the computation can be done close to the data within the nodes, while it is still in memory; (2) extract analysis components can (and should) be performed in remote staging and analysis nodes; (3) develop error-bound approximation methods for applications tolerable for small errors; (4) identify the type of derived distributions and statistics, for spatio-temporal data, that can be kept locally in order to both accelerate computations and meet energy constraints in subsequent iterations and phases; (5) use a self-describing data format so that data can be consistent and understood among local storage (memory and SSDs) and at staging and analysis nodes, thereby providing portability and flexibility; (6) develop service-oriented functions that can schedule in-situ and post-hoc analysis tasks based on the dynamic requirements of applications. Our development focus is to produce the parallel data analysis software/library that will be scalable, reusable, extensible, and generic for applications in different disciplines. The software will be able to run in-situ with the simulations as well as post-hoc analysis. This approach will satisfy many synergistic requirements for data intensive applications executed on data coming from instruments and experiments. In particular, the proposed multilevel approach is directly applicable to perform design tradeoffs for running part of the algorithms near the instruments and the rest on remote (analysis) systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Enhancements to VTK enabling Scientific Visualization in Immersive Environments

Modern scientific, engineering and medical computational sim- ulations, as well as experimental and observational data sens- ing/measuring devices, produce enormous amounts of data. While statistical analysis provides insight into this data, scientific vi- sualization is tactically important for scientific discovery, prod- uct design and data analysis. These benefits are impeded, how- ever, when scientific visualization algorithms are implemented from scratch—a time-consuming and redundant process in im- mersive application development. This process can greatly ben- efit from leveraging the state-of-the-art open-source Visualization Toolkit (VTK) and its community. Over the past two (almost three) decades, integrating VTK with a virtual reality (VR) environment has only been attempted to varying degrees of success. In this pa- per, we demonstrate two new approaches to simplify this amalga- mation of an immersive interface with visualization rendering from VTK. In addition, we cover several enhancements to VTK that pro- vide near real-time updates and efficient interaction. Finally, we demonstrate the combination of VTK with both Vrui and OpenVR immersive environments in example applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Scalable Data Management, Analysis, and Visualization (SDAV) Institute

The purpose of the SDAV institute is to provide tools and expertise in scientific data management, analysis, and visualization to DOE’s application scientists. Our goal is to actively work with application teams to assist them in achieving breakthrough science, and to provide technical solutions in the data management, analysis, and visualization regimes that are broadly used by the computational science community. Over the last 5 years members of our institute worked directly with application scientists and DOE leadership-class facilities to assist them by applying the best tools and technologies at our disposal. We also enhanced our tools based on input from scientists on their needs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

End-to-end online performance data capture and analysis for scientific workflows

With the increased prevalence of employing workflows for scientific computing and a push towards exascale computing, it has become paramount that we are able to analyze characteristics of scientific applications to better understand their impact on the underlying infrastructure and vice-versa. Such analysis can help drive the design, development, and optimization of these next generation systems and solutions. Here, we present the architecture, integrated with existing well-established and newly developed tools, to collect online performance statistics of workflow executions from various, heterogeneous sources and publish them in a distributed database (Elasticsearch). Using this architecture, we are able to correlate online workflow performance data, with data from the underlying infrastructure, and present them in a useful and intuitive way via an online dashboard. We have validated our approach by executing two classes of real-world workflows, both under normal and anomalous conditions. The first is an I/O-intensive genome analysis workflow; the second, a CPU- and memory-intensive material science workflow. Based on the data collected in Elasticsearch, we are able to demonstrate that we can correctly identify anomalies that we injected. The resulting end-to-end data collection of workflow performance data is an important resource of training data for automated machine learning analysis.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Scalable Data Management, Analysis, and Visualization (SDAV) Institute (Final Report)

The purpose of the SDAV institute is to provide tools and expertise in scientific data management, analysis, and visualization to application scientists on leadership-class computing facilities. Our goal is to actively work with application teams to assist them in achieving breakthrough science, and to provide technical solutions in the data management, analysis, and visualization regimes that are broadly used by the computational science community. Over the last 5 years members of our institute worked directly with application scientists to assist them by applying the best tools and technologies at our disposal. We also enhanced our tools based on input from scientists on their needs. Many of the applications we have been working with are based on connections with scientists established in previous years. However, we contacted additional scientists though our outreach activities, as well as engaging application teams running on leading DOE computing systems. Our approach is to employ an evolutionary development and deployment process: first considering the application of existing tools, followed by the customization necessary for each particular application, and then the deployment in real frameworks and infrastructures. The institute is organized into three areas, each with area leaders, who keep track of progress, engagement of application scientists, and results. The areas are: (1) Data Management, (2) Data Analysis, and (3) Visualization. This report is organized along these areas. However, often there are multiple technologies from these areas that are applied to a single application need. These are described on a case-by-case basis in the appropriate sections. This final report for the SDAV Institute covers the 5-year period from February 2012 to February 2017. This report covers activities in all SDAV institutions listed next. Laboratories: Argonne National Laboratory (ANL), Lawrence Berkeley National Laboratory (LBNL), Lawrence Livermore National Laboratory (LLNL), Los Alamos National Laboratory (LLNL), Oak Ridge National Laboratory (ORNL), Sandia National Laboratory (SNL). Universities: Georgia Institute of Technology, North Carolina State University, Northwestern University, Ohio State University, Oregon State University, Rutgers University, University of California at Davis, University of Utah. Industry: Kitware.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Performance analysis and data reduction for exascale scientific workflows

Chimbuko is the first in situ, scalable, workflow-level performance analysis tool for trace-level analysis and visualization of application performance. This tool was developed by the Co-design Center for Online Data Analysis and Reduction and funded by the U.S. Department of Energy’s Exascale Computing Project. We provide a detailed description of Chimbuko’s architecture and illustrate our online and offline visualization with multiple use cases. We also present results for the deployment and scalability of the tool as applied to a high-energy physics workflow running at large scale on the Frontier supercomputer.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Scalable data management, analysis and visualization (SDAV) Institute. Final Scientific/Technical Report

The purpose of the SDAV institute is to provide tools and expertise in scientific data management, analysis, and visualization to DOE’s application scientists. Our goal is to actively work with application teams to assist them in achieving breakthrough science, and to provide technical solutions in the data management, analysis, and visualization regimes that are broadly used by the computational science community. Over the last 5 years members of our institute worked directly with application scientists and DOE leadership-class facilities to assist them by applying the best tools and technologies at our disposal. We also enhanced our tools based on input from scientists on their needs. Many of the applications we have been working with are based on connections with scientists established in previous years. However, we contacted additional scientists though our outreach activities, as well as engaging application teams running on leading DOE computing systems. Our approach is to employ an evolutionary development and deployment process: first considering the application of existing tools, followed by the customization necessary for each particular application, and then the deployment in real frameworks and infrastructures. The institute is organized into three areas, each with area leaders, who keep track of progress, engagement of application scientists, and results. The areas are: (1) Data Management, (2) Data Analysis, and (3) Visualization. Kitware has been involved in the Visualization area. This report covers Kitware’s contributions over the last 5 years (February 2012 – February 2017). For details on the work performed by the SDAV institute as a whole, please see the SDAV final report.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1994

Scientific analysis of satellite ranging data

A network of satellite laser ranging (SLR) tracking systems with continuously improving accuracies is challenging the modelling capabilities of analysts worldwide. Various data analysis techniques have yielded many advances in the development of orbit, instrument and Earth models. The direct measurement of the distance to the satellite provided by the laser ranges has given us a simple metric which links the results obtained by diverse approaches. Different groups have used SLR data, often in combination with observations from other space geodetic techniques, to improve models of the static geopotential, the solid Earth, ocean tides, and atmospheric drag models for low Earth satellites. Radiation pressure models and other non-conservative forces for satellite orbits above the atmosphere have been developed to exploit the full accuracy of the latest SLR instruments. SLR is the baseline tracking system for the altimeter missions TOPEX/Poseidon, and ERS-1 and will play an important role in providing the reference frame for locating the geocentric position of the ocean surface, in providing an unchanging range standard for altimeter calibration, and for improving the geoid models to separate gravitational from ocean circulation signals seen in the sea surface. However, even with the many improvements in the models used to support the orbital analysis of laser observations, there remain systematic effects which limit the full exploitation of SLR accuracy today.

Smith, David E.↗
NASA NTRS2006

Tool for Analysis and Reduction of Scientific Data

The Automated Scheduling and Planning Environment (ASPEN) computer program has been updated to version 3.0. ASPEN as a whole (up to version 2.0) has been summarized, and selected aspects of ASPEN have been discussed in several previous NASA Tech Briefs articles. Restated briefly, ASPEN is a modular, reconfigurable, application software framework for solving batch problems that involve reasoning about time, activities, states, and resources. Applications of ASPEN can include planning spacecraft missions, scheduling of personnel, and managing supply chains, inventories, and production lines. ASPEN 3.0 can be customized for a wide range of applications and for a variety of computing environments that include various central processing units and randomaccess memories. Domain-specific reasoning modules (e.g., modules for determining orbits for spacecraft) can easily be plugged into ASPEN 3.0. Improvements over other, similar software that have been incorporated into ASPEN 3.0 include a provision for more expressive time-line values, new parsing capabilities afforded by an ASPEN language based on Extensible Markup Language, improved search capabilities, and improved interfaces to other, utility-type software (notably including MATLAB).

James, Mark↗