Emc aerospace systems analysis Interim scientific report
Analysis and data requirements for solving potential aerospace electromagnetic compatibility problems
Otero, R. J.↗
SEARCH · Engineering Papers
Results for “Scientific Data Analysis”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Kernels for scalable data analysis in science: Towards an architecture-portable future
In this paper, we pose and address some of the unique challenges in the analysis of scientific Big Data on supercomputing platforms. Our approach identifies, implements and scales numerical kernels that are critical to the instantiation of theory-inspired analytic workflows on modern computing architectures. We present the benefits of scalable kernels towards constructing algorithms such as principal component analysis and non-negative matrix factorization on an image-analysis use case at the Oak Ridge Leadership Computing Facility (OLCF). Based on experience with the use-case, we conclude that piecing scalable analytic kernels into user-defined analytic workflows are a flexible, modular and agile way to enable architecture-portable productivity for the data-intensive sciences.
Sukumar, Sreenivas R.↗
Gamma ray astronomy
The Burst and Transient Source Experiment (BATSE) is one of four instruments on the Compton observatory which was launched by the space shuttle Atlantis on April 5, 1991. As of mid-March, 1994, BATSE detected more than 925 cosmic gamma-ray bursts and more than 725 solar flares. Pulsed gamma rays have been detected from at least 16 sources and emission from at least 28 sources (including most of the pulsed sources) has been detected by the earth occultation technique. UAH participation in BATSE is extensive but can be divided into two main areas, operations and data analysis. The daily BATSE operations tasks represent a substantial level of effort and involve a large team composed of MSFC personnel as well as contractors such as UAH. The scientific data reduction and analysis of BATSE data is also a substantial level of effort in which UAH personnel have made significant contributions.
Paciesas, William S.↗
DataHub - Science data management in support of interactive exploratory analysis
DataHub addresses four areas of significant need: scientific visualization and analysis; science data management; interactions in a distributed, heterogeneous environment; and knowledge-based assistance for these functions. The fundamental innovation embedded within the DataHub is the integration of three technologies, viz. knowledge-based expert systems, science visualization, and science data management. This integration is based on a concept called the DataHub. With the DataHub concept, science investigators are able to apply a more complete solution to all nodes of a distributed system. Both computational nodes and interactive nodes are able to effectively and efficiently use the data services (access, retrieval, update, etc.) in a distributed, interdisciplinary information system in a uniform and standard way. This allows the science investigators to concentrate on their scientific endeavors, rather than to involve themselves in the intricate technical details of the systems and tools required to accomplish their work. Thus, science investigators need not be programmers. The emphasis is on the definition and prototyping of system elements with sufficient detail to enable data analysis and interpretation leading to information. The DataHub includes all the required end-to-end components and interfaces to demonstrate the complete concept.
Handley, Thomas H., Jr.↗
Issues in knowledge representation to support maintainability: A case study in scientific data preparation
Scientific data preparation is the process of extracting usable scientific data from raw instrument data. This task involves noise detection (and subsequent noise classification and flagging or removal), extracting data from compressed forms, and construction of derivative or aggregate data (e.g. spectral densities or running averages). A software system called PIPE provides intelligent assistance to users developing scientific data preparation plans using a programming language called Master Plumber. PIPE provides this assistance capability by using a process description to create a dependency model of the scientific data preparation plan. This dependency model can then be used to verify syntactic and semantic constraints on processing steps to perform limited plan validation. PIPE also provides capabilities for using this model to assist in debugging faulty data preparation plans. In this case, the process model is used to focus the developer's attention upon those processing steps and data elements that were used in computing the faulty output values. Finally, the dependency model of a plan can be used to perform plan optimization and runtime estimation. These capabilities allow scientists to spend less time developing data preparation procedures and more time on scientific analysis tasks. Because the scientific data processing modules (called fittings) evolve to match scientists' needs, issues regarding maintainability are of prime importance in PIPE. This paper describes the PIPE system and describes how issues in maintainability affected the knowledge representation used in PIPE to capture knowledge about the behavior of fittings.
Chien, Steve↗
DataHub: Science data management in support of interactive exploratory analysis
The DataHub addresses four areas of significant needs: scientific visualization and analysis; science data management; interactions in a distributed, heterogeneous environment; and knowledge-based assistance for these functions. The fundamental innovation embedded within the DataHub is the integration of three technologies, viz. knowledge-based expert systems, science visualization, and science data management. This integration is based on a concept called the DataHub. With the DataHub concept, science investigators are able to apply a more complete solution to all nodes of a distributed system. Both computational nodes and interactives nodes are able to effectively and efficiently use the data services (access, retrieval, update, etc), in a distributed, interdisciplinary information system in a uniform and standard way. This allows the science investigators to concentrate on their scientific endeavors, rather than to involve themselves in the intricate technical details of the systems and tools required to accomplish their work. Thus, science investigators need not be programmers. The emphasis on the definition and prototyping of system elements with sufficient detail to enable data analysis and interpretation leading to information. The DataHub includes all the required end-to-end components and interfaces to demonstrate the complete concept.
Handley, Thomas H., Jr.↗
Crosscut report: Exascale Requirements Reviews, March 9–10, 2017 – Tysons Corner, Virginia. An Office of Science review sponsored by: Advanced Scientific Computing Research, Basic Energy Sciences, Biological and Environmental Research, Fusion Energy Sciences, High Energy Physics, Nuclear Physics
The mission of the U.S. Department of Energy Office of Science (DOE SC) is the delivery of scientific discoveries and major scientific tools to transform our understanding of nature and to advance the energy, economic, and national security missions of the United States. To achieve these goals in today’s world requires investments in not only the traditional scientific endeavors of theory and experiment, but also in computational science and the facilities that support large-scale simulation and data analysis. The Advanced Scientific Computing Research (ASCR) program addresses these challenges in the Office of Science. ASCR’s mission is to discover, develop, and deploy computational and networking capabilities to analyze, model, simulate, and predict complex phenomena important to DOE. ASCR supports research in computational science, three high-performance computing (HPC) facilities — the National Energy Research Scientific Computing Center (NERSC) at Lawrence Berkeley National Laboratory and Leadership Computing Facilities at Argonne (ALCF) and Oak Ridge (OLCF) National Laboratories — and the Energy Sciences Network (ESnet) at Berkeley Lab. ASCR is guided by science needs as it develops research programs, computers, and networks at the leading edge of technologies. As we approach the era of exascale computing, technology changes are creating challenges for science programs in SC for those who need to use high performance computing and data systems effectively. Numerous significant modifications to today’s tools and techniques will be needed to realize the full potential of emerging computing systems and other novel computing architectures. To assess these needs and challenges, ASCR held a series of Exascale Requirements Reviews in 2015–2017, one with each of the six SC program offices,1 and a subsequent Crosscut Review that sought to integrate the findings from each. Participants at the reviews were drawn from the communities of leading domain scientists, experts in computer science and applied mathematics, ASCR facility staff, and DOE program managers in ASCR and the respective program offices. The purpose of these reviews was to identify mission-critical scientific problems within the DOE Office of Science (including experimental facilities) and determine the requirements for the exascale ecosystem that would be needed to address those challenges. The exascale ecosystem includes exascale computing systems, high-end data capabilities, efficient software at scale, libraries, tools, and other capabilities. This effort will contribute to the development of a strategic roadmap for ASCR compute and data facility investments and will help the ASCR Facility Division establish partnerships with Office of Science stakeholders. It will also inform the Office of Science research needs and agenda. The results of the six reviews have been published in reports available on the web at http://exascaleage.org/. This report presents a summary of the individual reports and of common and crosscutting findings, and it identifies opportunities for productive collaborations among the DOE SC program offices.
97 MATHEMATICS AND COMPUTING↗
Insufficient reporting of x-ray photoelectron spectroscopy instrumental and peak fitting parameters (metadata) in the scientific literature
This study was motivated by earlier observations. It is a systematic examination of the adequacy of reporting of information (metadata) necessary to understand x-ray photoelectron spectroscopy (XPS) data collection and data analysis in the scientific literature. The information for this study was obtained from papers published in three high-quality journals over a six-month period in 2019 and throughout 2021. Each paper was evaluated to determine whether the authors had reported (percentages of the papers properly providing the information are given in parentheses) the spectrometer (66%), fitting software (15%), x-ray source (40%), pass energy (10%), spot size (5%), synthetic peak shapes in fits (10%), backgrounds in fits (10%), whether the XPS data are shown in the main body of the paper or in the supporting information (or both), and whether fitted or unfitted spectra were shown (80% of published spectra are fit). The Shirley background is the most widely used background in XPS peak fitting. The Al Kα source is the most widely used x-ray source for XPS data collection. CASAXPS is the most widely used fitting program for XPS data analysis. Further, there is good agreement between the results gathered during the two years of our survey. There are some hints the situation may be improving. This study also provides a list of the information/parameters that should be reported when XPS is performed.
47 OTHER INSTRUMENTATION↗
Planetary astronomy of Mars. Mars data reduction
Grant #NAGW-1408 was specifically awarded to obtain telescopic visible and near-IR spectral imaging of Mars during the 1988 apparition (9/28/88). The observing program was highly successful producing approximately 2 Gbytes of data, but was only funded for one year and virtually all of the funds were spent in data acquisition. The follow-up grant was funded the following year for reduction of these data into a scientifically productive form, which because of the size and nature of our observations, was a non-trivial task. A more detailed scientific analysis of these data (fully reduced) is in progress now and will take a number of years. Extended geologic analyses of the astronomical data are being funded by the NASA Planetary Geology and Geophysics program. The objective was to produce detailed reflectance spectra for contiguous, spatially resolved surface elements covering most of the planet (about +50 degrees to -90 degrees latitude, all longitudes). A total of 6 observing runs, of 3-4 days duration each, were conducted on the University of Arizona's 1.5m telescope on Mt. Bigelow. We present a sketch map showing the approximate total extent of our spectral image coverage. Nearly all of Mars south of 40 degrees N was observed at least once. About half of the area shown was observed multiple times. South of 65 degrees S, including the south polar cap, our coverage is heavily redundant. The first run was conducted June 29 - July 1, 1988 (all dates are UT) to serve as a baseline prior to possible dust storm activity on Mars. The other observing runs were closer to opposition: Sept. 3-6, 13-15, 24-26, and Oct. 5-7 and 16-18. The September and October observations were scheduled to provide maximum longitudinal coverage. This was also intended to provide a balance between surface observations and observations of predicted dust storm activity. No global dust storm developed in 1988, so we have a large volume of data for the surface of Mars.
Singer, Robert B.↗
Building MCP-native hierarchical AI scientist ecosystems: a perspective on scaling multi-agent scientific discovery
Large language models (LLMs) are evolving from chatbots with limited tool-using capabilities to agentic AI systems that can perform deep research, assist in proposing hypotheses, help design experiments, automate data analysis, and draft scientific reports. However, there are currently two bottlenecks limiting LLMs' real-world impact on the broader scientific research community beyond academic demonstrations: lack of interoperability (repetitive manual tool-integration is required across scenarios) and the need for scalable coordination (unstructured communication and memory become brittle as the number of agents grows). In this Perspective, we argue that the next phase of agentic scientific discovery requires the development of an ecosystem of protocol-native agents and tools organized through hierarchies inspired by human society, beyond the current paradigm of a single monolithic “AI scientist”. We use Model Context Protocol (MCP) as a concrete example of an emerging interoperability layer for scientific tool and context exchange, and we propose three complementary pathways to increase the scaling capabilities of an MCP-native scientific ecosystem by addressing the composability issues: (1) MCP servers for high-value scientific tools maintained by domain experts, (2) automated transformation of existing code repositories into MCP services, and (3) autonomous invention and evolution of new agents and workflows. Finally, we provide a practical roadmap for scaling AI-driven scientific discovery by expanding tool supply and coordination in MCP-native scientific ecosystems.
97 MATHEMATICS AND COMPUTING↗
Reliability Lessons Learned From GPU Experience With The Titan Supercomputer at Oak Ridge Leadership Computing Facility
The high computational capability of graphics processing units (GPUs) is enabling and driving the scientific discovery process at large-scale. The world’s second fastest supercomputer for open science, Titan, has more than 18,000 GPUs that computational scientists use to perform scientific simu- lations and data analysis. Understanding of GPU reliability characteristics, however, is still in its nascent stage since GPUs have only recently been deployed at large-scale. This paper presents a detailed study of GPU errors and their impact on system operations and applications, describing experiences with the 18,688 GPUs on the Titan supercom- puter as well as lessons learned in the process of efficient operation of GPUs at scale. These experiences are helpful to HPC sites which already have large-scale GPU clusters or plan to deploy GPUs in the future.
Tiwari, Devesh↗
Uncovering interpretable relationships in high-dimensional scientific data through function preserving projections
Abstract In many fields of science and engineering, we frequently encounter experiments or simulations datasets that describe the behavior of complex systems and uncovering human interpretable patterns between their inputs and outputs via exploratory data analysis is essential for building intuition and facilitating discovery. Often, we resort to 2D embeddings for examining these high-dimensional relationships (e.g. dimensionality reduction). However, most existing embedding methods treat the dimensions as coordinates for samples in a high-dimensional space, which fail to capture the potential functional relationships, and the few methods that do take function into consideration either only focus on linear patterns or produce non-linear embeddings that are hard to interpret. To address these challenges, we proposed function preserving projections (FPP), which construct 2D linear embeddings optimized to reveal interpretable yet potentially non-linear patterns between the domain and the range of a high-dimensional function. The intuition here is that humans are good at understanding potentially non-linear patterns in 2D but unable to interpret non-linear mapping from high-dimensional space to 2D. Therefore, we should restrict the projection to linear but not the pattern we are seeking. Using FPP on real-world datasets, one can obtain fundamentally new insights about high-dimensional relationships in extremely large datasets that could not be processed with existing dimension reduction methods.
97 MATHEMATICS AND COMPUTING↗
High-performance End-to-End Integrity Verification on Big Data Transfer
The scale of scientific data generated by experimental facilities and simulations in high-performance computing facilities has been proliferating with the emergence of IoT-based big data. In many cases, this data must be transmitted rapidly and reliably to remote facilities for storage, analysis, or sharing, for the Internet of Things (IoT) applications. Simultaneously, IoT data can be verified using a checksum after the data has been written to the disk at the destination to ensure its integrity. However, this end-to-end integrity verification inevitably creates overheads (extra disk I/O and more computation). Thus, the overall data transfer time increases. In this article, we evaluate strategies to maximize the overlap between data transfer and checksum computation for astronomical observation data. Specifically, we examine file-level and block-level (with various block sizes) pipelining to overlap data transfer and checksum computation. We analyze these pipelining approaches in the context of GridFTP, a widely used protocol for scientific data transfers. Theoretical analysis and experiments are conducted to evaluate our methods. The results show that block-level pipelining is effective in maximizing the overlap mentioned above, and can improve the overall data transfer time with end-to-end integrity verification by up to 70% compared to the sequential execution of transfer and checksum, and by up to 60% compared to file-level pipelining.
data integrity↗
DataHub: Science Data Management in Support of Interactive Exploratory Analysis
The DataHub addresses four areas of significant need: scientific visualization and analysis; science data management; interactions in a distributed, heterogeneous environment; and knowledge-based assistance for these functions. The fundamental innovation embedded within the DataHub is the integration of three technologies, videlicet knowledge-based expert systems, science visualization, and science data management. With the DataHub concept, science investigators are able to apply a more complete solution to all nodes of a distributed system.
science↗
Data processing for a cosmic ray experiment onboard the solar probes Helios 1 and 2: Experiment 6
The data processing system for the Helios experiment 6, measuring energetic charged particles of solar, planetary and galactic origin in the inner solar system, is described. The aim of this experiment is to extend knowledge on origin and propagation of cosmic rays. The different programs for data reduction, analysis, presentation, and scientific evaluation are described as well as hardware and software of the data processing equipment. A chronological presentation of the data processing operation is given. Procedures and methods for data analysis which were developed can be used with minor modifications for analysis of other space research experiments.
Mueller-Mellin, R.↗
Analysis Report for Exascale Storage Requirements for Scientific Data.
Over the next 10 years, the Department of Energy will be transitioning from Petascale to Exascale Computing resulting in data storage, networking, and infrastructure requirements to increase by three orders of magnitude. The technologies and best practices used today are the result of a relatively slow evolution of ancestral technologies developed in the 1950s and 1960s. These include magnetic tape, magnetic disk, networking, databases, file systems, and operating systems. These technologies will continue to evolve over the next 10 to 15 years on a reasonably predictable path. Experience with the challenges involved in transitioning these fundamental technologies from Terascale to Petascale computing systems has raised questions about how these will scale another 3 or 4 orders of magnitude to meet the requirements imposed by Exascale computing systems. This report is focused on the most concerning scaling issues with data storage systems as they relate to High Performance Computing- and presents options for a path forward. Given the ability to store exponentially increasing amounts of data, far more advanced concepts and use of metadata will be critical to managing data in Exascale computing systems.
97 MATHEMATICS AND COMPUTING↗
OGO 3 data analysis - Dynamic spectra of solar bursts Final scientific report
Solar burst time profiles and dynamic spectra for calculating theoretical values for Type 3 burst characteristics
Graedel, T. E.↗
Implications of Numerical and Data Intensive Technology Trends on Scientific Visualization and Analysis
Abstract Not Provided
Computer Science↗