Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Scientific Data Analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
NASA NTRS1990

Planetary astronomy of Mars. Mars data reduction

Grant #NAGW-1408 was specifically awarded to obtain telescopic visible and near-IR spectral imaging of Mars during the 1988 apparition (9/28/88). The observing program was highly successful producing approximately 2 Gbytes of data, but was only funded for one year and virtually all of the funds were spent in data acquisition. The follow-up grant was funded the following year for reduction of these data into a scientifically productive form, which because of the size and nature of our observations, was a non-trivial task. A more detailed scientific analysis of these data (fully reduced) is in progress now and will take a number of years. Extended geologic analyses of the astronomical data are being funded by the NASA Planetary Geology and Geophysics program. The objective was to produce detailed reflectance spectra for contiguous, spatially resolved surface elements covering most of the planet (about +50 degrees to -90 degrees latitude, all longitudes). A total of 6 observing runs, of 3-4 days duration each, were conducted on the University of Arizona's 1.5m telescope on Mt. Bigelow. We present a sketch map showing the approximate total extent of our spectral image coverage. Nearly all of Mars south of 40 degrees N was observed at least once. About half of the area shown was observed multiple times. South of 65 degrees S, including the south polar cap, our coverage is heavily redundant. The first run was conducted June 29 - July 1, 1988 (all dates are UT) to serve as a baseline prior to possible dust storm activity on Mars. The other observing runs were closer to opposition: Sept. 3-6, 13-15, 24-26, and Oct. 5-7 and 16-18. The September and October observations were scheduled to provide maximum longitudinal coverage. This was also intended to provide a balance between surface observations and observations of predicted dust storm activity. No global dust storm developed in 1988, so we have a large volume of data for the surface of Mars.

Singer, Robert B.↗
DOE OSTI2026

Building MCP-native hierarchical AI scientist ecosystems: a perspective on scaling multi-agent scientific discovery

Large language models (LLMs) are evolving from chatbots with limited tool-using capabilities to agentic AI systems that can perform deep research, assist in proposing hypotheses, help design experiments, automate data analysis, and draft scientific reports. However, there are currently two bottlenecks limiting LLMs' real-world impact on the broader scientific research community beyond academic demonstrations: lack of interoperability (repetitive manual tool-integration is required across scenarios) and the need for scalable coordination (unstructured communication and memory become brittle as the number of agents grows). In this Perspective, we argue that the next phase of agentic scientific discovery requires the development of an ecosystem of protocol-native agents and tools organized through hierarchies inspired by human society, beyond the current paradigm of a single monolithic “AI scientist”. We use Model Context Protocol (MCP) as a concrete example of an emerging interoperability layer for scientific tool and context exchange, and we propose three complementary pathways to increase the scaling capabilities of an MCP-native scientific ecosystem by addressing the composability issues: (1) MCP servers for high-value scientific tools maintained by domain experts, (2) automated transformation of existing code repositories into MCP services, and (3) autonomous invention and evolution of new agents and workflows. Finally, we provide a practical roadmap for scaling AI-driven scientific discovery by expanding tool supply and coordination in MCP-native scientific ecosystems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

Reliability Lessons Learned From GPU Experience With The Titan Supercomputer at Oak Ridge Leadership Computing Facility

The high computational capability of graphics processing units (GPUs) is enabling and driving the scientific discovery process at large-scale. The world’s second fastest supercomputer for open science, Titan, has more than 18,000 GPUs that computational scientists use to perform scientific simu- lations and data analysis. Understanding of GPU reliability characteristics, however, is still in its nascent stage since GPUs have only recently been deployed at large-scale. This paper presents a detailed study of GPU errors and their impact on system operations and applications, describing experiences with the 18,688 GPUs on the Titan supercom- puter as well as lessons learned in the process of efficient operation of GPUs at scale. These experiences are helpful to HPC sites which already have large-scale GPU clusters or plan to deploy GPUs in the future.

Tiwari, Devesh↗
DOE OSTI2020

Uncovering interpretable relationships in high-dimensional scientific data through function preserving projections

Abstract In many fields of science and engineering, we frequently encounter experiments or simulations datasets that describe the behavior of complex systems and uncovering human interpretable patterns between their inputs and outputs via exploratory data analysis is essential for building intuition and facilitating discovery. Often, we resort to 2D embeddings for examining these high-dimensional relationships (e.g. dimensionality reduction). However, most existing embedding methods treat the dimensions as coordinates for samples in a high-dimensional space, which fail to capture the potential functional relationships, and the few methods that do take function into consideration either only focus on linear patterns or produce non-linear embeddings that are hard to interpret. To address these challenges, we proposed function preserving projections (FPP), which construct 2D linear embeddings optimized to reveal interpretable yet potentially non-linear patterns between the domain and the range of a high-dimensional function. The intuition here is that humans are good at understanding potentially non-linear patterns in 2D but unable to interpret non-linear mapping from high-dimensional space to 2D. Therefore, we should restrict the projection to linear but not the pattern we are seeking. Using FPP on real-world datasets, one can obtain fundamentally new insights about high-dimensional relationships in extremely large datasets that could not be processed with existing dimension reduction methods.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

High-performance End-to-End Integrity Verification on Big Data Transfer

The scale of scientific data generated by experimental facilities and simulations in high-performance computing facilities has been proliferating with the emergence of IoT-based big data. In many cases, this data must be transmitted rapidly and reliably to remote facilities for storage, analysis, or sharing, for the Internet of Things (IoT) applications. Simultaneously, IoT data can be verified using a checksum after the data has been written to the disk at the destination to ensure its integrity. However, this end-to-end integrity verification inevitably creates overheads (extra disk I/O and more computation). Thus, the overall data transfer time increases. In this article, we evaluate strategies to maximize the overlap between data transfer and checksum computation for astronomical observation data. Specifically, we examine file-level and block-level (with various block sizes) pipelining to overlap data transfer and checksum computation. We analyze these pipelining approaches in the context of GridFTP, a widely used protocol for scientific data transfers. Theoretical analysis and experiments are conducted to evaluate our methods. The results show that block-level pipelining is effective in maximizing the overlap mentioned above, and can improve the overall data transfer time with end-to-end integrity verification by up to 70% compared to the sequential execution of transfer and checksum, and by up to 60% compared to file-level pipelining.

data integrity↗
NASA NTRS1993

DataHub: Science Data Management in Support of Interactive Exploratory Analysis

The DataHub addresses four areas of significant need: scientific visualization and analysis; science data management; interactions in a distributed, heterogeneous environment; and knowledge-based assistance for these functions. The fundamental innovation embedded within the DataHub is the integration of three technologies, videlicet knowledge-based expert systems, science visualization, and science data management. With the DataHub concept, science investigators are able to apply a more complete solution to all nodes of a distributed system.

science↗
NASA NTRS1982

Data processing for a cosmic ray experiment onboard the solar probes Helios 1 and 2: Experiment 6

The data processing system for the Helios experiment 6, measuring energetic charged particles of solar, planetary and galactic origin in the inner solar system, is described. The aim of this experiment is to extend knowledge on origin and propagation of cosmic rays. The different programs for data reduction, analysis, presentation, and scientific evaluation are described as well as hardware and software of the data processing equipment. A chronological presentation of the data processing operation is given. Procedures and methods for data analysis which were developed can be used with minor modifications for analysis of other space research experiments.

Mueller-Mellin, R.↗
DOE OSTI2018

Analysis Report for Exascale Storage Requirements for Scientific Data.

Over the next 10 years, the Department of Energy will be transitioning from Petascale to Exascale Computing resulting in data storage, networking, and infrastructure requirements to increase by three orders of magnitude. The technologies and best practices used today are the result of a relatively slow evolution of ancestral technologies developed in the 1950s and 1960s. These include magnetic tape, magnetic disk, networking, databases, file systems, and operating systems. These technologies will continue to evolve over the next 10 to 15 years on a reasonably predictable path. Experience with the challenges involved in transitioning these fundamental technologies from Terascale to Petascale computing systems has raised questions about how these will scale another 3 or 4 orders of magnitude to meet the requirements imposed by Exascale computing systems. This report is focused on the most concerning scaling issues with data storage systems as they relate to High Performance Computing- and presents options for a path forward. Given the ability to store exponentially increasing amounts of data, far more advanced concepts and use of metadata will be critical to managing data in Exascale computing systems.

97 MATHEMATICS AND COMPUTING↗
NASA NTRSDate not supplied

Leveraging the 2024 Solar Eclipse to Enhance Science Learning through Citizen Science

The solar eclipse on April 8, 2024, provided a rare and compelling opportunity for educators and students to engage in hands-on citizen science. Two NASA Science Activation projects GLOBE Mission Earth (GME) and NASA Earth Science Education Collaborative (NESEC) partnered together to provide a unique professional development experience for educators interested in the eclipse. They invited educators to participate in a specialized 5-week online workshop designed to engage the educators in Global Learning and Observations to Benefit the Environment (GLOBE) citizen science about the eclipse. The workshop was designed to support their certification as GLOBE educators and develop their knowledge, skills, and confidence in conducting a GLOBE investigation. Over 60 educators from across the United States took part in this workshop, which focused on investigating the eclipse by collecting and analyzing atmospheric data. The GLOBE Program promotes environmental and scientific literacy by enabling participants to collect Earth science data and contribute it to a global database accessible for research. The 2024 GLOBE Eclipse workshop trained educators in specific GLOBE protocols—Clouds, Air Temperature, and Surface Temperature—using the GLOBE Eclipse tool integrated into the GLOBE Observer app. This training was supplemented with guidance on engaging students in authentic scientific research and creating research posters to present their findings. The workshop aimed to enhance educators’ skills and confidence in integrating GLOBE protocols into their teaching practices. It addressed several key areas outlined in the National Academies' report on Learning through Citizen Science, including scientific context, nature of participation, and project infrastructure. Educators learned about the atmospheric effects of solar eclipses and were trained to use scientific tools and data analysis methods relevant to their research. They also engaged in live sessions and asynchronous activities, practicing data collection and analysis with real-time feedback. Survey results from the workshop highlighted that participants were primarily motivated by the desire to better use data in their classrooms and improve their proficiency with the GLOBE Observer app. Post-workshop evaluations showed significant increases in educators' confidence regarding their ability to conduct and guide scientific research. Participants reported feeling well-prepared to use the GLOBE Observer app for data collection and were successful in integrating their eclipse observations into classroom activities. Participants also valued the opportunity to contribute to authentic science through GLOBE, which involved observing atmospheric changes such as air temperature fluctuations and cloud cover alterations during the eclipse. The success of the workshop is evident in the increased confidence and skill levels of educators, as well as the publication of research posters on the GLOBE Mission Earth Student Research webpage.This session will discuss how the GLOBE Eclipse workshop series effectively utilized the unique context of the solar eclipse to enhance science education through citizen science. By adhering to the principles outlined in the Learning through Citizen Science framework, the workshop successfully engaged educators and students in meaningful scientific practices, demonstrating the potential of citizen science to enrich science education and foster a deeper understanding of Earth systems.

Jessica Taylor↗
DOE OSTI2016

CHANDRA ACIS SURVEY OF X-RAY POINT SOURCES: THE SOURCE CATALOG

The Chandra archival data is a valuable resource for various studies on different X-ray astronomy topics. In this paper, we utilize this wealth of information and present a uniformly processed data set, which can be used to address a wide range of scientific questions. The data analysis procedures are applied to 10,029 Advanced CCD Imaging Spectrometer observations, which produces 363,530 source detections belonging to 217,828 distinct X-ray sources. This number is twice the size of the Chandra Source Catalog (Version 1.1). The catalogs in this paper provide abundant estimates of the detected X-ray source properties, including source positions, counts, colors, fluxes, luminosities, variability statistics, etc. Cross-correlation of these objects with galaxies shows that 17,828 sources are located within the D {sub 25} isophotes of 1110 galaxies, and 7504 sources are located between the D {sub 25} and 2 D {sub 25} isophotes of 910 galaxies. Contamination analysis with the log N –log S relation indicates that 51.3% of objects within 2 D {sub 25} isophotes are truly relevant to galaxies, and the “net” source fraction increases to 58.9%, 67.3%, and 69.1% for sources with luminosities above 10{sup 37}, 10{sup 38}, and 10{sup 39} erg s{sup −1}, respectively. Among the possible scientific uses of this catalog, we discuss the possibility of studying intra-observation variability, inter-observation variability, and supersoft sources (SSSs). About 17,092 detected sources above 10 counts are classified as variable in individual observation with the Kolmogorov–Smirnov (K–S) criterion ( P {sub K–S} < 0.01). There are 99,647 sources observed more than once and 11,843 sources observed 10 times or more, offering us a wealth of data with which to explore the long-term variability. There are 1638 individual objects (∼2350 detections) classified as SSSs. As a quite interesting subclass, detailed studies on X-ray spectra and optical spectroscopic follow-up are needed to categorize these SSSs and pinpoint their properties. In addition, this survey can enable a wide range of statistical studies, such as X-ray activity in different types of stars, X-ray luminosity functions in different types of galaxies, and multi-wavelength identification and classification of different X-ray populations.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2017

Management, Analysis, and Visualization of Experimental and Observational Data – The Convergence of Data and Computing

Scientific user facilities—particle accelerators, telescopes, colliders, supercomputers, light sources, sequencing facilities, and more—operated by the U.S. Department of Energy (DOE) Office of Science (SC) generate ever increasing volumes of data at unprecedented rates from experiments, observations, and simulations. At the same time there is a growing community of experimentalists that require real-time data analysis feedback, to enable them to steer their complex experimental instruments to optimized scientific outcomes and new discoveries. Recent efforts in DOE-SC have focused on articulating the data-centric challenges and opportunities facing these science communities. Key challenges include difficulties coping with data size, rate, and complexity in the context of both real-time and post-experiment data analysis and interpretation. Solutions will require algorithmic and mathematical advances, as well as hardware and software infrastructures that adequately support data-intensive scientific workloads. This paper presents the summary findings of a workshop held by DOE-SC in September 2015, convened to identify the major challenges and the research that is needed to meet those challenges.

Bethel, E. Wes↗
DOE OSTI2016

Management, Analysis, and Visualization of Experimental and Observational Data -- The Convergence of Data and Computing

Scientific user facilities---particle accelerators, telescopes, colliders, supercomputers, light sources, sequencing facilities, and more---operated by the U.S. Department of Energy (DOE) Office of Science (SC) generate ever increasing volumes of data at unprecedented rates from experiments, observations, and simulations. At the same time there is a growing community of experimentalists that require real-time data analysis feedback, to enable them to steer their complex experimental instruments to optimized scientific outcomes and new discoveries. Recent efforts in DOE-SC have focused on articulating the data-centric challenges and opportunities facing these science communities. Key challenges include difficulties coping with data size, rate, and complexity in the context of both real-time and post-experiment data analysis and interpretation. Solutions will require algorithmic and mathematical advances, as well as hardware and software infrastructures that adequately support data-intensive scientific workloads. This paper presents the summary findings of a workshop held by DOE-SC in September 2015, convened to identify the major challenges and the research that is needed to meet those challenges.

Bethel, E. Wes↗
DOE OSTI2019

Anomaly detection in scientific data using joint statistical moments

We propose an anomaly detection method for multi-variate scientific data based on analysis of high-order joint moments. Using kurtosis as a reliable measure of outliers, we suggest that principal kurtosis vectors, by analogy to principal component analysis (PCA) vectors, signify the principal directions along which outliers appear. The inception of an anomaly, then, manifests as a change in the principal values and vectors of kurtosis. Obtaining the principal kurtosis vectors requires decomposing a fourth order joint cumulant tensor for which we use a simple, computationally less expensive approach that involves performing a singular value decomposition (SVD) over the matricized tensor. We demonstrate the efficacy of this approach on synthetic data, and develop an algorithm to identify the occurrence of a spatial and/or temporal anomalous event in scientific phenomena. The algorithm decomposes the data into several spatial sub-domains and time steps to identify regions with such events. Feature moment metrics, based on the alignments of the principal kurtosis vectors, are computed at each sub-domain and time step for all features to quantify their relative importance towards the overall kurtosis in the data. Accordingly, spatial and temporal anomaly metrics for each sub-domain are proposed using the Hellinger distance of the feature moment metric distribution from a suitable nominal distribution. Finally, we apply the algorithm to two turbulent auto-ignition combustion cases and demonstrate that the anomaly metrics reliably capture the occurrence of auto-ignition in relevant spatial sub-domains at the right time steps.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Dark Matter and Super Symmetry: Exploring and Explaining the Universe with Simulations at the LHC

The Large Hadron Collider (LHC) at CERN in Geneva, Switzerland, is one of the largest machines on this planet. It is built to smash protons into each other at unprecedented energies to reveal the fundamental constituents of our universe. The 4 detectors at the LHC record multi-petabyte datasets every year. The scientific analysis of this data requires equally large simulation datasets of the collisions based on the theory of particle physics, the Standard Model. The goal is to verify the validity of the Standard Model or of theories that extend the Model like the concepts of Supersymmetry and an explanation of Dark Matter. I will give an overview of the nature of simulations needed to discover new particles like the Higgs boson in 2012, and review the different areas where simulations are indispensable: from the actual recording of the collisions to the extraction of scientific results to the conceptual design of improvements to the LHC and its experiments.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS2005

Research Opportunities Supporting the Vision for Space Exploration from the Transformation of the Former Microgravity Materials Science Program

The Microgravity Materials Science Program established a strong research capability through partnerships between NASA and the scientific research community. With the announcement of the vision for space exploration, additional emphasis in strategic materials science areas was necessary. The President's Commission recognized that achieving its exploration objectives would require significant technical innovation, research, and development in focal areas defined as "enabling technologies." Among the 17 enabling technologies identified for initial focus were: advanced structures, advanced power and propulsion; closed-loop life support and habitability; extravehicular activity systems; autonomous systems and robotics; scientific data collection and analysis, biomedical risk mitigation; and planetary in situ resource utilization. Mission success may depend upon use of local resources to fabricate a replacement part to repair a critical system. Future propulsion systems will require materials with a wide range of mechanical, thermophysical, and thermochemical properties, many of them well beyond capabilities of today's materials systems. Materials challenges have also been identified by experts working to develop advanced life support systems. In responding to the vision for space exploration, the Microgravity Materials Science Program aggressively transformed its research portfolio and focused materials science areas of emphasis to include space radiation shielding; in situ fabrication and repair for life support systems; in situ resource utilization for life support consumables; and advanced materials for exploration, including materials science for space propulsion systems and for life support systems. The purpose of this paper is to inform the scientific community of these new research directions and opportunities to utilize their materials science expertise and capabilities to support the vision for space exploration.

Clinton, R. G., Jr.↗