Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “scientific data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14
All sourcesNASA NTRSDOE OSTI
NASA NTRS2019

A Delay Tolerant Networking-Based Approach to a High Data Rate Architecture for Spacecraft

Historically, it has been the case that SWaP placed such severe constraints on radios that the links between spacecraft and the ground were relatively slow. This meant that the radio link was normally a significant bottleneck in returning scientific data. Over recent years, however, a combination of more efficient radio design, intelligent waveforms, and highly directed, high-frequency RF / optical systems have led to a rapid increase in the amount of data that can be pushed through radio and optical links. This has led to some cases where the radio links are capable of moving data much more quickly than the spacecraft and instruments are capable of actually generating it! In some instances, scientific data can therefore be lost not because the downlink is too slow to support the data rate, but instead because the spacecraft was not designed in a way that would let it fully utilize both the radio and the networking services available to it.The High Data Rate Architecture (HiDRA) project describes a packet-based approach to building modern, distributed spacecraft systems. It presents a means for spacecraft and other assets to participate in both present and future Delay Tolerant Networks (DTN), while simultaneously ensuring that the asset is able to fully utilize the new, high-speed links that have been seeing more widespread development and deployment in recent years. With this in mind, this paper begins with a discussion regarding HiDRA's evolution. Next, it discusses the capabilities and limitations of NASA's present DTN-enabled networks. Of particular note is the way in which principles of network design at the terrestrial level (e.g. use of programmable networks / software-defined networks, separation between data and control plane, infusion of COTS Ethernet switch chips, etc.) can all be translated into the space environment as well. After this, the paper discusses the design and implementation of a present prototype reference implementation of High-Rate DTN (HDTN), which is intended to demonstrate future high-rate networking concepts as part of a coherent demonstration on the International Space Station (ISS). The goal, of both the research and of this implementation, is to help develop a ready-made toolbox of ideas, approaches, and examples from which mission designers can draw when putting together new missions. Assuming all goes as planned, this should not only work to reduce the cost of individual mission design, but also improve the rate at which science data can be returned for mission participants to review.

Hylton, Alan↗
NASA NTRS2018

Improving NASA Earth Science Data and Information Access Through Natural Language Processing Based Data Analysis and Visualization

NASA: The Research Access initiative is part of the agency's framework for increasing public access to scientific publications and digital scientific data. The initiative follows the release of White House Office of Science and Technology Policy's (OSTP) memorandum "Increasing Access to the Results of Federally Funded Research," to ensure federally funded research is available to the public within one year of publication. NASA answered the mandate by creating an agency plan entitled "NASA Plan for Increasing Access to the Results of Scientific Research" and associated policy, NPD 2230.1, Research Data and Publication Access. Principles in NASA SMD Strategic Plan for Scientific Data and Computing: Continued free and open access to scientific data for any use. Improved ease of use and discoverability. Enhanced science applications and new use cases. Incorporates best practices and "state of the art" through partnerships. Earth Data and Systems are Evolving: Increasing archive and file sizes. More complicated data structures. More user-friendly and data services. What is the future direction?

Liu, Zhong↗
NASA NTRS1999

Interdisciplinary Research Scenario Testing of EOSDIS

During the reporting period, the Principle Investigator (PI) has continued to serve on numerous review panels, task forces and committees with the goal of providing input and guidance for the Earth Observing System Data and Information System (EOSDIS) program at NASA Headquarters and NASA GSFC. In addition, the PI has worked together with personnel at the University of Virginia and the subcontractor (Simpson Weather Associates (SWA)) to continue to evaluate the latest releases of various versions of the user interfaces to the EOSDIS. Finally, as part of the subcontract, SWA has created an on-line Hierarchial Data Format (HDF) tutorial for non-HDF experts, particularly those that will be using EOSDIS and future EOS data products. A summary of these three activities is provided. The topics include: 1) Participation on EODIS Panels and Committees; 2) Evaluation and Tire Kicking of EODIS User Interfaces; and 3) An On-line HDF Tutorial. The report also includes attachments A, B, and C. Attachment A: Report From the May 1999 Science Data Panel. The topics include: 1) Summary of Data Panel Meeting; and 2) Panel's Comments/Recommendations. Attachment B: Survey Requesting Integrated Design Systems (IDS) Teams Input on the Descoping and Rescoping of the EODIS; and Attachment C: An HDF Tutorial for Beginners: EODIS Users and Small Data Providers (HTML Version). The topics include: 1) Tutorial Overview; 2) An introduction to HDF; 3) The HDF Library: Software and Hardware; 4) Methods of Working with HDF Files; 5) Scientific Data API; 6) Attributes and Metadata; 7) Writing a SDS to an HDF file; 8) Obtaining Information on Existing HDF Files; 9) Reading a Scientific Data Set from an HDF file: 10) Example Programs; 11) Browsing and Visualizing HDF Data; and 12) Laboratory (Question and Answer).

Emmitt, G. D.↗
NASA NTRS2019

A Delay Tolerant Networking-Based Approach to a High Data Rate Architecture for Spacecraft

Historically, it has been the case that SWaP (Size, Weigh a lot, And consume Power) placed such severe constraints on radios that the links between spacecraft and the ground were relatively slow. This meant that the radio link was normally a significant bottleneck in returning scientific data. Over recent years, however, a combination of more efficient radio design, intelligent waveforms, and highly directed, high-frequency RF / optical systems have led to a rapid increase in the amount of data that can be pushed through radio and optical links. This has led to some cases where the radio links are capable of moving data much more quickly than the spacecraft and instruments are capable of actually generating it! In some instances, scientific data can therefore be lost not because the downlink is too slow to support the data rate, but instead because the spacecraft was not designed in a way that would let it fully utilize both the radio and the networking services available to it. The High Data Rate Architecture (HiDRA) project describes a packet-based approach to building modern, distributed spacecraft systems. It presents a means for spacecraft and other assets to participate in both present and future Delay Tolerant Networks (DTN), while simultaneously ensuring that the asset is able to fully utilize the new, high-speed links that have been seeing more widespread development and deployment in recent years. With this in mind, this paper begins with a discussion regarding HiDRA's evolution. Next, it discusses the capabilities and limitations of NASA's present DTN-enabled networks. Of particular note is the way in which principles of network designate the terrestrial level (e.g., use of programmable networks / software-defined networks, separation between data and control plane, infusion of COTS (Commercial Off-The-Shelf) Ethernet switch chips, etc.) can all be translated into the space environment as well. After this, the paper discusses the design and implementation of a present prototype reference implementation of High-Rate DTN (HDTN),which is intended to demonstrate future high-rate networking concepts as part of a coherent demonstration on the International Space Station (ISS). The goal, of both the research and of this implementation, is to help develop a ready-made toolbox of ideas, approaches, and examples from which mission designers can draw when putting together new missions. Assuming all goes as planned, this should not only work to reduce the cost of individual mission design, but also improve the rate at which science data can be returned for mission participants to review.

Hylton, Alan↗
DOE OSTI2023

Exploring Future Storage Options for ATLAS at the BNL/SDCC facility

The ATLAS experiment is expected to deliver an unprecedented amount of scientific data in the High Luminosity(HL-LHC) era. As the demand for disk storage capacity in ATLAS continues to rise steadily, the BNL Scientific Data and Computing Center (SDCC) faces challenges in terms of cost implications for maintaining multiple disk copies and adapting to the coming ATLAS storage requirements. To address these challenges, the SDCC Storage team has undertaken a thorough analysis of the ATLAS experiment's requirements, matching them to suitable storage options and strategies, and has explored alternatives to enhance or replace the current storage solution. This paper aims to present the main challenges encountered while supporting big data experiments such as ATLAS. We describe the experiment's specific requirements and priorities, particularly focusing on the critical storage system characteristics of the high-luminosity run and how the key storage components provided by the Storage team work together: the dCache disk storage system; its archival back-end, HPSS; and its OS-level backend Storage. Specifically, we investigate a novel approach to integrate Lustre and XRootD. In this setup, Lustre serves as backend storage and XRootD acts as an access layer frontend, supporting various grid access protocols. Additionally, we also describe the validation and commissioning tests, including the performance comparison between dCache and XRootd. Furthermore, we provide a performance and cost analysis comparing OpenZFS and LINUX MD RAID, evaluate different storage software stacks, and showcase stress tests conducted to validate Third Party Copy (TPC) functionality.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2024

Real-Time Neuromorphic Processing of Spatiotemporal Data for Scientific Discovery

Spiking Neural Networks (SNNs) are brain-inspired computing models incorporating unique temporal dynamics and event-driven processing. Rich dynamics in both space and time offer great challenges and opportunities for efficient processing of sparse spatiotemporal data compared with conventional artificial neural networks (ANNs). Under this context, the goal of this project is to develop spiking neural network based neuromorphic computing to enable energy-efficient real-time learning and processing of spatiotemporal data. This report summarizes the key results on network architecture design, training methods, and SNN hardware acceleration achieved under this project, demonstrating the promise of spiking neural networks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

In situ multi-tier auto-ignition detection applied to dual-fuel combustion simulations

Here we use an anomaly detection methodology that is centered on analyzing fourth-order joint moments (co-kurtosis), particularly focusing on its application in auto-ignition of combustion problems with large numbers of species. Unsupervised anomaly detection is challenging to generalize across problem types and domains. A recent technique, centered on analyzing information in the fourth-order joint moment co-kurtosis, has shown promise, especially for high-dimensional scientific data. In this work we present developments to the co-kurtosis based anomaly detection method needed to make it effective and scalable for large-scale distributed scientific data, such as those generated by massively parallel simulations. An in situ co-kurtosis algorithm is employed as the anomaly detection method for identifying ignition kernels in simulations of turbulent combustion. Here, we extend an existing methodology which identifies regions of the domain where anomalies are present, and add another tier of anomaly detection where the individual samples contributing to the anomaly are identified. We apply this algorithm on-the-fly to a variety of turbulent reacting flow problems and compare it to the widely used (but significantly more expensive) chemical explosive mode analysis (CEMA). We demonstrate the ability of the method to detect and identify the onset of low and high temperature ignition which can be used for computational steering, as chemical and combustion anomalies occur intermittently at spatio-temporal locations unknown a priori. Finally, we apply our lightweight in situ algorithm to an exascale high-fidelity simulation with a total of 2.4 Trillion degrees of freedom, performed using an adaptive mesh refinement solver. Furthermore, through a scalability analysis, we show that the relative computational cost of this in-situ anomaly detection algorithm compared to an iteration of the reacting flow solver is negligible.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1965

Science data processing.

Processing technique for scientific data telemetered to Earth from Mariner IV Mars probe

MARINER IV SPACE PROBE↗
NASA NTRSDate not supplied

A Science-Focused Artificial Intelligence (AI) Responding in Real-Time to New Information: Capability Demonstration for Ocean World Missions

Introduction: Artificial intelligence (AI) has long been considered a potential mechanism to explore increasingly challenging environments, including those with extreme temperatures and pressures, limited communication capabilities, or those with demanding terrain. We posit that missions in extreme environments could deploy an onboard AI focused on science observations and goals in order to augment a traditional concept(s) of operations (ConOps). An onboard AI capability could perform functions such as data analysis in order to make high-level decisions, including prioritized data transmission for analysis by ground-based teams or autonomously-guided follow-on analyses that maximize science return. Such a capability would empower missions to respond to scientific data of interest in real-time; a mission could make observations and perform a preliminary analysis to alert ground-based scientists to an observation of interest, enabling an informed, rapid response from Earth-based teams. Enceladus Case Study for Onboard AI: We are developing an onboard AI capability for real-time telemetry response that formulates and carries-out informed decisions in service to established mission goals, enabling increased science return of a mission. We focus our AI development for use on a constellation of SmallSats orbiting Enceladus. Our Enceladus case study tests autonomous decision-making capabilities in scenarios with complex orbital dynamics, plume ejecta, extreme cold environments, power restrictions, and a requirement to maximize science return for a potential positive detection of life, while critically evaluating the potential for false positives. Telemetry includes simulated scientific data, spacecraft onboard operational data (e.g., position, velocity, and rotation), and engineering hardware performance data. Enceladus SmallSat Constellation. Our constellation includes eight SmallSat spacecraft in an 8:35 resonant orbit-based formation, leveraging Saturn’s gravitational forces to maintain stable orbits with global coverage around Enceladus. To our knowledge, we simulate the first stable configuration of multiple spacecraft in closed orbits around Enceladus, using a full ephemeris force model (Russell and Lara, 2009). Each spacecraft’s orbit will precess, causing an eastward ground track shift (from an orbiter’s perspective) of each spacecraft for each orbit. However, all spacecraft return to their original positions relative to Enceladus after eight Enceladus revolutions around Saturn. We model communication pathways between SmallSats to understand how information would need to be transmitted across the constellation to enable AI-driven decision-making and resource allocation across the fleet. Capability Demonstration. Our simulated capability demonstration inputs position, velocity, and rotation telemetry from our Enceladus-focused constellation simulations, and mass spectrometry data collected from abiotic and biotic laboratory-analog ocean world experiments (Theiling et al., 2018; Theiling, 2021; Da Poian et al., 2023). Data from these experiments are used to simulate MS measurements and different scenarios of science observations for onboard analysis performed on each of the eight spacecraft. For these demonstrations, we integrate 24 machine learning (ML) algorithms into an onboard intelligence as a ‘knowledge base’, including algorithms evaluating data quality and those predicting (with % confidence) gas composition, ocean aqueous chemistry, and whether the sample was influenced by microbial life. The onboard AI capability is designed to use the knowledge base to come to a consensus-based decision in the interpretation of the observed data in order to request additional action outside of a pre-defined ConOps. Requested actions could include e.g., prioritized downlink to Earth (for analysis by ground-based teams) or follow-on analyses performed across the constellation. The spacecraft’s intelligent onboard planner must then determine whether sufficient resources (e.g., time, power, etc.) are available and weigh the request with mission priorities. In our simulation, the constellation is able to identify potential biosignatures using onboard ML algorithms, evaluate the confidence of that prediction, and perform follow-on analyses across the fleet to confirm the detection, in order to best prepare a transmission of these data to Earth-based teams.

astrobiology↗
NASA NTRS1985

International Ultraviolet Explorer Observatory operations

This volume contains the final report for the International Ultraviolet Explorer IUE Observatory Operations contract. The fundamental operational objective of the International Ultraviolet Explorer (IUE) program is to translate competitively selected observing programs into IUE observations, to reduce these observations into meaningful scientific data, and then to present these data to the Guest Observer in a form amenable to the pursuit of scientific research. The IUE Observatory is the key to this objective since it is the central control and support facility for all science operations functions within the IUE Project. In carrying out the operation of this facility, a number of complex functions were provided beginning with telescope scheduling and operation, proceeding to data processing, and ending with data distribution and scientific data analysis. In support of these critical-path functions, a number of other significant activities were also provided, including scientific instrument calibration, systems analysis, and software support. Routine activities have been summarized briefly whenever possible.

Source record↗
DOE OSTI2025

Label-based Virtual Directories In dCache

Traditional filesystems organize data in directories. These directories are typically a collection of files whose grouping is based on a single criterion, e.g., the starting date of an experiment, experiment name, beamline ID, measurement device, or instrument. However, each file in a directory can belong to several logical groups, such as a special event type, experiment condition, or a part of a selected dataset. dCache is a storage system developed to store large amounts of scientific data, used by many HEP and Photon Science experiments. With recent developments in dCache, we have introduced a concept of file tagging, which dynamically groups files with the same label into virtual directories. The file labels can be added, removed, renamed, and deleted through the admin interface or via REST API. The files in virtual directories are exposed through all protocols supported by dCache. This contribution will describe the details of the implementation for file tagging in dCache and present our future development plans on automatic metadata extractions, a feature that will significantly simplify data management. Additionally, we are exploring the future use of virtual directories as a way to translate scientific data catalogs into filesystem views for direct data analysis.

Sahakyan, Marina [DESY]↗
DOE OSTI2021

Robustness of deep learning algorithms in astronomy -- galaxy morphology studies

Deep learning models are being increasingly adopted in wide array of scientific domains, especially to handle high-dimensionality and volume of the scientific data. However, these models tend to be brittle due to their complexity and overparametrization, especially to the inadvertent adversarial perturbations that can appear due to common image processing such as compression or blurring that are often seen with real scientific data. It is crucial to understand this brittleness and develop models robust to these adversarial perturbations. To this end, we study the effect of observational noise from the exposure time, as well as the worst case scenario of a one-pixel attack as a proxy for compression or telescope errors on performance of ResNet18 trained to distinguish between galaxies of different morphologies in LSST mock data. We also explore how domain adaptation techniques can help improve model robustness in case of this type of naturally occurring attacks and help scientists build more trustworthy and stable models.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS2009

Interference-Detection Module in a Digital Radar Receiver

A digital receiver in a 1.26-GHz spaceborne radar scatterometer now undergoing development includes a module for detecting radio-frequency interference (RFI) that could contaminate scientific data intended to be acquired by the scatterometer. The role of the RFI-detection module is to identify time intervals during which the received signal is likely to be contaminated by RFI and thereby to enable exclusion, from further scientific data processing, of signal data acquired during those intervals. The underlying concepts of detection of RFI and rejection of RFI-contaminated signal data are also potentially applicable in advanced terrestrial radio receivers, including software-defined radio receivers in general, receivers in cellular telephones and other wireless consumer electronic devices, and receivers in automotive collision-avoidance radar systems.

Fischman, Mark↗
DOE OSTI2021

Improving Performance of Tape Restore Request Scheduling in the Storage System dCache

Given the anticipated increase in the amount of scientific data, it is widely accepted that primarily disk based storage will become prohibitively expensive. Tape based storage, on the other hand, provides a viable and affordable solution for the ever increasing demand for storage space. Coupled with a disk caching layer that temporarily holds a small fraction of the total data volume to allow for low latency access, it turns tape based systems into active archival storage (write once, read many) that imposes additional demands on data flow optimization compared to traditional backup setups (write once, read never). In order to preserve the lifetime of tapes and minimize the inherently higher access latency, different tape usage strategies are being evaluated. As an important disk storage system for scientific data that transparently handles tape access, dCache is making efforts to evaluate its recall optimization potential and is introducing a proof-of-concept, high-level stage request scheduling component within its SRM implementation.

Morschel, Lea↗
NASA NTRS2002

Radiometric Characterization of IKONOS Multispectral Imagery

A radiometric characterization of Space Imaging's IKONOS 4-m multispectral imagery has been performed by a NASA funded team from the John C. Stennis Space Center (SSC), the University of Arizona Remote Sensing Group (UARSG), and South Dakota State University (SDSU). Both intrinsic radiometry and the effects of Space Imaging processing on radiometry were investigated. Relative radiometry was examined with uniform Antarctic and Saharan sites. Absolute radiometric calibration was performed using reflectance-based vicarious calibration methods on several uniform sites imaged by IKONOS, coincident with ground-based surface and atmospheric measurements. Ground-based data and the IKONOS spectral response function served as input to radiative transfer codes to generate a Top-of-Atmosphere radiance estimate. Calibration coefficients derived from each vicarious calibration were combined to generate an IKONOS radiometric gain coefficient for each multispectral band assuming a linear response over the full dynamic range of the instrument. These calibration coefficients were made available to Space Imaging, which subsequently adopted them by updating its initial set of calibration coefficients. IKONOS imagery procured through the NASA Scientific Data Purchase program is processed with or without a Modulation Transfer Function Compensation kernel. The radiometric effects of this kernel on various scene types was also investigated. All imagery characterized was procured through the NASA Scientific Data Purchase program.

Pagnutti, Mary↗
DOE OSTI2024

Tuning the Interpolation Basis in a Multigrid Decomposition for Local Error Control

In the compression of scientific data, error-controlled compressors enable to considerably decrease the size of the dataset while maintaining adequate levels of accuracy. In this paper, we note that multi-level refactoring scheme such as MGARD i) rely on an approximation of the data based on the interpolation of coefficients, ii) estimate the resulting error with global metrics on the dataset. To improve on these two aspects, we propose a method that aims to divide the original dataset into blocks based on their smoothness and refactors each block separately with the most relevant interpolation order. We show the relevance of such a method on tailored datasets and the benefits and challenges when applying it to large scientific data.

Vidal, Nicolas [ORNL]↗
DOE OSTI2021

Fostering Remote Visualization: Experiences in Two Different HPC Sites

Visualization of scientific data is crucial for scientific discovery to gain insight into the results of simulations and experiments. Remote visualization is of crucial importance to access infrastructure, data and computational resources and, to avoid data movement from where data is produced and to where data will be analyzed. Remote visualization enables geographically diverse collaboration and enhances user experience through graphical user interfaces. This paper presents two approaches deployed by two different HPC centers: The SC3 - Supercomputación y Cálculo Científico Center in Colombia and the Oak Ridge Leadership Computing Facility in USA. We overview our remote visualization experiences, adopted technologies, use cases, and challenges encountered. Our contribution is to signal the commonality between approaches in terms of the end goal, showing their fitness for their contexts, while not focusing only on attempting to provide a general picture of remote visualization, given the differences between centers in terms of purposes, needs, resources, and national impact.

Gelvez Cortes, Sergio Augusto↗
NASA NTRS1981

Video image processor on the Spacelab 2 Solar Optical Universal Polarimeter /SL2 SOUP/

The SOUP instrument is designed to obtain diffraction-limited digital images of the sun with high photometric accuracy. The Video Processor originated from the requirement to provide onboard real-time image processing, both to reduce the telemetry rate and to provide meaningful video displays of scientific data to the payload crew. This original concept has evolved into a versatile digital processing system with a multitude of other uses in the SOUP program. The central element in the Video Processor design is a 16-bit central processing unit based on 2900 family bipolar bit-slice devices. All arithmetic, logical and I/O operations are under control of microprograms, stored in programmable read-only memory and initiated by commands from the LSI-11. Several functions of the Video Processor are described, including interface to the High Rate Multiplexer downlink, cosmetic and scientific data processing, scan conversion for crew displays, focus and exposure testing, and use as ground support equipment.

Lindgren, R. W.↗