Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Distribution”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

HEPnOS: a Specialized Data Service for High Energy Physics Analysis

In this paper, we present HEPnOS, a distributed data service for managing data produced by high-energy physics (HEP) experiments. Using HEPnOS, HEP applications can use HPC resources more effciently than traditional fle-based applications. The fle-based model leads to a rigid, chunk-based allocation of computational resources and limits the number of cores that can be used concurrently by an HEP application. The fundamental problem is that organizing domain-specifc data into fles inadvertently introduces a single, artifcial, confated tuning parameter that puts key optimization goals into confict: larger fle sizes reduce metadata overhead and thus improve I/O effciency, but smaller fle sizes provide more opportunity for workfow parallelism and load balancing. In this work, we introduce a domain-specifc data service that decouples that constraint so that data can be accessed and processed in its natural granularity while still maintaining I/O effciency. By removing the constraints introduced by fle handling we are able to obtain better scaling and make effcient use of more cores for processing a fxed-sized data sample. We demonstrate the improved scalability by using an application developed in the fle-based paradigm and comparing it to a version modifed to use HEPnOS.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS1986

Visibility related to backscatter at 1.54 micron

The lidar process was shown to have the necessary potential to fulfill the need for a remote measurement of visibility. Visibility can be inferred from a lidar return optical extinction. The wavelength 1.54 micron was chosen, being near the visible wavelength region and having a high eye safety threshol, 200,000 times higher than 1.06 micron; 1.54 is the erbium laser wavelength. This research utilized 105 measured height profiles of natural droplet size distributions data, taken in clouds, fog, and haze. These profiles were examined to determine the completeness of the droplet counting data. It was found that the particle spectrometer data were incomplete in the very light ford and haze so this portion of the data was eliminated. Utilizing the Mie theory, these droplet size distribution profiles were converted to backscatter at 1.54 micron and extinction in the visible region, 0.55 micron. Using Koschmeider's relationship, the extinction profiles were converted to visibility. The visibility and backscatter profiles were compared to develop a relationship between visibility and backscatter at 1.54 micron.

Barber, T. L.↗
NASA NTRS1991

Optical space-to-ground link availability assessment and diversity requirements

The application of optical space-to-ground links (SGLs) for high speed data distribution from geosynchronous and low earth orbiting satellites (e.g., sensor data from the planned Earth Observing System), for lunar and Mars links, and for links from interplanetary probes has been a topic of considerable recent interest. These optical SGLs could conceivably represent the system's operational baseline, or could represent backup links in the event of a GEO relay terminal failure. In this paper the availability of optical SGLs for various system/orbit configurations is considered. Single CONUS sites are assessed for their probability of cloud free line of sight (PCFLOS), and cloud free field of view (PCFFOV). PCFLOS represents an availability metric for geosynchronous platforms, while PCFFOV is a relevant performance metric for non-geostationary platforms (e.g., low earth orbiting satellites). Additionally, the availability of multiple ground terminals utilized in a diversity configuration is considered. Availability statistics vs. the number of diversity sites are derived from climatological data bases for CONUS sites.

Chapman, William↗
NASA NTRS1993

The consistency of standard cosmology and the BATSE number versus brightness relation

The integrated number-peak-flux relation measured by the Burst and Transient Source Experiment (BATSE) on board the Compton Gamma Ray Observatory is compared with several standard cosmological distributions for gamma-ray bursts (GRBs). Friedmann-Robertson-Walker models were used along with the assumption that the bursts are standard candles and have no number or luminosity evolution. For a given Omega spectral shape, we used a free parameter, essentially the comoving number density of bursts, to generate a best fit between the cosmology and the measured relation. Our results are shown for a subsample of the first 260 GRBs recorded by BATSE. We find acceptable fits between simple cosmological models and the brightness distribution data, as determined by the Kolmogorov-Smirnov one-distribution statistical test. One cannot distinguish a single best cosmological model from the goodness of the fits. The best fit implies that BATSE GRBs are complete out to a redshift of about unity. However, significantly higher and lower redshifts, by as much as a factor of 2, are possible for other marginally acceptable fits.

Wickramasinghe, W. A. D. T.↗
NASA NTRS2011

Land and Atmosphere Near-Real-Time Capability for Earth Observing System

The past decade has seen a rapid increase in availability and usage of near-real-time data from satellite sensors. The EOSDIS (Earth Observing System Data and Information System) was not originally designed to provide data with sufficiently low latency to satisfy the requirements for near-real-time users. The EOS (Earth Observing System) instruments aboard the Terra, Aqua and Aura satellites make global measurements daily, which are processed into higher-level 'standard' products within 8-40 hours of observation and then made available to users, primarily earth science researchers. However, applications users, operational agencies, and even researchers desire EOS products in near-real-time to support research and applications, including numerical weather and climate prediction and forecasting, monitoring of natural hazards, ecological/invasive species, agriculture, air quality, disaster relief and homeland security. These users often need data much sooner than routine science processing allows, usually within 3 hours, and are willing to trade science product quality for timely access. While Direct Broadcast provides more timely access to data, it does not provide global coverage. In 2002, a joint initiative between NASA (National Aeronautics and Space Administration), NOAA (National Oceanic and Atmospheric Administration), and the DOD (Department of Defense) was undertaken to provide data from EOS instruments in near-real-time. The NRTPE (Near Real Time Processing Effort) provided products within 3 hours of observation on a best-effort basis. As the popularity of these near-real-time products and applications grew, multiple near-real-time systems began to spring up such as the Rapid Response System. In recognizing the dependence of customers on this data and the need for highly reliable and timely data access, NASA's Earth Science Division sponsored the Earth Science Data and Information System Project (ESDIS)-led development of a new near-real-time system called LANCE (Land, Atmosphere Near-Real-Time Capability for EOS) in 2009. LANCE consists of special processing elements, co-located with selected EOSDIS data centers and processing facilities. A primary goal of LANCE is to bring multiple near-real-time systems under one umbrella, offering commonality in data access, quality control, and latency. LANCE now processes and distributes data from the Moderate Resolution Imaging Spectroradiometer (MODIS), Atmospheric Infrared Sounder (AIRS), Advanced Microwave Scanning Radiometer Earth Observing System (AMSR-E), Microwave Limb Sounder (MLS) and Ozone Monitoring Instrument (OMI) instruments within 3 hours of satellite observation. The Rapid Response System and the Fire Information for Resource Management System (FIRMS) capabilities will be incorporated into LANCE in 2011. LANCE maintains a central website to facilitate easy access to data and user services. LANCE products are extensively tested and compared with science products before being made available to users. Each element also plans to implement redundant network, power and server infrastructure to ensure high availability of data and services. Through the user registration system, users are informed of any data outages and when new products or services will be available for access. Building on a significant investment by NASA in developing science algorithms and products, LANCE creates products that have a demonstrated utility for applications requiring near-real-time data. From lower level data products such as calibrated geolocated radiances to higher-level products such as sea ice extent, snow cover, and cloud cover, users have integrated LANCE data into forecast models and decision support systems. The table above shows the current near-real-time product categories by instrument. The ESDIS Project continues to improve the LANCE system and use the experience gained through practice to seek adjustments to improve the quality and performance of the system. For example, anGC-compliant Web Map Service (WMS) will be added shortly that will allow users to download geo-referenced MODIS images for arbitrary bounding boxes. Further, an OGC-compliant Web Coverage Service (WCS) will be added later this year that will expedite user access to arbitrary data subsets or re-formatted products. AIRS images are now served through WMS and available in multiple formats (PNG, GeoTIFF, KMZ). NASA has established a LANCE User Working Group to steer the development of the system and create a forum for sharing ideas and experiences that are expected to further improve the LANCE capabilities. The LANCE system has proved a success by satisfying the growing needs of the applications and operational communities for land and atmosphere data in near-real-time. NASA's Earth Sciences Division was able to leverage existing science research capabilities to provide the near-real-time community with products and imagery that support monitoring of disasters in a timely manner.

Murphy, Kevin J.↗
NASA NTRS2011

SPASE: The Connection Among Solar and Space Physics Data Centers

The Space Physics Archive Search and Extract (SPASE) project is an international collaboration among Heliophysics (solar and space physics) groups concerned with data acquisition and archiving. Within this community there are a variety of old and new data centers, resident archives, "virtual observatories", etc. acquiring, holding, and distributing data. A researcher interested in finding data of value for his or her study faces a complex data environment. The SPASE group has simplified the search for data through the development of the SPASE Data Model as a common method to describe data sets in the various archives. The data model is an XML-based schema and is now in operational use. There are both positives and negatives to this approach. The advantage is the common metadata language enabling wide-ranging searches across the archives, but it is difficult to inspire the data holders to spend the time necessary to describe their data using the Model. Software tools have helped, but the main motivational factor is wide-ranging use of the standard by the community. The use is expanding, but there are still other groups who could benefit from adopting SPASE. The SPASE Data Model is also being expanded in the sense of providing the means for more detailed description of data sets with the aim of enabling more automated ingestion and use of the data through detailed format descriptions. We will discuss the present state of SPASE usage and how we foresee development in the future. The evolution is based on a number of lessons learned - some unique to Heliophysics, but many common to the various data disciplines.

Thieman, James R.↗
NASA NTRS2011

SPASE: Current Uses, Tools, and Plans

The Space Physics Archive Search and Extract (SPASE) project is an international collaboration among Heliophysics (solar and space physics) groups concerned with data acquisition and archiving. Within this community there are a variety of old and new data centers, resident archives, "virtual observatories", etc. acquiring, holding, and distributing data. The main product of the SPASE group is an XML-based SPASE Data Model now in operational use to enable searches for and ultimate acquisition of data of interest to a researcher. The SPASE Data Model defines the content of resource descriptions (metadata). The intent is to describe all SCientifically usable Heliophysics data sets using the Data Model. Another product of the SPASE group, in collaboration with NASA's Virtual Observatories, is a set of tools and services which work with SPASE meta data. This includes Registry Services which can retrieve and render metadata using resource identifiers and facilitate the downloading of the data referenced by the meta data. The SPASE Data Model has also been used as a vocabulary in specialized data models. One example is the Heliophysics Event List Manager (HELM) model. The SPASE Data Model is also being expanded to provide the means for more detailed description of data sets with the aim of enabling more automated ingestion and use of the data through detailed format descriptions. The evolution is based on a number of lessons learned and feedback from our community. Some of the lessons learned are unique to Heliophysics, and some are common to the various data diSCiplines. We will discuss the present state of SPASE usage, the role the SPASE Data Model can play in speCialized data models and how we foresee the development direction in the future.

Thieman, J. R.↗
DOE OSTI2025

A Risk-Informed Approach to Trustworthiness Assessment in Digital Twins-Based Autonomous Control

In autonomous control systems, digital twins (DTs) are used to perform diagnostic and prognostic functions. The trustworthiness of these DTs is dependent on quality and coverage of the training data, model accuracy and integrity of sensor data. This work introduces a methodology to determine the trustworthiness of a DT system given faulty sensor data using a risk informed approach. Bayesian Belief Networks (BBNs) are used to propagate uncertainties and determine the probability of trustable recommendations. The decision to trust the control action provided by the DT is based on the DT output, expert opinion, and severity of problems. The performance of DTs is reliant on the data they are trained on. When they encounter out of distribution data, the trustworthiness of the recommendations decreases. To address this issue, we include an expert component that provides input on sensor degradation. For this, we utilize a generative artificial intelligence (AI) model, such as Generative Pretrained Transformer (GPT). The GPT functions as an expert with broad knowledge. The GPT is fine-tuned to understand and discriminate sensor degradation scenarios using manufactured data. This methodology is demonstrated through a case study on a Nearly Autonomous Management and Control System (NAMAC) during a steady state scenario. Various sensor degradation types with different severity levels are considered. Degraded sensor data is processed by the DT system and the fine-tuned GPT. Finally, using the BBN, we combine the GPT information and the DT output with its sources of uncertainty. This provides an output regarding the trustworthiness of the DT recommendation.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS1990

Advanced systems engineering and network planning support

The objective of this task was to take a fresh look at the NASA Space Network Control (SNC) element for the Advanced Tracking and Data Relay Satellite System (ATDRSS) such that it can be made more efficient and responsive to the user by introducing new concepts and technologies appropriate for the 1997 timeframe. In particular, it was desired to investigate the technologies and concepts employed in similar systems that may be applicable to the SNC. The recommendations resulting from this study include resource partitioning, on-line access to subsets of the SN schedule, fluid scheduling, increased use of demand access on the MA service, automating Inter-System Control functions using monitor by exception, increase automation for distributed data management and distributed work management, viewing SN operational control in terms of the OSI Management framework, and the introduction of automated interface management.

Walters, David H.↗
DOE OSTI2021

A Multi-criteria CCUS Screening Evaluation of the Gulf of Mexico, USA - Supplementary Data

The Wendt et al. study aims to incorporate multiple and disparate carbon capture, utilization, and storage (CCUS) decision-making criteria into a systematic, quantitative analytical approach to help identify areas with potentially high suitability to serve as offshore CO2 storage or EOR regions. Spatially-distributed data from publicly-available sources within the Gulf of Mexico (GOM) study area (limited to federal waters; state waters were not evaluated) was compiled using the U.S. Department of Energy's (DOE) National Energy Technology Laboratory (NETL) Cumulative Spatial Impact Layers™ (CSIL) tool to easily aggregate data based on evenly-distributed grids across the study region set at a resolution of approximately 25 square miles (65 square kilometers). The data included in this Microsoft Excel™ workbook provide the aggregated scores for each grid point across the study domain as well the weighting for each criterion under the four scenarios evaluated.

CCUS↗
DOE OSTI2020

Improving Enzyme Optimum Temperature Prediction with Resampling Strategies and Ensemble Learning

Accurate prediction of the optimal catalytic temperature ( T opt ) of enzymes is vital in biotechnology, as enzymes with high T opt values are desired for enhanced reaction rates. Recently, a machine learning method (temperature optima for microorganisms and enzymes, TOME) for predicting T opt was developed. TOME was trained on a normally distributed data set with a median T opt of 37 °C and less than 5% of T opt values above 85 °C, limiting the method’s predictive capabilities for thermostable enzymes. Due to the distribution of the training data, the mean squared error on T opt values greater than 85 °C is nearly an order of magnitude higher than the error on values between 30 and 50 °C. Here, we apply ensemble learning and resampling strategies that tackle the data imbalance to significantly decrease the error on high T opt values (>85 °C) by 60% and increase the overall R 2 value from 0.527 to 0.632. The revised method, temperature optima for enzymes with resampling (TOMER), and the resampling strategies applied in this work are freely available to other researchers as Python packages on GitHub.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS1993

Simulation of a data archival and distribution system at GSFC

A version-0 of a Data Archive and Distribution System (DADS) is being developed at GSFC to support existing and pre-EOS Earth science datasets and test Earth Observing System Data and Information System (EOSDIS) concepts. The performance of DADS is predicted using a discrete event simulation model. The goals of the simulation were to estimate the amount of disk space needed and the time required to fulfill the DADS requirements for ingestion (14 GB/day) and distribution (48 GB/day). The model has demonstrated that 4 mm and 8 mm stackers can play a critical role in improving the performance of the DADS, since it takes, on average, 3 minutes to manually mount/dismount tapes compared to less than a minute with stackers. With two 4 mm stackers and two 8 mm stackers, and a single operator per shift, the DADS requirements can be met within 16 hours using a total of 9 GB of disk space. When the DADS has no stacker, and the DADS depends entirely on operators to handle the distribution tapes, the simulation has shown that the DADS requirements can still be met within 16 hours, but a minimum of 4 operators per shift were required. The compression/decompression of data sets is very CPU intensive, and relatively slow when performed in software, thereby contributing to an increase in the amount of disk space needed.

Bedet, Jean-Jacques↗
NASA NTRS1982

Evaluation of the Seasat altimeter time tag bias

Two methods were used in analyses of Seasat altimeter data, aimed at the corroboration of an inherent altimeter microprocessor delay compensation value of -79.4 msec, which benefited from a global data distribution in the oceanic areas: (1) the crossover method, using altimeter data differenced at points where the Seasat ground track intersected with itself, and (2) the direct use of the altimeter data. Because the former method is independent of errors in the geoid model, it is considered the more reliable. For all crossover method results, the adopted value of -79.4 msec is within the bounds of the standard deviation associated with the estimates, of which -78.1 + or - 2.0 msec is considered the best representative.

Schutz, B. E.↗
NASA NTRS1982

PMP-1: A coordinated study of the behavior of the middle atmosphere in winter

The following observations of the middle atmosphere were available regularly: radiosonde data distributed through arrangements made by the World Meteorological Organization (WMO); radiosonde data; and the infrared radiances of the SSU (stratospheric sounding unit onboard the operational NOAA satellites). Other data of more experimental nature which are or will become available are, data from the Nimbus-7 satellite, especially from the Stratosphere and Mesospheric Sounder (SAMS) and the Limb Infrared Monitor of the Stratosphere (LIMS), but only for the winter of 1978 to 1979. These data are collected and integrated into the large-scale meteorological field analysis. Parameters necessary for the understanding of the large-scale dynamics of the middle atmosphere are derived.

Labitzke, K.↗
NASA NTRS1992

Data management for community research projects: A JGOFS case study

Since the mid 1980s, much of the marine science research effort in the United Kingdom has been focused into large scale collaborative projects involving public sector laboratories and university departments, termed Community Research Projects. Two of these, the Biogeochemical Ocean Flux Study (BOFS) and the North Sea Project incorporated large scale data collection to underpin multidisciplinary modeling efforts. The challenge of providing project data sets to support the science was met by a small team within the British Oceanographic Data Centre (BODC) operating as a topical data center. The role of the data center was to both work up the data from the ship's sensors and to combine these data with sample measurements into online databases. The working up of the data was achieved by a unique symbiosis between data center staff and project scientists. The project management, programming and data processing skills of the data center were combined with the oceanographic experience of the project communities to develop a system which has produced quality controlled, calibrated data sets from 49 research cruises in 3.5 years of operation. The data center resources required to achieve this were modest and far outweighed by the time liberated in the scientific community by the removal of the data processing burden. Two online project databases have been assembled containing a very high proportion of the data collected. As these are under the control of BODC their long term availability as part of the UK national data archive is assured. The success of the topical data center model for UK Community Research Project data management has been founded upon the strong working relationships forged between the data center and project scientists. These can only be established by frequent personal contact and hence the relatively small size of the UK has been a critical factor. However, projects covering a larger, even international scale could be successfully supported by a network of topical data centers managing online databases which are interconnected by object oriented distributed data management systems over wide area networks.

Lowry, Roy K.↗
NASA NTRS2013

RUSHMAPS: Real-Time Uploadable Spherical Harmonic Moment Analysis for Particle Spectrometers

RUSHMAPS is a new onboard data reduction scheme that gives real-time access to key science parameters (e.g. moments) of a class of heliophysics science and/or solar system exploration investigation that includes plasma particle spectrometers (PPS), but requires moments reporting (density, bulk-velocity, temperature, pressure, etc.) of higher-level quality, and tolerates a lowpass (variable quality) spectral representation of the corresponding particle velocity distributions, such that telemetry use is minimized. The proposed methodology trades access to the full-resolution velocity distribution data, saving on telemetry, for real-time access to both the moments and an adjustable-quality (increasing quality increases volume) spectral representation of distribution functions. Traditional onboard data storage and downlink bandwidth constraints severely limit PPS system functionality and drive cost, which, as a consequence, drives a limited data collection and lower angular energy and time resolution. This prototypical system exploit, using high-performance processing technology at GSFC (Goddard Space Flight Center), uses a SpaceCube and/or Maestro-type platform for processing. These processing platforms are currently being used on the International Space Station as a technology demonstration, and work is currently ongoing in a new onboard computation system for the Earth Science missions, but they have never been implemented in heliospheric science or solar system exploration missions. Preliminary analysis confirms that the targeted processor platforms possess the processing resources required for realtime application of these algorithms to the spectrometer data. SpaceCube platforms demonstrate that the target architecture possesses the sort of compact, low-mass/power, radiation-tolerant characteristics needed for flight. These high-performing hybrid systems embed unprecedented amounts of onboard processing power in the CPU (central processing unit), FPGAs (field programmable gate arrays), and DSP (digital signal processing) elements. The fundamental computational algorithm de constructs 3D velocity distributions in terms of spherical harmonic spectral coefficients (which are analogous to a Fourier sine-cosine decomposition), but uses instead spherical harmonics Legendre polynomial orthogonal functions as a basis for the expansion, portraying each 2D angular distribution at every energy or, geometrically, spherical speed-shell swept by the particle spectrometer. Optionally, these spherical harmonic spectral coefficients may be telemetered to the ground. These will provide a smoothed description of the velocity distribution function whose quality will depend on the number of coefficients determined. Successfully implemented on the GSFC-developed processor, the capability to integrate the proposed methodology with both heritage and anticipated future plasma particle spectrometer designs is demonstrated (with sufficiently detailed design analysis to advance TRL) to show specific science relevancy with future HSD (Heliophysics Science Division) solar-interplanetary, planetary missions, sounding rockets and/or CubeSat missions.

Figueroa-Vinas, Adolfo↗
NASA NTRS1993

Role of formats in the life cycle of data

This paper's perspective is based on the author's experience generating, analyzing, archiving, and distributing data obtained from satellites, and on the experience gained in data modeling and the development of standards for data understanding under the Consultative Committee for Space Data Systems (CCSDS). Data formats are used to represent all information in digital form, and thus play a major role in all interchanges and access to this information. The need to more efficiently manage and process rapidly growing quantities of data, and to preserve the information contained therein, continue to drive a great interest in data formats. The purpose of this paper is to examine the role of formats as they support the use of data within a space agency. The life-cycle identified is only one of many variations that would be recognized by those familiar with the 'space business', however it is expected that most of the issues raised will be pertinent to other 'space business' life cycles and to other 'non-space' disciplines as well.

Sawyer, Don↗
NASA NTRS2004

Complexity Computational Environment: Data Assimilation SERVOGrid

We are using Web (Grid) service technology to demonstrate the assimilation of multiple distributed data sources (a typical data grid problem) into a major parallel high-performance computing earthquake forecasting code. Such a linkage of Geoinformatics with Geocomplexity demonstrates the value of the Solid Earth Research Virtual Observatory (SERVO) Grid concept, and advance Grid technology by building the first real-time large-scale data assimilation grid Here we develop the next steps for both the SERVO concept and the identified need for a Solid Earth problem-solving environment. We use a challenging motivating problem of importance to NASA namely integrating NASA space geodetic observations with numerical simulations of a changing earth.

data assimiliation↗