Results for “Data access” — Page 90

NASA NTRS1994

Automated mainframe data collection in a network environment

The progress and direction of the computer industry have resulted in widespread use of dissimilar and incompatible mainframe data systems. Data collection from these multiple systems is a labor intensive task. In the past, data collection had been restricted to the efforts of personnel specially trained on each system. Information is one of the most important resources an organizations has. Any improvement in an organization's ability to access and manage that information provides a competitive advantage. This problem of data collection is compounded at NASA sites by multi-center and contractor operations. The Centralized Automated Data Retrieval System (CADRS) is designed to provide a common interface that would permit data access, query, and retrieval from multiple contractor and NASA systems. The methods developed for CADRS have a strong commercial potential in that they would be applicable for any industry that needs inter-department, inter-company, or inter-agency data communications. The widespread use of multi-system data networks, that combine older legacy systems with newer decentralized networks, has made data retrieval a critical problem for information dependent industries. Implementing the technology discussed in this paper would reduce operational expense and improve data collection on these composite data systems.

Gross, David L.↗

DOE OSTI2021

Design and Performance of Kokkos Staging Space toward Scalable Resilient Application Couplings

With the growing number of applications designed for heterogeneous HPC devices, application programmers and users are finding it challenging to compose scalable workflows as ensembles of these applications, that are portable, performant and resilient. The Kokkos C++ library has been designed to simplify this cumbersome procedure by providing an intra-application uniform programming model and portable performance. However, assembling multiple Kokkos-enabled applications into a complex workflow is still a challenge. Although Kokkos enables a uniform programming model, the inter-application data exchange still remains a challenge from both performance and software development cost perspectives. In order to address this issue, we propose Kokkos data staging memory space, an extension of Kokkos' data abstraction (memory space) for heterogeneous computing systems. This new abstraction allows to express data on a virtual shared-space for multiple Kokkos applications, thus extending Kokkos to support inter-application data exchange to build an efficient application workflow. Additionally, we study the effectiveness of asynchronous data layout conversions for applications requiring different memory access patterns for the shared data. Our preliminary evaluation with a synthetic benchmark indicate the effectiveness of this conversion adapted to three different scenarios representing access frequency and use patterns of the shared data.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1991

The Pilot Land Data System (PLDS) at the Ames Research Center manages aircraft data in collaboration with an ecosystem research project

The Pilot Land Data System (PLDS) is a data and information system serving NASA-supported investigators in the land science community. The three nodes of the PLDS, one each at the Ames Research Center (ARC), the Goddard Space Flight Center (GSFC) and the Jet Propulsion Laboratory (JPL), cooperate in providing consistent information describing the various data holding in the hardware and software (accessible via network and modem) that provide information about and access to PLDS-held data, which is available for distribution. A major new activity of the PLDS node at the Ames Research Center involves the interaction of the PLDS with an active NASA ecosystem science project, the Oregon Transect Ecosystems Research involves the management of, access to, and distribution of the large volume of widely-varying aircraft data collected by OTTER. The OTTER project, is managed by researchers at the Ames Research Center and Oregon State University. Its principal objective is to estimate major fluxes of carbon, nitrogen, and water of forest ecosystems using an ecosystem process model driven by remote sensing data. Ten researchers at NASA centers and universities are analyzing data for six sites along a temperature-moisture gradient across the western half of central Oregon (called the Oregon Transect). Sensors mounted on six different aircraft have acquired data over the Oregon Transect in support of the OTTER project.

Angelici, Gary↗

NASA NTRSDate not supplied

Transformation of the NASA Life Sciences Portal to a FAIR Data Point

The FAIR principles emphasize optimizing metadata, the vast majority of which are textual in nature, and often organized into attribute name-value pairs. This uniformity has led to the development of guidelines and best practices for providing programmatic access to scientific data through their metadata, yielding the first iteration of the FAIR Data Point Specifications (FDPS). A key feature of the FDPS is its support for automated agents seeking and fetching data without first needing to learn a plethora of different application programming interfaces. These software agents can interrogate metadata catalogs that adhere to FDPS in a uniform manner because each catalog describes itself and its metadata schema consistently. This approach enhances the sustainability of data retrieval support, allowing systems to refine and update their metadata schemas as needed and without requiring data-seeking software agents to change how they interrogate FDPS catalogs. An essential aspect of the FDPS is the standardization of data catalog semantics, which formalizes concepts such as “metadata” and “metadata service” and links them to other concepts specifications including the Data Catalog Vocabulary (DCAT), a W3C standard that is also the basis of NASA-STD-2831 “Metadata Standard for Data Discoverability,” authored by NASA’s Office of the Chief Information Officer. The FDPS references DCAT (version 2) elements which focus on the distribution of datasets and support the goal of stream-lined catalog integration across repositories for improved data discovery. Additionally, the FDPS also prescribe the use of Linked Data Platform elements for data catalog-metadata record containment descriptions, allowing users to ascertain which data and metadata belong to which catalogs. NASA’s Life Sciences Portal is implementing the FDPS while formalizing its metadata schema to support the accelerated synthesis of knowledge from space life sciences investigations.

platform↗

NASA NTRS2000

EOSDIS: Archive and Distribution Systems in the Year 2000

Earth Science Enterprise (ESE) is a long-term NASA research mission to study the processes leading to global climate change. The Earth Observing System (EOS) is a NASA campaign of satellite observatories that are a major component of ESE. The EOS Data and Information System (EOSDIS) is another component of ESE that will provide the Earth science community with easy, affordable, and reliable access to Earth science data. EOSDIS is a distributed system, with major facilities at seven Distributed Active Archive Centers (DAACs) located throughout the United States. The EOSDIS software architecture is being designed to receive, process, and archive several terabytes of science data on a daily basis. Thousands of science users and perhaps several hundred thousands of non-science users are expected to access the system. The first major set of data to be archived in the EOSDIS is from Landsat-7. Another EOS satellite, Terra, was launched on December 18, 1999. With the Terra launch, the EOSDIS will be required to support approximately one terabyte of data into and out of the archives per day. Since EOS is a multi-mission program, including the launch of more satellites and many other missions, the role of the archive systems becomes larger and more critical. In 1995, at the fourth convening of NASA Mass Storage Systems and Technologies Conference, the development plans for the EOSDIS information system and archive were described. Five years later, many changes have occurred in the effort to field an operational system. It is interesting to reflect on some of the changes driving the archive technology and system development for EOSDIS. This paper principally describes the Data Server subsystem including how the other subsystems access the archive, the nature of the data repository, and the mass-storage I/O management. The paper reviews the system architecture (both hardware and software) of the basic components of the archive. It discusses the operations concept, code development, and testing phase of the system. Finally, it describes the future plans for the archive.

Behnke, Jeanne↗

DOE OSTI2025

A machine-learning-driven data labeling pipeline for scientific analysis in MLExchange

This study introduces a novel labeling pipeline to accelerate the labeling process of scientific data sets by using artificial intelligence (AI)-guided tagging techniques. This pipeline includes a set of interconnected web-based graphical user interfaces (GUIs), where Data Clinic and MLCoach enable the preparation of machine learning (ML) models for data reduction and classification, respectively, while Label Maker is used for label assignment. Throughout this pipeline, data can be accessed through a direct connection to a file system or through Tiled for access through Hypertext Transfer Protocol (HTTP). Our experimental results present three use cases where this labeling pipeline has been instrumental for the study of large X-ray scattering data sets in the area of pattern recognition, the remote analysis of resonant soft X-ray scattering data and the fine-tuning process of foundation models. These use cases highlight the labeling capabilities of this pipeline, including the ability to label large data sets in a short period of time, to perform remote data analysis while minimizing data movement and to enhance the fine-tuning process of complex ML models with human involvement.

Chavez, Tanny (ORCID:0000000193172896)↗

NASA NTRS1980

Snow mapping from space platforms

The paper considers problems of optimum resolution, periodicity, and wavelength bands used for snow mapping. Analog and digital methods were used for application of satellite data; techniques were developed for producing steamflow forecasts, hydroelectric power generation regulation data, irrigation potentials, and information on the availability of drinking water supplies. Future systems will utilize improved spectral band selection, new spectral regions, higher repetition rates, and more rapid access to satellite data.

Itten, K. I.↗

NASA NTRS1991

Solar-A reformatted data files and observing log

An overview is presented of the Solar-A telemetry data files which are to be created and the format and organization which the files are to use. The organization chosen is to be efficient in space, to facilitate access to the data, and to allow the data to be transportable to different machines. An observing log file is to be created automatically, using the reformatted data files as the input. It will be possible to perform searches with the observing log to list cases where instruments are in certain modes and/or seeing certain signal levels. A user will be able to search the observing log and obtain a list of all cases where a given set of conditions are satisfied. An event log will be created listing the times when the instrument or spacecraft modes change.

Morrison, M. D.↗

NASA NTRS2007

Characterizing Aerosols over Southeast Asia using the AERONET Data Synergy Tool

Biomass burning, urban pollution and dust aerosols have significant impacts on the radiative forcing of the atmosphere over Asia. In order to better quanti@ these aerosol characteristics, the Aerosol Robotic Network (AERONET) has established over 200 sites worldwide with an emphasis in recent years on the Asian continent - specifically Southeast Asia. A total of approximately 15 AERONET sun photometer instruments have been deployed to China, India, Pakistan, Thailand, and Vietnam. Sun photometer spectral aerosol optical depth measurements as well as microphysical and optical aerosol retrievals over Southeast Asia will be analyzed and discussed with supporting ground-based instrument, satellite, and model data sets, which are freely available via the AERONET Data Synergy tool at the AERONET web site (http://aeronet.gsfc.nasa.gov). This web-based data tool provides access to groundbased (AERONET and MPLNET), satellite (MODIS, SeaWiFS, TOMS, and OMI) and model (GOCART and back trajectory analyses) databases via one web portal. Future development of the AERONET Data Synergy Tool will include the expansion of current data sets as well as the implementation of other Earth Science data sets pertinent to advancing aerosol research.

Giles, David M.↗

DOE OSTI2026

Transportation Secure Data Center: Frequently Asked Questions for Data Owners/Contributors

The Transportation Secure Data Center is a centralized repository for detailed transportation data from travel and transit surveys and studies conducted across the nation. It makes vital transportation data broadly available to users while preserving the privacy of survey participants. Hundreds of datasets from surveys and studies of household travel and transit passenger travel are archived in the TSDC, including surveys and studies conducted by state departments of transportation, metropolitan planning organizations, transit agencies, cities, and other public agencies. Detailed data from travel surveys and studies are extremely valuable for research purposes. However, the fine-grained information they contain could potentially be misused to identify individual travelers, so access to these data should only be granted with safeguards in place to protect participant privacy. The TSDC was created to address this challenge and to relieve public agencies from the burden of archiving their data and responding to data requests.

33 ADVANCED PROPULSION SYSTEMS↗

DOE OSTI2024

AI Surrogate Model for Distributed Computing Workloads

Explore the source record for details and available documents.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

ARM Aerial Facility (AAF) Merged Value-Added Product Report for Historical G-1 Field Campaigns

For 30 years, the U.S. Department of Energy (DOE) Office of Science supported an instrumented Grumman Gulfstream-1 (G-1) aircraft for atmospheric field campaigns. Data from the final decade of G-1 operations were archived by the Atmospheric Radiation Measurement (ARM) user facility Data Center (ADC) and made publicly available at no cost to all registered users. To ensure a consistent data format and to improve the accessibility of the ARM airborne data, an integrated data set was recently developed covering the final six years of G-1 operations (2013 to 2018). The integrated data set includes data collected from 236 flights (766.4 hours). Four of the seven field campaigns were based in the U.S. One campaign collected data from the wildfires in the U.S. Pacific Northwest and agricultural burns in the lower Mississippi River valley as part of the Biomass Burning Observation Project (BBOP) in 2013. In 2015, the ARM Cloud Aerosol Precipitation Experiment provided data on atmospheric rivers and associated aerosol-cloud interactions that produce heavy precipitation on the U.S. west coast during the early spring. Research data from Airborne Carbon Measurements-V (ACME-V), collected during the summer of 2015, gave scientists insight into trends and variability of trace gases in the atmosphere over the North Slope of Alaska to improve arctic climate models. In the early summer and autumn of 2016, the Holistic Interactions of Shallow Clouds, Aerosols, and Land-Ecosystems (HI-SCALE) campaign provided an extensive data set geared toward coupled processes that affect the life cycle of shallow clouds through the interaction among aerosol, cloud, land surface, and ecosystems. In 2014 (March and October), the airborne sampling moved outside of the U.S. to the city of Manaus in central Amazonia, Brazil, where residential and industrial emissions were extensively characterized by flights of the G-1. The GoAmazon2014/15 aircraft campaign data are being integrated with aquatic and terrestrial ecosystem measurements to quantify anthropogenic perturbations to a usually pristine tropical environment. Another international airborne mission was carried out in the Eastern North Atlantic region. The Aerosol and Cloud Experiments in the Eastern North Atlantic (ACE-ENA) campaign saw the G-1 aircraft fly from Terceira Island in the Azores during the summer of 2017 and the winter of 2018. The campaign studied both seasons to measure key aerosol and cloud processes under various meteorological and cloud conditions with different aerosol sources. Then the G-1 deployed to the Sierras de Córdoba range in central Argentina from October to November 2018 for the Cloud, Aerosol, and Complex Terrain Interactions (CACTI) campaign to study orographic convective cloud interactions with their surrounding environment. These comprehensive datastreams provide much-needed insight into spatiotemporal variability of thermodynamic quantities, aerosol and cloud states, and properties for addressing essential science questions in Earth system process studies.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

2020 Budget Request for the DOE Computational Science Graduate Fellowship (CSGF) Grant

The Department of Energy Computational Science Graduate Fellowship (DOE CSGF) is essential for addressing the increasingly complex national workforce demands stemming from the growth of computational science and engineering challenges. Computational science and engineering (CSE) takes a multidisciplinary approach that utilizes scientific computing to tackle practical problems and provide technical tools across the spectrum of scientific discovery. The DOE CSGF specifically highlights high-performance computing (HPC) as a critical enabling technology in CSE, driving advancements in science and engineering that are vital to both the DOE and the broader economy. Over the past half-century, HPC has been an essential tool for DOE’s success. During this period, important missions, such as nuclear stockpile stewardship, have turned to HPC as an essential technology. Entire science disciplines have been transformed through the augmentation of scientific observation via HPC. At government laboratories, academic institutions, and in industry, DOE CSGF alumni are helping push traditional HPC boundaries while contributing to discoveries in high-energy physics, quantum information systems, fusion-reactor design, machine learning, additive manufacturing, nano materials for next-generation batteries and transistors, and advanced nuclear reactor modeling. In addition, HPC is used to address national health needs that will eventually point to cures both by helping cancer researchers manage and analyze huge troves of data, by simulating biological mechanisms, and by accelerating drug development. A 2023 report from the ASCAC Subcommittee on American Competitiveness and Innovation to the ASCR office, “Can the United States Maintain Its Leadership in High-Performance Computing?” says of the Program, “The CSGF program provides a barometer for disciplines that will be of interest to future DOE computing. Computational biology, machine learning, and quantum computing are among the subjects that began to swell in the ranks of CSGF applicants before the labs were hiring as high a percentage of employees in these categories.” The explosion of scientific and technological data has heightened the demand for advanced high-performance computing (HPC) to transform these data into meaningful scientific insights. As access to vast amounts of data increases, the fields of Machine Learning and Artificial Intelligence are experiencing a resurgence, enhancing the established practices of computational modeling and simulation. In its September 2020 subcommittee report on "AI/ML, Data Intensive Science, and High-Performance Computing," the DOE Advanced Scientific Computing Advisory Committee (ASCAC) specifically called for a fellowship program to train computational and data scientists to address exascale and data-intensive computing challenges. This integration of empirical and theoretical modeling will increasingly guide federal policymakers in making decisions that impact American society and future generations. It demands a workforce of highly skilled and intellectually agile computational scientists capable of navigating the rapid advancements in scientific computing within the DOE National Laboratory research environment. The DOE CSGF program has consistently addressed this critical need.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

ARM Aerial Facility (AAF) Merged Value-Added Product Report for Historical G-1 Field Campaigns

For 30 years, the U.S. Department of Energy (DOE) Office of Science supported an instrumented Grumman Gulfstream-1 (G-1) aircraft for atmospheric field campaigns. Data from the final decade of G-1 operations were archived by the Atmospheric Radiation Measurement (ARM) user facility Data Center (ADC) and made publicly available at no cost to all registered users. To ensure a consistent data format and to improve the accessibility of the ARM airborne data, an integrated data set was recently developed covering the final six years of G-1 operations (2013 to 2018). The integrated data set includes data collected from 236 flights (766.4 hours). Four of the seven field campaigns were based in the U.S. One campaign collected data from the wildfires in the U.S. Pacific Northwest and agricultural burns in the lower Mississippi River valley as part of the Biomass Burning Observation Project (BBOP) in 2013. In 2015, the ARM Cloud Aerosol Precipitation Experiment provided data on atmospheric rivers and associated aerosol-cloud interactions that produce heavy precipitation on the U.S. west coast during the early spring. Research data from Airborne Carbon Measurements-V (ACME-V), collected during the summer of 2015, gave scientists insight into trends and variability of trace gases in the atmosphere over the North Slope of Alaska to improve arctic climate models. In the early summer and autumn of 2016, the Holistic Interactions of Shallow Clouds, Aerosols, and Land-Ecosystems (HI-SCALE) campaign provided an extensive data set geared toward coupled processes that affect the life cycle of shallow clouds through the interaction among aerosol, cloud, land surface, and ecosystems. In 2014 (March and October), the airborne sampling moved outside of the U.S. to the city of Manaus in central Amazonia, Brazil, where residential and industrial emissions were extensively characterized by flights of the G-1. The GoAmazon2014/15 aircraft campaign data are being integrated with aquatic and terrestrial ecosystem measurements to quantify anthropogenic perturbations to a usually pristine tropical environment. Another international airborne mission was carried out in the Eastern North Atlantic region. The Aerosol and Cloud Experiments in the Eastern North Atlantic (ACE-ENA) campaign saw the G-1 aircraft fly from Terceira Island in the Azores during the summer of 2017 and the winter of 2018. The campaign studied both seasons to measure key aerosol and cloud processes under various meteorological and cloud conditions with different aerosol sources. Then the G-1 deployed to the Sierras de Córdoba range in central Argentina from October to November 2018 for the Cloud, Aerosol, and Complex Terrain Interactions (CACTI) campaign to study orographic convective cloud interactions with their surrounding environment. These comprehensive datastreams provide much-needed insight into spatiotemporal variability of thermodynamic quantities, aerosol and cloud states, and properties for addressing essential science questions in Earth system process studies.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

Atmospheric Radiation Measurement (ARM) airborne field campaign data products between 2013 and 2018

Airborne measurements are pivotal for providing detailed, spatiotemporally resolved information about atmospheric parameters and aerosol and cloud properties, thereby enhancing our understanding of dynamic atmospheric processes. For 30 years, the US Department of Energy (DOE) Office of Science supported an instrumented Gulfstream 1 (G-1) aircraft for atmospheric field campaigns. Data from the final decade of G-1 operations were archived by the Atmospheric Radiation Measurement (ARM) Data Center and made publicly available at no cost to all registered users. To ensure a consistent data format and to improve the accessibility of the ARM airborne data, an integrated dataset was recently developed covering the final 6 years of G-1 operations (2013 to 2018, https://doi.org/10.5439/1999133; Mei and Gaustad, 2024). The integrated dataset includes data collected from 236 flights (766.4 h), which covered the Arctic, the US Southern Great Plains (SGP), the US West Coast, the eastern North Atlantic (ENA), the Amazon Basin in Brazil, and the Sierras de Córdoba range in Argentina. These comprehensive data streams provide much-needed insight into spatiotemporal variability in the thermodynamic quantities and aerosol and cloud properties for addressing essential science questions in Earth system process studies. This paper describes the DOE ARM merged G-1 datasets, including information on the acquisition, data collection challenges and future potentials, and quality control processes. It further illustrates the usage of this merged dataset to evaluate the Energy Exascale Earth System Model (E3SM) with the Earth System Model Aerosol–Cloud Diagnostics (ESMAC Diags) package.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS1978

Actinometric measurement of solar ultraviolet and development of a weighted solar UV integral

An actinometer has been developed to measure outdoor irradiance in the range 295-400 nm. Actinometric measurements of radiation are based on determination of photochemical reaction rates for reactions of known quantum efficiency. Actinometers have the advantage of providing irradiance data over surfaces of difficult accessibility; in addition, actinometrically determined irradiance data are wavelength weighted and therefore provide a useful means of assessing the degradation rates of polymers employed in solar energy systems.

Gupta, A.↗

NASA NTRS1983

Acoustic environmental accuracy requirements for response determination

A general purpose computer program was developed for the prediction of vehicle interior noise. This program, named VIN, has both modal and statistical energy analysis capabilities for structural/acoustic interaction analysis. The analytic models and their computer implementation were verified through simple test cases with well-defined experimental results. The model was also applied in a space shuttle payload bay launch acoustics prediction study. The computer program processes large and small problems with equal efficiency because all arrays are dynamically sized by program input variables at run time. A data base is built and easily accessed for design studies. The data base significantly reduces the computational costs of such studies by allowing the reuse of the still-valid calculated parameters of previous iterations.

Pettitt, M. R.↗

NASA NTRS1988

High-speed data acquisition for the Princeton University Dynamic Model Track

Real time analysis of data can reduce the time involved in exploring dynamic systems. The failure of the data acquisition system at the Princeton Dynamic Model Track prompted its replacement with a real time data acquisition system. Data can be obtained from an experiment and analyzed during and immediately following a data run. The new system employs high speed analog to digital conversion and a small computer to collect data. Sampling rates of 1000 hertz over 44 channels (44,000 words/sec) are obtainable. The data can be accessed as it enters the computer's environment where it may be displayed or stored for later processing. The system was tested on a helicopter rotor steep descent experiment. The data collected compares with previous data from a similar experiment.

Demko, Michael G.↗