SEARCH · Engineering Papers

Results for “science data systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30

All sources NASA NTRS DOE OSTI

NASA NTRS2017

NASA's EOSDIS Cumulus: Ingesting, Archiving, Managing, and Distributing Earth Science Data from the Commercial Cloud

NASA's Earth Observing System Data and Information System (EOSDIS) has been working towards a vision of a cloud-based, highly-flexible, ingest, archive, management, and distribution system for its ever-growing and evolving data holdings. This system, Cumulus, is emerging from its prototyping stages and is poised to make a huge impact on how NASA manages and disseminates its Earth science data. This talk will outline the motivation for this work, present the achievements and hurdles of the past 18 months and will chart a course for the future expansion of the Cumulus expansion. We will explore on not just the technical, but also the socio-technical challenges that we face in evolving a system of this magnitude into the cloud and how we are rising to meet those challenges through open collaboration and intentional stakeholder engagement.

Earth Scienc↗

DOE OSTI2019

OpenSolar: Promoting the Openness and Accessibility of Diverse Public Solar Datasets

Observational solar data is the foundation of data-driven research in solar power grid integration and power system operations. Compared to other fields in data science, the openness and accessibility of solar data is lacking, which prevents solar data science from catching up with the emerging trends of data science (e.g., deep learning). In this paper, OpenSolar, a package with both R and Python versions, is developed to enhance the openness and accessibility of publicly available solar datasets. The OpenSolar package provides access to multiple types of solar data, primarily from four datasets: (1) the National Renewable Energy Laboratory (NREL) Solar Power Data for Integration Studies dataset, (2) the NREL Solar Radiation Research Laboratory dataset, (3) the Sheffield Solar-Microgen database, and (4) the Dataport database. Unlike other open solar datasets that only contain meteorological data, the four datasets in the OpenSolar package also contain behind-the-meter power data, sky images, and solar power data with satisfactory temporal and spatial resolution and coverage. The overview, quality-control methods, and potential usage of the datasets, in conjunction with sample code implementing the OpenSolar functions, are described in this paper. The package is expected to assist in bridging the gaps between the research fields of solar energy, power systems, and data science.

14 SOLAR ENERGY↗

NASA NTRS2010

Multi-Instrument Tools and Services to Access NASA Earth Science Data from the GSFC Earth Sciences Data and Information Services Center

The presentation purpose is to describe multi-instrument tools and services that facilitate access and usability of NASA Earth science data at Goddard Space Flight Center (GSFC). NASA's Earth observing system includes 14 satellites. Topics include EOSDIS facilities and system architecture, and overview of GSFC Earth Science Data and Information Services Center (GES DISC) mission, Mirador data search, Giovanni, multi-instrument data exploration, Google Earth[TM], data merging, and applications.

Kempler, Steve↗

NASA NTRS2014

Cultivating an Emergent Earth Observation Analytics Ecosystem in the Cloud

A diverse set of data analytics systems for Earth Observations are sprouting up in the Earth Science community, with a wealth of processing algorithms and analysis methods. There is a similar wealth of data resources available via myriad data providers and clearinghouses, including large institutional systems like the Earth Observing System Data and Information System, Comprehensive Large Scale Array-data Stewardship System, and Federated Earth Observation Missions gateway. With Earth system science driving a need to work with more datasets together, and the community developing more analysis tools (some of them dataset-specific), how can we develop analysis workflows that incorporate far-flung datasets and leverage analysis resources from multiple organizations? Cloud computing points the way toward a solution in two different respects. Firstly, the access to and abstraction of virtually unlimited storage and computing power provides an environment that enables more straightforward means of pulling datasets and analysis resources together. Just as importantly, however, cloud computing serves as an example of an "ecosystem" of interoperating services, since the essence of cloud computing is the presentation of all resources as a service, from hardware to infrastructure to platform to software. This enables the combination of off-the-shelf, diverse services to construct entire systems that emerge out of an equally diverse community of architects and developers. This approach can be similarly applied to the data and analysis resources in the Earth Observation community. By exposing these resources via well understood services, and consuming resources in the same way, different organizations can construct bespoke analysis workflows and systems for their own purposes. The key leap the community needs to make is to develop analysis systems in components that interact with other components via services. The result would be a rich ecosystem of analytics components that can be combined to analyze datasets at scale and in conjunction with other datasets from other sources.

chaos↗

NASA NTRS2020

Famine Early Warning Systems Network (FEWS NET) Land Data Assimilation System (LDAS) and Other Assimilated Hydrological Data at NASA GES DISC

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) provides science support for several data sets relevant to agriculture and food security, including the Famine Early Warning Systems Network (FEWS NET) Land Data Assimilation System (LDAS), or FLDAS data set. The GES DISC is one of twelve NASA Earth Observing System (EOS) data centers that process, archive, document, and distribute data from Earth science missions and related projects. The GES DISC hosts a wide range of remote sensing and model data, and provides reliable and robust data access and other services to users worldwide. Beyond data archive and access, the GES DISC offers many services to visualize and analyze the data. This presentation provides a summary of the hydrological data available at the GES DISC, along with an overview of related data services. Specifically, the FLDAS data set has been adapted to work with domains, data streams, and monitoring and forecast requirements associated with food security assessment in data-sparse, developing country settings. The FLDAS global monthly data have a 0.1 x 0.1 degree spatial resolution covering the period from January 1982 to present. Global FLDAS monthly anomaly and monthly climatology data are also available at the GES DISC to evaluate how current conditions compare to averages over the FLDAS 35-year period. Several case studies using the FLDAS soil moisture, evapotranspiration, rainfall, runoff, and surface temperature data will be presented.

Loeser, Carlee↗

NASA NTRS1992

NSSDC Conference on Mass Storage Systems and Technologies for Space and Earth Science Applications, volume 3

This report contains copies of nearly all of the technical papers and viewgraphs presented at the National Space Science Data Center (NSSDC) Conference on Mass Storage Systems and Technologies for Space and Earth Science Applications. This conference served as a broad forum for the discussion of a number of important issues in the field of mass storage systems. Topics include magnetic disk and tape technologies, optical disk and tape, software storage and file management systems, and experiences with the use of a large, distributed storage system. The technical presentations describe, among other things, integrated mass storage systems that are expected to be available commercially. Also included is a series of presentations from Federal Government organizations and research institutions covering their mass storage requirements for the 1990s.

Kobler, Ben↗

NASA NTRS2022

The Mars 2020 Ground Data System Architecture

The Mars 2020 Mission’s primary objective is to collect 20 geographically unique samples during its prime mission of one and a quarter Martian years, or just over 2 Earth years. Mission planners determined the project needed to develop a system that would enable the operations team to analyze engineering and science data, make science decisions, select viable rover targets at a millimeter resolution and validate an uplink bundle for a car sized rover with more complex science instruments than any previous Mars surface mission. All this had to be done within a five hour time frame. Doing this with a small team would be a challenge, but this had to be accomplished by a large team of engineers and scientists located across North America and Europe. Achieving this level of operational efficiency was unheard of in the prime mission. In addition, the mission had another set of requirements that had nothing to do with surface operations; the Mars 2020 Ground Data System (GDS) was also expected to comply with a new set of security requirements to keep up with the ever changing cybersecurity landscape. The Mars 2020 Ground Data System (GDS) is a re-architected version of the Mars Science Laboratory GDS. The primary goal was to integrate the lessons learned from previous Mars surface missions, accommodate a set of new requirements and capabilities required to ensure mission success, and comply with a new set of cybersecurity controls. The new architecture includes several unique qualities including a data lake, language-agnostic system-wide event-based operations, containerization, automated deployment, network segmentation, infrastructure-as-code, API-driven interfaces, and the first Mars surface GDS to operate primarily in the cloud. The new architecture enabled greater access to the system’s data, tighter integration with the operations team, and a higher level of traceability. The availability of the data also enabled a new set of capabilities previously not possible on surface missions. These new capabilities include an autonomous data to information, pipeline for downlink analysis, horizontal scaling of science data processing capabilities, autonomous round trip data tracking of science and engineering data, integration of flight system state into the tactical planning cycle, high fidelity targeting utilizing kinematic data, and hierarchical image and 3d meshes data representations. This paper will introduce the requirements for the Mars 2020 Mission, the heritage architecture, and the rationale for the changes to achieve the new architecture. The paper will continue to describe the fundamental changes made to the GDS architecture, how these changes enabled a more tightly integrated GDS, and the new capabilities that were enabled by the new architecture. The paper will conclude with the lessons learned from the process of rearchitecting a heritage GDS system and from the first 200 days of operations supporting over 800 users from around the world.

Lopez-Roig, Reynaldo↗

NASA NTRS1991

Information data systems for a global change technology initiative architecture trade study

The Global Change Technology Initiative (GCTI) was established to develop technology which will enable use of satellite systems of Earth observations on a global scale, enable use of the observations to predictively model Earth's changes, and provide scientists, government, business, and industry with quick access to the resulting information. At LaRC, a GCTI Architecture Trade Study was undertaken to develop and evaluate the architectural implications to meet the requirements of the global change studies and the eventual implementation of a global change system. The output of the trade study are recommended technologies for the GCTI. That portion of the study concerned with the information data system is documented. The information data system for an earth global change modeling system can be very extensive and beyond affordability in terms of today's costs. Therefore, an incremental approach to gaining a system is most likely. An options approach to levels of capability versus needed technologies was developed. The primary drivers of the requirements for the information data system evaluation were the needed science products, the science measurements, the spacecraft orbits, the instruments configurations, and the spacecraft configurations and their attendant architectures. The science products requirements were not studied here; however, some consideration of the product needs were included in the evaluation results. The information data system technology items were identified from the viewpoint of the desirable overall information system characteristics.

Murray, Nicholas D.↗

NASA NTRS2003

Data Access Tools And Services At The Goddard Distributed Active Archive Center (GDAAC)

As one of the largest providers of Earth Science data from the Earth Observing System, GDAAC provides the latest data from the Moderate Resolution Imaging Spectroradiometer (MODIS), Atmospheric Infrared Sounder (AIRS), Solar Radiation and Climate Experiment (SORCE) data products via GDAAC's data pool (50TB of disk cache). In order to make this huge volume of data more accessible to the public and science communities, the GDAAC offers multiple data access tools and services: Open Source Project for Network Data Access Protocol (OPeNDAP), Grid Analysis and Display System (GrADS/DODS) (GDS), Live Access Server (LAS), OpenGlS Web Map Server (WMS) and Near Archive Data Mining (NADM). The objective is to assist users in retrieving electronically a smaller, usable portion of data for further analysis. The OPeNDAP server, formerly known as the Distributed Oceanographic Data System (DODS), allows the user to retrieve data without worrying about the data format. OPeNDAP is capable of server-side subsetting of HDF, HDF-EOS, netCDF, JGOFS, ASCII, DSP, FITS and binary data formats. The GrADS/DODS server is capable of serving the same data formats as OPeNDAP. GDS has an additional feature of server-side analysis. Users can analyze the data on the server there by decreasing the computational load on their client's system. The LAS is a flexible server that allows user to graphically visualize data on the fly, to request different file formats and to compare variables from distributed locations. Users of LAS have options to use other available graphics viewers such as IDL, Matlab or GrADS. WMS is based on the OPeNDAP for serving geospatial information. WMS supports OpenGlS protocol to provide data in GIs-friendly formats for analysis and visualization. NADM is another access to the GDAAC's data pool. NADM gives users the capability to use a browser to upload their C, FORTRAN or IDL algorithms, test the algorithms, and mine data in the data pool. With NADM, the GDAAC provides an environment physically close to the data source. NADM will benefit users with mining or offer data reduction algorithms by reducing large volumes of data before transmission over the network to the user.

Pham, Long↗

NASA NTRS2000

The Mars Microprobe Mission: Advanced Micro-Avionics for Exploration Surface

The Mars Microprobe Mission is the second spacecraft developed as part of the New Millennium Program deep space missions. The objective of the Microprobe Project is to demonstrate the applicability of key technologies for future planetary missions by developing two probes for deployment on Mars. The probes are designed with a single stage entry, descent, and landing system and impact the Martian surface at speeds of approximately 200 meters per second. The microprobes are composed of two main sections, a forebody section that penetrates to a depth below the Martian surface of 0.5 to 2 meters, and an aftbody section that remains on the surface. Each probe system consists of a number of advanced technology components developed specifically for this mission. These include a non-erosive aeroshell for entry into. the atmosphere, a set of low temperature batteries to supply probe power, an advanced microcontroller to execute the mission sequence, collect the science data, and react to possible system fault conditions, a telecommunications subsystem implemented on a set of custom integrated circuits, and instruments designed to provide science measurements from above and below the Martian surface. All of the electronic components have been designed and fabricated to withstand the severe impact shock environment and to operate correctly at predicted temperatures below -100 C.

Blue, Randel↗

DOE OSTI2018

Advance reservation access control using software-defined networking and tokens

Advance reservation systems allow users to reserve dedicated bandwidth connection resources from advanced high-speed networks. A common use case for such systems is data transfers in distributed science environments in which a user wants exclusive access to the reservation. However, current advance network reservation methods cannot ensure exclusive access of a network reservation to the specific flow for which the user made the reservation. We present here a novel network architecture that addresses this limitation and ensures that a reservation is used only by the intended flow. We achieve this by leveraging software-defined networking (SDN) and token-based authorization. We use SDN to orchestrate and automate the reservation of networking resources, end-to-end and across multiple administrative domains, and tokens to create a strong binding between the user or application that requested the reservation and the flows provisioned by SDN. We conducted experiments on the ESNet 100G SDN testbed, and demonstrated that our system effectively protects authorized flows from competing traffic in the network.

admission control↗

DOE OSTI2020

Rapid data-driven model reduction of nonlinear dynamical systems including chemical reaction networks using ℓ 1 -regularization

We develop a new data-driven paradigm for efficient model reduction of a broad class of nonlinear dynamical systems. Our model reduction method directly enables the interpretation of key components of the dynamical system, unlike traditional projection-based model reduction methods that focus on reducing computational complexity more than interpretability. Our method is not application specific and is simple to implement on nonlinear dynamical systems arising from a variety of different fields. It requires minimal parameterization using a single parameter to trade-off between model complexity and estimation error. We use a data-driven paradigm to formulate model reduction as an efficient convex optimization problem that scales polynomially in the original size of the complex system, enabling systems with as many as thousands of components to be reduced in a matter of minutes.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2024

DMTN-065: Data Management for LSST Special Programs

Special Programs are additional survey areas and/or observing strategies that are driven by specific science goals which build on, or are beyond, the core science pillars of the Wide Fast Deep Main Survey. In order to meet the requirements and enable science related to Special Programs, this document provides recommendations for Rubin Data Management regarding processing and serving data products for Special Programs. Hardware and processing boundaries on the potential diversity of data from Special Programs are discussed along with scenarios in which user-generated processing and data products might be needed to meet Special Programs' science goals.

79 ASTRONOMY AND ASTROPHYSICS↗

NASA NTRS1984

The Pilot Land Data System: Report of the Program Planning Workshops

An advisory report to be used by NASA in developing a program plan for a Pilot Land Data System (PLDS) was developed. The purpose of the PLDS is to improve the ability of NASA and NASA sponsored researchers to conduct land-related research. The goal of the planning workshops was to provide and coordinate planning and concept development between the land related science and computer science disciplines, to discuss the architecture of the PLDs, requirements for information science technology, and system evaluation. The findings and recommendations of the Working Group are presented. The pilot program establishes a limited scale distributed information system to explore scientific, technical, and management approaches to satisfying the needs of the land science community. The PLDS paves the way for a land data system to improve data access, processing, transfer, and analysis, which land sciences information synthesis occurs on a scale not previously permitted because of limits to data assembly and access.

Source record↗

NASA NTRS2012

Using Quality Attributes to Bridge Systems Engineering Gaps : A Juno Ground Data Systems Case Study

The Juno Mission to Jupiter is the second mission selected by the NASA New Frontiers Program. Juno launched August 2011 and will reach Jupiter July 2016. Juno's payload system is composed of nine instruments plus a gravity science experiment. One of the primary functions of the Juno Ground Data System (GDS) is the assembly and distribution of the CFDP (CCSDS File Delivery Protocol) product telemetry, also referred to as raw science data, for eight out of the nine instruments. The GDS accomplishes this with the Instrument Data Pipeline (IDP). During payload integration, the first attempt to exercise the IDP in a flight like manner revealed that although the functional requirements were well understood, the system was unable to meet latency requirements with the as-is heritage design. A systems engineering gap emerged between Juno instrument data delivery requirements and the assumptions behind the heritage flight-ground interactions. This paper describes the use of quality attributes to measure and overcome this gap by introducing a new systems engineering activity, and a new monitoring service architecture that successfully delivered the performance metrics needed to validate Juno IDP.

Ground Data Systems (GDS)↗

NASA NTRS1993

Embedded Data Processor and Portable Computer Technology testbeds

Attention is given to current activities in the Embedded Data Processor and Portable Computer Technology testbed configurations that are part of the Advanced Data Systems Architectures Testbed at the Information Sciences Division at NASA Ames Research Center. The Embedded Data Processor Testbed evaluates advanced microprocessors for potential use in mission and payload applications within the Space Station Freedom Program. The Portable Computer Technology (PCT) Testbed integrates and demonstrates advanced portable computing devices and data system architectures. The PCT Testbed uses both commercial and custom-developed devices to demonstrate the feasibility of functional expansion and networking for portable computers in flight missions.

Alena, Richard↗

NASA NTRSDate not supplied

Biospecimen and Data Sharing: NASA Institutional Scientific Collection at Ames Research Center (ISC-ARC), and the Ames Life Sciences Data Archive (ALSDA)

For decades, NASA and international partners have conducted biological experiments in space to understand effects of spaceflight and address potential hazards. To enable spaceflight back to the Moon, and then to Mars and beyond, it is imperative to further understand basic science and health risks associated with spaceflight, along with developing countermeasures. The sending of experiments and organisms into space is a costly endeavor. To maximize scientific return, sharing with the scientific community both space-flown biospecimens and data from completed experiments is essential. New fundamental, applied, and bioinformatic science insights can be gained from specimen and data sharing efforts. Data reuse enables spaceflight health risk modeling, analyzing adverse outcomes across spaceflight hazards, and deep space autonomous support for the flight medical officer.

Data↗

DOE OSTI2019

SDN-NGenIA, a Software Defined Next Generation Integrated Architecture for HEP and Data Intensive Science

This project aims to enable the LHC and other leading programs in high energy physics and other global science domains funded by the DOE to operate with a new level of efficiency and control, through the development of a Next Generation Integrated Architecture (NGenIA) based on intelligent software defined network (SDN)-driven network systems coupled to high throughput applications. While the initial focus is on the challenging LHC use case, the systems and products being developed are general, and apply to many fields of data intensive science ranging from astrophysical sky surveys to bioinformatics and earth observation, as well as other organizations facing the challenges of extracting knowledge from distributed multi-petabyte data stores. A central concept in this development program is a new paradigm ‘consistent network operations’ among widely distributed computing and storage facilities, where stable high throughput flows, at set rates, cross load-balanced network paths, up to flexible high water marks that are adjusted in real time to accommodate other network traffic. The large smooth flows are launched and managed by SDN services that act in concert with the experiments’ site-resident data distribution and management systems, to meet the expanding needs of the science programs. The technical goals include the construction of autonomous, intelligent site-resident services that interact dynamically with network-resident services, and with the science programs’ principal data distribution and management tools, to request or command network resources in support of high throughput petascale to exascale workflow Specific work items include: 1. Developing compact Data Transfer Nodes (DTNs) with auto-tuning functions that support data transfer rates in the 100 Gbps to the 1 Tbps range when used with high throughput data transfer applications 2. Deep site orchestration among virtualized clusters, storage subsystems and subnets to successfully coschedule CPU, storage and network resources 3. Science-program designed site architectures, operational modes, and policy and resource usage priorities, adjudicated across multiple network domains and virtual organizations, using the orchestration functions and methods 4. Seamlessly extending end-to-end operation across both extra-site and intra-site boundaries through the use of Open vSwitch (OVS), FireQoS + next generation Science DMZs 5. Novel methods of system integration that enable granular control of extreme scale long distance transfers through flow matching of scattered source-destination address pairs to multi-domain dynamic circuits 6. Funneling massive sets of streams to DTNs at the site edge hosting petascale buffer pools configured for flows of 100 Gbps and up, exploiting state of the art data transfers 7. Adaptive scheduling based on pervasive end-to-end monitoring, including DTN or compute-node resident agents providing comprehensive end-system profiling 8. Developing unsupervised and supervised machine learning and modeling methods to optimize the workflow involving terabyte to multi-petabyte datasets.

47 OTHER INSTRUMENTATION↗