Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Web service”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10

Hydrological Data at the NASA GES DISC: Current Capabilities and New Opportunities

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is one of twelve NASA Earth science data centers that document, process, archive and distribute data from Earth observation missions and projects. GES DISC maintains an archive of several hydrology datasets, including the Land Data Assimilation Systems (LDAS) and the Gravity Recovery and Climate Experiment (GRACE) Data Assimilation for Drought Monitoring (GRACE-DA-DM) data products. These datasets include model output of heat fluxes, rain, snow, soil temperature, soil moisture, and runoff; and observational forcing data, including surface pressure, temperature, precipitation, downward shortwave and longwave radiation, humidity, and wind. The temporal resolution of the hydrology data at GES DISC ranges from hourly to monthly, and spatial resolutions range from 0.1° to 1.0°. The GES DISC provides services which enable users to aggregate, temporally and spatially subset, regrid, and visualize archived data including the GES DISC Subsetter, Hydrology Data Rods, and the Geospatial Interactive Online Visualization and Analysis Infrastructure (GIOVANNI). The Hydrology Data Rods service optimally reorganizes large hydrological data sets as extended time series, providing more efficient access for the hydrological community. The time series data (aka “data rods”) were integrated into hydrology community tools, such as the Data Rods Explorer on HydroShare. Furthermore, the GES DISC is in the process of migrating its data and services to the cloud. Hydrological data available at the GES DISC are now available in the Amazon Web Services (AWS) cloud (us-west-2 region) providing users Direct S3 data access and the capability for cloud computing operations. In this presentation, the hydrology data products and services currently available at the GES DISC will be summarized. Also discussed are the migration to the cloud, user support through this transition, and the status of migrating the data rods service to the cloud.

Ashley Heath↗

Toward an Autonomous Workflow for Single Crystal Neutron Diffraction

The operation of the neutron facility relies heavily on beamline scientists. Some experiments can take one or two days with experts making decisions along the way. Leveraging the computing power of HPC platforms and AI advances in image analyses, here we demonstrate an autonomous workflow for the single-crystal neutron diffraction experiments. The workflow consists of three components: an inference service that provides real-time AI segmentation on the image stream from the experiments conducted at the neutron facility, a continuous integration service that launches distributed training jobs on Summit to update the AI model on newly collected images, and a frontend web service to display the AI tagged images to the expert. Ultimately, the feedback can be directly fed to the equipment at the edge in deciding the next-step experiment without requiring an expert in the loop. With the analyses of the requirements and benchmarks of the performance for each component, this effort serves as the first step toward an autonomous workflow for real-time experiment steering at ORNL neutron facilities.

Yin, Junqi↗

Leveraging Technology Services for Public Good

As the technological world expands, vulnerabilities of our critical infrastructure are becoming clear. Fortunately, emerging services provide an opportunity to improve the efficiency and security of current practices. In particular, serverless computing (such as Amazon Web Services and REDFISHs Acequia) provide opportunities to improve current practices. However, the critical infrastructure needs to evolve and that will require due diligence to ensure that transferring aspects of its practices onto the internet is done in a secure manner.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

INCREASING THE TRANSPARENCY AND REPRODUCIBILITY OF SPACE RADIATION SCIENCE: THE RADIATION BIOLOGY ONTOLOGY

Among the primary objectives of the Open/Open-Source Science paradigm are making scientific investigation data transparent and results reproducible [1], objectives shared by the FAIR principles [2]. To accomplish this, the conceptual framework that includes all the investigation objects needs to be accurately captured and communicated to all data consumers. A large part of this requires using metadata standards to annotate data collected. These standards should be readily accessible, informed by scientific community consensus and sufficiently specific to encompass all of the important aspects of the investigation. Starting in 2020 we have been co-leading an open consortium to develop a new metadata standard, the Radiation Biology Ontology (RBO), through the Open Biological and Biomedical Ontologies (OBO) Foundry [3]. We began by transforming many of the terms from the National Council on Radiation Protection and Measurement into concepts that can be formally related to existing OBO Foundry classes or attributes. We then identified and imported into the RBO existing OBO Foundry classes that have obvious relevance for radiation biomedicine (for example, concepts from the Environment Ontology that describe radiative processes, and concepts from the Gene Ontology dealing with molecular and cellular responses to radiation). Finally, we scrutinized datasets from investigations of radiation effects held in NASA GeneLab and LSDA repositories and added additional classes, instances, and attributes into the RBO that should be used to annotate these data. We developed the RBO using the open-source tools of GitHub and publish the RBO periodically through the NIH/NCBI BioPortal website, so systems worldwide can leverage the knowledge it contains [4]. This initial phase of concept modeling has yielded an RBO that at present has more than 300 declared concepts, with more than 3500 additional concepts imported from other OBO Foundry ontologies. While this first phase has focused on concepts for annotating samples, environments, exposures, and measurements, the next phase will center on supporting annotation of results and findings, such as concept models of molecular, cellular and tissue effects. The value of the RBO will be determined in part by our ability to engage the community in its development, and we have established a Radiobiology Informatics Consortium with unrestricted membership as the owner of the RBO in order to encourage investigators, system owners and other to join in this effort. Anyone can report issues or request new concept modeling or other features directly on GitHub. By using the BioPortal application programming interface, systems can pose dynamic queries to the latest version of the RBO for information on individual classes or entire hierarchies; this design eliminates the need for systems to be updated in order to use newer versions of the RBO. We hope to contribute to the advancement of open radiobiological science through the continued, open development of the RBO, that will provide more precise, machine-interpretable descriptions of investigations, as well as support data meta-analysis through machine learning or other artificial intelligence methods. REFERENCES [1] Open science in space. Nature Medicine, 2021. 27(9): p. 1485-1485. [2] Wilkinson, M.D., et al., The FAIR Guiding Principles for scientific data management and stewardship. Sci Data, 2016. 3: p. 160018. [3] Smith, B., et al., The OBO Foundry: coordinated evolution of ontologies to support biomedical data integration. Nat Biotechnol, 2007. 25(11): p. 1251-5. [4] Whetzel, P.L., et al., BioPortal: enhanced functionality via new Web services from the National Center for Biomedical Ontology to access and use ontologies in software applications. Nucleic Acids Res, 2011. 39(Web Server issue): p. W541-5.

informatics↗

Excel to NGDS Services Tool

This site allows state data contributors to standardize their data in a USIGN content model, ensure that it meets all specifications, reprojects spatial data, and creates a geodatabase that is ready for web services deployment for tier 3 data.

Good, Jessica↗

Enhancing In-Flight Transoceanic Communications Using Swift-64 Packet Mode Service

Current aeronautical communications can be divided into two segments. The first provides state of the art, packet switched technology to the cabin passengers so that they have access to e-mail and web services. The second provides basic circuit switch communication technology to the cockpit, which does not use bandwidth as efficiently as packet switching nor promotes resource sharing. This paper explores the research efforts currently being conducted by the NASA/Glenn Research Center (GRC) for transoceanic communications. The goal is to bring packet mode services to both the cabin and the cockpit of the aircraft and be able to attain benefits by sharing the data link with cabin services. First, this paper will outline the goals of the program and detail the benefits and issues related to this research. We will explain our current laboratory setup and show an architecture implemented in the testbed. Finally, we will present a work plan that will show the progression of research over the next year. This plan will describe a complete cycle from conceptual design and laboratory implementation to the final flight testing.

Slywczak, Richard A.↗

Excel to NGDS Services Tool

This site allows state data contributors to standardize their data in a USIGN content model, ensure that it meets all specifications, reprojects spatial data, and creates a geodatabase that is ready for web services deployment for tier 3 data.

Good, Jessica↗

Enabling High Availability Service with oVirt Virtualization and CephFS

The online computing environment at STAR has generated demand for high availability of services (HAS) and a resilient uptime guarantee. Such services include databases, webservers, and storage systems that user and sub-systems tend to rely on for their critical workflows. Standard deployment of services on bare metal creates a problem if the fundamental hardware fails or loses connectivity. Additionally, the configuration of redundant failover nodes (a secondary Web service for example) requires constant syncing of configuration and content and sometimes manual interaction for switching hardware (DNS name comes to mind). For this project, we will focus on two tools: oVirt and Ceph. oVirt is an OpenSource virtualization management application that enables central management of hardware nodes, storage, and network resources used to deploy and monitor virtual machines. oVirt supports the deployment of a virtual environment for your data center leveraging automatic provisioning, live migration, and the ability to easily scale the number of hypervisors. oVirt enables the use of multiple storage technologies where you can store virtual machines, images, and templates within one or multiple storage systems. STAR's recent efforts focused on deploying a CephFS POSIX compliant distributed storage system, we would enable the ability to couple our Ceph storage with the oVirt virtualization management system. When designing an intricate system for hosting critical services, it is a requirement to circumvent single points of failure. This work will involve the testing and viability of such an approach along with test cases for high availability, live migration, and service on demand.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

The Astronomy Commons Platform: A Deployable Cloud-based Analysis Platform for Astronomy

Abstract We present a scalable, cloud-based science platform solution designed to enable next-to-the-data analyses of terabyte-scale astronomical tabular data sets. The presented platform is built on Amazon Web Services (over Kubernetes and S3 abstraction layers), utilizes Apache Spark and the Astronomy eXtensions for Spark for parallel data analysis and manipulation, and provides the familiar JupyterHub web-accessible front end for user access. We outline the architecture of the analysis platform, provide implementation details and rationale for (and against) technology choices, verify scalability through strong and weak scaling tests, and demonstrate usability through an example science analysis of data from the Zwicky Transient Facility’s 1Bn+ light-curve catalog. Furthermore, we show how this system enables an end user to iteratively build analyses (in Python) that transparently scale processing with no need for end-user interaction. The system is designed to be deployable by astronomers with moderate cloud engineering knowledge, or (ideally) IT groups. Over the past 3 yr, it has been utilized to build science platforms for the DiRAC Institute, the ZTF partnership, the LSST Solar System Science Collaboration, and the LSST Interdisciplinary Network for Collaboration and Computing, as well as for numerous short-term events (with over 100 simultaneous users). In a live demo instance, the deployment scripts, source code, and cost calculators are accessible. 4 4 http://hub.astronomycommons.org/

79 ASTRONOMY AND ASTROPHYSICS↗

Satellite Estimation of Fractional Cover in Several California Specialty Crops

Past research in California and elsewhere has revealed strong relationships between satellite NDVI, photosynthetically active vegetation fraction (Fc), and crop evapotranspiration (ETc). Estimation of ETc can support efficiency of irrigation practice, which enhances water security and may mitigate nitrate leaching. The U.C. Cooperative Extension previously developed the CropManage (CM) web application for evaluation of crop water requirement and irrigation scheduling for several high-value specialty crops. CM currently uses empirical equations to predict daily Fc as a function of crop type, planting date and expected harvest date. The Fc prediction is transformed to fraction of reference ET and combined with reference data from the California Irrigation Management Information System to estimate daily ETc. In the current study, atmospherically-corrected Landsat NDVI data were compared with in-situ Fc estimates on several crops in the Salinas Valley during 2011-2014. The satellite data were observed on day of ground collection or were linearly interpolated across no more than an 8-day revisit period. Results will be presented for lettuce, spinach, celery, broccoli, cauliflower, cabbage, peppers, and strawberry. An application programming interface (API) allows CM and other clients to automatically retrieve NDVI and associated data from NASA's Satellite Irrigation Management Support (SIMS) web service. The SIMS API allows for queries both by individual points or user-defined polygons, and provides data for individual days or annual timeseries. Updates to the CM web app will convert these NDVI data to Fc on a crop-specific basis. The satellite observations are expected to play a support role in Salinas Valley, and may eventually serve as a primary data source as CM is extended to crop systems or regions where Fc is less predictable.

satellite↗

Data Quality Screening Service

A report describes the Data Quality Screening Service (DQSS), which is designed to help automate the filtering of remote sensing data on behalf of science users. Whereas this process often involves much research through quality documents followed by laborious coding, the DQSS is a Web Service that provides data users with data pre-filtered to their particular criteria, while at the same time guiding the user with filtering recommendations of the cognizant data experts. The DQSS design is based on a formal semantic Web ontology that describes data fields and the quality fields for applying quality control within a data product. The accompanying code base handles several remote sensing datasets and quality control schemes for data products stored in Hierarchical Data Format (HDF), a common format for NASA remote sensing data. Together, the ontology and code support a variety of quality control schemes through the implementation of the Boolean expression with simple, reusable conditional expressions as operands. Additional datasets are added to the DQSS simply by registering instances in the ontology if they follow a quality scheme that is already modeled in the ontology. New quality schemes are added by extending the ontology and adding code for each new scheme.

Strub, Richard↗

Economic Impact Assessments (EIA) of application of GEOGLOWS in Ecuador: Data Gaps, Limitations and Recommendations

In 2022, the United Nations launched the Early Warnings for All (EW4ALL) Program to establish global early warning systems by 2027. To assess the impact of the substantial $3.1 billion annual investment over five years, EW4ALL will consider factors that will require national coordination for the data needed for these assessments. In 2023, Ecuador was identified as one of the world's most climate-vulnerable countries, emphasizing the need to enhance its early warning systems. In 2020, the SERVIR Amazonia hub implemented the GEOGLOWS streamflow forecast service in collaboration with Ecuador's national meteorological agency (INAMHI). GEOGloWS provides 15-day ensemble forecasts and 80 years of historical streamflow data for every river worldwide through a free web service. The World Meteorological Organization has recognized this initiative as essential in contributing to the UN's call to ensure an 'Early Warning for All' by 2027. In 2023, as part of NASA's continuous efforts to fund research for Policy-Relevant Implementations, an economic impact assessment (EIA) was performed to understand the potential socioeconomic benefits of Early streamflow predictions in Ecuador using the GEOGLOWS service. Preliminary findings highlighted that gaps remain in effectively integrating socioeconomic and Earth observation (EO) data to capture the total value of these predictions. Implementing GEOGLOWS has led to valuable hydrological forecasts; however, the total economic benefits have yet to be documented. This study addresses the gaps and makes recommendations for future work that should focus on capturing the socio-economic benefits and costs associated with these forecasts, including their impact on decision-making at national and local levels. Despite the daily use of GEOGLOWS by key figures, including the President of Ecuador, the need for comprehensive recommendations and assessments is urgent.

Reetwika Basu↗

FASTQ quality control dashboard

FQCDB builds up existing open source software, FastQC, implementing a modern web interface for across parsed output of FastQC. In addition, FQCDB is extensible as a web service to include additional plots of type line, boxplot, or heatmap, across data formatted according to guidelines. The interface is also configurable via more readable JSON format, enabling customization by non-web programmers.

Source record↗

What Do Our Users Want? Perspectives on Understanding and Meeting User Needs for Multi-Mission Data Services

The Sun-Earth Connection Active Archive (SECAA) project of NASA's Space Physics Data Facility operates a range of unique and heavily used multi-mission data services in support of the large-scale science objectives of the Great Observatory, including services such as CDAWeb, the CDAWeb Plus client, SSCWeb, OMNIweb and the CDF data format. In developing and operating these services, we have encountered and continue to struggle with a wide range of issues such as balancing scope and functionality with simplicity and ease of use, understanding the effectiveness of our choices and identifying areas most important for further improvement. In this paper, we will review our key services and then discuss some of our observations and new approaches to understanding and meeting user data service requirements. Some observations are obvious but may still have substantial implications; e.g. functionality without information content is of little user interest, which has led to our recent emphasis on development of web services interfaces, so the content and functionality we already serve is readily and fully available as a building block for new services. Some observations require careful design and tradeoffs; e.g. users will complain when they are offered interfaces with limited options but users are also easily intimidated and become lost when offered extensive options for customization. Some observations remain highly challenging; e.g. a comprehensive multi-mission, multi-source view of all data and services available easily produces a daunting list, but a more selective view can easily lead users to overlook available and relevant data. It is often difficult to obtain and meaningfully interpret measures of true productive usage and overall user satisfaction, even with a variety of techniques including statistics, citations, case studies, user feedback and advisory committees. Most of these issues will apply to and may even be more acute for distributed implementation architectures.

McGuire, Robert E.↗

FASTQ quality control dashboard

FQCDB builds up existing open source software, FastQC, implementing a modern web interface for across parsed output of FastQC. In addition, FQCDB is extensible as a web service to include additional plots of type line, boxplot, or heatmap, across data formatted according to guidelines. The interface is also configurable via more readable JSON format, enabling customization by non-web programmers.

Source record↗

The architecture of an event correlation service for adaptive middleware-based applications

Loosely coupled component communication driven by events is a key mechanism for building middleware- based applications that must achieve reliable qualities of service in an adaptive manner. In such a system, events that encapsulate state snapshots of a running system are generated by monitoring components. Hence, an event correlation service is necessary for correlating monitored events from multiple sources. The requirements for the event correlation raise two challenges: to seamlessly integrate event correlation services with other services and applications; and to provide reliable event management with minimal delay. This paper describes our experience in the design and implementation of an event correlation service. The design encompasses an event correlator and an event proxy that are integrated with an architecture for adaptive middleware components. The implementation utilizes the common-based event (CBE) specification and stateful Web service technologies to support the deployment of the event correlation service in a distributed architecture. We evaluate the performance of the overall solution in a test bed and present the results in terms of the trade-off between the flexibility and the performance overhead of the architecture

99 GENERAL AND MISCELLANEOUS↗

Virtual Solar Observatory Distributed Query Construction

Through a prototype implementation (Tian et al., this meeting) the VSO has already demonstrated the capability of unifying geographically distributed data sources following the Web Services paradigm and utilizing mechanisms such as the Simple Object Access Protocol (SOAP). So far, four participating sites (Stanford, Montana State University, National Solar Observatory and the Solar Data Analysis Center) permit Web-accessible, time-based searches that allow browse access to a number of diverse data sets. Our latest work includes the extension of the simple, time-based queries to include numerous other searchable observation parameters. For VSO users, this extended functionality enables more refined searches. For the VSO, it is a proof of concept that more complex, distributed queries can be effectively constructed and that results from heterogeneous, remote sources can be synthesized and presented to users as a single, virtual data product.

Gurman, J. B.↗

Use of Schema on Read in Earth Science Data Archives

Traditionally, NASA Earth Science data archives have file-based storage using proprietary data file formats, such as HDF and HDF-EOS, which are optimized to support fast and efficient storage of spaceborne and model data as they are generated. The use of file-based storage essentially imposes an indexing strategy based on data dimensions. In most cases, NASA Earth Science data uses time as the primary index, leading to poor performance in accessing data in spatial dimensions. For example, producing a time series for a single spatial grid cell involves accessing a large number of data files. With exponential growth in data volume due to the ever-increasing spatial and temporal resolution of the data, using file-based archives poses significant performance and cost barriers to data discovery and access. Storing and disseminating data in proprietary data formats imposes an additional access barrier for users outside the mainstream research community. At the NASA Goddard Earth Sciences Data Information Services Center (GES DISC), we have evaluated applying the schema-on-read principle to data access and distribution. We used Apache Parquet to store geospatial data, and have exposed data through Amazon Web Services (AWS) Athena, AWS Simple Storage Service (S3), and Apache Spark. Using the schema-on-read approach allows customization of indexing spatially or temporally to suit the data access pattern. The storage of data in open formats such as Apache Parquet has widespread support in popular programming languages. A wide range of solutions for handling big data lowers the access barrier for all users. This presentation will discuss formats used for data storage, frameworks with This presentation will discuss formats used for data storage, frameworks with support for schema-on-read used for data access, and common use cases covering data usage patterns seen in a geospatial data archive.

cloud applications↗