Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “service migration”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

LAADS DAAC Migrates to the Cloud: Lessons Learned from Communicating About Earth Science Data on the Cloud

The Level-1 and Atmosphere Archive Distribution System (LAADS) Distributed Active Archive Center (DAAC) is migrating data to the cloud. As one of twelve DAACS supported by NASA’s Earth Science Data and Information System (ESDIS), LAADS is using moving away from on premise data storage facilities to migrating to Amazon Web Services, where the massive archive of data from the Moderate Imaging Spectroradiometer (MODIS) and the Visible Infrared Imaging Radiometer Suite (VIIRS) will be available for download and post-processing transformations online. The migration is happening in three phases and LAADS is concluding its beta testing period. This poster shows the lessons learned from communicating with a select group of users about how to effectively educate data users on using data in the cloud.

Tassia Owen↗

New Metadata Capabilities within NASA's Common Metadata Repository (CMR)

This talk will convey the new capabilities and features of the UMM-Variables and the UMM-Services metadata model within the Common Metadata Repository (CMR) and how community engagement through the ESDIS Standards Office (ESO) review shaped the new versions of the models. The Unified Metadata Model (UMM) provides a common metadata model to unify legacy systems (i.e. GCMD (Global Change Master Directory), ECHO (Earth Observing System (EOS) Clearinghouse)) with new systems (i.e. CMR). The rationale and migration process of the Service Entry Resource Format (SERF) to the UMM-S will also be conveyed. The talk will conclude with discussing issues and lessons learned from the review process and how future reviews will be conducted to ensure a more targeted, meaningful review with a faster turn-around time for triage and implementation.

Earth Observing System↗

ON THE IMPACT OF SUPER RESOLUTION WSR-88D DOPPLER RADAR DATA ASSIMILATION ON HIGH RESOLUTION NUMERICAL MODEL FORECASTS

Assimilation of radar velocity and precipitation fields into high-resolution model simulations can improve precipitation forecasts with decreased 'spin-up' time and improve short-term simulation of boundary layer winds (Benjamin, 2004 & 2007; Xiao, 2008) which is critical to improving plume transport forecasts. Accurate description of wind and turbulence fields is essential to useful atmospheric transport and dispersion results, and any improvement in the accuracy of these fields will make consequence assessment more valuable during both routine operation as well as potential emergency situations. During 2008, the United States National Weather Service (NWS) radars implemented a significant upgrade which increased the real-time level II data resolution to 8 times their previous 'legacy' resolution, from 1 km range gate and 1.0 degree azimuthal resolution to 'super resolution' 250 m range gate and 0.5 degree azimuthal resolution (Fig 1). These radar observations provide reflectivity, velocity and returned power spectra measurements at a range of up to 300 km (460 km for reflectivity) at a frequency of 4-5 minutes and yield up to 13.5 million point observations per level in super-resolution mode. The migration of National Weather Service (NWS) WSR-88D radars to super resolution is expected to improve warning lead times by detecting small scale features sooner with increased reliability; however, current operational mesoscale model domains utilize grid spacing several times larger than the legacy data resolution, and therefore the added resolution of radar data is not fully exploited. The assimilation of super resolution reflectivity and velocity data into high resolution numerical weather model forecasts where grid spacing is comparable to the radar data resolution is investigated here to determine the impact of the improved data resolution on model predictions.

54 ENVIRONMENTAL SCIENCES↗

Coupling of Nuclear Waste Form Corrosion and Radionuclide Transports in Presence of Relevant Repository Sediments

Assessments of waste form and disposal options start with the degradation of the waste forms and consequent mobilization of radionuclides. Long-term static tests, single-pass flow-through tests, and the pressurized unsaturated flow test are often employed to study the durability of potential waste forms and to help create models that predict their durability throughout the lifespan of the disposal site. These tests involve the corrosion of the material in the presence of various leachants, with different experimental designs yielding desired information about the behavior of the material. Though these tests have proved instrumental in elucidating various mechanisms responsible for material corrosion, the chemical environment to which the material is subject is often not representative of a potential radioactive waste repository where factors such as pH and leachant composition will be controlled by the near-field environment. Near-field materials include, but are not limited to, the original engineered barriers, their resulting corrosion products, backfill materials, and the natural host rock. For an accurate performance assessment of a nuclear waste repository, realistic waste corrosion experimental data ought to be modeled to allow for a better understanding of waste form corrosion mechanisms and the effect of immediate geochemical environment on these mechanisms. Additionally, the migration of radionuclides in the resulting chemical environment during and after waste form corrosion must be quantified and mechanisms responsible for migrations understood. The goal of this research was to understand the mechanisms responsible for waste form corrosion in the presence of relevant repository sediments to allow for accurate radionuclide migration quantifications. The rationale for this work is that a better understanding of waste form corrosion in relevant systems will enable increased reliance on waste form performance in repository environments and potentially decrease the need for expensive engineered barriers.Our current work aims are 1) quantifying and understanding the processes associated with glass alteration in contact with Fe-bearing materials; 2) quantifying and understanding the processes associated with glass alteration in presence of MgO (example of engineered barrier used in WIPP); 3) identifying glass alteration suppressants and the processes involved to reach glass alteration suppression; 4) quantifying and understanding the processes associated with Saltstone and Cast Stone (SRS and Hanford cementitious waste forms) in various representative groundwaters; 5) investigating positron annihilation as a new tool for the study of glass alteration; and 6) quantifying and understanding the processes associated with glass alteration under gamma irradiation.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Cloud-Based Demonstration of the Eastern Interconnection Situational Awareness Monitoring System (ESAMS)

This report describes a cloud-based implementation and field demonstration of the Eastern Interconnection Situational Awareness and Monitoring System (ESAMS). ESAMS was developed to support the detection and source localization of forced oscillations using synchrophasor measurements from tie-lines connecting areas served by different reliability coordinators (RCs), so that RCs could better coordinate their response to wide-area events. A previous effort had identified deployment barriers associated with hosting shared situational awareness tools at a single RC. To address these barriers, ESAMS was migrated to Amazon Web Services and evaluated in a six-month field demonstration. ISO New England (ISO-NE) and PJM streamed data to the platform using AWS Direct Connect and a site-to-site VPN, respectively. The resulting multi-utility measurement footprint enabled regional source localization across major portions of the U.S. Eastern Interconnection and supported routine identification of oscillation events. During the final three months of the trial, 24 events above 2 MW/MVAR were detected. The largest detected oscillation approached a 25 MW peak-to-peak amplitude, and the longest persisted intermittently for more than 11 hours. The demonstration also assessed operational considerations—including data transfer volumes, end-to-end latency, and cloud computing costs—and found that network and compute requirements were modest relative to typical cloud capabilities while providing performance comparable to prior on-premises deployments. Overall, the results indicate that cloud hosting can provide a practical path to shared interconnection-wide oscillation monitoring. The cloud ESAMS demonstration establishes a foundation for broader utility participation and for building future wide-area analytics that leverage measurements across organizational boundaries.

Follum, James D.↗

Trade Study: Storing NASA HDF5/netCDF-4 Data in the Amazon Cloud and Retrieving Data via Hyrax Server / THREDDS Data Server

As part of the overall effort to understand implications of migrating ESDIS data and services to the cloud we are testing several common OPeNDAP and HDF use cases against three architectures for general performance and cost characteristics. The architectures include retrieving entire files, retrieving datasets using HTTP range gets, and retrieving elements of datasets (chunks) with HTTP range gets. We will describe these architectures and discuss our approach to estimating cost.

HDF↗

WGISS-46 ACCESS Section: IDN Report

The objective of this presentation is to provide IDN updates on features and activities to the Committee on Earth Observation Satellites (CEOS) Working Group on Information Systems and Services (WGISS) and provider community. The following topics will be will be discussed during the presentation: Overview of NASA's Vision of Services (UMM-S), Migration of IDN SERFs to UMM-S, Metadata Management Tool (MMT): Successor to DocBuilder, DocBuilder-10: Create/Update Dataset Records, Transition of DIF-9 to DIF-10, Schedule for UMM-C and GCMD/IDN Keywords, IDN's WGISS/WGCV Joint Session: Collaborative Actions.

Committee on Earth Observation Satellites↗

"Sensor Web Evolution - Webs of Webs for NASA Science - Focus on small Uninhabited Aerial Systems (sUAS)"

This paper will describe the evolution of information collection, derivation and delivery mechanisms in webs of NASA sensor webs, with a focus on recent advancements in small Uninhabited Aerial Systems (sUAS). I will discuss the movement to "Fog Computing", also known as Edge Computing. Fog Computing facilitates the distribution of common operations and networking between edge devices and cloud computing facilities, optimizing the production of actionable intelligence. Initially, sUASs utilized onboard data collection as standard, with minimal data downloaded directly. Information products were derived in conventional computational environments, generally desk top computers, and information products made available to the Science Community in weeks or months. With the increased availability, and increasingly lower costs, of beyond line of sight (BLOS) satellite based communication, transmission rates and data volumes increased, and processing migrated to Cloud based services. Contemporary sUASs are moving some of that information product derivation to on vehicle services, and are creating a distributed Cloud/Fog environment. I will describe the technological advances that have made this possible, including low power multi-core Central Processing Units (CPU), and, more recently, the availability of high end Graphical Processing Units (GPU) that consume only a few watts. Intelligent system software, leveraging these hardware advances, finally allows for information product generation on-board, rather than simple data collection. Additionally, intelligent flight control systems now support mutual vehicle to vehicle collaboration, allowing sUASs to create ad-hoc sensor webs on demand, as required. Also discussed will be the lessons learned by the Authors' development of data systems for NASA's large High Altitude Long Endurance (HALE) UASs like Predator and Global Hawk, and how those lessons are being applied to sUAS development. This paper will focus on application, rather a deep dive into the technology, and will highlight improving data management through these new technologies.

Sensor Web↗

UASs in the VOG/Edge/FOG Sensor Web Environment

This paper will describe the evolution of information collection, derivation and delivery mechanisms in sensor webs utilizing Uninhabited Aerial Systems (UAS).We will discuss the movement to "Fog Computing", also known as Edge Computing. Fog Computing facilitates the distribution of common operations and networking between edge devices and cloud computing facilities, optimizing the production of actionable intelligence. Initially, UASs utilized onboard data collection as standard, with minimal data downloaded directly. Information products were derived in conventional computational environments, generally desk top computers, and information products made available to the Science Community in weeks or months. With the increased availability, and increasingly lower costs, of beyond line of sight (BLOS) satellite based communication, transmission rates and data volumes increased, and processing migrated to Cloud based services. Contemporary UASs are moving some of that information product derivation to on vehicle services, and are creating a distributed Cloud/Fog environment. The Author will describe the technological advances that have made this possible, including low power multi-core Central Processing Units (CPU), and, more recently, the availability of high end Graphical Processing Units (GPU) that consume only a few watts. Intelligent system software, leveraging these hardware advances, finally allows for information product generation on-board, rather than simple data collection. Additionally, intelligent flight control systems now support mutual vehicle to vehicle collaboration, allowing UASs to create ad-hoc sensor webs on demand, as required. Also discussed will be the lessons learned by the Authors' development of data systems for NASA's large High Altitude Long Endurance (HALE) UASs like Predator and Global Hawk, and how those lessons are being applied to other UAS development This paper will focus on applications, rather a deep dive into the technology, and will highlight improving data management through these new technologies.

UAS↗

Technical Challenges and Lessons from the Migration of the GLOBE Data and Information System to Utilize Cloud Computing Service

The Global Learning and Observation to Benefit the Environment (GLOBE) Data and Information System supports an international science and education program with capabilities to accept local environment observations, archive, display and visualize them along with global satellite observations. Since its inception twenty years ago, the Web and database system has been upgraded periodically to accommodate the changes in technology and the steady growth of GLOBEs education community and collection of observations. Recently, near the end-of-life of the system hardware, new commercial computer platform options were explored and a decision made to utilize Cloud services. Now the GLOBE DIS has been fully deployed and maintained using Amazon Cloud services for over two years now. This paper reviews the early risks, actual challenges, and some unexpected findings as a result of the GLOBE DIS migration. We describe the plans, cost drivers and estimates, highlight adjustments that were made and suggest improvements. We present the trade studies for provisioning, for load balancing, networks, processing, storage, as well as production, staging and backup systems. We outline the migration teams skills and required level of effort for transition, and resulting changes in the overall maintenance and operations activities. Examples include incremental adjustments to processing capacity and frequency of backups, and efforts previously expended on hardware maintenance that were refocused onto application-specific enhancements.

GLOBE↗

Technical Challenges and Lessons from the Migration of the GLOBE Data and Information System to Utilize Cloud Computing Service

The Global Learning and Observation to Benefit the Environment (GLOBE) Data and Information System supports an international science and education program with capabilities to accept local environment observations, archive, display and visualize them along with global satellite observations. Since its inception twenty years ago, the Web and database system has been upgraded periodically to accommodate the changes in technology and the steady growth of GLOBEs education community and collection of observations. Recently, near the end-of-life of the system hardware, new commercial computer platform options were explored and a decision made to utilize Cloud services. Now the GLOBE DIS has been fully deployed and maintained using Amazon Cloud services for over two years now. This paper reviews the early risks, actual challenges, and some unexpected findings as a result of the GLOBE DIS migration. We describe the plans, cost drivers and estimates, highlight adjustments that were made and suggest improvements. We present the trade studies for provisioning, for load balancing, networks, processing, storage, as well as production, staging and backup systems. We outline the migration teams skills and required level of effort for transition, and resulting changes in the overall maintenance and operations activities. Examples include incremental adjustments to processing capacity and frequency of backups, and efforts previously expended on hardware maintenance that were refocused onto application-specific enhancements.

GLOBE Data↗

Exploiting Kubernetes to Simplify the Deployment and Management of the Multi-purpose CMS Pilot Job Factory

GlideinWMS, a widely utilized workload management system in high-energy physics (HEP) research, serves as the backbone for efficient job provisioning across distributed computing resources. It is utilized by various experiments and organizations, including CMS, OSG, Dune, and FIFE, to create HTCondor pools as large as 600k cores. In particular, a shared factory service historically deployed at UCSD has been configured to interface with more than 500 routes to compute clusters. As part of our team’s initiative to modernize infrastructure and enhance scalability, we undertook the migration of the GlideinWMS factory service into the Kubernetes environment. Leveraging the flexibility and orchestration capabilities of Kubernetes, we successfully deployed the factory service within the OSG Tiger Kubernetes cluster. The major benefits Kubernetes gives us is it streamlines the management and monitoring of the factory infrastructure, and improves fault tolerance through its resilient deployment strategies. Through this case study, we aim to share insights, challenges, and best practices encountered during the migration process. Our experience underscores the benefits of embracing containerization and Kubernetes orchestration for HEP computing infrastructure, paving the way for scalability and resilience in distributed computing environments.

Dost, Jeffrey Michael [UC, San Diego (main)]↗

Cloud-Based Time Series Analysis of Extremes: Use Cases and Applications

"Extreme weather events, such as hurricanes, tornadoes, floods, droughts, heatwaves, and blizzards, can cause widespread damage, disrupting ecosystems, agricultural production, and economies. The frequency and intensity of these events have been increasing, likely due to climate change, raising concerns and the need for more accurate analysis and predictions. NASA's Goddard Earth Sciences Data and Information Services Center (GES DISC) has migrated its long-term historical datasets, including precipitation data from MERRA-2 reanalysis, GLDAS land data assimilation, and IMERG satellite observations, to the cloud. This cloud-stored data enables scientists and researchers to utilize cloud computing for advanced modeling and forecasting of extreme weather events, eliminating the need to download large datasets. In this presentation, we will provide an overview of the cloud-based data and services managed by GES DISC; demonstrate methods for accessing and analyzing time series data stored in the cloud; and compare results across various datasets to address critical questions related to extreme precipitation. We will present use cases including: 1. Determining the average total precipitation in California during January and February from 2000 to 2024, and identifying anomalous precipitation in 2021. 2. Calculating the 10, 20, 50, and 100-year return periods for maximum daily rainfall based on 25 years of historical precipitation data (2000-2024) for Maryland.

time series↗

Next generation experimental data access at NSLS-II

The NSLS-II network and computing infrastructure has been significantly updated recently. The re-IP process in 2020-2021 enabled the NSLS-II network to be routable to the rest of the BNL campus. Then, standardization of the operating systems and deployment procedures helped to deliver a consistent environment to workstations and servers used by all NSLS-II beamlines. In particular, the RedHat Enterprise Linux 8 was deployed to 700+ machines using the RedHat Satellite infrastructure management product, and all critical services (IOCs, databases, etc.) were migrated to the new OS. NFS users’ home directories are consistent across all of the machines, which eliminates the need for the individual configuration of the user environment on each host. The standard suite of software packages is available to the beamline staff and users, which includes the system packages (deployed via RPM) as well as the conda environments for data acquisition and analysis. Security measures were implemented to comply with the industry standards, which include multi-factor authentication (using Duo), secure screen lock for the beamline machines, and advanced access control to the experimental data that is stored in shared central storage available on all hosts. These major enhancements facilitated sharing the experimental data (currently for a number of selected beamlines, with a plan to extend it to the whole facility in the nearest future) with the users via an externally facing JupyterHub instance. The beamlines keep using the Bluesky data acquisition framework to orchestrate their experiments, and the new infrastructure enabled them to use a next-generation data access library called tiled.

36 MATERIALS SCIENCE↗

Semantic Property Graph for Scalable Knowledge Graph Analytics

Graphs are a natural and fundamental representation to describe entities, relationships, activities, and evolution of complex systems. Many domains such as communication, citation, procurement, biology, social media, and transportation can be modeled as a set of entities and their relationships. Resource Description Framework (RDF) and Labeled Property Graph (LPG) are two of the most used data models to encode information in a graph. Both models are similar in terms of using basic graph elements such as nodes and edges but differ in terms of the modeling approach, expressibility, serialization, and target applications. RDF is a flexible data exchange model for expressing information about entities but it tends to a have high memory footprint and inefficient storage, which does not make it a natural choice to perform scalable graph analytics. In contrast, LPG has gained traction as a reliable model to perform scalable graph analytic tasks such as sub-graph matching, network alignment, and real-time knowledge graph query. It provides efficient storage, fast traversal, and flexibility to model various real-world domains. At the same time, the LPG lacks the support of a formal knowledge representation such as an ontology to provide automated knowledge inference. We propose Semantic Property Graph (SPG) as a logical projection of reified RDF into the LPG model. SPG continues to use RDF ontology to define the type hierarchy of the projected graph and validate it against a given ontology. We present a framework to convert reified RDF graphs into SPG using two different computing environments. We also present cloud-based graph migration capabilities using Amazon Web Services.

Purohit, Sumit↗

Transparent process migration: Design alternatives and the Sprite implementation

The Sprite operating system allows executing processes to be moved between hosts at any time. We use this process migration mechanism to offload work onto idle machines, and also to evict migrated processes when idle workstations are reclaimed by their owners. Sprite's migration mechanism provides a high degree of transparency both for migrated processes and for users. Idle machines are identified, and eviction is invoked, automatically by daemon processes. On Sprite it takes up to a few hundred milliseconds on SPARCstation 1 workstations to perform a remote exec, while evictions typically occur in a few seconds. The pmake program uses remote invocation to invoke tasks concurrently. Compilations commonly obtain speedup factors in the range of three to six; they are limited primarily by contention for centralized resources such as file servers. CPU-bound tasks such as simulations can make more effective use of idle hosts, obtaining as much as eight-fold speedup over a period of hours. Process migration has been in regular service for over two years.

Douglis, Fred↗

Trends in Local Therapy Utilization and Cost for Early-Stage Breast Cancer in Older Women: Implications for Payment and Policy Reform

Purpose: Older women with early-stage disease constitute the most rapidly growing breast cancer demographic, yet it is not known which local therapy strategies are most favored by this population in the current era. Understanding utilization trends and cost of local therapy is important for informing the design of bundled payment models as payers migrate away from fee-for-service models. We therefore used the Surveillance, Epidemiology, and End Results Medicare database to determine patterns of care and costs for local therapy among older women with breast cancer. Methods and Materials: Treatment strategy and covariables were determined in 55,327 women age ≥66 with Tis-T2N0-1M0 breast cancer who underwent local therapy between 2000 and 2008. Trends in local therapy were characterized using Joinpoint. Polychotomous logistic regression determined predictors of local therapy. The median aggregate cost over the first 24 months after diagnosis was determined from Medicare claims through 2010 and reported in 2014 dollars. Results: The median age was 75. Local therapy distribution was as follows: 27,896 (50.3%) lumpectomy with external beam radiation, 18,356 (33.1%) mastectomy alone, 6159 (11.1%) lumpectomy alone, 1488 (2.7%) mastectomy with reconstruction, and 1455 (2.6%) lumpectomy with brachytherapy. Mastectomy alone declined from 39.0% in 2000 to 28.2% in 2008, and the use of breast conserving local therapies rose from 58.7% to 68.2%. Mastectomy with reconstruction was more common among the youngest, healthiest patients, whereas mastectomy alone was more common among patients living in rural low-income regions. By 2008, the costs were $36,749 for lumpectomy with brachytherapy, $35,030 for mastectomy with reconstruction, $31,388 for lumpectomy with external beam radiation, $21,993 for mastectomy alone, and $19,287 for lumpectomy alone. Conclusions: The use of mastectomy alone in older women declined in favor of breast conserving strategies between 2000 and 2008. Using these cost estimates, price points for local therapy bundles can be constructed to incentivize the treatment strategies that confer the highest value.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Secure Network-Centric Aviation Communication (SNAC)

The existing National Airspace System (NAS) communications capabilities are largely unsecured, are not designed for efficient use of spectrum and collectively are not capable of servicing the future needs of the NAS with the inclusion of new operators in Unmanned Aviation Systems (UAS) or On Demand Mobility (ODM). SNAC will provide a ubiquitous secure, network-based communications architecture that will provide new service capabilities and allow for the migration of current communications to SNAC over time. The necessary change in communication technologies to digital domains will allow for the adoption of security mechanisms, sharing of link technologies, large increase in spectrum utilization, new forms of resilience and redundancy and the possibly of spectrum reuse. SNAC consists of a long term open architectural approach with increasingly capable designs used to steer research and development and enable operating capabilities that run in parallel with current NAS systems.

Communications↗