Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Cloud Computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 883 records · Page 49
All sourcesNASA NTRSDOE OSTI
NASA NTRS2013

Surface Irradiances Consistent With CERES-Derived Top-of-Atmosphere Shortwave and Longwave Irradiances

The estimate of surface irradiance on a global scale is possible through radiative transfer calculations using satellite-retrieved surface, cloud, and aerosol properties as input. Computed top-of-atmosphere (TOA) irradiances, however, do not necessarily agree with observation-based values, for example, from the Clouds and the Earth's Radiant Energy System (CERES). This paper presents amethod to determine surface irradiances using observational constraints of TOA irradiance from CERES. A Lagrange multiplier procedure is used to objectively adjust inputs based on their uncertainties such that the computed TOA irradiance is consistent with CERES-derived irradiance to within the uncertainty. These input adjustments are then used to determine surface irradiance adjustments. Observations by the Atmospheric Infrared Sounder (AIRS), Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO), CloudSat, andModerate Resolution Imaging Spectroradiometer (MODIS) that are a part of the NASA A-Train constellation provide the uncertainty estimates. A comparison with surface observations from a number of sites shows that the bias [root-mean-square (RMS) difference] between computed and observed monthlymean irradiances calculated with 10 years of data is 4.7 (13.3) W/sq m for downward shortwave and 22.5 (7.1) W/sq m for downward longwave irradiances over ocean and 21.7 (7.8) W m22 for downward shortwave and 21.0 (7.6) W/sq m for downward longwave irradiances over land. The bias andRMS error for the downward longwave and shortwave irradiances over ocean are decreased from those without constraint. Similarly, the bias and RMS error for downward longwave over land improves, although the constraint does not improve downward shortwave over land. This study demonstrates how synergetic use of multiple instruments (CERES,MODIS, CALIPSO, CloudSat, AIRS, and geostationary satellites) improves the accuracy of surface irradiance computations.

Kato, Seiji↗
DOE OSTI2023

Snowmass 2021 Computational Frontier CompF4 Topical Group Report Storage and Processing Resource Access

Computing plays a significant role in all areas of high energy physics. The Snowmass 2021 CompF4 topical group's scope is facilities R&D, where we consider "facilities" as the computing hardware and software infrastructure inside the data centers plus the networking between data centers, irrespective of who owns them, and what policies are applied for using them. In other words, it includes commercial clouds, federally funded High Performance Computing (HPC) systems for all of science, and systems funded explicitly for a given experimental or theoretical program. This topical group report summarizes the findings and recommendations for the storage, processing, networking and associated software service infrastructures for future high energy physics research, based on the discussions organized through the Snowmass 2021 community study.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS1979

Scattering by randomly oriented ellipsoids: Application to aerosol and cloud problems

A program was developed for computing the scattering and absorption by arbitrarily oriented and randomly oriented prolate and oblate spheroids. This permits examination of the effect of particle shape for cases ranging from needles through spheres to platelets. Applications of this capability to aerosol and cloud problems are discussed. Initial results suggest that the effect of nonspherical particle shape on transfer of radiation through aerosol layers and cirrus clouds, as required for many climate studies, can be readily accounted for by defining an appropriate effective spherical particle radius.

Asano, S.↗
NASA NTRS1978

Sampling analysis for the Earth Radiation Budget Satellite System mission based on orbital coverage and cloud variability

It is pointed out that accurate computation of the earth's radiation budget from satellite measurements requires spatial and temporal sampling which accounts for variations in cloud and surface conditions. It is in this connection important that cloud radiative properties and areal cloud cover be accurately determined for the desired time period. The reported investigation represents an extension of a study by Harrison et al. (1976). The time and space coverage capabilities of various sampling schemes are analyzed for the proposed Earth Radiation Budget Satellite System. The effects of cloud cover variability on the measured monthly mean reflected shortwave (0.2-5.0 micrometers) irradiance are also analyzed for several spatial scales.

Harrison, E. F.↗
DOE OSTI2017

Integration of end-user Cloud storage for CMS analysis

End-user Cloud storage is increasing rapidly in popularity in research communities thanks to the collaboration capabilities it offers, namely synchronisation and sharing. CERN IT has implemented a model of such storage named, CERNBox, integrated with the CERN AuthN and AuthZ services. To exploit the use of the end-user Cloud storage for the distributed data analysis activity, the CMS experiment has started the integration of CERNBox as a Grid resource. This will allow CMS users to make use of their own storage in the Cloud for their analysis activities as well as to benefit from synchronisation and sharing capabilities to achieve results faster and more effectively. It will provide an integration model of Cloud storages in the Grid, which is implemented and commissioned over the world’s largest computing Grid infrastructure, Worldwide LHC Computing Grid (WLCG). In this paper, we present the integration strategy and infrastructure changes needed in order to transparently integrate end-user Cloud storage with the CMS distributed computing model. We describe the new challenges faced in data management between Grid and Cloud and how they were addressed, along with details of the support for Cloud storage recently introduced into the WLCG data movement middleware, FTS3. Finally, the commissioning experience of CERNBox for the distributed data analysis activity is also presented.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1983

Cloud-particle galactic gas dynamics and star formation

Galactic gas dynamics, spiral structure, and star formation are discussed in relation to N-body computational studies based on a cloud-particle model of the interstellar medium. On the small scale, the interstellar medium is seen as cloud-dominated and supernova-perturbed. It is noted that the cloud-particle model simulates cloud-cloud collisions, the formation of stellar associations, and supernova explosions as dominant local processes. On the large scale, in response to a spiral galactic gravitational field, global density waves and galactic shocks develop having large-scale characteristics similar to those found in continuum gas dynamical studies. Both the system of gas clouds and the system of young stellar associations forming from the clouds figure in the global spiral structure. However, with the attributes of neither assuming a continuum of gas (as in continuum gas dynamical studies) or requiring a prescribed equation of state (such as the isothermal condition), the cloud-particle picture retains much of the detail lost in earlier work. By detail is meant the small-scale features and structures so important in understanding the local, turbulent state of the interstellar medium as well as the degree of raggedness often seen to be superposed on the global spiral structure.

Roberts, W. W., Jr.↗
DOE OSTI2016

The FIFE Project at Fermilab

The FabrIc for Frontier Experiments (FIFE) project is an initiative within the Fermilab Scientific Computing Division designed to steer the computing model for non-LHC Fermilab experiments across multiple physics areas. FIFE is a collaborative effort between experimenters and computing professionals to design and develop integrated computing models for experiments of varying size, needs, and infrastructure. The major focus of the FIFE project is the development, deployment, and integration of solutions for high throughput computing, data management, database access and collaboration management within an experiment. To accomplish this goal, FIFE has developed workflows that utilize Open Science Grid compute sites along with dedicated and commercial cloud resources. The FIFE project has made significant progress integrating into experiment computing operations several services including a common job submission service, software and reference data distribution through CVMFS repositories, flexible and robust data transfer clients, and access to opportunistic resources on the Open Science Grid. The progress with current experiments and plans for expansion with additional projects will be discussed. FIFE has taken the leading role in defining the computing model for Fermilab experiments, aided in the design of experiments beyond those hosted at Fermilab, and will continue to define the future direction of high throughput computing for future physics experiments worldwide.

Box, D.↗
DOE OSTI2016

Adoption of High Performance Computational (HPC) Modeling Software for Widespread Use in the Manufacture of Welded Structures

This report summarizes the final product developed for the US DOE Small Business Innovation Research (SBIR) Phase II grant made to Engineering Mechanics Corporation of Columbus (Emc 2 ) between April 16, 2014 and August 31, 2016 titled ‘Adoption of High Performance Computational (HPC) Modeling Software for Widespread Use in the Manufacture of Welded Structures’. Many US companies have moved fabrication and production facilities off shore because of cheaper labor costs. A key aspect in bringing these jobs back to the US is the use of technology to render US-made fabrications more cost-efficient overall with higher quality. One significant advantage that has emerged in the US over the last two decades is the use of virtual design for fabrication of small and large structures in weld fabrication industries. Industries that use virtual design and analysis tools have reduced material part size, developed environmentally-friendly fabrication processes, improved product quality and performance, and reduced manufacturing costs. Indeed, Caterpillar Inc. (CAT), one of the partners in this effort, continues to have a large fabrication presence in the US because of the use of weld fabrication modeling to optimize fabrications by controlling weld residual stresses and distortions and improving fatigue, corrosion, and fracture performance. This report describes Emc 2 ’s DOE SBIR Phase II final results to extend an existing, state-of-the-art software code, Virtual Fabrication Technology (VFT®), currently used to design and model large welded structures prior to fabrication - to a broader range of products with widespread applications for small and medium-sized enterprises (SMEs). VFT® helps control distortion, can minimize and/or control residual stresses, control welding microstructure, and pre-determine welding parameters such as weld-sequencing, pre-bending, thermal-tensioning, etc. VFT® uses material properties, consumable properties, etc. as inputs. Through VFT®, manufacturing companies can avoid costly design changes after fabrication. This leads to the concept of joint design/fabrication where these important disciplines are intimately linked to minimize fabrication costs. Finally service performance (such as fatigue, corrosion, and fracture/damage) can be improved using this product. Emc 2 ’s DOE SBIR Phase II effort successfully adapted VFT® to perform efficiently in an HPC environment independent of commercial software on a platform to permit easy and cost effective access to the code. This provides the key for SMEs to access this sophisticated and proven methodology that is quick, accurate, cost effective and available “on-demand” to address weld-simulation and fabrication problems prior to manufacture. In addition, other organizations, such as Government agencies and large companies, may have a need for spot use of such a tool. The open source code, WARP3D, a high performance finite element code used in fracture and damage assessment of structures, was significantly modified so computational weld problems can be solved efficiently on multiple processors and threads with VFT®. The thermal solver for VFT®, based on a series of closed form solution approximations, was extensively enhanced for solution on multiple processors greatly increasing overall speed. In addition, the graphical user interface (GUI) was re-written to permit SMEs access to an HPC environment at the Ohio Super Computer Center (OSC) to integrate these solutions with WARP3D. The GUI is used to define all weld pass descriptions, number of passes, material properties, consumable properties, weld speed, etc. for the structure to be modeled. The GUI was enhanced to make it more user-friendly so that non-experts can perform weld modeling. Finally, an extensive outreach program to market this capability to fabrication companies was performed. This access will permit SMEs to perform weld modeling to improve their competitiveness at a reasonable cost.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
NASA NTRS2020

An Examination of the Impact of Grid Spacing on WRF Simulations of Wintertime Precipitation in the Mid-Atlantic United States

A large set of deterministic and ensemble forecasts was produced to identify the optimal spacing for forecasting U.S. East Coast snowstorms. WRF forecasts were produced on cloud-allowing (~1-km grid spacing) and convection-allowing (3–4 km) grids, and compared against forecasts with parameterized convection (>~10 km). Performance diagrams were used to evaluate 19 deterministic forecasts from the winter of 2013–14. Ensemble forecasts of five disruptive snowstorms spanning the years 2015–18 were evaluated using various methods to evaluate probabilistic forecasts. While deterministic forecasts using cloud-allowing grids were not better than convection-allowing forecasts, both had lower bias and higher success ratios than forecasts with parameterized convection. All forecasts were underdispersive. Nevertheless, forecasts on the higher-resolution grids were more reliable than those with parameterized convection. Forecasts on the cloud-allowing grid were best able to discriminate areas that received heavy snow and those that did not, while the forecasts with parameterized convection were least able to do so. It is recommended to use convection-resolving and (if computationally possible) to use cloud-allowing forecast grids when predicting East Coast winter storms.

Snowstorms↗
DOE OSTI2016

Virtual Facility at Fermilab: Infrastructure and Services Expand to Public Clouds

In preparation for its new Virtual Facility Project, Fermilab has launched a program of work to determine the requirements for running a computation facility on-site, in public clouds, or a combination of both. This program builds on the work we have done to successfully run experimental workflows of 1000-VM scale both on an on-site private cloud and on Amazon AWS. To do this at scale we deployed dynamically launched and discovered caching services on the cloud. We are now testing the deployment of more complicated services on Amazon AWS using native load balancing and auto scaling features they provide. The Virtual Facility Project will design and develop a facility including infrastructure and services that can live on the site of Fermilab, off-site, or a combination of both. We expect to need this capacity to meet the peak computing requirements in the future. The Virtual Facility is intended to provision resources on the public cloud on behalf of the facility as a whole instead of having each experiment or Virtual Organization do it on their own. We will describe the policy aspects of a distributed Virtual Facility, the requirements, and plans to make a detailed comparison of the relative cost of the public and private clouds. Furthermore, this talk will present the details of the technical mechanisms we have developed to date, and the plans currently taking shape for a Virtual Facility at Fermilab.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2015

Enabling opportunistic resources for CMS Computing Operations

With the increased pressure on computing brought by the higher energy and luminosity from the LHC in Run 2, CMS Computing Operations expects to require the ability to utilize opportunistic resources resources not owned by, or a priori configured for CMS to meet peak demands. In addition to our dedicated resources we look to add computing resources from non CMS grids, cloud resources, and national supercomputing centers. CMS uses the HTCondor/glideinWMS job submission infrastructure for all its batch processing, so such resources will need to be transparently integrated into its glideinWMS pool. Bosco and parrot wrappers are used to enable access and bring the CMS environment into these non CMS resources. Finally, we describe our strategy to supplement our native capabilities with opportunistic resources and our experience so far using them.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1974

Altitude determination and descriptive analysis of clouds on ERTS-1 multispectral photography

A simple method to determine the approximate altitude of clouds is described, with the objective of refining their classification using only marginal data from the photographs. Results of the application of this method on photographs of the Goajira Peninsula, Paraguana Peninsula and the Central Coast of Venezuela are presented. Here, the altitudes computed are used to classify clouds and to identify the genus of others without typical form. Instability of air masses through clouds vertical development, and wind direction as well as other local climatic characteristics such as moisture content, loci of condensation, area, etc. are determined using repetitive coverage for the time interval of the photography. Applications for the regional and urban planning (including airport location and flights schedule) and natural resources evaluation are suggested.

Albrizzio, C.↗
DOE OSTI2020

Seismology in the Cloud: A New Streaming Workflow

Data-intensive research in seismology is experiencing a recent boom, driven in part by large volumes of available data and advances in the growing field of data science. However, there are significant barriers to processing large data volumes, such as long retrieval times from data repositories, complex data management, and limited computational resources. New tools and platforms have reduced the barriers to entry for scientific cluster computing, including the maturation of the commercial cloud as an accessible instrument for research. Here, we build a customized research cluster in the cloud to test a new workflow for large-scale seismic analysis, in which data are processed as a stream (retrieved on-the-fly and acted upon without storing), with data from the Incorporated Research Institutions for Seismology Data Management Center. We use this workflow to deploy a spectral peak detection algorithm over 5.6 TB of compressed continuous seismic data from 2074 stations of the USArray Transportable Array EarthScope network. Using a 50-node cluster in the cloud, we completed the noise survey in 80 hr, with an average data throughput of 1.7 GB per minute. By varying cluster sizes, we find the scaling of our analysis to be sublinear, due to a combination of algorithmic limitations and data center response times. The cloud-based streaming workflow represents an order-of-magnitude increase in acquisition and processing speed compared to a traditional download-store-process workflow, and offers the additional benefits of employing a flexible, accessible, and widely used computing architecture. It is limited, however, due to its reliance on Internet transfer speeds and data center service capacity, and may not work well for repeated analyses or those for which even higher data throughputs are needed. These research applications will require a new class of cloud-native approaches in which both data and analysis are in the cloud.

58 GEOSCIENCES↗
DOE OSTI2022

Design and implementation of dynamic I/O control scheme for large scale distributed file systems

In this paper, we have analyzed the input/output (I/O) activities of Cori, which is a high-performance computing system at the National Energy Research Scientific Computing Center at Lawrence Berkeley National Laboratory. Our analysis results indicate that most users do not adjust storage configurations but rather use the default settings. In addition, owing to the interference from many applications running simultaneously, the performance varies based on the system status. To configure file systems autonomously in complex environments, we developed DCA-IO, a dynamic distributed file system configuration adjustment algorithm that utilizes the system log information to adjust storage configurations automatically. Our scheme aims to improve the application performance and avoid interference from other applications without user intervention. Moreover, DCA-IO uses the existing system logs and does not require code modifications, an additional library, or user intervention. To demonstrate the effectiveness of DCA-IO, we performed experiments using I/O kernels of real applications in both an isolated small-sized Lustre environment and Cori. Our experimental results shows that our scheme can improve the performance of HPC applications by up to 263% with the default Lustre configuration.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

An Improved Algorithm for Estimating Surface Shortwave Radiation: Preliminary Evaluation With MODIS Products

Cloud parameters, as key inputs in radiative transfer algorithms, have a critical impact on surface shortwave radiation (SSR) computation. By introducing a parameterization of cloud transmittance and reflectance, based on radiative transfer simulations, this study improves the accuracy of an existing physically based model which severely underestimates SSR under thick cloud conditions. The cloud parameterization adopts the single-layer cloud model and simulates cloud transmittances and reflectances by varying cloud optical thickness, cloud particle size, and solar zenith angle. The revised model is applied to estimate instantaneous SSR using Moderate-resolution Imaging Spectroradiometer (MODIS) atmospheric and land products. The retrieved SSR is evaluated against observation data from 41 Baseline Surface Radiation Network (BSRN) stations and is also compared with the MODIS official SSR product. The root mean square error (RMSE) of the estimated instantaneous radiation is approximately 52 and 98 W m -2 under clear-sky and all-sky conditions, respectively. The accuracy of the improved parameterization is higher than that of the original model, and there is no obvious underestimation of SSR in the case of high cloud optical thickness. Therefore, the new algorithm improves the accuracy of SSR estimates in the presence of thick clouds. Retrievals with the improved model also achieve higher accuracy than the MODIS official SSR product (MCD18A1). To conclude, the reliable performance of the scheme at most BSRN stations illustrates that the improved model can be used to map SSR on a global scale.

42 ENGINEERING↗