Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “CLOUD COMPUTING”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 883 records · Page 49

Integration of end-user Cloud storage for CMS analysis

End-user Cloud storage is increasing rapidly in popularity in research communities thanks to the collaboration capabilities it offers, namely synchronisation and sharing. CERN IT has implemented a model of such storage named, CERNBox, integrated with the CERN AuthN and AuthZ services. To exploit the use of the end-user Cloud storage for the distributed data analysis activity, the CMS experiment has started the integration of CERNBox as a Grid resource. This will allow CMS users to make use of their own storage in the Cloud for their analysis activities as well as to benefit from synchronisation and sharing capabilities to achieve results faster and more effectively. It will provide an integration model of Cloud storages in the Grid, which is implemented and commissioned over the world’s largest computing Grid infrastructure, Worldwide LHC Computing Grid (WLCG). In this paper, we present the integration strategy and infrastructure changes needed in order to transparently integrate end-user Cloud storage with the CMS distributed computing model. We describe the new challenges faced in data management between Grid and Cloud and how they were addressed, along with details of the support for Cloud storage recently introduced into the WLCG data movement middleware, FTS3. Finally, the commissioning experience of CERNBox for the distributed data analysis activity is also presented.

97 MATHEMATICS AND COMPUTING↗

Cloud-particle galactic gas dynamics and star formation

Galactic gas dynamics, spiral structure, and star formation are discussed in relation to N-body computational studies based on a cloud-particle model of the interstellar medium. On the small scale, the interstellar medium is seen as cloud-dominated and supernova-perturbed. It is noted that the cloud-particle model simulates cloud-cloud collisions, the formation of stellar associations, and supernova explosions as dominant local processes. On the large scale, in response to a spiral galactic gravitational field, global density waves and galactic shocks develop having large-scale characteristics similar to those found in continuum gas dynamical studies. Both the system of gas clouds and the system of young stellar associations forming from the clouds figure in the global spiral structure. However, with the attributes of neither assuming a continuum of gas (as in continuum gas dynamical studies) or requiring a prescribed equation of state (such as the isothermal condition), the cloud-particle picture retains much of the detail lost in earlier work. By detail is meant the small-scale features and structures so important in understanding the local, turbulent state of the interstellar medium as well as the degree of raggedness often seen to be superposed on the global spiral structure.

Roberts, W. W., Jr.↗

The FIFE Project at Fermilab

The FabrIc for Frontier Experiments (FIFE) project is an initiative within the Fermilab Scientific Computing Division designed to steer the computing model for non-LHC Fermilab experiments across multiple physics areas. FIFE is a collaborative effort between experimenters and computing professionals to design and develop integrated computing models for experiments of varying size, needs, and infrastructure. The major focus of the FIFE project is the development, deployment, and integration of solutions for high throughput computing, data management, database access and collaboration management within an experiment. To accomplish this goal, FIFE has developed workflows that utilize Open Science Grid compute sites along with dedicated and commercial cloud resources. The FIFE project has made significant progress integrating into experiment computing operations several services including a common job submission service, software and reference data distribution through CVMFS repositories, flexible and robust data transfer clients, and access to opportunistic resources on the Open Science Grid. The progress with current experiments and plans for expansion with additional projects will be discussed. FIFE has taken the leading role in defining the computing model for Fermilab experiments, aided in the design of experiments beyond those hosted at Fermilab, and will continue to define the future direction of high throughput computing for future physics experiments worldwide.

Box, D.↗

Adoption of High Performance Computational (HPC) Modeling Software for Widespread Use in the Manufacture of Welded Structures

This report summarizes the final product developed for the US DOE Small Business Innovation Research (SBIR) Phase II grant made to Engineering Mechanics Corporation of Columbus (Emc 2 ) between April 16, 2014 and August 31, 2016 titled ‘Adoption of High Performance Computational (HPC) Modeling Software for Widespread Use in the Manufacture of Welded Structures’. Many US companies have moved fabrication and production facilities off shore because of cheaper labor costs. A key aspect in bringing these jobs back to the US is the use of technology to render US-made fabrications more cost-efficient overall with higher quality. One significant advantage that has emerged in the US over the last two decades is the use of virtual design for fabrication of small and large structures in weld fabrication industries. Industries that use virtual design and analysis tools have reduced material part size, developed environmentally-friendly fabrication processes, improved product quality and performance, and reduced manufacturing costs. Indeed, Caterpillar Inc. (CAT), one of the partners in this effort, continues to have a large fabrication presence in the US because of the use of weld fabrication modeling to optimize fabrications by controlling weld residual stresses and distortions and improving fatigue, corrosion, and fracture performance. This report describes Emc 2 ’s DOE SBIR Phase II final results to extend an existing, state-of-the-art software code, Virtual Fabrication Technology (VFT®), currently used to design and model large welded structures prior to fabrication - to a broader range of products with widespread applications for small and medium-sized enterprises (SMEs). VFT® helps control distortion, can minimize and/or control residual stresses, control welding microstructure, and pre-determine welding parameters such as weld-sequencing, pre-bending, thermal-tensioning, etc. VFT® uses material properties, consumable properties, etc. as inputs. Through VFT®, manufacturing companies can avoid costly design changes after fabrication. This leads to the concept of joint design/fabrication where these important disciplines are intimately linked to minimize fabrication costs. Finally service performance (such as fatigue, corrosion, and fracture/damage) can be improved using this product. Emc 2 ’s DOE SBIR Phase II effort successfully adapted VFT® to perform efficiently in an HPC environment independent of commercial software on a platform to permit easy and cost effective access to the code. This provides the key for SMEs to access this sophisticated and proven methodology that is quick, accurate, cost effective and available “on-demand” to address weld-simulation and fabrication problems prior to manufacture. In addition, other organizations, such as Government agencies and large companies, may have a need for spot use of such a tool. The open source code, WARP3D, a high performance finite element code used in fracture and damage assessment of structures, was significantly modified so computational weld problems can be solved efficiently on multiple processors and threads with VFT®. The thermal solver for VFT®, based on a series of closed form solution approximations, was extensively enhanced for solution on multiple processors greatly increasing overall speed. In addition, the graphical user interface (GUI) was re-written to permit SMEs access to an HPC environment at the Ohio Super Computer Center (OSC) to integrate these solutions with WARP3D. The GUI is used to define all weld pass descriptions, number of passes, material properties, consumable properties, weld speed, etc. for the structure to be modeled. The GUI was enhanced to make it more user-friendly so that non-experts can perform weld modeling. Finally, an extensive outreach program to market this capability to fabrication companies was performed. This access will permit SMEs to perform weld modeling to improve their competitiveness at a reasonable cost.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

An Examination of the Impact of Grid Spacing on WRF Simulations of Wintertime Precipitation in the Mid-Atlantic United States

A large set of deterministic and ensemble forecasts was produced to identify the optimal spacing for forecasting U.S. East Coast snowstorms. WRF forecasts were produced on cloud-allowing (~1-km grid spacing) and convection-allowing (3–4 km) grids, and compared against forecasts with parameterized convection (>~10 km). Performance diagrams were used to evaluate 19 deterministic forecasts from the winter of 2013–14. Ensemble forecasts of five disruptive snowstorms spanning the years 2015–18 were evaluated using various methods to evaluate probabilistic forecasts. While deterministic forecasts using cloud-allowing grids were not better than convection-allowing forecasts, both had lower bias and higher success ratios than forecasts with parameterized convection. All forecasts were underdispersive. Nevertheless, forecasts on the higher-resolution grids were more reliable than those with parameterized convection. Forecasts on the cloud-allowing grid were best able to discriminate areas that received heavy snow and those that did not, while the forecasts with parameterized convection were least able to do so. It is recommended to use convection-resolving and (if computationally possible) to use cloud-allowing forecast grids when predicting East Coast winter storms.

Snowstorms↗

Virtual Facility at Fermilab: Infrastructure and Services Expand to Public Clouds

In preparation for its new Virtual Facility Project, Fermilab has launched a program of work to determine the requirements for running a computation facility on-site, in public clouds, or a combination of both. This program builds on the work we have done to successfully run experimental workflows of 1000-VM scale both on an on-site private cloud and on Amazon AWS. To do this at scale we deployed dynamically launched and discovered caching services on the cloud. We are now testing the deployment of more complicated services on Amazon AWS using native load balancing and auto scaling features they provide. The Virtual Facility Project will design and develop a facility including infrastructure and services that can live on the site of Fermilab, off-site, or a combination of both. We expect to need this capacity to meet the peak computing requirements in the future. The Virtual Facility is intended to provision resources on the public cloud on behalf of the facility as a whole instead of having each experiment or Virtual Organization do it on their own. We will describe the policy aspects of a distributed Virtual Facility, the requirements, and plans to make a detailed comparison of the relative cost of the public and private clouds. Furthermore, this talk will present the details of the technical mechanisms we have developed to date, and the plans currently taking shape for a Virtual Facility at Fermilab.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Enabling opportunistic resources for CMS Computing Operations

With the increased pressure on computing brought by the higher energy and luminosity from the LHC in Run 2, CMS Computing Operations expects to require the ability to utilize opportunistic resources resources not owned by, or a priori configured for CMS to meet peak demands. In addition to our dedicated resources we look to add computing resources from non CMS grids, cloud resources, and national supercomputing centers. CMS uses the HTCondor/glideinWMS job submission infrastructure for all its batch processing, so such resources will need to be transparently integrated into its glideinWMS pool. Bosco and parrot wrappers are used to enable access and bring the CMS environment into these non CMS resources. Finally, we describe our strategy to supplement our native capabilities with opportunistic resources and our experience so far using them.

97 MATHEMATICS AND COMPUTING↗

Altitude determination and descriptive analysis of clouds on ERTS-1 multispectral photography

A simple method to determine the approximate altitude of clouds is described, with the objective of refining their classification using only marginal data from the photographs. Results of the application of this method on photographs of the Goajira Peninsula, Paraguana Peninsula and the Central Coast of Venezuela are presented. Here, the altitudes computed are used to classify clouds and to identify the genus of others without typical form. Instability of air masses through clouds vertical development, and wind direction as well as other local climatic characteristics such as moisture content, loci of condensation, area, etc. are determined using repetitive coverage for the time interval of the photography. Applications for the regional and urban planning (including airport location and flights schedule) and natural resources evaluation are suggested.

Albrizzio, C.↗

Seismology in the Cloud: A New Streaming Workflow

Data-intensive research in seismology is experiencing a recent boom, driven in part by large volumes of available data and advances in the growing field of data science. However, there are significant barriers to processing large data volumes, such as long retrieval times from data repositories, complex data management, and limited computational resources. New tools and platforms have reduced the barriers to entry for scientific cluster computing, including the maturation of the commercial cloud as an accessible instrument for research. Here, we build a customized research cluster in the cloud to test a new workflow for large-scale seismic analysis, in which data are processed as a stream (retrieved on-the-fly and acted upon without storing), with data from the Incorporated Research Institutions for Seismology Data Management Center. We use this workflow to deploy a spectral peak detection algorithm over 5.6 TB of compressed continuous seismic data from 2074 stations of the USArray Transportable Array EarthScope network. Using a 50-node cluster in the cloud, we completed the noise survey in 80 hr, with an average data throughput of 1.7 GB per minute. By varying cluster sizes, we find the scaling of our analysis to be sublinear, due to a combination of algorithmic limitations and data center response times. The cloud-based streaming workflow represents an order-of-magnitude increase in acquisition and processing speed compared to a traditional download-store-process workflow, and offers the additional benefits of employing a flexible, accessible, and widely used computing architecture. It is limited, however, due to its reliance on Internet transfer speeds and data center service capacity, and may not work well for repeated analyses or those for which even higher data throughputs are needed. These research applications will require a new class of cloud-native approaches in which both data and analysis are in the cloud.

58 GEOSCIENCES↗

Design and implementation of dynamic I/O control scheme for large scale distributed file systems

In this paper, we have analyzed the input/output (I/O) activities of Cori, which is a high-performance computing system at the National Energy Research Scientific Computing Center at Lawrence Berkeley National Laboratory. Our analysis results indicate that most users do not adjust storage configurations but rather use the default settings. In addition, owing to the interference from many applications running simultaneously, the performance varies based on the system status. To configure file systems autonomously in complex environments, we developed DCA-IO, a dynamic distributed file system configuration adjustment algorithm that utilizes the system log information to adjust storage configurations automatically. Our scheme aims to improve the application performance and avoid interference from other applications without user intervention. Moreover, DCA-IO uses the existing system logs and does not require code modifications, an additional library, or user intervention. To demonstrate the effectiveness of DCA-IO, we performed experiments using I/O kernels of real applications in both an isolated small-sized Lustre environment and Cori. Our experimental results shows that our scheme can improve the performance of HPC applications by up to 263% with the default Lustre configuration.

97 MATHEMATICS AND COMPUTING↗

An Improved Algorithm for Estimating Surface Shortwave Radiation: Preliminary Evaluation With MODIS Products

Cloud parameters, as key inputs in radiative transfer algorithms, have a critical impact on surface shortwave radiation (SSR) computation. By introducing a parameterization of cloud transmittance and reflectance, based on radiative transfer simulations, this study improves the accuracy of an existing physically based model which severely underestimates SSR under thick cloud conditions. The cloud parameterization adopts the single-layer cloud model and simulates cloud transmittances and reflectances by varying cloud optical thickness, cloud particle size, and solar zenith angle. The revised model is applied to estimate instantaneous SSR using Moderate-resolution Imaging Spectroradiometer (MODIS) atmospheric and land products. The retrieved SSR is evaluated against observation data from 41 Baseline Surface Radiation Network (BSRN) stations and is also compared with the MODIS official SSR product. The root mean square error (RMSE) of the estimated instantaneous radiation is approximately 52 and 98 W m -2 under clear-sky and all-sky conditions, respectively. The accuracy of the improved parameterization is higher than that of the original model, and there is no obvious underestimation of SSR in the case of high cloud optical thickness. Therefore, the new algorithm improves the accuracy of SSR estimates in the presence of thick clouds. Retrievals with the improved model also achieve higher accuracy than the MODIS official SSR product (MCD18A1). To conclude, the reliable performance of the scheme at most BSRN stations illustrates that the improved model can be used to map SSR on a global scale.

42 ENGINEERING↗

The GONG Site Survey

The Global Oscillation Network Group (GONG) project is planning to place six observing stations around the world to observe the solar oscillations as continuously as possible. The procedures that are being used to select the six sites are described. Results of measurements of cloud cover obtained by networks of 6 (out of 10) radiometers show a duty cycle of over 93 percent, with the first diurnal sidelobe in the window power spectrum suppressed by a factor of 400. The results are in good agreement with the predictions of a computer model of the expected cloud cover at individual sites.

Hill, Frank↗

Radar sensitivity and antenna scan pattern study for a satellite-based Radar Wind Sounder (RAWS)

Modeling global atmospheric circulations and forecasting the weather would improve greatly if worldwide information on winds aloft were available. Recognition of this led to the inclusion of the LAser Wind Sounder (LAWS) system to measure Doppler shifts from aerosols in the planned for Earth Observation System (EOS). However, gaps will exist in LAWS coverage where heavy clouds are present. The RAdar Wind Sensor (RAWS) is an instrument that could fill these gaps by measuring Doppler shifts from clouds and rain. Previous studies conducted at the University of Kansas show RAWS as a feasible instrument. This thesis pertains to the signal-to-noise ratio (SNR) sensitivity, transmit waveform, and limitations to the antenna scan pattern of the RAWS system. A dop-size distribution model is selected and applied to the radar range equation for the sensitivity analysis. Six frequencies are used in computing the SNR for several cloud types to determine the optimal transmit frequency. the results show the use of two frequencies, one higher (94 GHz) to obtain sensitivity for thinner cloud, and a lower frequency (24 GHz) to obtain sensitivity for thinner cloud, and a lower frequency (24 GHz) for better penetration in rain, provide ample SNR. The waveform design supports covariance estimation processing. This estimator eliminates the Doppler ambiguities compounded by the selection of such high transmit frequencies, while providing an estimate of the mean frequency. the unambiguous range and velocity computation shows them to be within acceptable limits. The design goal for the RAWS system is to limit the wind-speed error to less than 1 ms(exp -1). Due to linear dependence between vectors for a three-vector scan pattern, a reasonable wind-speed error is unattainable. Only the two-vector scan pattern falls within the wind-error limits for azimuth angles between 16 deg to 70 deg. However, this scan only allows two components of the wind to be determined. As a result, a technique is then shown, based on the Z-R-V relationships, that permit the vertical component (i.e., rain) to be computed. Thus the horizontal wind components may be obtained form the covariance estimator and the vertical component from the reflectivity factor. Finally, a new candidate system is introduced which summarizes the parameters taken from previous RAWS studies, or those modified in this thesis.

Stuart, Michael A.↗

Feasibility and Demonstration of a Cloud-Based RIID Analysis System

A significant limitation in the operational utility of handheld and backpack radioisotope identifiers (RIIDs) is the inability of their onboard algorithms to accurately and reliably identify the isotopic sources of the measured gamma-ray energy spectrum. A possible solution is to move the spectral analysis computations to an external device, the cloud, where significantly greater capabilities are available. The implementation and demonstration of a prototype cloud-based RIID analysis system have shown this type of system to be feasible with currently available communication and computational technology. A system study has shown that the potential user community could derive significant benefits from an appropriately implemented cloud-based analysis system and has identified the design and operational characteristics required by the users and stakeholders for such a system. A general description of the hardware and software necessary to implement reliable cloud-based analysis, the value of the cloud expressed by the user community, and the aspects of the cloud implemented in the demonstrations are discussed.

Wright, Michael C↗

A Novel Method for Estimating Shortwave Direct Radiative Effect of Above-Cloud Aerosols Using CALIOP and MODIS Data

This paper describes an efficient and unique method for computing the shortwave direct radiative effect (DRE) of aerosol residing above low-level liquid-phase clouds using CALIOP and MODIS data. It accounts for the overlapping of aerosol and cloud rigorously by utilizing the joint histogram of cloud optical depth and cloud top pressure. Effects of sub-grid scale cloud and aerosol variations on DRE are accounted for. It is computationally efficient through using grid-level cloud and aerosol statistics, instead of pixel-level products, and a pre-computed look-up table in radiative transfer calculations. We verified that for smoke over the southeast Atlantic Ocean the method yields a seasonal mean instantaneous shortwave DRE that generally agrees with more rigorous pixel-level computation within 4. We have also computed the annual mean instantaneous shortwave DRE of light-absorbing aerosols (i.e., smoke and polluted dust) over global ocean based on 4 yr of CALIOP and MODIS data. We found that the variability of the annual mean shortwave DRE of above-cloud light-absorbing aerosol is mainly driven by the optical depth of the underlying clouds.

radiative forcing↗

The meteorological effects on microwave apparent temperatures looking downward over a smooth sea

The effects of clouds and rain on microwave apparent temperatures for a flat sea surface are examined. The presence of clouds and rain can be expressed as a change of absorption coefficient and the total absorption is computed as the sum of individual effects. Various cloud and rain models proposed by meteorologists are employed to compute the microwave apparent temperature when viewing downward through these model atmospheres. It is shown that stratus, cumulus, overcast, and rain all contribute significantly to the observed temperature. Larger sensitivities to clouds and rain are observed for horizontally polarized apparent temperature at large nadir angles than for vertically polarized apparent temperature.

Wu, S.↗

A Computationally Efficient Linear Semi-Lagrangian Scheme for the Advection of Microphysical Variables in Cloud-Resolving Models

An efficient semi-Lagrangian (SL) numerical scheme for the advection of positive-definite microphysical variables in cloud-resolving models is proposed. The scheme is a linear combination of SL schemes of the first and second order. The proposed scheme is compared with two high-order, monotonic, nonlinear Eulerian schemes, in two 2D tests. In the first test, advection of a positive scalar field with large gradients is performed within complicated idealized steering flows. In the second test, a hail storm is simulated using the 2D Hebrew University Cloud Model, describing the interactions between 15 size distributions of different microphysical quantities (each distribution is described by 43 mass bins). The proposed scheme is computationally efficient, has a low numerical diffusion and a high level of mass conservation accuracy, and preserves the sum of multiple advected variables as well as the linear relationships between the advected scalar variables. Although the proposed SL scheme is not exactly positive definite, the negative values and their number, which may appear only in the case of very strong gradients, are negligible. The scheme produces results similar to those of nonlinear Eulerian advection schemes while reducing computation time by approximately a factor of 10.

54 ENVIRONMENTAL SCIENCES↗