Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “CLOUD COMPUTING”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 991 records · Page 55

Nearly a Decade of CALIPSO Observations of Asian and Saharan Dust Properties Near Source and Transport Regions

The lidar on the Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) mission, makes robust measurements of dust and has generated a length of record that is significant both seasonally and inter-annually. We exploit this record to determine a multi-year climatology of the properties of Asian and Saharan dust, in particular seasonal optical depths, layer frequencies, and layer heights of dust gridded in accordance with the Level 3 data products protocol, between 2006-2015. The data are screened using standard CALIPSO quality assurance flags, cloud aerosol discrimination (CAD) scores, overlying features and layer properties. To evaluate the effects of transport on the morphology, vertical extent and size of the dust layers, we compare probability distribution functions of the layer integrated volume depolarization ratios, geometric depths and integrated attenuated color ratios near the source to the same distributions in the far field or transport region. CALIPSO is collaboration between NASA and Centre National D'études Spatiales (CNES), was launched in April 2006 to provide vertically resolved measurements of cloud and aerosol distributions. The primary instrument on the CALIPSO satellite is the Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP), a near-nadir viewing two-wavelength polarization-sensitive instrument. The unique nature of CALIOP measurements make it quite challenging to validate backscatter profiles, aerosol type, and cloud phase, all of which are used to retrieve extinction and optical depth. To evaluate the uncertainty in the lidar ratios, we compare the values computed from dust layers overlying opaque water clouds, considered nominal, with the constant lidar ratio value used in the CALIOP algorithms for dust. We also explore the effects of noise on the CALIOP retrievals at daytime by comparing the distributions of the properties at daytime to the nighttime distributions.

Omar, Ali H.↗

Resource Management for Processing Wide Area Data Streams on Supercomputers

Modern scientific instruments generate enormous amount of data. Typically, the data collected from the instruments are stored in one or more files that are then moved to a distant supercomputer for processing. The final results are sent back to the user. In order to make effective use of the time on expensive instruments, experimenters want to process the data as they are generated. They want to stream the data from instruments' memory directly to a supercomputer's memory for analysis. Since the compute nodes in a supercomputer are not connected directly to the wide area network, the data streams need to be passed through intermediate gateway nodes. As opposed to the best effort file transfers, data streaming applications require resources at a specific time for a specific period. In this paper, we present a system model for enabling data streaming through gateway nodes and an algorithm to efficiently allocate gateway node resources along with compute nodes. We evaluate the algorithm using real-world traces on the Chameleon Cloud. The results show that our system can schedule compute and gateway resources efficiently for streaming analysis.

data streaming↗

Validation of GOES-10 Satellite-derived Cloud and Radiative Properties for the MASRAD ARM Mobile Facility Deployment

This poster presentation reviews the process used to validate the GOES-10 satellite derived cloud and radiative properties. The ARM Mobile Facility (AMF) deployment at Pt Reyes, CA as part of the Marine Stratus Radiation Aerosol and Drizzle experiment (MASRAD), 14 March - 14 September 2005 provided an excellent chance to validate satellite cloud-property retrievals with the AMF's flexible suite of ground-based remote sensing instruments. For this comparison, NASA LaRC GOES10 satellite retrievals covering this region and period were re-processed using an updated version of the Visible Infrared Solar-Infrared Split-Window Technique (VISST), which uses data taken at 4 wavelengths (0.65, 3.9,11 and 12 m resolution), and computes broadband fluxes using improved CERES (Clouds and Earth's Radiant Energy System)-GOES-10 narrowband-to-broadband flux conversion coefficients. To validate MASRAD GOES-10 satellite-derived cloud property data, VISST-derived cloud amounts, heights, liquid water paths are compared with similar quantities derived from available ARM ground-based instrumentation and with CERES fluxes from Terra.

Khaiyer, M. M.↗

Evaluation of a Cloud Resolving Model Using TRMM Observations for Multiscale Modeling Applications

The climate change simulation community is moving toward use of global cloud resolving models (CRMs), however, current computational resources are not sufficient to run global CRMs over the hundreds of years necessary to produce climate change estimates. As an intermediate step between conventional general circulation models (GCMs) and global CRMs, many climate analysis centers are embedding a CRM in each grid cell of a conventional GCM. These Multiscale Modeling Frameworks (MMFs) represent a theoretical advance over the use of conventional GCM cloud and convection parameterizations, but have been shown to exhibit an overproduction of precipitation in the tropics during the northern hemisphere summer. In this study, simulations of clouds, precipitation, and radiation over the South China Sea using the CRM component of the NASA Goddard MMF are evaluated using retrievals derived from the instruments aboard the Tropical Rainfall Measuring Mission (TRMM) satellite platform for a 46-day time period that spans 5 May - 20 June 1998. The NASA Goddard Cumulus Ensemble (GCE) model is forced with observed largescale forcing derived from soundings taken during the intensive observing period of the South China Sea Monsoon Experiment. It is found that the GCE configuration used in the NASA Goddard MMF responds too vigorously to the imposed large-scale forcing, accumulating too much moisture and producing too much cloud cover during convective phases, and overdrying the atmosphere and suppressing clouds during monsoon break periods. Sensitivity experiments reveal that changes to ice cloud microphysical parameters have a relatively large effect on simulated clouds, precipitation, and radiation, while changes to grid spacing and domain length have little effect on simulation results. The results motivate a more detailed and quantitative exploration of the sources and magnitude of the uncertainty associated with specified cloud microphysical parameters in the CRM components of MMFs.

Posselt, Derek J.↗

Massively scalable workflows for quantum chemistry: BigChem and ChemCloud

Electronic structure theory, i.e., quantum chemistry, is the fundamental building block for many problems in computational chemistry. Here we present a new distributed computing framework (BigChem), which allows for an efficient solution of many quantum chemistry problems in parallel. BigChem is designed to be easily composable and leverages industry-standard middleware (e.g., Celery, RabbitMQ, and Redis) for distributed approaches to large scale problems. BigChem can harness any collection of worker nodes, including ones on cloud providers (such as AWS or Azure), local clusters, or supercomputer centers (and any mixture of these). BigChem builds upon MolSSI packages, such as QCEngine to standardize the operation of numerous computational chemistry programs, demonstrated here with Psi4, xtb, geomeTRIC, and TeraChem. BigChem delivers full utilization of compute resources at scale, offers a programable canvas for designing sophisticated quantum chemistry workflows, and is fault tolerant to node failures and network disruptions. We demonstrate linear scalability of BigChem running computational chemistry workloads on up to 125 GPUs. Finally, we present ChemCloud, a web API to BigChem and successor to TeraChem Cloud. ChemCloud delivers scalable and secure access to BigChem over the Internet.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

FPGA-Accelerated Machine Learning Inference as a Service for Particle Physics Computing

Large-scale particle physics experiments face challenging demands for high-throughput computing resources both now and in the future. New heterogeneous computing paradigms on dedicated hardware with increased parallelization, such as Field Programmable Gate Arrays (FPGAs), offer exciting solutions with large potential gains. The growing applications of machine learning algorithms in particle physics for simulation, reconstruction, and analysis are naturally deployed on such platforms. We demonstrate that the acceleration of machine learning inference as a web service represents a heterogeneous computing solution for particle physics experiments that potentially requires minimal modification to the current computing model. As examples, we retrain the ResNet-50 convolutional neural network to demonstrate state-of-the-art performance for top quark jet tagging at the LHC and apply a ResNet-50 model with transfer learning for neutrino event classification. Using Project Brainwave by Microsoft to accelerate the ResNet-50 image classification model, we achieve average inference times of 60 (10) ms with our experimental physics software framework using Brainwave as a cloud (edge or on-premises) service, representing an improvement by a factor of approximately 30 (175) in model inference latency over traditional CPU inference in current experimental hardware. A single FPGA service accessed by many CPUs achieves a throughput of 600–700 inferences per second using an image batch of one, comparable to large batch-size GPU throughput and significantly better than small batch-size GPU throughput. Deployed as an edge or cloud service for the particle physics computing model, coprocessor accelerators can have a higher duty cycle and are potentially much more cost-effective.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

FPGAs as a Service to Accelerate Machine Learning Inference [PowerPoint]

Large-scale particle physics experiments face challenging demands for high-throughput computing resources both now and in the future. New heterogeneous computing paradigms on dedicated hardware with increased parallelization, such as Field Programmable Gate Arrays (FPGAs), offer exciting solutions with large potential gains. The growing applications of machine learning algorithms in particle physics for simulation, reconstruction, and analysis are naturally deployed on such platforms. We demonstrate that the acceleration of machine learning inference as a web service represents a heterogeneous computing solution for particle physics experiments that requires minimal modification to the current computing model. As examples, we retrain the ResNet-50 convolutional neural network to demonstrate state-of-the-art performance for top quark jet tagging at the LHC, and apply a ResNet-50 model with transfer learning for neutrino event classification. Using Project Brainwave b y Microsoft to accelerate the ResNet-50 image classification model, we achieve average inference times of 60 (10) milliseconds with our experimental physics software framework using Brainwave as a cloud (edge or on-premises) service, representing an improvement by a factor of approximately 30 (175) in model inference latency over traditional CPU inference in current experimental hardware. A single FPGA service accessed by many CPUs achieves a throughput of 600--700 inferences per second using an image batch of one, comparable to large batch-size GPU throughput and significantly better than small batch-size GPU throughput. Deployed as an edge or cloud service for the particle physics computing model, coprocessor accelerators can have a higher duty cycle and are potentially much more cost-effective.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Non-conservation and conservation for different formulations of moist potential vorticity

Potential vorticity (PV) is one of the most important quantities in atmospheric science. In the absence of dissipative processes, the PV of each fluid parcel is known to be conserved, for a dry atmosphere. However, a parcel's PV is not conserved if clouds or phase changes of water occur. Recently, PV conservation laws were derived for a cloudy atmosphere, where each parcel's PV is not conserved but parcel-integrated PV is conserved, for integrals over certain volumes that move with the flow. Hence a variety of different statements are now possible for moist PV conservation and non-conservation, and in comparison to the case of a dry atmosphere, the situation for moist PV is more complex. Here, in light of this complexity, several different definitions of moist PV are compared for a cloudy atmosphere. Numerical simulations are shown for a rising thermal, both before and after the formation of a cloud. These simulations include the first computational illustration of the parcel-integrated, moist PV conservation laws. The comparisons, both theoretical and numerical, serve to clarify and highlight the different statements of conservation and non-conservation that arise for different definitions of moist PV.

54 ENVIRONMENTAL SCIENCES↗

Privacy Preserving Federated Learning for Advanced Scientific Ecosystems

We present a framework to provide privacy preserving (PP) federating learning (FL) across multiple computational and experimental facilities. This work joins the compute capabilities of National Energy Research Scientific Computing Center (NERSC) and Oak Ridge National Laboratory Research Cloud (ORC) with simulated experimental data, such as those produced at the SLAC National Accelerator Laboratory and Spallation Neutron Source (SNS). We describe the software infrastructure developed to provide privacy for computational and experimental networks. We developed algorithmic privacy across the federated system by embedding database security, computation, and communication into the federation architecture, utilizing scientific tools developed by the experimental community.

Archibald, Rick [ORNL] (ORCID:0000000245389780)↗

Effectiveness of Deep Learning Trained on SynthCity Data for Urban Point-Cloud Classification

3D object recognition is one of the most popular areas of study in computer vision. Many of the more recent algorithms focus on indoor point clouds, classifying 3D geometric objects, and segmenting outdoor 3D scenes. One of the challenges of the classification pipeline is finding adequate and accurate training data. Hence, this article seeks to evaluate the accuracy of a synthetically generated data set called SynthCity, tested on two mobile laser-scan data sets. Varying levels of noise were applied to the training data to reflect varying levels of noise in different scanners. The chosen deep-learning algorithm was Kernel Point Convolution, a convolutional neural network that uses kernel points in Euclidean space for convolution weights.

Geology↗

Cumulus clouds - Numerical models, observations and entrainment

The first computer simulation of the organization phase of a buoyant atmospheric thermal is described. Although crude, it showed the spontaneous development of a rounded tight-gradient 'cap' and internal vortical circulation. The complexities involved in these 'field of motion' models in part motivated the development of entity models, based upon laboratory thermals. These one-dimensional models and their uses with observations are briefly described as well as their limitations. Finally, an application of Schlesinger's three-dimensional model to a GATE cumulus situation clarifies many apparently conflicting observations and postulates, thereby raising further challenging questions to be addressed jointly by the more sophisticated measuring and modeling tools available in the 1980's.

Simpson, J.↗

The effect of brokenness on cloud-climate sensitivity

A study has been made of the effect of brokenness on the infrared and albedo feedback of clouds in climate models using a simplified treatment of broken cloudiness. It is shown that the individual feedback terms computed using the plane-parallel assumption differ markedly from the computations performed for a regular array of cuboidal clouds. If the cloudiness changes such that clear areas become cloudy but the nature of the broken cloud field remains unchanged, then both the feedbacks are magnified, the infrared more so than the albedo. However, if individual elements in the cloud field are increased horizontally in a constrained area, the feedbacks can be enhanced or diminished depending on the cloud fraction and aspect ratio of the elements. It is also shown that, in the global average, the ratio of the change in outgoing infrared flux to global albedo, as deduced from satellite measurements, will be larger than model simulations using planiform clouds.

Harshvardhan, MR.↗

Development of an end state vision to implement digital monitoring in nuclear plants

Transitioning from an onsite Maintenance & Diagnostics Center to cloud-based services offers many new opportunities with computing power and storage, but also new challenges in terms of networking and security. This report will cover everything required for that transition including data processing and uploading to cloud services, feature selection, model creation, and result visualization for decision making. Although there are several other cloud-based services (e.g. Amazon Web Services and Google Cloud), this report explores Microsoft Azure to simplify nomenclature and maintain a consistent focus. Many of the services offered by Microsoft Azure are also available in the other cloud-based services, and their differences have been recorded in other literature. The Azure services most important to a nuclear power plant including networking & security, storage & databases, and Artificial Intelligence (AI) are reviewed here. Networking covers all aspects related to communication to Azure resources including security, privacy, and redundancy. Storage & databases includes data storage, upgrading, patching, backups, and monitoring. The AI services allows the user access to the machine learning (ML) techniques developed with Azure including automated ML, anomaly detection, computer vision, and natural language processing. This report summaries the features, capabilities, and challenges when using cloud-based services in a user-friendly manner.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Deep learning tackles cloud detection

To better understand Earth systems, atmospheric scientists use cloud data collected by a remote sensing tool called lidar. These data are an important part of global forecasting, but analyzing lidar images by hand is time-consuming. Researchers at Pacific Northwest National Laboratory teamed up to see if a deep learning model could process lidar data better than current automatic detection techniques.

54 ENVIRONMENTAL SCIENCES↗

Dynamics and Control of a Disordered System in Space

In this paper, we present some ideas regarding the modeling, dynamics and control aspects of granular spacecraft. Granular spacecraft are complex multibody systems composed of a spatially disordered distribution of a large number of elements, for instance a cloud of N grains in orbit, with N greater than 10(exp 3). These grains can be large (Cubesat-size) or small (mm-size), and can be active, i.e., a fully equipped vehicle capable sensing their own position and attitude, and enabled with propulsion means, or entirely passive. The ultimate objective would be to study the behavior of the single grains and of large ensembles of grains in orbit and to identify ways to guide and control the shape of a cloud composed of these grains so that it can perform a useful function in space, for instance, as an element of an optical imaging system for astrophysical applications. This concept, in which the aperture does not need to be continuous and monolithic, would increase the aperture size several times compared to large NASA observatories such as ATLAST, allowing for a true Terrestrial Planet Imager that would be able to resolve exo-planet details and do meaningful spectroscopy on distant world. In the paper, we address the modeling and autonomous operation of a distributed assembly (the cloud) of large numbers of highly miniaturized space-borne elements (the grains). A multi-scale, multi-physics model is proposed of the dynamics of the cloud in orbit, as well as a control law for cloud shape maintenance, and preliminary simulation studies yield an estimate of the computational effort, indicating a scale factor of approximately N(exp 1.4) as a function of the number of grains. A granular spacecraft can be defined as a collection of a large number of space-borne elements (in the 1000s) designed and controlled such that a desirable collective behavior emerges, either from the interactions among neighboring grains, and/or between the grains and the environment. In this paper, each grain is considered to be a highly miniaturized spacecraft which has limited size and mass, hence it has limited actuation, limited propulsive capability, limited power, limited sensing, limited communication, limited computational resources, limited range of motion, limited lifetime, and may be expendable. The modeling and dynamics of clouds of vehicles is more challenging than with conventional vehicles because we are faced with a probabilistic vehicle composed of a large number of physically disconnected vehicles. First, different scales of motion occur simultaneously in a cloud: translations and rotations of the cloud as a whole (macro-dynamics), relative rotation and translation of one cloud member with respect to another (meso-dynamics), and individual cloud member dynamics (micro-dynamics). Second, the control design needs to be tolerant of the system complexity, of the system architecture (centralized vs. decentralized large scale system control) as well as robust to un-modeled dynamics and noise sources. Figure 1, top left, shows the kinematic parameters of a 1000 element cloud in orbit. The motion of the system is described with respect to a local vertical-local horizontal (LV-LH) orbiting reference frame (x,y,z)=F(sub ORF) of origin O(sub ORF) which rotates with mean motion omega and orbital semi-major axis R(sub 0). The orbital geometry at the initial time is defined in terms of its six orbital elements, and the orbital dynamics equation for point O(sub ORF) is propagated forward in time under the influence of the gravitational field of the primary and other external perturbations, described below. The origin of this frame coincides with the initial position of the center of mass of the system, and the coordinate axes are z along the local vertical, x toward the flight direction, and y in the orbit normal direction. The assumptions we used to model the dynamics are as follows: 1) The inertial frame is fixed at Earth's center. 2) The orbiting Frame ORF follows Keplerian orbit. 3) the cloud system dynamics is referred to ORF. 4) the attitude of each grain uses the principal body frame as body fixed frame. 5) the atmosphere is assumed to be rigidly rotating with the Earth. Regarding the grains forming the cloud: 1) each grain is modeled as a rigid body; 2) a simple attitude estimator provides attitude estimates, 3) a simple guidance logic commands the position and attitude of each grain, 4) a simple local feedback controller based on PD control of local states is used to stabilize the attitude of the vehicle. Regarding the cloud: 1) the cloud as a whole is modeled as an equivalent rigid body in orbit, and 2) an associated graph establishes agent connectivity and enables coupling between modes of motion at the micro and macro scales; 3) a simple guidance and estimation logic is modeled to estimate and command the attitude of this equivalent rigid body; 4) a cloud shape maintenance controller is based on the dynamics of a stable virtual truss in the orbiting frame. Regarding the environmental perturbations acting on the cloud: 1) a non-spherical gravity field including JO (Earth's spherical field) zonal component, J2 (Earth's oblateness) and J3 zonal components is implemented; 2) atmospheric drag is modeled with an exponential model; 3) solar pressure is modeled assuming the Sun is inertially fixed; and 4) the Earth's magnetic field is model using an equivalent dipole model. The equations of motion are written in a referential system with respect to the origin of the orbiting frame and the state is propagated forward in time using an incremental predictor-corrector scheme. A representative cloud with varying number of grains is simulated to identify the limitations in computation time as the number of grains grows. We derive a control law to track a desired surface in the ORF (equivalently to maintain a reference cloud shape) by defining an error from a desired surface shape, and designing a control law that is exponentially stable and reduces the tracking error to zero. Figure 1 (top right) shows a comparison of various requirements for simulation of single spacecraft vs. granular spacecraft, indicating the high degree of complexity that needs to be taken into consideration. The ORF components of control force required by one of the grains is, for this particular case, in the micro-Newton range. However, no attempt has been made yet to reconfigure (or re-orient) the cloud configuration internally, for which forces in the milli-Newton level are expected, depending on the time required to do the reconfiguration. Figure 1, bottom, shows the computation time as a function of the number of grains, indicating an order N(exp 1.43) scaling on a 8 Gb, 1067 MHz RAM MacOSX computer with a 3.06 GHz Intel Core 2 Duo processor. With this metric, the same simulation for a system of N=1000 grains would take 5.4 hours, and 146 hours (i.e., 6 days) for a system with N=10,000 grains. Therefore, efficient ways to simulate this complex system, where not only the time scales of natural system dynamics, but also the sampling times of the Guidance, Navigation, and Control are included, remain to be explored. Additional details on the cloud modeling, dynamics, and control will be described in the paper.

simulation↗

Hybrid Storage Solution

With the rise of artificial intelligence and machine learning, data sets used to train models have become increasingly large. The availability, accessibility and integrity of large data sets has become important to the research conducted at Los Alamos National Laboratory. Ceph is a storage solution suitable for use with critical data because of its distributed nature and ability to keep multiple copies of a file in different locations. The amount of data means that bandwidth, latency, and cost are important factors and the reason most storage solutions are on-premises. However, there are distinct advantages to hosting services in the cloud, namely scalability and ease-of-use. In this paper, we explore the possibility of provisioning a hybrid Ceph cluster that leverages the benefits of both cloud architectures and on-premise performance.

97 MATHEMATICS AND COMPUTING↗

Jupiter's atmospheric composition and cloud structure deduced from absorption bands in reflected sunlight

The spectrum of sunlight reflected by Jupiter is analyzed by comparing observations of Woodman (1979) with multiple-scattering computations. The analysis yields information on the vertical cloud structure at several latitudes and on the abundance of CH4 and NH3 in the atmosphere of Jupiter. The abundances of CH4 and NH3 suggest that all ices and rocks are overabundant on Jupiter by a factor of 2 or more, providing an important constraint on models for the formation of Jupiter from the primitive solar nebula. The pressure level of the clouds, the gaseous NH3 abundance, the mean temperature profile, and the Clausius-Clapeyron relation suggest that these clouds are predominantly ammonia crystals with the cloud bottom at 600-700 mb. A diffuse distribution of aerosols exists between 150 and 500 mb, and the spectral variation of albedo reflects a changing bulk absorption coefficient of the material composing the aerosols and is diagnostic of the aerosol composition.

Sato, M.↗

Integration of Total-Sky Imager Data with a Physics-Based Smart Persistence Model for Intra-Hour Forecasting of Solar Radiation

Short-term solar forecasting models based solely on global horizontal irradiance (GHI) measurements are often unable to discriminate the forecasting of the factors affecting GHI from those that can be precisely computed by atmospheric models. Our previous study introduced a Physics-based Smart Persistence model for Intra-hour forecasting of solar radiation (PSPI) that decomposed the forecasting of GHI into the computation of extraterrestrial solar radiation and solar zenith angle and the forecasting of cloud albedo and cloud fraction. The extraterrestrial solar radiation and solar zenith angle were accurately computed by the Solar Position Algorithm (SPA) developed at the National Renewable Energy Laboratory (NREL). A cloud retrieval technique was used to estimate cloud albedo and cloud fraction from surface-based observations of GHI. With the assumption of persistent cloud structures, the cloud albedo and cloud fraction were predicted for future time steps using a two-stream approximation and a 5-minute exponential weighted moving average, respectively. The model evaluation indicated the estimation and forecast of cloud fraction mostly contributed to the uncertainty of the PSPI though it overcame the persistence and smart persistence models in all forecast time horizons between 5 and 60 minutes. This study aims to enhance the PSPI by ingesting surface-based observations of cloud fraction from a total sky imager (TSI). The estimation and forecast of cloud albedo is correspondingly improved by utilizing the cloud fraction observations and thus leads to more accurate GHI forecast. Various time-series analysis methods are also investigated on the forecasting of cloud fraction and cloud albedo for further improving the GHI forecast. These improvements are valuable for many applications, such as forecasting energy use for buildings, grid operations, and ultimately bringing down the cost of solar energy.

14 SOLAR ENERGY↗