Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Cloud Computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 991 records · Page 55
All sourcesNASA NTRSDOE OSTI
NASA NTRS2012

Effects of Cloud Horizontal Inhomogeneity and Drizzle on Remote Sensing of Cloud Droplet Effective Radius: Case Studies Based on Large-eddy Simulations

This study investigates effects of drizzle and cloud horizontal inhomogeneity on cloud effective radius (re) retrievals from the Moderate Resolution Imaging Spectroradiometer (MODIS). In order to identify the relative importance of various factors, we developed a MODIS cloud property retrieval simulator based on the combination of large-eddy simulations (LES) and radiative transfer computations. The case studies based on synthetic LES cloud fields indicate that at high spatial resolution (100 m) 3-D radiative transfer effects, such as illumination and shadowing, can induce significant differences between retrievals ofre based on reflectance at 2.1 m (re,2.1) and 3.7 m (re,3.7). It is also found that 3-D effects tend to have stronger impact onre,2.1 than re,3.7, leading to positive difference between the two (re,3.72.1) from illumination and negative re,3.72.1from shadowing. The cancellation of opposing 3-D effects leads to overall reasonable agreement betweenre,2.1 and re,3.7 at high spatial resolution as far as domain averages are concerned. At resolutions similar to MODIS, however, re,2.1 is systematically larger than re,3.7when averaged over the LES domain, with the difference exhibiting a threshold-like dependence on bothre,2.1and an index of the sub-pixel variability in reflectance (H), consistent with MODIS observations. In the LES cases studied, drizzle does not strongly impact reretrievals at either wavelength. It is also found that opposing 3-D radiative transfer effects partly cancel each other when cloud reflectance is aggregated from high spatial resolution to MODIS resolution, resulting in a weaker net impact of 3-D radiative effects onre retrievals. The large difference at MODIS resolution between re,3.7 and re,2.1 for highly inhomogeneous pixels with H 0.4 can be largely attributed to what we refer to as the plane-parallelrebias, which is attributable to the impact of sub-pixel level horizontal variability of cloud optical thickness onre retrievals and is greater for re,2.1 than re,3.7. These results suggest that there are substantial uncertainties attributable to 3-D radiative effects and plane-parallelre bias in the MODIS re,2.1retrievals for pixels with strong sub-pixel scale variability, and theH index can be used to identify these uncertainties.

Zhang, Zhibo↗
NASA NTRS2019

Deep Learning Emulation of Atmospheric Correction for Geostationary Sensors

New generation geostationary satellites make reflectance observations available at a continental scale with unprecedented spatiotemporal resolution and spectral range. Generating Earth monitoring products from these observations requires retrieval of the basic parameter, surface reflectance (SR), by atmospheric correction (AC). Algorithms for atmospheric correction, including Multi-Angle Implementation of Atmospheric Correction (MAIAC), are adapted for each sensor and are too computationally complex to be run in real time, relying instead on look-up tables with precomputed values. Machine learning methods, including convolutional neural networks, have demonstrated performance in learning complex, nonlinear mappings and extracting insight from high-dimensional remote sensing data. In this work, we present a deep learning emulator of MAIAC to retrieve both SR and cloud products. Using this adaptation of deep learning-based emulation to remote sensing, we demonstrate stable SR retrieval over a variety of land covers and viewing conditions and accurate cloud detection. Further, a comparison of computation time suggests emulation as a compelling alternative for expensive physical simulation, especially for applications benefited by near-real time data, such as agricultural management and disaster response.

Duffy, Kate↗
NASA NTRS1994

Independent pixel and Monte Carlo estimates of stratocumulus albedo

Monte Carlo radiative transfer methods are employed here to estimate the plane-parallel albedo bias for marine stratocumulus clouds. This is the bias in estimates of the mesoscale-average albedo, which arises from the assumption that cloud liquid water is uniformly distributed. The authors compare such estimates with those based on a more realistic distribution generated from a fractal model of marine stratocumulus clouds belonging to the class of 'bounded cascade' models. In this model the cloud top and base are fixed, so that all variations in cloud shape are ignored. The model generates random variations in liquid water along a single horizontal direction, forming fractal cloud streets while conserving the total liquid water in the cloud field. The model reproduces the mean, variance, and skewness of the vertically integrated cloud liquid water, as well as its observed wavenumber spectrum, which is approximately a power law. The Monte Carlo method keeps track of the three-dimensional paths solar photons take through the cloud field, using a vectorized implementation of a direct technique. The simplifications in the cloud field studied here allow the computations to be accelerated. The Monte Carlo results are compared to those of the independent pixel approximation, which neglects net horizontal photon transport. Differences between the Monte Carlo and independent pixel estimates of the mesoscale-average albedo are on the order of 1% for conservative scattering, while the plane-parallel bias itself is an order of magnitude larger. As cloud absorption increases, the independent pixel approximation agrees even more closely with the Monte Carlo estimates. This result holds for a wide range of sun angles and aspect ratios. Thus, horizontal photon transport can be safely neglected in estimates of the area-average flux for such cloud models. This result relies on the rapid falloff of the wavenumber spectrum of stratocumulus, which ensures that the smaller-scale variability, where the radiative transfer is more three-dimensional, contributes less to the plane-parallel albedo bias than the larger scales, which are more variable. The lack of significant three-dimensional effects also relies on the assumption of a relatively simple geometry. Even with these assumptions, the independent pixel approximation is accurate only for fluxes averaged over large horizontal areas, many photon mean free paths in diameter, and not for local radiance values, which depend strongly on the interaction between neighboring cloud elements.

Cahalan, Robert F.↗
NASA NTRS2011

Graphics Processing Units (GPU) and the Goddard Earth Observing System atmospheric model (GEOS-5): Implementation and Potential Applications

Earth system models like the Goddard Earth Observing System model (GEOS-5) have been pushing the limits of large clusters of multi-core microprocessors, producing breath-taking fidelity in resolving cloud systems at a global scale. GPU computing presents an opportunity for improving the efficiency of these leading edge models. A GPU implementation of GEOS-5 will facilitate the use of cloud-system resolving resolutions in data assimilation and weather prediction, at resolutions near 3.5 km, improving our ability to extract detailed information from high-resolution satellite observations and ultimately produce better weather and climate predictions

Putnam, William M.↗
NASA NTRS2015

Nearly a Decade of CALIPSO Observations of Asian and Saharan Dust Properties Near Source and Transport Regions

The lidar on the Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) mission, makes robust measurements of dust and has generated a length of record that is significant both seasonally and inter-annually. We exploit this record to determine a multi-year climatology of the properties of Asian and Saharan dust, in particular seasonal optical depths, layer frequencies, and layer heights of dust gridded in accordance with the Level 3 data products protocol, between 2006-2015. The data are screened using standard CALIPSO quality assurance flags, cloud aerosol discrimination (CAD) scores, overlying features and layer properties. To evaluate the effects of transport on the morphology, vertical extent and size of the dust layers, we compare probability distribution functions of the layer integrated volume depolarization ratios, geometric depths and integrated attenuated color ratios near the source to the same distributions in the far field or transport region. CALIPSO is collaboration between NASA and Centre National D'études Spatiales (CNES), was launched in April 2006 to provide vertically resolved measurements of cloud and aerosol distributions. The primary instrument on the CALIPSO satellite is the Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP), a near-nadir viewing two-wavelength polarization-sensitive instrument. The unique nature of CALIOP measurements make it quite challenging to validate backscatter profiles, aerosol type, and cloud phase, all of which are used to retrieve extinction and optical depth. To evaluate the uncertainty in the lidar ratios, we compare the values computed from dust layers overlying opaque water clouds, considered nominal, with the constant lidar ratio value used in the CALIOP algorithms for dust. We also explore the effects of noise on the CALIOP retrievals at daytime by comparing the distributions of the properties at daytime to the nighttime distributions.

Omar, Ali H.↗
DOE OSTI2020

Resource Management for Processing Wide Area Data Streams on Supercomputers

Modern scientific instruments generate enormous amount of data. Typically, the data collected from the instruments are stored in one or more files that are then moved to a distant supercomputer for processing. The final results are sent back to the user. In order to make effective use of the time on expensive instruments, experimenters want to process the data as they are generated. They want to stream the data from instruments' memory directly to a supercomputer's memory for analysis. Since the compute nodes in a supercomputer are not connected directly to the wide area network, the data streams need to be passed through intermediate gateway nodes. As opposed to the best effort file transfers, data streaming applications require resources at a specific time for a specific period. In this paper, we present a system model for enabling data streaming through gateway nodes and an algorithm to efficiently allocate gateway node resources along with compute nodes. We evaluate the algorithm using real-world traces on the Chameleon Cloud. The results show that our system can schedule compute and gateway resources efficiently for streaming analysis.

data streaming↗
NASA NTRS2007

Validation of GOES-10 Satellite-derived Cloud and Radiative Properties for the MASRAD ARM Mobile Facility Deployment

This poster presentation reviews the process used to validate the GOES-10 satellite derived cloud and radiative properties. The ARM Mobile Facility (AMF) deployment at Pt Reyes, CA as part of the Marine Stratus Radiation Aerosol and Drizzle experiment (MASRAD), 14 March - 14 September 2005 provided an excellent chance to validate satellite cloud-property retrievals with the AMF's flexible suite of ground-based remote sensing instruments. For this comparison, NASA LaRC GOES10 satellite retrievals covering this region and period were re-processed using an updated version of the Visible Infrared Solar-Infrared Split-Window Technique (VISST), which uses data taken at 4 wavelengths (0.65, 3.9,11 and 12 m resolution), and computes broadband fluxes using improved CERES (Clouds and Earth's Radiant Energy System)-GOES-10 narrowband-to-broadband flux conversion coefficients. To validate MASRAD GOES-10 satellite-derived cloud property data, VISST-derived cloud amounts, heights, liquid water paths are compared with similar quantities derived from available ARM ground-based instrumentation and with CERES fluxes from Terra.

Khaiyer, M. M.↗
NASA NTRS2007

Evaluation of a Cloud Resolving Model Using TRMM Observations for Multiscale Modeling Applications

The climate change simulation community is moving toward use of global cloud resolving models (CRMs), however, current computational resources are not sufficient to run global CRMs over the hundreds of years necessary to produce climate change estimates. As an intermediate step between conventional general circulation models (GCMs) and global CRMs, many climate analysis centers are embedding a CRM in each grid cell of a conventional GCM. These Multiscale Modeling Frameworks (MMFs) represent a theoretical advance over the use of conventional GCM cloud and convection parameterizations, but have been shown to exhibit an overproduction of precipitation in the tropics during the northern hemisphere summer. In this study, simulations of clouds, precipitation, and radiation over the South China Sea using the CRM component of the NASA Goddard MMF are evaluated using retrievals derived from the instruments aboard the Tropical Rainfall Measuring Mission (TRMM) satellite platform for a 46-day time period that spans 5 May - 20 June 1998. The NASA Goddard Cumulus Ensemble (GCE) model is forced with observed largescale forcing derived from soundings taken during the intensive observing period of the South China Sea Monsoon Experiment. It is found that the GCE configuration used in the NASA Goddard MMF responds too vigorously to the imposed large-scale forcing, accumulating too much moisture and producing too much cloud cover during convective phases, and overdrying the atmosphere and suppressing clouds during monsoon break periods. Sensitivity experiments reveal that changes to ice cloud microphysical parameters have a relatively large effect on simulated clouds, precipitation, and radiation, while changes to grid spacing and domain length have little effect on simulation results. The results motivate a more detailed and quantitative exploration of the sources and magnitude of the uncertainty associated with specified cloud microphysical parameters in the CRM components of MMFs.

Posselt, Derek J.↗
DOE OSTI2024

Massively scalable workflows for quantum chemistry: BigChem and ChemCloud

Electronic structure theory, i.e., quantum chemistry, is the fundamental building block for many problems in computational chemistry. Here we present a new distributed computing framework (BigChem), which allows for an efficient solution of many quantum chemistry problems in parallel. BigChem is designed to be easily composable and leverages industry-standard middleware (e.g., Celery, RabbitMQ, and Redis) for distributed approaches to large scale problems. BigChem can harness any collection of worker nodes, including ones on cloud providers (such as AWS or Azure), local clusters, or supercomputer centers (and any mixture of these). BigChem builds upon MolSSI packages, such as QCEngine to standardize the operation of numerous computational chemistry programs, demonstrated here with Psi4, xtb, geomeTRIC, and TeraChem. BigChem delivers full utilization of compute resources at scale, offers a programable canvas for designing sophisticated quantum chemistry workflows, and is fault tolerant to node failures and network disruptions. We demonstrate linear scalability of BigChem running computational chemistry workloads on up to 125 GPUs. Finally, we present ChemCloud, a web API to BigChem and successor to TeraChem Cloud. ChemCloud delivers scalable and secure access to BigChem over the Internet.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2019

FPGA-Accelerated Machine Learning Inference as a Service for Particle Physics Computing

Large-scale particle physics experiments face challenging demands for high-throughput computing resources both now and in the future. New heterogeneous computing paradigms on dedicated hardware with increased parallelization, such as Field Programmable Gate Arrays (FPGAs), offer exciting solutions with large potential gains. The growing applications of machine learning algorithms in particle physics for simulation, reconstruction, and analysis are naturally deployed on such platforms. We demonstrate that the acceleration of machine learning inference as a web service represents a heterogeneous computing solution for particle physics experiments that potentially requires minimal modification to the current computing model. As examples, we retrain the ResNet-50 convolutional neural network to demonstrate state-of-the-art performance for top quark jet tagging at the LHC and apply a ResNet-50 model with transfer learning for neutrino event classification. Using Project Brainwave by Microsoft to accelerate the ResNet-50 image classification model, we achieve average inference times of 60 (10) ms with our experimental physics software framework using Brainwave as a cloud (edge or on-premises) service, representing an improvement by a factor of approximately 30 (175) in model inference latency over traditional CPU inference in current experimental hardware. A single FPGA service accessed by many CPUs achieves a throughput of 600–700 inferences per second using an image batch of one, comparable to large batch-size GPU throughput and significantly better than small batch-size GPU throughput. Deployed as an edge or cloud service for the particle physics computing model, coprocessor accelerators can have a higher duty cycle and are potentially much more cost-effective.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2019

FPGAs as a Service to Accelerate Machine Learning Inference [PowerPoint]

Large-scale particle physics experiments face challenging demands for high-throughput computing resources both now and in the future. New heterogeneous computing paradigms on dedicated hardware with increased parallelization, such as Field Programmable Gate Arrays (FPGAs), offer exciting solutions with large potential gains. The growing applications of machine learning algorithms in particle physics for simulation, reconstruction, and analysis are naturally deployed on such platforms. We demonstrate that the acceleration of machine learning inference as a web service represents a heterogeneous computing solution for particle physics experiments that requires minimal modification to the current computing model. As examples, we retrain the ResNet-50 convolutional neural network to demonstrate state-of-the-art performance for top quark jet tagging at the LHC, and apply a ResNet-50 model with transfer learning for neutrino event classification. Using Project Brainwave b y Microsoft to accelerate the ResNet-50 image classification model, we achieve average inference times of 60 (10) milliseconds with our experimental physics software framework using Brainwave as a cloud (edge or on-premises) service, representing an improvement by a factor of approximately 30 (175) in model inference latency over traditional CPU inference in current experimental hardware. A single FPGA service accessed by many CPUs achieves a throughput of 600--700 inferences per second using an image batch of one, comparable to large batch-size GPU throughput and significantly better than small batch-size GPU throughput. Deployed as an edge or cloud service for the particle physics computing model, coprocessor accelerators can have a higher duty cycle and are potentially much more cost-effective.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Non-conservation and conservation for different formulations of moist potential vorticity

Potential vorticity (PV) is one of the most important quantities in atmospheric science. In the absence of dissipative processes, the PV of each fluid parcel is known to be conserved, for a dry atmosphere. However, a parcel's PV is not conserved if clouds or phase changes of water occur. Recently, PV conservation laws were derived for a cloudy atmosphere, where each parcel's PV is not conserved but parcel-integrated PV is conserved, for integrals over certain volumes that move with the flow. Hence a variety of different statements are now possible for moist PV conservation and non-conservation, and in comparison to the case of a dry atmosphere, the situation for moist PV is more complex. Here, in light of this complexity, several different definitions of moist PV are compared for a cloudy atmosphere. Numerical simulations are shown for a rising thermal, both before and after the formation of a cloud. These simulations include the first computational illustration of the parcel-integrated, moist PV conservation laws. The comparisons, both theoretical and numerical, serve to clarify and highlight the different statements of conservation and non-conservation that arise for different definitions of moist PV.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Privacy Preserving Federated Learning for Advanced Scientific Ecosystems

We present a framework to provide privacy preserving (PP) federating learning (FL) across multiple computational and experimental facilities. This work joins the compute capabilities of National Energy Research Scientific Computing Center (NERSC) and Oak Ridge National Laboratory Research Cloud (ORC) with simulated experimental data, such as those produced at the SLAC National Accelerator Laboratory and Spallation Neutron Source (SNS). We describe the software infrastructure developed to provide privacy for computational and experimental networks. We developed algorithmic privacy across the federated system by embedding database security, computation, and communication into the federation architecture, utilizing scientific tools developed by the experimental community.

Archibald, Rick [ORNL] (ORCID:0000000245389780)↗
DOE OSTI2022

Effectiveness of Deep Learning Trained on SynthCity Data for Urban Point-Cloud Classification

3D object recognition is one of the most popular areas of study in computer vision. Many of the more recent algorithms focus on indoor point clouds, classifying 3D geometric objects, and segmenting outdoor 3D scenes. One of the challenges of the classification pipeline is finding adequate and accurate training data. Hence, this article seeks to evaluate the accuracy of a synthetically generated data set called SynthCity, tested on two mobile laser-scan data sets. Varying levels of noise were applied to the training data to reflect varying levels of noise in different scanners. The chosen deep-learning algorithm was Kernel Point Convolution, a convolutional neural network that uses kernel points in Euclidean space for convolution weights.

Geology↗
NASA NTRS1983

Cumulus clouds - Numerical models, observations and entrainment

The first computer simulation of the organization phase of a buoyant atmospheric thermal is described. Although crude, it showed the spontaneous development of a rounded tight-gradient 'cap' and internal vortical circulation. The complexities involved in these 'field of motion' models in part motivated the development of entity models, based upon laboratory thermals. These one-dimensional models and their uses with observations are briefly described as well as their limitations. Finally, an application of Schlesinger's three-dimensional model to a GATE cumulus situation clarifies many apparently conflicting observations and postulates, thereby raising further challenging questions to be addressed jointly by the more sophisticated measuring and modeling tools available in the 1980's.

Simpson, J.↗
NASA NTRS1982

The effect of brokenness on cloud-climate sensitivity

A study has been made of the effect of brokenness on the infrared and albedo feedback of clouds in climate models using a simplified treatment of broken cloudiness. It is shown that the individual feedback terms computed using the plane-parallel assumption differ markedly from the computations performed for a regular array of cuboidal clouds. If the cloudiness changes such that clear areas become cloudy but the nature of the broken cloud field remains unchanged, then both the feedbacks are magnified, the infrared more so than the albedo. However, if individual elements in the cloud field are increased horizontally in a constrained area, the feedbacks can be enhanced or diminished depending on the cloud fraction and aspect ratio of the elements. It is also shown that, in the global average, the ratio of the change in outgoing infrared flux to global albedo, as deduced from satellite measurements, will be larger than model simulations using planiform clouds.

Harshvardhan, MR.↗
DOE OSTI2022

Development of an end state vision to implement digital monitoring in nuclear plants

Transitioning from an onsite Maintenance & Diagnostics Center to cloud-based services offers many new opportunities with computing power and storage, but also new challenges in terms of networking and security. This report will cover everything required for that transition including data processing and uploading to cloud services, feature selection, model creation, and result visualization for decision making. Although there are several other cloud-based services (e.g. Amazon Web Services and Google Cloud), this report explores Microsoft Azure to simplify nomenclature and maintain a consistent focus. Many of the services offered by Microsoft Azure are also available in the other cloud-based services, and their differences have been recorded in other literature. The Azure services most important to a nuclear power plant including networking & security, storage & databases, and Artificial Intelligence (AI) are reviewed here. Networking covers all aspects related to communication to Azure resources including security, privacy, and redundancy. Storage & databases includes data storage, upgrading, patching, backups, and monitoring. The AI services allows the user access to the machine learning (ML) techniques developed with Azure including automated ML, anomaly detection, computer vision, and natural language processing. This report summaries the features, capabilities, and challenges when using cloud-based services in a user-friendly manner.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2018

Deep learning tackles cloud detection

To better understand Earth systems, atmospheric scientists use cloud data collected by a remote sensing tool called lidar. These data are an important part of global forecasting, but analyzing lidar images by hand is time-consuming. Researchers at Pacific Northwest National Laboratory teamed up to see if a deep learning model could process lidar data better than current automatic detection techniques.

54 ENVIRONMENTAL SCIENCES↗