SEARCH · Engineering Papers

Results for “CLOUD COMPUTING”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 901 records · Page 50

All sources NASA NTRS DOE OSTI

DOE OSTI2016

A hybrid computational strategy to address WGS variant analysis in >5000 samples

Abstract Background The decreasing costs of sequencing are driving the need for cost effective and real time variant calling of whole genome sequencing data. The scale of these projects are far beyond the capacity of typical computing resources available with most research labs. Other infrastructures like the cloud AWS environment and supercomputers also have limitations due to which large scale joint variant calling becomes infeasible, and infrastructure specific variant calling strategies either fail to scale up to large datasets or abandon joint calling strategies. Results We present a high throughput framework including multiple variant callers for single nucleotide variant (SNV) calling, which leverages hybrid computing infrastructure consisting of cloud AWS, supercomputers and local high performance computing infrastructures. We present a novel binning approach for large scale joint variant calling and imputation which can scale up to over 10,000 samples while producing SNV callsets with high sensitivity and specificity. As a proof of principle, we present results of analysis on Cohorts for Heart And Aging Research in Genomic Epidemiology (CHARGE) WGS freeze 3 dataset in which joint calling, imputation and phasing of over 5300 whole genome samples was produced in under 6 weeks using four state-of-the-art callers. The callers used were SNPTools, GATK-HaplotypeCaller, GATK-UnifiedGenotyper and GotCloud. We used Amazon AWS, a 4000-core in-house cluster at Baylor College of Medicine, IBM power PC Blue BioU at Rice and Rhea at Oak Ridge National Laboratory (ORNL) for the computation. AWS was used for joint calling of 180 TB of BAM files, and ORNL and Rice supercomputers were used for the imputation and phasing step. All other steps were carried out on the local compute cluster. The entire operation used 5.2 million core hours and only transferred a total of 6 TB of data across the platforms. Conclusions Even with increasing sizes of whole genome datasets, ensemble joint calling of SNVs for low coverage data can be accomplished in a scalable, cost effective and fast manner by using heterogeneous computing platforms without compromising on the quality of variants.

59 BASIC BIOLOGICAL SCIENCES↗

NASA NTRSDate not supplied

Implementation of A New Microwave Scattering Database and A Forward Model for Active Microwave Sensors in CRTM

Radiative transfer models are extensively used for the assimilation of satellite observations into NWP models as well as retrieving geophysical products from satellite measurements. CRTM is a community model developed by NOAA JCSDA and widely used for different purposes requiring RT calculations. CRTM requires bulk scattering lookup tables in order to perform all-sky RT calculations. However, the current CRTM lookup tables for microwave frequencies were generated based on the Mie theory by assuming spherical frozen particles. The scattering lookup tables generated using the DDA technique has shown to largely improve the RT scattering calculations in the MW region. This presentation targets (i) the implementation and validation of a DDA database that was originally developed for the ARTS RT model into CRTM, and (ii) developing the CRTM active sensor module that takes advantage of the backscattering coefficients computed using the DDA method. The DDA database only provides single scattering properties of different habits, while CRTM requires bulk scattering properties. The CRTM cloud coefficients were previously generated based on the effective radius for representing the size of the particles. However, effective radius is neither measurable nor provided by the NWP models, thus need to be estimated from other geophysical variables such as water content. Therefore, in addition to calculating the CRTM bulk scattering properties from the DDA single scattering database, the CRTM was also largely modified to use cloud water content (kg.m-3), instead of effective radius, for performing the interpolation over size/mass of the particles. CRTM already requires water content as input, thus no extra variables are required for performing scattering calculations using the new ARTS DDA database. The CRTM scattering modules search for effective radius in cloud coefficient files and will use the cloud water content if the effective radius dimension is not found in the cloud coefficient files. Figure 1 shows the CRTM simulated brightness temperatures computed using different cloud coefficients versus ATMS observed values over Hurricane Irma on September 7, 2017 at 18:00 UTC. We used all the cloud water content values included in ERA5 with default CRTM/DDA habits for water, rain, snow, ice, hail, and graupel. ERA5 does not provide separate water content values for ice, hail, and graupel, thus the ice water content values were divided between ice, hail, and graupel clouds similar to what was explained in the previous section. In channels with a frequency lower than 90 GHz, emission from water and rain clouds can compensate for cloud scattering so that cloud contaminated Tbs are larger than corresponding clear sky Tbs. The DDA simulations for channels 1-7 largely perform better than the Mie simulations. The DDA simulations show a mix of small negative and positive simulated minus observed values, while the Mie results show large negative biases. The weighting functions for some of the ATMS temperature sounding channels (channels 9-15) peak mostly above the clouds, therefore the measured Tbs become less sensitive to clouds so that the results of both Mie and DDA become very similar. The Mie lookup tables generate excessive scattering for channel 16, but not enough scattering for the water vapor channels. In the specific case of Hurricane Maria, the DDA lookup tables do not generate enough scattering for channel 16, but the DDA results are much more consistent with observations for water vapor channels than for channel 16. It should be noted that the results may vary if we use other habits to represent snow, hail, and graupel in the DDA simulations. Although these results clearly show the advantage of the DDA database over the Mie dataset, different error sources such as error in the observations, displacement of clouds in the ERA5 reanalysis, and also lack of convective clouds or in general errors in the input atmospheric and cloud profiles contribute to the differences between the simulated and observed values. Aside from the improvements in the simulations, a major advantage of the new dataset is a large number of habits that can be used to tune the data assimilation systems to perform well in different weather conditions.

Isaac Moradi↗

DOE OSTI2016

A Fast All-sky Radiation Model for Solar applications (FARMS): Algorithm and performance evaluation

Not Available

Xie, Yu↗

NASA NTRS2007

Nonspherical and Spherical Characterization of Ice in Hurricane Erin for Wideband Passive Microwave Comparisons

In order to better understand the characteristics of frozen cloud particles in hurricane systems, computed brightness temperatures were compared with radiometric observations of Hurricane Erin (2001) from the NASA ER-2 aircraft. The focus was oil the frozen particle microphysics and the high frequencies (2 85 GHz) that are particularly sensitive to frozen particles. Frozen particles in hurricanes are an indicator of increasing hurricane intensity. In fact "hot towers" associated with increasing hurricane intensity are composed of frozen ice cloud particles. (They are called hot towers because their column of air is warmer than the surrounding air temperature, but above about 5-7 km to the tops of the towers at 15-19 km, the cloud particles are frozen.) This work showed that indeed, one can model information about cloud ice particle characteristics and indicated that nonspherical ice shapes, instead of spherical particles, provided the best match to the observations. Overall, this work shows that while non-spherical particles show promise, selecting and modeling a proper ice particle parameterization can be difficult and additional in situ measurements are needed to define and validate appropriate parameterizations. This work is important for developing Global Precipitation Measurement (GPM) mission satellite algorithms for the retrieval of ice characteristics both above the melting layer, as in Hurricane Erin, and for ice particles that reach the surface as falling snow.

Skofronick-Jackson, Gail↗

DOE OSTI2025

Simulating Droplet-Resolved Haze and Cloud Chemistry Forming Secondary Organic Aerosols in Turbulent Conditions within Laboratory and Cloud Parcels

Most of our existing knowledge of cloud chemistry in regards to forming secondary organic aerosols (SOA) is based on measurements in bulk aqueous solutions. However, SOA reaction kinetics derived from bulk solution measurements might differ from the kinetics in actual cloud droplets, since turbulent mixing and ionic strengths, and ratio of surface area to volume in individual cloud droplets might vary substantially from bulk solutions in the real atmosphere. Three-dimensional models at various scales have been used to simulate aqueous chemistry. However, most of these models do not resolve turbulence down to the smallest length scales of 1 mm and do not simulate cloud chemistry in individual cloud droplets due to large computational costs. Here we incorporate the formation of isoprene epoxydiol SOA (IEPOX-SOA) in individual droplets within a one-dimensional explicit mixing parcel model (EMPM-Chem). We apply EMPM-Chem to simulate turbulence and droplet-resolved IEPOX-SOA formation using a configuration based on the Michigan Tech Pi chamber. We find that the dissolution of IEPOX gases is weighted more towards larger cloud droplets due to their large liquid water content (compared to smaller droplets), while the conversion of dissolved IEPOX to IEPOX-SOA is much greater within smaller deliquesced haze particles due to their higher acidity and ionic strengths compared to cloud droplets. We also find that as droplet residence times increase in the chamber, e.g., due to increasing ammonium bisulfate seed aerosol injection rates and/or increasing heights of the chamber, formation of IEPOX-SOA increases substantially. Thus, our EMPM-Chem model could be used to design future cloud chambers to maximize SOA production from cloud chemistry. We also apply the EMPM-Chem model to simulate how IEPOX-SOA formation evolves in individual cloud droplets within rising cloudy parcels in the atmosphere. We find that as subsaturated air is entrained into and turbulently mixed with the cloud parcel, evaporation causes a reduction in droplet sizes, which leads to corresponding increases in per droplet ionic strength and acidity. Increased droplet acidity in turn greatly accelerates the kinetics of IEPOX-SOA formation. In conclusion, our results provide key insights into single-cloud-droplet chemistry, suggesting that entrainment mixing may be an important process that increases SOA formation in the real atmosphere.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2020

Practical quantum computation of chemical and nuclear energy levels using quantum imaginary time evolution and Lanczos algorithms

Various methods have been developed for the quantum computation of the ground and excited states of physical and chemical systems, but many of them require either large numbers of ancilla qubits or high-dimensional optimization in the presence of noise. The quantum imaginary-time evolution (QITE) and quantum Lanczos (QLanczos) methods proposed in Motta et al. (2020) eschew the aforementioned issues. In this study, we demonstrate the practical application of these algorithms to challenging quantum computations of relevance for chemistry and nuclear physics, using the deuteron-binding energy and molecular hydrogen binding and excited state energies as examples. With the correct choice of initial and final states, we show that the number of timesteps in QITE and QLanczos can be reduced significantly, which commensurately simplifies the required quantum circuit and improves compatibility with NISQ devices. We have performed these calculations on cloud-accessible IBM Q quantum computers. With the application of readout-error mitigation and Richardson error extrapolation, we have obtained ground and excited state energies that agree well with exact results obtained from diagonalization.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

NASA NTRS1976

Investigation of mesoscale cloud features viewed by LANDSAT

The author has identified the following significant results. Some 50 LANDSAT images displaying mesoscale cloud features were analyzed. This analysis was based on the Rayleigh-Kuettner model describing the formation of that type of mesoscale cloud feature. This model lends itself to computation of the average wind speed in northerly flow from the dimensions of the cloud band configurations measured from a LANDSAT image. In nearly every case, necessary conditions of a curved wind profile and orientation of the cloud streets within 20 degrees of the direction of the mean wind in the convective layer were met. Verification of the results by direct observation was hampered, however, by the incompatibility of the resolution of conventional rawinsonde observations with the scale of the banded cloud patterns measured from LANDSAT data. Comparison seems to be somewhat better in northerly flows than in southerly flows, with the largest discrepancies in wind speed being within 8m/sec, or a factor of two.

Sherr, P. E.↗

NASA NTRS1976

CTRANS: A Monte Carlo program for radiative transfer in plane parallel atmospheres with imbedded finite clouds: Development, testing and user's guide

The program called CTRANS is described which was designed to perform radiative transfer computations in an atmosphere with horizontal inhomogeneities (clouds). Since the atmosphere-ground system was to be richly detailed, the Monte Carlo method was employed. This means that results are obtained through direct modeling of the physical process of radiative transport. The effects of atmopheric or ground albedo pattern detail are essentially built up from their impact upon the transport of individual photons. The CTRANS program actually tracks the photons backwards through the atmosphere, initiating them at a receiver and following them backwards along their path to the Sun. The pattern of incident photons generated through backwards tracking automatically reflects the importance to the receiver of each region of the sky. Further, through backwards tracking, the impact of the finite field of view of the receiver and variations in its response over the field of view can be directly simulated.

Source record↗

NASA NTRS1981

Ionospheric remote sensing of medium-scale gravity waves and tornadic storms

Gravity waves associated with severe storms are investigated on the basis of ionospheric sounding using a ground-based Doppler system. Reverse-group ray tracing computations are used to determine the origin of over 20 gravity waves detected within 800 km of Huntsville, Alabama in association with a group of tornadoes, isolated tornadoes in the presence of a squall line, and isolated tornadoes in the absence of a squall line. Gravity waves associated with tornadoes are found either to be generated by thunderstorms with enhanced convection embedded in a squall line, or in an isolated cloud with enhanced convection. The computed wave sources are observed in all cases to be located near the points where the tornadoes touched down more than an hour after wave excitation. Results show that gravity waves play an important role in troposphere-ionosphere coupling during times of intense convection associated with tornadic storm activity.

Hung, R. J.↗

NASA NTRS2014

Assessment of the Accuracy of the Conventional Ray-Tracing Technique: Implications in Remote Sensing and Radiative Transfer Involving Ice Clouds.

A fundamental problem in remote sensing and radiative transfer simulations involving ice clouds is the ability to compute accurate optical properties for individual ice particles. While relatively simple and intuitively appealing, the conventional geometric-optics method (CGOM) is used frequently for the solution of light scattering by ice crystals. Due to the approximations in the ray-tracing technique, the CGOM accuracy is not well quantified. The result is that the uncertainties are introduced that can impact many applications. Improvements in the Invariant Imbedding T-matrix method (II-TM) and the Improved Geometric-Optics Method (IGOM) provide a mechanism to assess the aforementioned uncertainties. The results computed by the II-TMþIGOM are considered as a benchmark because the IITM solves Maxwell's equations from first principles and is applicable to particle size parameters ranging into the domain at which the IGOM has reasonable accuracy. To assess the uncertainties with the CGOM in remote sensing and radiative transfer simulations, two independent optical property datasets of hexagonal columns are developed for sensitivity studies by using the CGOM and the II-TMþIGOM, respectively. Ice cloud bulk optical properties obtained from the two datasets are compared and subsequently applied to retrieve the optical thickness and effective diameter from Moderate Resolution Imaging Spectroradiometer (MODIS) measurements. Additionally, the bulk optical properties are tested in broadband radiative transfer (RT) simulations using the general circulation model (GCM) version of the Rapid Radiative Transfer Model (RRTMG) that is adopted in the National Center for Atmospheric Research (NCAR) Community Atmosphere Model (CAM, version 5.1). For MODIS retrievals, the mean bias of uncertainties of applying the CGOM in shortwave bands (0.86 and 2.13 micrometers) can be up to 5% in the optical thickness and as high as 20% in the effective diameter, depending on cloud optical thickness and effective diameter. In the MODIS infrared window bands centered at 8.5, 11, and 12 micrometers biases in the optical thickness and effective diameter are up to 12% and 10%, respectively. The CGOM-based simulation errors in ice cloud radiative forcing calculations are on the order of 10Wm(exp 2).

remote sensing↗

DOE OSTI2021

Fast All-sky Radiation Model for Solar applications (FARMS) [SWR-16-18]

The Fast All-sky Radiation Model for Solar applications (FARMS) is used to compute cloudy irradiance. Radiative transfer (RT) models simulating broadband solar radiation have been widely used by atmospheric scientists to model solar resources for various energy applications such as operational forecasting. Due to the complexity of solving the RT equation, the computation under cloudy conditions can be extremely time consuming though many approximations (e.g. two-stream approach and delta-M truncation scheme) have been utilized. Thus, a more efficient RT model is crucial for model developers as a new option for approximating solar radiation at the land surface with minimal loss of accuracy. We have developed a fast all-sky radiation model for solar applications (FARMS) using the simplified clear-sky RT model, REST2, and simulated cloud transmittances and reflectances from the Rapid Radiation Transfer Model (RRTM) with a sixteen-stream Discrete Ordinates Radiative Transfer (DISORT). Simulated lookup tables (LUTs) of cloud transmittances and reflectances were created by varying cloud optical thicknesses, cloud particle sizes, and solar zenith angles. Equations with optimized parameters were fitted to the cloud transmittances and reflectances to develop the model. Using this model the all-sky solar irradiance at the land surface can be computed rapidly by combining REST2 with the cloud transmittances and reflectances. This new RT model is more than 1000 times faster than those currently utilized in solar resource assessment and forecasting since it does not explicitly solve the RT equation for each individual cloud condition. Our results indicate the accuracy of the fast radiative transfer model is comparable to or better than two-stream approximation in term of computing cloud transmittance and solar radiation.

Xie, Yu↗

DOE OSTI2023

Fast All-sky Radiation Model for Solar applications (FARMS) 2023 [SWR-16-18]

Xie, Yu↗

DOE OSTI2017

Modeling Condensation in Deep Convection

Cloud-scale models apply two drastically different methods to represent condensation of water vapor to form and grow cloud droplets. Maintenance of water saturation inside liquid clouds is assumed in the computationally efficient saturation adjustment approach used in most bulk microphysics schemes. When super- or subsaturations are allowed, condensation/evaporation can be calculated using the predicted saturation ratio and (either predicted or prescribed) mean droplet radius and concentration. Here, we investigate differences between simulations of deep unorganized convection applying a saturation adjustment condensation scheme (SADJ) and a scheme with supersaturation prediction (SPRE). A double-moment microphysics scheme with CCN activation parameterized as a function of the local vertical velocity is applied to compare cloud fields simulated applying SPRE and SADJ. Clean CCN conditions are assumed to demonstrate upper limits of the SPRE and SADJ difference. Microphysical piggybacking is used to extract the impacts with confidence. Results show a significant impact on deep convection dynamics, with SADJ featuring more cloud buoyancy and thus stronger updrafts. This leads to around a 3% increase of the surface rain accumulation in SADJ. Upper-tropospheric anvil cloud fractions are much larger in SPRE than in SADJ because of the higher ice concentrations and thus longer residence times of anvil particles in SPRE, as demonstrated by sensitivity tests. Higher ice concentrations in SPRE come from significantly larger ice supersaturations in strong convective updrafts that feature water supersaturations of several percent.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2021

Two–Dimensional Idealized Hadley Circulation Simulation for Global High Resolution Model Development

A new framework for the development of global high resolution models using a vertical- meridional, two-dimensional pole-to-pole domain is proposed. Compared with the three-dimensional global model, the two-dimensional framework can simulate cloud-scale convection at significantly reduced computational cost, data volume, and turn-around time, and accelerate the development phase. Although idealized, it allows for simulation of various clouds and scale interactions under a wide range of environmental conditions, even though zonal waves such as mid latitude storms cannot be simulated. Physics schemes can be tested and assessed over a full array of simultaneously occurring cloud regimes. The framework can serve as an intermediate step between a one-dimensional simulation and a full three-dimensional global model. Using this framework, we analyze the resolution dependencies of simulated clouds and the Hadley circulation in a range of global high resolution models by performing multiple 1000-day simulations; for some resolutions tested, for example, 2 km horizontal resolution, three-dimensional simulation is not feasible at the present time. Analysis shows that both horizontal and vertical resolution determine the properties of the Hadley circulation and clouds, and that the differences among the simulated Hadley circulations are understood as differences in scale interactions influenced by resolution.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2015

Defense of Cyber Infrastructures Against Cyber-Physical Attacks Using Game-Theoretic Models

The operation of cyber infrastructures relies on both cyber and physical components, which are subject to incidental and intentional degradations of different kinds. Within the context of network and computing infrastructures, we study the strategic interactions between an attacker and a defender using game-theoretic models that take into account both cyber and physical components. The attacker and defender optimize their individual utilities expressed as sums of cost and system terms. First, we consider a Boolean attack-defense model, wherein the cyber and physical sub-infrastructures may be attacked and reinforced as individual units. Second, we consider a component attack-defense model wherein their components may be attacked and defended, and the infrastructure requires minimum numbers of both to function. We show that the Nash equilibrium under uniform costs in both cases is computable in polynomial time, and it provides high-level deterministic conditions for the infrastructure survival. When probabilities of successful attack and defense, and of incidental failures are incorporated into the models, the results favor the attacker but otherwise remain qualitatively similar. This approach has been motivated and validated by our experiences with UltraScience Net infrastructure, which was built to support high-performance network experiments. In conclusion, the analytical results, however, are more general, and we apply them to simplified models of cloud and high-performance computing infrastructures.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2013

A Novel Method for Estimating Shortwave Direct Radiative Effect of Above-cloud Aerosols over Ocean Using CALIOP and MODIS Data

This paper describes an efficient and unique method for computing the shortwave direct radiative effect (DRE) of aerosol residing above low-level liquid-phase clouds using CALIOP and MODIS data. It accounts for the overlapping of aerosol and cloud rigorously by utilizing the joint histogram of cloud optical depth and cloud top pressure. Effects of sub-grid scale cloud and aerosol variations on DRE are accounted for. It is computationally efficient through using grid-level cloud and aerosol statistics, instead of pixel-level products, and a pre-computed look-up table in radiative transfer calculations. We verified that for smoke over the southeast Atlantic Ocean the method yields a seasonal mean instantaneous shortwave DRE that generally agrees with more rigorous pixel-level computation within 4%. We have also computed the annual mean instantaneous shortwave DRE of light-absorbing aerosols (i.e., smoke and polluted dust) over global ocean based on 4 yr of CALIOP and MODIS data. We found that the variability of the annual mean shortwave DRE of above-cloud light-absorbing aerosol is mainly driven by the optical depth of the underlying clouds.

CALIOP and MODIS↗

DOE OSTI2024

Portable Acceleration of CMS Computing Workflows with Coprocessors as a Service

Computing demands for large scientific experiments, such as the CMS experiment at the CERN LHC, will increase dramatically in the next decades. To complement the future performance increases of software running on central processing units (CPUs), explorations of coprocessor usage in data processing hold great potential and interest. Coprocessors are a class of computer processors that supplement CPUs, often improving the execution of certain functions due to architectural design choices. We explore the approach of Services for Optimized Network Inference on Coprocessors (SONIC) and study the deployment of this as-a-service approach in large-scale data processing. In the studies, we take a data processing workflow of the CMS experiment and run the main workflow on CPUs, while offloading several machine learning (ML) inference tasks onto either remote or local coprocessors, specifically graphics processing units (GPUs). With experiments performed at Google Cloud, the Purdue Tier-2 computing center, and combinations of the two, we demonstrate the acceleration of these ML algorithms individually on coprocessors and the corresponding throughput improvement for the entire workflow. This approach can be easily generalized to different types of coprocessors and deployed on local CPUs without decreasing the throughput performance. We emphasize that the SONIC approach enables high coprocessor usage and enables the portability to run workflows on different types of coprocessors.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2014

The Fabric for Frontier Experiments Project at Fermilab

The FabrIc for Frontier Experiments (FIFE) project is a new, far-reaching initiative within the Fermilab Scientific Computing Division to drive the future of computing services for experiments at FNAL and elsewhere. It is a collaborative effort between computing professionals and experiment scientists to produce an end-to-end, fully integrated set of services for computing on the grid and clouds, managing data, accessing databases, and collaborating within experiments. FIFE includes 1) easy to use job submission services for processing physics tasks on the Open Science Grid and elsewhere, 2) an extensive data management system for managing local and remote caches, cataloging, querying, moving, and tracking the use of data, 3) custom and generic database applications for calibrations, beam information, and other purposes, 4) collaboration tools including an electronic log book, speakers bureau database, and experiment membership database. All of these aspects will be discussed in detail. FIFE sets the direction of computing at Fermilab experiments now and in the future, and therefore is a major driver in the design of computing services worldwide.

Kirby, Michael↗