Simulating the consequence of cloud cover on earth-viewing space missions
Cloud cover effect on earth photographing space missions by Monte Carlo computer simulation, using worldwide cloud statistics
Brown, S. C.↗
SEARCH · Engineering Papers
Results for “CLOUD COMPUTING”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Advances in Grid Computing for the FabrIc for Frontier Experiments Project at Fermialb
The FabrIc for Frontier Experiments (FIFE) project is a major initiative within the Fermilab Scientic Computing Division charged with leading the computing model for Fermilab experiments. Work within the FIFE project creates close collaboration between experimenters and computing professionals to serve high-energy physics experiments of diering size, scope, and physics area. The FIFE project has worked to develop common tools for job submission, certicate management, software and reference data distribution through CVMFS repositories, robust data transfer, job monitoring, and databases for project tracking. Since the projects inception the experiments under the FIFE umbrella have signicantly matured, and present an increasingly complex list of requirements to service providers. To meet these requirements, the FIFE project has been involved in transitioning the Fermilab General Purpose Grid cluster to support a partitionable slot model, expanding the resources available to experiments via the Open Science Grid, assisting with commissioning dedicated high-throughput computing resources for individual experiments, supporting the eorts of the HEP Cloud projects to provision a variety of back end resources, including public clouds and high performance computers, and developing rapid onboarding procedures for new experiments and collaborations. The larger demands also require enhanced job monitoring tools, which the project has developed using such tools as ElasticSearch and Grafana. in helping experiments manage their large-scale production work ows. This group in turn requires a structured service to facilitate smooth management of experiment requests, which FIFE provides in the form of the Production Operations Management Service (POMS). POMS is designed to track and manage requests from the FIFE experiments to run particular work ows, and support troubleshooting and triage in case of problems. Recently a new certicate management infrastructure called Distributed Computing Access with Federated Identities (DCAFI) has been put in place that has eliminated our dependence on a Fermilab-specic third-party Certicate Authority service and better accommodates FIFE collaborators without a Fermilab Kerberos account. DCAFI integrates the existing InCommon federated identity infrastructure, CILogon Basic CA, and a MyProxy service using a new general purpose open source tool. We will discuss the general FIFE onboarding strategy, progress in expanding FIFE experiments presence on the Open Science Grid, new tools for job monitoring, the POMS service, and the DCAFI project.
Herner, K.↗
Radiometric Assessment of the First Three Years of the NOAA-20 VIIRS Reflective Solar Bands Calibration
NASA’s Clouds and the Earth’s Radiant Energy System (CERES) SYN1deg Ed4.1 product utilizes geostationary (GEO) satellite measured radiances and retrieved cloud properties to account for the regional diurnal fluctuations in the Earth’s radiant broadband fluxes for times between the CERES measurements gathered from the Aqua (1:30 PM) and Terra (10:30 AM) sun-synchronous satellites. In CERES Edition 4 products, a global uniformity in the cloud properties and computed surface fluxes across the GEO satellite domains is maintained by scaling the radiance observations from more than twenty GEO visible imagers in the CERES record to a common radiometric reference scale, i.e., Aqua-MODIS. With the new-generation GEO (Himawari-8/9 and GOES-16/17) imagers having multiple reflective solar bands (RSB) that are spectrally similar to those of VIIRS, the CERES Imager and Geostationary Calibration Group (IGCG) is preparing to use NOAA-20 VIIRS as the reference imager for maintaining the radiometric uniformity across the GEO imager constellation. Given the recent Aqua satellite anomaly, this transition may happen sooner than the projected date of Aqua’s de-orbit. This paper presents an independent performance evaluation of the first three years of the NOAA-20 VIIRS RSB calibration in the NASA VIIRS Land Science Investigator-led Processing System (Land SIPS) L1b Collection 2 dataset. The temporal radiometric stability is assessed using multiple invariant Earth targets, including tropical deep convective clouds and the Saharan desert. The invariant target anisotropic reflectance at the top of atmosphere was modeled using five years of stable satellite observations acquired from the previous VIIRS instrument onboard the Suomi National Polar-orbiting Partnership (S-NPP) satellite. The anisotropic corrections are essential for detecting temporal trends with a high statistical confidence. In addition, the radiometric consistency between the RSB of the two VIIRS instruments will be evaluated.
Rajendra Bhatt↗
Radiometric Assessment of the First Three Years of the NOAA-20 VIIRS Reflective Solar Bands Calibration
NASA’s Clouds and the Earth’s Radiant Energy System (CERES) SYN1deg Ed4.1 product utilizes geostationary (GEO) satellite measured radiances and retrieved cloud properties to account for the regional diurnal fluctuations in the Earth’s radiant broadband fluxes for times between the CERES measurements gathered from the Aqua (1:30 PM) and Terra (10:30 AM) sun-synchronous satellites. In CERES Edition 4 products, a global uniformity in the cloud properties and computed surface fluxes across the GEO satellite domains is maintained by scaling the radiance observations from more than twenty GEO visible imagers in the CERES record to a common radiometric reference scale, i.e., Aqua-MODIS. With the new-generation GEO (Himawari-8/9 and GOES-16/17) imagers having multiple reflective solar bands (RSB) that are spectrally similar to those of VIIRS, the CERES Imager and Geostationary Calibration Group (IGCG) is preparing to use NOAA-20 VIIRS as the reference imager for maintaining the radiometric uniformity across the GEO imager constellation. Given the recent Aqua satellite anomaly, this transition may happen sooner than the projected date of Aqua’s de-orbit. This paper presents an independent performance evaluation of the first three years of the NOAA-20 VIIRS RSB calibration in the NASA VIIRS Land Science Investigator-led Processing System (Land SIPS) L1b Collection 2 dataset. The temporal radiometric stability is assessed using multiple invariant Earth targets, including tropical deep convective clouds and the Saharan desert. The invariant target anisotropic reflectance at the top of the atmosphere was modeled using five years of stable satellite observations acquired from the previous VIIRS instrument onboard the Suomi National Polar-orbiting Partnership (S-NPP) satellite. The anisotropic corrections are essential for detecting temporal trends with high statistical confidence. In addition, the radiometric consistency between the RSB of the two VIIRS instruments will be evaluated.
Rajendra Bhatt↗
Report on the Second ARM Mobile Facility (AMF2) Roll, Pitch, and Heave (RPH) Stabilization Platform: Design and Evaluation
One of the primary objectives of the U.S. Department of Energy’s Atmospheric Radiation Measurement (ARM) Climate Research Facility’s second Mobile Facility (AMF2) is to obtain reliable measurements of solar, surface, and atmospheric radiation, as well as cloud and atmospheric properties, from ocean-going vessels. To ensure that these climatic measurements are representative and accurate, many AMF2 instrument systems are designed to collect data in a zenith orientation. A pillar of the AMF2 strategy in this effort is the use of a stable platform. The purpose of the platform is to 1) mitigate vessel motion for instruments that require a truly vertical orientation and keep them pointed in the zenith direction, and 2) allow for accurate positioning for viewing or shading of the sensors from direct sunlight. Numerous ARM instruments fall into these categories, but perhaps the most important are the vertically pointing cloud radars, for which vertical motions are a critical parameter. During the design and construction phase of AMF2, an inexpensive stable platform was purchased to perform the stabilization tasks for some of these instruments. The first table compensated for roll, pitch, and yaw (RPY) and was reported upon in a previous technical report (Kafle and Coulter, 2012). Subsequently, a second table was purchased specifically for operation with the Marine W-band cloud radar (MWACR). Computer programs originally developed for RPY were modified to communicate with the new platform controller and with an inertial measurements platform that measures true ship motion components (roll, pitch, yaw, surge, sway, and heave). This platform could not be tested dynamically for RPY because of time constraints requiring its deployment aboard the container ship Horizon Spirit in September 2013. Hence the initial motion tests were conducted on the initial cruise. Subsequent cruises provided additional test results. The platform, as tested, meets all the design and performance criteria established for its use. This is a report of the results of those efforts and the critical points in moving forward
54 ENVIRONMENTAL SCIENCES↗
Use of TRMM Data to Test an Improved Parameterization of Stratiform Precipitation
We successfully implemented our EAUcup parameterization of convection in our most recent geodesic-grid version of the Colorado State University General Circulation Model (CSU GCM). Tendencies due to convection and large-scale cloud microphysics are computed using the same time-step, allowing smoother interactions between convective and large-scale moist processes. Figures illustrate our most recent progress on the simulation of convective and stratiform precipitations for January. Comparisons against TRMM precipitation radar (PR) data would reveal a systematic overestimation of the simulated monthly-mean convective rainfall over well-known areas of deep convection. One major deficiency in the global distribution of stratiform precipitation simulated in the CSU GCM is the lack of stratiform precipitation over land in the tropics. This result strongly disagree with Fowler and Randall who show that convective and stratiform precipitations always coexist and that their ratio to the total precipitation is about 0.5. We believe that stratiform precipitation over land evaporates before reaching the ground because the lower troposphere remains too dry in response to the lack of convective detrainment occurring immediately above the Planetary Boundary Layer (PBL). This issue is further investigated at present. We made significant progress in the development of our parameterization of fractional cloudiness. As an example of our ongoing research effort, a figure shows the global distribution of the cloud ice mixing ratio and corresponding fractional area of ice clouds at 250 hPa. These results were obtained using our latitude-longitude grid version of the CSU GCM. There is a nice correspondence between areas of high (low) values of the cloud ice mixing ratio and the cloud fraction. A nice correspondence between the global distribution of the cloud water mixing ratio and fraction of water clouds in the lower troposphere can also be found. We are pursuing this effort, but in the geodesic version of the GCM. We hope to be able to make comparison of the simulated precipitation against TRMM precipitation data in the near future.
Randall, David A.↗
Cloud identification using genetic algorithms and massively parallel computation
As a Guest Computational Investigator under the NASA administered component of the High Performance Computing and Communication Program, we implemented a massively parallel genetic algorithm on the MasPar SIMD computer. Experiments were conducted using Earth Science data in the domains of meteorology and oceanography. Results obtained in these domains are competitive with, and in most cases better than, similar problems solved using other methods. In the meteorological domain, we chose to identify clouds using AVHRR spectral data. Four cloud speciations were used although most researchers settle for three. Results were remarkedly consistent across all tests (91% accuracy). Refinements of this method may lead to more timely and complete information for Global Circulation Models (GCMS) that are prevalent in weather forecasting and global environment studies. In the oceanographic domain, we chose to identify ocean currents from a spectrometer having similar characteristics to AVHRR. Here the results were mixed (60% to 80% accuracy). Given that one is willing to run the experiment several times (say 10), then it is acceptable to claim the higher accuracy rating. This problem has never been successfully automated. Therefore, these results are encouraging even though less impressive than the cloud experiment. Successful conclusion of an automated ocean current detection system would impact coastal fishing, naval tactics, and the study of micro-climates. Finally we contributed to the basic knowledge of GA (genetic algorithm) behavior in parallel environments. We developed better knowledge of the use of subpopulations in the context of shared breeding pools and the migration of individuals. Rigorous experiments were conducted based on quantifiable performance criteria. While much of the work confirmed current wisdom, for the first time we were able to submit conclusive evidence. The software developed under this grant was placed in the public domain. An extensive user's manual was written and distributed nationwide to scientists whose work might benefit from its availability. Several papers, including two journal articles, were produced.
Buckles, Bill P.↗
Improving Cloud Optical Property Retrievals for Partly Cloudy Pixels Using Coincident Higher-Resolution Single Band Measurements: A Feasibility Study Using ASTER Observations
Clear-sky contamination is a challenging and long-lasting problem for cloud optical thickness (τ) and effective droplet radius (r eff ) retrievals using passive satellite sensors. This study explores the feasibility of improving both τ and r eff retrievals for partly cloudy (PCL) pixels by using available subpixel samples in a visible to near-infrared band, which many satellite sensors offer. Data are provided by high-resolution reflectance (R ) observations and cloud property retrievals by the Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) at horizontal resolutions between 30-960m. For partly cloudy 960-m observations, the clear-sky component of the pixels induces significant underestimations of up to 58% for τ, while overestimations in r eff can exceed 41%. This yields underestimations in the derived liquid water path and cloud droplet number concentration of up to 68% and 72%, respectively. By means of three different assumptions it is shown that subpixel R observations in the visible to near-infrared band can be used to estimate higher-resolution R for the second band in the retrieval scheme, as well as the subpixel cloud cover. Here, the estimated values compare well to actually observed ASTER results and are used to retrieve cloud properties, which are unbiased by the clear-sky component of PCL pixels. While the presented retrieval approach is only evaluated for marine boundary layer clouds, it is computationally efficient and can be easily applied to observations from different imagers. As an example, the PCL retrieval scheme is applied to data by the Moderate Resolution Imaging Spectroradiometer, where similar biases for PCL pixels are observed.
54 ENVIRONMENTAL SCIENCES↗
An open source analysis framework for large-scale building energy modeling
Full integration of building energy modelling into the design and retrofit process has long been a goal of building scientists and practitioners. However, significant barriers still exist. Among them are the lack of available: (1) configurable technology stacks for performing both small- and large-scale analyses, (2) different classes of algorithms compatible with common design workflows, and (3) analysis tools for effectively visualizing large-scale simulation results. This article discusses the OpenStudio® Analysis Framework: a scalable analysis framework for building energy modelling that was developed to overcome the three barriers listed above. The framework is open-source and scalable to facilitate wider adoption and has a clearly defined application programming interface upon which other applications can be built. It runs on high-performance computing systems, within cloud infrastructure, and on laptops, and uses a common workflow to enable different classes of algorithms. Lessons learned from previous development efforts are also discussed.
32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
Leveraging the checkpoint-restart technique for optimizing CPU efficiency of ATLAS production applications on opportunistic platforms
Data processing applications of the ATLAS experiment, such as event simulation and reconstruction, spend considerable amount of time in the initialization phase. This phase includes loading a large number of shared libraries, reading detector geometry and condition data from external databases, building a transient representation of the detector geometry and initializing various algorithms and services. In some cases the initialization step can take as long as 10-15 minutes. Such slow initialization has a significant negative impact on overall CPU efficiency of the production job, especially when the job is executed on opportunistic, often short-lived, resources such as commercial clouds or volunteer computing. In order to improve this situation, we can take advantage of the fact that ATLAS runs large numbers of production jobs with similar configuration parameters (e.g. jobs within the same production task). This allows us to checkpoint one job at the end of its configuration step and then use the generated checkpoint image for rapid startup of thousands of production jobs. By applying this technique we can bring the initialization time of a job from tens of minutes down to just a few seconds. In addition to that we can leverage container technology for restarting checkpointed applications on the variety of computing platforms, in particular of platforms different from the one on which the checkpoint image was created. We will describe the mechanism of creating checkpoint images of Geant4 simulation jobs with AthenaMP (the multi-process version of the ATLAS data simulation, reconstruction and analysis framework Athena) and the usage of these images for running ATLAS Simulation production jobs on volunteer computing resources (ATLAS@Home) and on Supercomputers.
72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
Benchmarking Information Scrambling
Information scrambling refers to the rapid spreading of initially localized information over an entire system, via the generation of global entanglement. This effect is usually detected by measuring a temporal decay of the out-of-time order correlators. However, in experiments, decays of these correlators suffer from fake positive signals from various sources, e.g., decoherence due to inevitable couplings to the environment, or errors that cause mismatches between the purported forward and backward evolutions. Here, in this Letter, we provide a simple and robust approach to single out the effect of genuine scrambling. This allows us to benchmark the scrambling process by quantifying the degree of the scrambling from the noisy backgrounds. We also demonstrate our protocol with simulations on IBM cloud-based quantum computers.
97 MATHEMATICS AND COMPUTING↗
Impact of Data Placement on Resilience in Large-Scale Object Storage Systems
Distributed object storage architectures have become the de facto standard for high-performance storage in big data, cloud, and HPC computing. Object storage deployments using commodity hardware to reduce costs often employ object replication as a method to achieve data resilience. Repairing object replicas after failure is a daunting task for systems with thousands of servers and billions of objects, however, and it is increasingly difficult to evaluate such scenarios at scale on realworld systems. Resilience and availability are both compromised if objects are not repaired in a timely manner. In this work we leverage a high-fidelity discrete-event simulation model to investigate replica reconstruction on large-scale object storage systems with thousands of servers, billions of objects, and petabytes of data. We evaluate the behavior of CRUSH, a well-known object placement algorithm, and identify configuration scenarios in which aggregate rebuild performance is constrained by object placement policies. After determining the root cause of this bottleneck, we then propose enhancements to CRUSH and the usage policies atop it to enable scalable replica reconstruction. We use these methods to demonstrate a simulated aggregate rebuild rate of 410 GiB/s (within 5% of projected ideal linear scaling) on a 1,024-node commodity storage system. We also uncover an unexpected phenomenon in rebuild performance based on the characteristics of the data stored on the system.
Carns, Philip↗
Development and Validation of Southeast Asia Solar Resource Data [Slides]
Lack of access to high-quality, publicly available, time series solar data to inform decisions that will transform energy sectors in Southeast Asia is a challenge. The solution is to level the playing field by offering free, high-quality, robust solar data to inform private sector investment and policymaking. This is done by (1) leveraging deep NREL expertise in atmospheric science, solar resource assessment, high-performance computing, and cloud-based data dissemination, (2) producing and validating high spatial and temporal resolution solar resource data, (3) making data available on the USAID-funded global Renewable Energy Data Explorer platform, (4) providing capacity building for data and applications, and (5) informing future demand-driven tool development.
14 SOLAR ENERGY↗
GlideinBenchmark: collecting resource information to optimize provisioning
Choosing the right resource can speedup jobs completion, better utilize the available hardware and visibly reduce costs, especially when renting computers on the cloud. This was demonstrated in earlier studies on HEPCloud. But the benchmarking of the resources proved to be a laborious and time-consuming process. This paper presents GlideinBenchmark, a new Web application leveraging the pilot infrastructure of GlideinWMS to benchmark resources, and shows how to use the data collected and published by GlideinBenchmark to automate the optimal selection of resources. An experiment can select the benchmark or the set of benchmarks that most closely evaluate the performance of its workflows. With GlideinBenchmark and the help of the GldieinWMS Factory it controls the benchmark execution. Finally, a scheduler like HEPCloud’s Decision Engine can use the results to optimize resource provisioning.
Mambelli, Marco↗
GlideinBenchmark: collecting resource information to optimize provisioning
Choosing the right resource can speed up job completion, better utilize the available hardware, and visibly reduce costs, especially when renting computers in the cloud. This was demonstrated in earlier studies on HEPCloud. However, the benchmarking of the resources proved to be a laborious and time-consuming process. This paper presents GlideinBenchmark, a new Web application leveraging the pilot infrastructure of GlideinWMS to benchmark resources, and it shows how to use the data collected and published by GlideinBenchmark to automate the optimal selection of resources. An experiment can select the benchmark or the set of benchmarks that most closely evaluate the performance of its workflows. GlideinBenchmark, with the help of the GlideinWMS Factory, controls the benchmark execution. Finally, a scheduler like HEPCloud's Decision Engine can use the results to optimize resource provisioning.
Mambelli, Marco [Fermilab] (ORCID:0000000294892681↗
The remarkable inefficiency of stratocumulus
Marine stratocumulus clouds play a central role in Earth's climate system by reflecting incoming solar radiation and exerting a strong cooling effect. Their organization into open and closed mesoscale cellular morphologies can be thought of as an example of bistable dynamics driven by aerosol–cloud interactions and mesoscale processes. From the perspective of non-equilibrium thermodynamics, these structures are an example of a far-from-equilibrium open system that continuously produces and exports entropy. While entropy production has been studied in idealized deep convective systems, it has not yet been quantified for shallow clouds. Here, we compute and decompose the internal entropy production of open- and closed-cell stratocumulus using an ensemble of large-eddy simulations. We show that the overall entropy production of stratocumulus is low, reflecting the limited vertical extent and corresponding reduced ability to utilize the energy fluxes at the system's boundaries. Moist processes dominate the overall irreversibility, which, combined with their low entropy production, leads to a mechanical efficiency about an order of magnitude smaller than in deep convective systems. Although the dominant irreversible processes differ between open- and closed-cell regimes, the distributions of total entropy production largely overlap across the ensemble, limiting the ability to distinguish the dynamics of individual cases based solely on total entropy production.
54 ENVIRONMENTAL SCIENCES↗
A geospatial risk analysis graphical user interface for identifying hazardous chemical emission sources
Background: Performing back trajectory and forward trajectory using the Hybrid Single-Particle Lagrangian Integrated Trajectory Model (HYSPLIT) is a reliable approach for assessing particle transport after release among mid-field atmospheric models. HYSPLIT has an externally facing online interface that allows non-expert users to run the model trajectories without requiring extensive training or programming. However, the existing HYSPLIT interface is limited if simulations have a large amount of meteorological data and timesteps that are not coincident. The objective of this study is to design and develop a more robust tool to rapidly evaluate hazard transport conditions and to perform risk analysis, while still maintaining an intuitive and user-friendly interface. Methods: HYSPLIT calculates forward and backward trajectories of particles based on wind speed, wind direction, and the corresponding location, timestamp, and Pasquill stability classes of the regions of the atmosphere in terms of the wind speed, the amount of solar radiation, and the fractional cloud cover. The computed particle transport trajectories, combined with the online Proton Transfer Reaction-Mass Spectrometry (PTR-MS) data (https://figshare.com/articles/dataset/ARL_Data_from_PROS_station_at_Hanford_site/19993964), can be used to identify and quantify the sources and affected area of the hazardous chemicals’ emission using the potential source distribution function (PSDF). PSDF is an improved statistical function based on the well-known potential source contribution function (PSCF) in establishing the air pollutant source and receptor relationship. Performing this analysis requires a range of meteorological and pollutant concentration measurements to be statistically meaningful. The existing HYSPLIT graphical user interface (GUI) does not easily permit computations of trajectories of a dataset of meteorological data in high temporal frequency. To improve the performance of HYSPLIT computations from a large dataset and enhance risk analysis of the accidental release of material at risk, a geospatial risk analysis tool (GRAT-GUI) is created to allow large data sets to be processed instantaneously and to provide ease of visualization. Results: The GRAT-GUI is a native desktop-based application and can be run in any Windows 10 system without any internet access requirements, thus providing a secure way to process large meteorological datasets even on a standalone computer. GRAT-GUI has features to import, integrate, and convert meteorological data with various formats for hazardous chemical emission source identification and risk analysis as a self-explanatory user interface. The tool is available at https://figshare.com/articles/software/GRAT/19426742.
97 MATHEMATICS AND COMPUTING↗
Extension of surface data by use of meteorological satellites
Ways of using meteorological satellite data to extend surface data are summarized. Temperature models are prepared from infrared data from ITOS/NOAA, NIMBUS, SMS/GOES, or future LANDSAT satellites. Using temperatures for surface meteorological stations as anchors, an adjustment is made to temperature values for each pixel in the model. The result is an image with an estimated temperature for each pixel. This provides an economical way of producing detailed temperature information for data-sparse areas, such as are found in underdeveloped countries. Related uses of these satellite data are also given, including the use of computer prepared cloud-free composites to extend climatic zones, and their use in discrimination of reflectivity-thermal regime zones.
Giddings, L. E.↗