Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Cloud Computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 685 records · Page 38

Joint Genome Institute Analysis Workflow Service (JAWS) v2.7

The U.S. Department of Energy Joint Genome Institute (JGI) has developed the JGI Analysis Workflow Service (JAWS) as a distributed framework to run computational workflows across diverse high-performance computing (HPC) and cloud environments. JAWS enhances the reusability, scalability, and robustness of scientific workflows by orchestrating data movement, code execution, and results retrieval across multiple DOE facilities. At its core, JAWS integrates the Cromwell workflow engine to run workflows expressed in the Workflow Description Language (WDL), ensuring portability and interoperability. To provide consistent runtime environments, JAWS employs container technologies such as Shifter, Apptainer, and Docker. Workflow tasks are managed via HTCondor on HPC backends, while Globus ensures secure and efficient data transfer between sites. JAWS is deployed as a multi-site workflow manager across national laboratory computing facilities, with dedicated instances supporting community projects such as the National Microbiome Data Collaborative (NMDC) and KBase. This distributed, service-oriented architecture enables users to "write once, run anywhere," providing scalable, production-quality workflow execution.

Kirton, Edward↗

Using Multi-Scale Modeling Systems to Study the Precipitation Processes

In recent years, exponentially increasing computer power has extended Cloud Resolving Model (CRM) integrations from hours to months, the number of computational grid points from less than a thousand to close to ten million. Three-dimensional models are now more prevalent. Much attention is devoted to precipitating cloud systems where the crucial 1-km scales are resolved in horizontal domains as large as 10,000 km in two-dimensions, and 1,000 x 1,000 km2 in three-dimensions. Cloud resolving models now provide statistical information useful for developing more realistic physically based parameterizations for climate models and numerical weather prediction models. It is also expected that NWP and mesoscale model can be run in grid size similar to cloud resolving model through nesting technique. Recently, a multi-scale modeling system with unified physics was developed at NASA Goddard. It consists of (1) a cloud-resolving model (Goddard Cumulus Ensemble model, GCE model), (2) a regional scale model (a NASA unified weather research and forecast, WRF), (3) a coupled CRM and global model (Goddard Multi-scale Modeling Framework, MMF), and (4) a land modeling system. The same microphysical processes, long and short wave radiative transfer and land processes and the explicit cloud-radiation, and cloud-land surface interactive processes are applied in this multi-scale modeling system. This modeling system has been coupled with a multi-satellite simulator to use NASA high-resolution satellite data to identify the strengths and weaknesses of cloud and precipitation processes simulated by the model. In this talk, a review of developments and applications of the multi-scale modeling system will be presented. In particular, the results from using multi-scale modeling system to study the interactions between clouds, precipitation, and aerosols will be presented. Also how to use of the multi-satellite simulator to improve precipitation processes will be discussed.

Tao, Wei-Kuo↗

Using Multi-Scale Modeling Systems and Satellite Data to Study the Precipitation Processes

In recent years, exponentially increasing computer power has extended Cloud Resolving Model (CRM) integrations from hours to months, the number of computational grid points from less than a thousand to close to ten million. Three-dimensional models are now more prevalent. Much attention is devoted to precipitating cloud systems where the crucial 1-km scales are resolved in horizontal domains as large as 10,000 km in two-dimensions, and 1,000 x 1,000 km2 in three-dimensions. Cloud resolving models now provide statistical information useful for developing more realistic physically based parameterizations for climate models and numerical weather prediction models. It is also expected that NWP and mesoscale model can be run in grid size similar to cloud resolving model through nesting technique. Recently, a multi-scale modeling system with unified physics was developed at NASA Goddard. It consists of (l) a cloud-resolving model (Goddard Cumulus Ensemble model, GCE model), (2) a regional scale model (a NASA unified weather research and forecast, WRF), (3) a coupled CRM and global model (Goddard Multi-scale Modeling Framework, MMF), and (4) a land modeling system. The same microphysical processes, long and short wave radiative transfer and land processes and the explicit cloud-radiation, and cloud-land surface interactive processes are applied in this multi-scale modeling system. This modeling system has been coupled with a multi-satellite simulator to use NASA high-resolution satellite data to identify the strengths and weaknesses of cloud and precipitation processes simulated by the model. In this talk, the recent developments and applications of the multi-scale modeling system will be presented. In particular, the results from using multi-scale modeling system to study the precipitating systems and hurricanes/typhoons will be presented. The high-resolution spatial and temporal visualization will be utilized to show the evolution of precipitation processes. Also how to use of the multi-satellite simulator tqimproy~ precipitation processes will be discussed.

Tao, Wei-Kuo↗

Microphysics in the Multi-Scale Modeling Systems with Unified Physics

In recent years, exponentially increasing computer power has extended Cloud Resolving Model (CRM) integrations from hours to months, the number of computational grid points from less than a thousand to close to ten million. Three-dimensional models are now more prevalent. Much attention is devoted to precipitating cloud systems where the crucial 1-km scales are resolved in horizontal domains as large as 10,000 km in two-dimensions, and 1,000 x 1,000 km2 in three-dimensions. Cloud resolving models now provide statistical information useful for developing more realistic physically based parameterizations for climate models and numerical weather prediction models. It is also expected that NWP and mesoscale model can be run in grid size similar to cloud resolving model through nesting technique. Recently, a multi-scale modeling system with unified physics was developed at NASA Goddard. It consists of (l) a cloud-resolving model (Goddard Cumulus Ensemble model, GCE model), (2) a regional scale model (a NASA unified weather research and forecast, WRF), (3) a coupled CRM and global model (Goddard Multi-scale Modeling Framework, MMF), and (4) a land modeling system. The same microphysical processes, long and short wave radiative transfer and land processes and the explicit cloud-radiation, and cloud-surface interactive processes are applied in this multi-scale modeling system. This modeling system has been coupled with a multi-satellite simulator to use NASA high-resolution satellite data to identify the strengths and weaknesses of cloud and precipitation processes simulated by the model. In this talk, the microphysics developments of the multi-scale modeling system will be presented. In particular, the results from using multi-scale modeling system to study the heavy precipitation processes will be presented.

Tao, Wei-Kuo↗

Advances in Grid Computing for the FabrIc for Frontier Experiments Project at Fermialb

The FabrIc for Frontier Experiments (FIFE) project is a major initiative within the Fermilab Scientic Computing Division charged with leading the computing model for Fermilab experiments. Work within the FIFE project creates close collaboration between experimenters and computing professionals to serve high-energy physics experiments of diering size, scope, and physics area. The FIFE project has worked to develop common tools for job submission, certicate management, software and reference data distribution through CVMFS repositories, robust data transfer, job monitoring, and databases for project tracking. Since the projects inception the experiments under the FIFE umbrella have signicantly matured, and present an increasingly complex list of requirements to service providers. To meet these requirements, the FIFE project has been involved in transitioning the Fermilab General Purpose Grid cluster to support a partitionable slot model, expanding the resources available to experiments via the Open Science Grid, assisting with commissioning dedicated high-throughput computing resources for individual experiments, supporting the eorts of the HEP Cloud projects to provision a variety of back end resources, including public clouds and high performance computers, and developing rapid onboarding procedures for new experiments and collaborations. The larger demands also require enhanced job monitoring tools, which the project has developed using such tools as ElasticSearch and Grafana. in helping experiments manage their large-scale production work ows. This group in turn requires a structured service to facilitate smooth management of experiment requests, which FIFE provides in the form of the Production Operations Management Service (POMS). POMS is designed to track and manage requests from the FIFE experiments to run particular work ows, and support troubleshooting and triage in case of problems. Recently a new certicate management infrastructure called Distributed Computing Access with Federated Identities (DCAFI) has been put in place that has eliminated our dependence on a Fermilab-specic third-party Certicate Authority service and better accommodates FIFE collaborators without a Fermilab Kerberos account. DCAFI integrates the existing InCommon federated identity infrastructure, CILogon Basic CA, and a MyProxy service using a new general purpose open source tool. We will discuss the general FIFE onboarding strategy, progress in expanding FIFE experiments presence on the Open Science Grid, new tools for job monitoring, the POMS service, and the DCAFI project.

Herner, K.↗

Radiometric Assessment of the First Three Years of the NOAA-20 VIIRS Reflective Solar Bands Calibration

NASA’s Clouds and the Earth’s Radiant Energy System (CERES) SYN1deg Ed4.1 product utilizes geostationary (GEO) satellite measured radiances and retrieved cloud properties to account for the regional diurnal fluctuations in the Earth’s radiant broadband fluxes for times between the CERES measurements gathered from the Aqua (1:30 PM) and Terra (10:30 AM) sun-synchronous satellites. In CERES Edition 4 products, a global uniformity in the cloud properties and computed surface fluxes across the GEO satellite domains is maintained by scaling the radiance observations from more than twenty GEO visible imagers in the CERES record to a common radiometric reference scale, i.e., Aqua-MODIS. With the new-generation GEO (Himawari-8/9 and GOES-16/17) imagers having multiple reflective solar bands (RSB) that are spectrally similar to those of VIIRS, the CERES Imager and Geostationary Calibration Group (IGCG) is preparing to use NOAA-20 VIIRS as the reference imager for maintaining the radiometric uniformity across the GEO imager constellation. Given the recent Aqua satellite anomaly, this transition may happen sooner than the projected date of Aqua’s de-orbit. This paper presents an independent performance evaluation of the first three years of the NOAA-20 VIIRS RSB calibration in the NASA VIIRS Land Science Investigator-led Processing System (Land SIPS) L1b Collection 2 dataset. The temporal radiometric stability is assessed using multiple invariant Earth targets, including tropical deep convective clouds and the Saharan desert. The invariant target anisotropic reflectance at the top of atmosphere was modeled using five years of stable satellite observations acquired from the previous VIIRS instrument onboard the Suomi National Polar-orbiting Partnership (S-NPP) satellite. The anisotropic corrections are essential for detecting temporal trends with a high statistical confidence. In addition, the radiometric consistency between the RSB of the two VIIRS instruments will be evaluated.

Rajendra Bhatt↗

Radiometric Assessment of the First Three Years of the NOAA-20 VIIRS Reflective Solar Bands Calibration

NASA’s Clouds and the Earth’s Radiant Energy System (CERES) SYN1deg Ed4.1 product utilizes geostationary (GEO) satellite measured radiances and retrieved cloud properties to account for the regional diurnal fluctuations in the Earth’s radiant broadband fluxes for times between the CERES measurements gathered from the Aqua (1:30 PM) and Terra (10:30 AM) sun-synchronous satellites. In CERES Edition 4 products, a global uniformity in the cloud properties and computed surface fluxes across the GEO satellite domains is maintained by scaling the radiance observations from more than twenty GEO visible imagers in the CERES record to a common radiometric reference scale, i.e., Aqua-MODIS. With the new-generation GEO (Himawari-8/9 and GOES-16/17) imagers having multiple reflective solar bands (RSB) that are spectrally similar to those of VIIRS, the CERES Imager and Geostationary Calibration Group (IGCG) is preparing to use NOAA-20 VIIRS as the reference imager for maintaining the radiometric uniformity across the GEO imager constellation. Given the recent Aqua satellite anomaly, this transition may happen sooner than the projected date of Aqua’s de-orbit. This paper presents an independent performance evaluation of the first three years of the NOAA-20 VIIRS RSB calibration in the NASA VIIRS Land Science Investigator-led Processing System (Land SIPS) L1b Collection 2 dataset. The temporal radiometric stability is assessed using multiple invariant Earth targets, including tropical deep convective clouds and the Saharan desert. The invariant target anisotropic reflectance at the top of the atmosphere was modeled using five years of stable satellite observations acquired from the previous VIIRS instrument onboard the Suomi National Polar-orbiting Partnership (S-NPP) satellite. The anisotropic corrections are essential for detecting temporal trends with high statistical confidence. In addition, the radiometric consistency between the RSB of the two VIIRS instruments will be evaluated.

Rajendra Bhatt↗

Report on the Second ARM Mobile Facility (AMF2) Roll, Pitch, and Heave (RPH) Stabilization Platform: Design and Evaluation

One of the primary objectives of the U.S. Department of Energy’s Atmospheric Radiation Measurement (ARM) Climate Research Facility’s second Mobile Facility (AMF2) is to obtain reliable measurements of solar, surface, and atmospheric radiation, as well as cloud and atmospheric properties, from ocean-going vessels. To ensure that these climatic measurements are representative and accurate, many AMF2 instrument systems are designed to collect data in a zenith orientation. A pillar of the AMF2 strategy in this effort is the use of a stable platform. The purpose of the platform is to 1) mitigate vessel motion for instruments that require a truly vertical orientation and keep them pointed in the zenith direction, and 2) allow for accurate positioning for viewing or shading of the sensors from direct sunlight. Numerous ARM instruments fall into these categories, but perhaps the most important are the vertically pointing cloud radars, for which vertical motions are a critical parameter. During the design and construction phase of AMF2, an inexpensive stable platform was purchased to perform the stabilization tasks for some of these instruments. The first table compensated for roll, pitch, and yaw (RPY) and was reported upon in a previous technical report (Kafle and Coulter, 2012). Subsequently, a second table was purchased specifically for operation with the Marine W-band cloud radar (MWACR). Computer programs originally developed for RPY were modified to communicate with the new platform controller and with an inertial measurements platform that measures true ship motion components (roll, pitch, yaw, surge, sway, and heave). This platform could not be tested dynamically for RPY because of time constraints requiring its deployment aboard the container ship Horizon Spirit in September 2013. Hence the initial motion tests were conducted on the initial cruise. Subsequent cruises provided additional test results. The platform, as tested, meets all the design and performance criteria established for its use. This is a report of the results of those efforts and the critical points in moving forward

54 ENVIRONMENTAL SCIENCES↗

Use of TRMM Data to Test an Improved Parameterization of Stratiform Precipitation

We successfully implemented our EAUcup parameterization of convection in our most recent geodesic-grid version of the Colorado State University General Circulation Model (CSU GCM). Tendencies due to convection and large-scale cloud microphysics are computed using the same time-step, allowing smoother interactions between convective and large-scale moist processes. Figures illustrate our most recent progress on the simulation of convective and stratiform precipitations for January. Comparisons against TRMM precipitation radar (PR) data would reveal a systematic overestimation of the simulated monthly-mean convective rainfall over well-known areas of deep convection. One major deficiency in the global distribution of stratiform precipitation simulated in the CSU GCM is the lack of stratiform precipitation over land in the tropics. This result strongly disagree with Fowler and Randall who show that convective and stratiform precipitations always coexist and that their ratio to the total precipitation is about 0.5. We believe that stratiform precipitation over land evaporates before reaching the ground because the lower troposphere remains too dry in response to the lack of convective detrainment occurring immediately above the Planetary Boundary Layer (PBL). This issue is further investigated at present. We made significant progress in the development of our parameterization of fractional cloudiness. As an example of our ongoing research effort, a figure shows the global distribution of the cloud ice mixing ratio and corresponding fractional area of ice clouds at 250 hPa. These results were obtained using our latitude-longitude grid version of the CSU GCM. There is a nice correspondence between areas of high (low) values of the cloud ice mixing ratio and the cloud fraction. A nice correspondence between the global distribution of the cloud water mixing ratio and fraction of water clouds in the lower troposphere can also be found. We are pursuing this effort, but in the geodesic version of the GCM. We hope to be able to make comparison of the simulated precipitation against TRMM precipitation data in the near future.

Randall, David A.↗

Cloud identification using genetic algorithms and massively parallel computation

As a Guest Computational Investigator under the NASA administered component of the High Performance Computing and Communication Program, we implemented a massively parallel genetic algorithm on the MasPar SIMD computer. Experiments were conducted using Earth Science data in the domains of meteorology and oceanography. Results obtained in these domains are competitive with, and in most cases better than, similar problems solved using other methods. In the meteorological domain, we chose to identify clouds using AVHRR spectral data. Four cloud speciations were used although most researchers settle for three. Results were remarkedly consistent across all tests (91% accuracy). Refinements of this method may lead to more timely and complete information for Global Circulation Models (GCMS) that are prevalent in weather forecasting and global environment studies. In the oceanographic domain, we chose to identify ocean currents from a spectrometer having similar characteristics to AVHRR. Here the results were mixed (60% to 80% accuracy). Given that one is willing to run the experiment several times (say 10), then it is acceptable to claim the higher accuracy rating. This problem has never been successfully automated. Therefore, these results are encouraging even though less impressive than the cloud experiment. Successful conclusion of an automated ocean current detection system would impact coastal fishing, naval tactics, and the study of micro-climates. Finally we contributed to the basic knowledge of GA (genetic algorithm) behavior in parallel environments. We developed better knowledge of the use of subpopulations in the context of shared breeding pools and the migration of individuals. Rigorous experiments were conducted based on quantifiable performance criteria. While much of the work confirmed current wisdom, for the first time we were able to submit conclusive evidence. The software developed under this grant was placed in the public domain. An extensive user's manual was written and distributed nationwide to scientists whose work might benefit from its availability. Several papers, including two journal articles, were produced.

Buckles, Bill P.↗

Improving Cloud Optical Property Retrievals for Partly Cloudy Pixels Using Coincident Higher-Resolution Single Band Measurements: A Feasibility Study Using ASTER Observations

Clear-sky contamination is a challenging and long-lasting problem for cloud optical thickness (τ) and effective droplet radius (r eff ) retrievals using passive satellite sensors. This study explores the feasibility of improving both τ and r eff retrievals for partly cloudy (PCL) pixels by using available subpixel samples in a visible to near-infrared band, which many satellite sensors offer. Data are provided by high-resolution reflectance (R ) observations and cloud property retrievals by the Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) at horizontal resolutions between 30-960m. For partly cloudy 960-m observations, the clear-sky component of the pixels induces significant underestimations of up to 58% for τ, while overestimations in r eff can exceed 41%. This yields underestimations in the derived liquid water path and cloud droplet number concentration of up to 68% and 72%, respectively. By means of three different assumptions it is shown that subpixel R observations in the visible to near-infrared band can be used to estimate higher-resolution R for the second band in the retrieval scheme, as well as the subpixel cloud cover. Here, the estimated values compare well to actually observed ASTER results and are used to retrieve cloud properties, which are unbiased by the clear-sky component of PCL pixels. While the presented retrieval approach is only evaluated for marine boundary layer clouds, it is computationally efficient and can be easily applied to observations from different imagers. As an example, the PCL retrieval scheme is applied to data by the Moderate Resolution Imaging Spectroradiometer, where similar biases for PCL pixels are observed.

54 ENVIRONMENTAL SCIENCES↗

An open source analysis framework for large-scale building energy modeling

Full integration of building energy modelling into the design and retrofit process has long been a goal of building scientists and practitioners. However, significant barriers still exist. Among them are the lack of available: (1) configurable technology stacks for performing both small- and large-scale analyses, (2) different classes of algorithms compatible with common design workflows, and (3) analysis tools for effectively visualizing large-scale simulation results. This article discusses the OpenStudio® Analysis Framework: a scalable analysis framework for building energy modelling that was developed to overcome the three barriers listed above. The framework is open-source and scalable to facilitate wider adoption and has a clearly defined application programming interface upon which other applications can be built. It runs on high-performance computing systems, within cloud infrastructure, and on laptops, and uses a common workflow to enable different classes of algorithms. Lessons learned from previous development efforts are also discussed.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Leveraging the checkpoint-restart technique for optimizing CPU efficiency of ATLAS production applications on opportunistic platforms

Data processing applications of the ATLAS experiment, such as event simulation and reconstruction, spend considerable amount of time in the initialization phase. This phase includes loading a large number of shared libraries, reading detector geometry and condition data from external databases, building a transient representation of the detector geometry and initializing various algorithms and services. In some cases the initialization step can take as long as 10-15 minutes. Such slow initialization has a significant negative impact on overall CPU efficiency of the production job, especially when the job is executed on opportunistic, often short-lived, resources such as commercial clouds or volunteer computing. In order to improve this situation, we can take advantage of the fact that ATLAS runs large numbers of production jobs with similar configuration parameters (e.g. jobs within the same production task). This allows us to checkpoint one job at the end of its configuration step and then use the generated checkpoint image for rapid startup of thousands of production jobs. By applying this technique we can bring the initialization time of a job from tens of minutes down to just a few seconds. In addition to that we can leverage container technology for restarting checkpointed applications on the variety of computing platforms, in particular of platforms different from the one on which the checkpoint image was created. We will describe the mechanism of creating checkpoint images of Geant4 simulation jobs with AthenaMP (the multi-process version of the ATLAS data simulation, reconstruction and analysis framework Athena) and the usage of these images for running ATLAS Simulation production jobs on volunteer computing resources (ATLAS@Home) and on Supercomputers.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Anatomy of relativistic pion loop corrections to the electromagnetic nucleon coupling

We present a relativistic formulation of pion loop corrections to the coupling of photons with nucleons on the light-front. Vertex and wave function renormalization constants are computed to lowest order in the pion field, including their nonanalytic behavior in the chiral limit, and studied numerically as a function of the ultraviolet cutoff. Particular care is taken to explicitly verify gauge invariance and Ward-Takahashi identity constraints to all orders in the m_\pi expansion. The results are used to compute the chiral corrections to matrix elements of local operators, related to moments of deep-inelastic structure functions. Finally, comparisons of results for pseudovector and pseudoscalar coupling allows the resolution of a long-standing puzzle in the computation of pion cloud corrections to structure function moments.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Benchmarking Information Scrambling

Information scrambling refers to the rapid spreading of initially localized information over an entire system, via the generation of global entanglement. This effect is usually detected by measuring a temporal decay of the out-of-time order correlators. However, in experiments, decays of these correlators suffer from fake positive signals from various sources, e.g., decoherence due to inevitable couplings to the environment, or errors that cause mismatches between the purported forward and backward evolutions. Here, in this Letter, we provide a simple and robust approach to single out the effect of genuine scrambling. This allows us to benchmark the scrambling process by quantifying the degree of the scrambling from the noisy backgrounds. We also demonstrate our protocol with simulations on IBM cloud-based quantum computers.

97 MATHEMATICS AND COMPUTING↗

Impact of Data Placement on Resilience in Large-Scale Object Storage Systems

Distributed object storage architectures have become the de facto standard for high-performance storage in big data, cloud, and HPC computing. Object storage deployments using commodity hardware to reduce costs often employ object replication as a method to achieve data resilience. Repairing object replicas after failure is a daunting task for systems with thousands of servers and billions of objects, however, and it is increasingly difficult to evaluate such scenarios at scale on realworld systems. Resilience and availability are both compromised if objects are not repaired in a timely manner. In this work we leverage a high-fidelity discrete-event simulation model to investigate replica reconstruction on large-scale object storage systems with thousands of servers, billions of objects, and petabytes of data. We evaluate the behavior of CRUSH, a well-known object placement algorithm, and identify configuration scenarios in which aggregate rebuild performance is constrained by object placement policies. After determining the root cause of this bottleneck, we then propose enhancements to CRUSH and the usage policies atop it to enable scalable replica reconstruction. We use these methods to demonstrate a simulated aggregate rebuild rate of 410 GiB/s (within 5% of projected ideal linear scaling) on a 1,024-node commodity storage system. We also uncover an unexpected phenomenon in rebuild performance based on the characteristics of the data stored on the system.

Carns, Philip↗

Development and Validation of Southeast Asia Solar Resource Data [Slides]

Lack of access to high-quality, publicly available, time series solar data to inform decisions that will transform energy sectors in Southeast Asia is a challenge. The solution is to level the playing field by offering free, high-quality, robust solar data to inform private sector investment and policymaking. This is done by (1) leveraging deep NREL expertise in atmospheric science, solar resource assessment, high-performance computing, and cloud-based data dissemination, (2) producing and validating high spatial and temporal resolution solar resource data, (3) making data available on the USAID-funded global Renewable Energy Data Explorer platform, (4) providing capacity building for data and applications, and (5) informing future demand-driven tool development.

14 SOLAR ENERGY↗