Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Cloud computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

TeraChem Cloud: A High-Performance Computing Service for Scalable Distributed GPU-Accelerated Electronic Structure Calculations

The encapsulation and commoditization of electronic structure arise naturally as interoperability, and the use of nontraditional compute resources (e.g., new hardware accelerators, cloud computing) remains important for the computational chemistry community. Here, we present TERACHEM CLOUD, a high-performance computing service (HPCS) that offers on-demand electronic structure calculations on both traditional HPC clusters and cloud-based hardware. The framework is designed using off-the-shelf web technologies and containerization to be extremely scalable and portable. Within the HPCS model, users can quickly develop new methods and algorithms in an interactive environment on their laptop while allowing TERACHEM CLOUD to distribute ab initio calculations across all available resources. This approach greatly increases the accessibility of hardware accelerators such as graphics processing units (GPUs) and flexibility for the development of new methods as additional electronic structure packages are integrated into the framework as alternative backends. Cost-performance analysis indicates that traditional nodes are the most cost-effective long-term solution, but commercial cloud providers offer cutting-edge hardware with competitive rates for short-term large-scale calculations. We demonstrate the power of the TERACHEM CLOUD framework by carrying out several showcase calculations, including the generation of 300,000 density functional theory energy and gradient evaluations on medium-sized organic molecules and reproducing 300 fs of nonadiabatic dynamics on the B800-B850 antenna complex in LH2, with the latter demonstration using over 50 Tesla V100 GPUs in a commercial cloud environment in 8 h for approximately $1250.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

MemXCT: Design, Optimization, Scaling, and Reproducibility of X-Ray Tomography Imaging

Here, this work extends our previous research entitled "MemXCT: Memory-centric X-ray CT Reconstruction with Massive Parallelization" that was originally published at SC19 conference (Hidayetoglu et al., 2019) with reproducibility of the computational imaging performance. X-ray computed tomography (XCT) is regularly used at synchrotron light sources to study the internal morphology of materials at high resolution. However, experimental constraints, such as radiation sensitivity, can result in noisy or undersampled measurements. Further, depending on the resolution, sample size and data acquisition rates, the resulting noisy dataset can be in the order of terabytes. Advanced iterative reconstruction techniques can produce high-quality images from noisy measurements, but their computational requirements have made their use an exception rather than the rule. We propose a novel memory-centric approach that avoids redundant computations at the expense of additional memory complexity. We develop a memory-centric iterative reconstruction system, MemXCT, that uses an optimized SpMV implementation with two-level pseudo-Hilbert ordering and multi-stage input buffering. We evaluate MemXCT on various supercomputer architectures involving KNL and GPU. MemXCT can reconstruct a large (11Kx11K) mouse brain tomogram in 10 seconds using 4096 KNL nodes (256K cores). The results presented in our original article at the SC19 were based on large-scale supercomputing resources. The MemXCT application was selected for the Student Cluster Competition (SCC) Reproducibility Challenge and evaluated on a variety of cloud computing resources by universities around the world in the SC20 conference. We summarize the results of the top-ranked SCC Reproducibility Challenge teams and identify the most pertinent measures for ensuring the reproducibility of our experiments in this article.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

Machine learning in nuclear materials research

Nuclear materials are often demanded to function for extended time in extreme environments, including high radiation fluxes with associated transmutations, high temperature and temperature gradients, mechanical stresses, and corrosive coolants. They also have a wide range of microstructural and chemical makeups, resulting in multifaceted and often out-of-equilibrium interactions. Machine learning (ML) is increasingly being used to tackle these complex time-dependent interactions and aid researchers in developing models and making predictions, sometimes with better accuracy than traditional modeling that focuses on one or two parameters at a time. Conventional practices of acquiring new experimental data in nuclear materials research are often slow and expensive, limiting the opportunity for data-centric ML, but new methods are changing that paradigm. Here we review high-throughput computational and experimental data approaches, especially robotic experimentation and active learning that is based on Gaussian process and Bayesian optimization. We show ML examples in structural materials (e.g., reactor pressure vessel (RPV) alloys and radiation detecting scintillating materials) and highlight new techniques of high-throughput sample preparation and characterizations, and automated radiation/environmental exposures and real-time online diagnostics. Herein, this review suggests that ML models of material constitutive relations in plasticity, damage, and even electronic and optical responses to radiation are likely to become powerful tools as they develop. Finally, we speculate on how the recent trends of using natural language processing (NLP) to aid the collection and analysis of literature data, interpretable artificial intelligence (AI), and the use of streamlined scripting, database, workflow management, and cloud computing platforms that will soon make the utilization of ML techniques as commonplace as the spreadsheet curve-fitting practices of today.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Measuring success for a future vision: Defining impact in science gateways/virtual research environments

Scholars worldwide leverage science gateways/virtual research environments (VREs) for a wide variety of research and education endeavors spanning diverse scientific fields. Evaluating the value of a given science gateway/VRE to its constituent community is critical in obtaining the financial and human resources necessary to sustain operations and increase adoption in the user community. In this article, we feature a variety of exemplar science gateways/VREs and detail how they define impact in terms of, for example, their purpose, operation principles, and size of user base. Further, the exemplars recognize that their science gateways/VREs will continuously evolve with technological advancements and standards in cloud computing platforms, web service architectures, data management tools and cybersecurity. We also present a number of technology advances that could be incorporated in next-generation science gateways/VREs to enhance their scope and scale of their operations for greater success/impact. The exemplars are selected from owners of science gateways in the Science Gateways Community Institute (SGCI) clientele in the United States, and from the owners of VREs in the International Virtual Research Environment Interest Group (VRE-IG) of the Research Data Alliance. Thus, community-driven best practices and technology advances are compiled from diverse expert groups with an international perspective to envisage futuristic science gateway/VRE innovations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Evaluating integration and performance of containerized climate applications on a Hewlett Packard Enterprise Cray system

Containers have taken over large swaths of cloud computing as the most convenient way of packaging and deploying applications. The features that containers offer for packaging and deploying applications translate to high performance computing (HPC) as well. At The National Oceanic and Atmospheric Administration, containers provide an easy way to build and distribute complex HPC applications, allowing faster collaboration, portability, and experiment computer environment reproducibility amongst the scientific community. The challenge arises when applications rely on message passing interface (MPI). This necessitates investigation into how to properly run these applications with their own unique requirements and produce performance on par with native runs. We investigate the MPI performance for benchmarks and containerized climate models for various containers covering selection of compiler and MPI library combinations from the Cray provided programming environments on the Cray XC supercomputer GAEA. Performance from the benchmarks and the climate models shows that for the most part containerized applications perform on par with the natively built applications when the system optimized Cray MPICH libraries are bound into the container, and the hybrid model containers have poor performance in comparison. We also describe several challenges and our solutions in running these containers, particularly challenges with heterogeneous jobs for the containerized model runs.

Abraham, Subil↗
DOE OSTI2024

MatRIS: Addressing the Challenges for Portability and Heterogeneity Using Tasking for Matrix Decomposition (Cholesky)

The ubiquitous in-node heterogeneity of HPC and cloud computing platforms makes software portability and performance optimization extremely challenging. Described here, the MatRIS multilevel math library abstraction framework employs tasking to alleviate these difficulties. MatRIS includes the IRIS task-based runtime on the bottom level and exposes different layers of abstraction to render algorithms architecturally agnostic. MatRIS ensures the decomposition and creation of tasks that represent the necessary encapsulation of the optimized kernels from both vendor and open-source math libraries. Once built, MatRIS can select different combinations of accelerators at runtime, making it portable even on diverse heterogeneous architectures. By leveraging the IRIS runtime’s features for managing heterogeneity, MatRIS deploys algorithms that remove the need to specify orchestration and data transfer. This study describes how the serial task abstraction of a tiled Cholesky factorization is made portable and scalable in the case of multi-device and multi-vendor heterogeneity on a node with NVIDIA and AMD GPUs by using MatRIS. First, we demonstrate that Cholesky in MatRIS provides multi-GPU scalability that offers competitive performance versus cuSolverMG. Then, we present the challenges and opportunities for heterogeneous execution.

Monil, M. A. H.↗
DOE OSTI2021

Land cover change-induced decline in terrestrial gross primary production over the conterminous United States from 2001 to 2016

As one of the most dynamic aspects of global environmental change, land cover change (LCC) has a profound impact on terrestrial carbon sequestration. However, LCC-induced carbon fluxes are still the most uncertain terms in global and regional carbon budgets. Ecosystem gross primary production (GPP) is the total carbon uptake by vegetation through photosynthesis, serving as a major control on ecosystem function and land carbon balance during and after the modification of the land surface. However, accurately capturing LCC-induced GPP changes requires both high-quality land cover data and controlling for variation driven by other environmental factors such as climate. In this study, we comprehensively examined the effects of LCC on annual GPP trends over the conterminous United States (CONUS) from 2001 to 2016 using the USGS National Land Cover Database, a remote sensing-driven ecosystem model, and the Google Earth Engine cloud computing platform. We designed a series of model experiments to identify LCC effects on GPP by controlling climate effects. During the study period, LCC exerted a strong negative effect on total GPP across the CONUS ([-2.2, -1.8] Tg C yr -2 ), while climate had smaller positive effects ([0.17, 0. 92] Tg C yr -2 ). The LCC-induced reduction of GPP was mainly caused by net forest loss ([-1.98, -1.39] Tg C yr -2 ) and urban expansion ([-2.03, -1.92] Tg C yr -2 ), but was partially offset by increases in crop area ([+0.66, +0.79] Tg C yr -2 ). Ensemble simulations from TRENDY did not capture the strong negative LCC influences on GPP, likely due to limitations of the adopted land use/cover data. Overall, our study provides a novel perspective on LCC-induced GPP changes, which could help to improve our understanding of ecosystem function changes and constrain the estimation of land carbon balance in the context of anthropogenic activity and climate change.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

A Fine-grained Asynchronous Bulk Synchronous parallelism model for PGAS applications

The Partitioned Global Address Space (PGAS) model is well suited for executing irregular applications on cluster-based systems, due to its efficient support for short, one-sided messages. Separately, the actor model has been gaining popularity as a productive asynchronous message-passing approach for distributed objects in enterprise and cloud computing platforms, typically implemented in languages such as Erlang, Scala or Rust. To the best of our knowledge, there has been no past work on using the actor model to deliver both productivity and scalability to irregular PGAS applications with large number of small messages. In this paper, we introduce a new programming system for PGAS applications, in which point-to-point remote operations can be expressed as fine-grained asynchronous actor messages. In our approach, the programmer does not need to worry about programming complexities related to message aggregation and termination detection. Our approach can be viewed as extending the classical Bulk Synchronous Parallelism model with fine-grained asynchronous communications within a phase or superstep. Here, we believe that our approach offers a desirable point in the productivity-performance space for PGAS applications, with more scalable performance and higher productivity relative to past approaches. Specifically, for seven irregular mini-applications from the Bale Kernels and three graph kernels executed using 2048 cores in the NERSC Cori system, our approach shows geometric mean performance improvements of ≥ 20X relative to standard PGAS versions (UPC and OpenSHMEM) while maintaining comparable productivity to those versions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

A new framework to map fine resolution cropping intensity across the globe: Algorithm, validation, and implication

We report accurate estimation of cropping intensity (CI), an indicator of food production, is well aligned with the ongoing efforts to achieve sustainable development goals (SDGs) under diminishing natural resources. The advancement in satellite remote sensing provides unprecedented opportunities for capturing CI information in a spatially continuous manner. However, challenges remain due to the lack of generalizable algorithms for accurately and efficiently mapping global CI with a fine spatial resolution. In this study, we developed a 30-m planetary-scale CI mapping framework with the reconstructed time series of Normalized Difference Vegetation Index (NDVI) from multiple satellite images. Using a binary crop phenophase profile indicating growing and non-growing periods, we estimated pixel-by-pixel CI by enumerating the total number of valid cropping cycles during the study years. Based on the Google Earth Engine cloud computing platform, we implemented the framework to estimate CI during 2016–2018 in eight geographic regions across continents that are representative of global cropping system diversity. Comparison with PhenoCam network data in four cropland sites suggests that the proposed framework is capable of capturing the seasonal dynamics of cropping practices. Spatially, overall accuracies based on validation samples range from 80.0% to 98.9% across different regions worldwide. Regarding the CI classes, single cropping systems are associated with more robust and less biased estimations than multiple cropping systems. Finally, our CI estimates reveal high agreement with two widely used land surface phenology products, including Vegetation Index and Phenology V004 (VIP4) and Moderate Resolution Imaging Spectroradiometer Land Cover Dynamics (MCD12Q2), meanwhile providing much more spatial details. Due to its robustness, the developed CI framework can be potentially generalized to produce global fine resolution CI products for food security and other applications.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Analysis of hydrogen infrastructure for the feasibility, economics, and sustainability of a fuel cell powered data center

Data centers used for internet data services, cloud computing, and/or data storage consume vast amounts of electricity and are increasing rapidly in capacity. Consequently, their power consumption has raised concerns about energy sustainability and environmental impacts. Large-scale, on-site renewable energy could help reduce data centers’ carbon footprint; however, wind and solar power alone cannot provide an uninterrupted power supply to computer servers due to their natural variability. Instead, reliable power integration can be achieved by using fuel cells powered by hydrogen from sustainable resources (e.g., wind and solar energy). Establishing a hydrogen infrastructure will be critical for realizing these benefits and establishing fuel cells as a viable power source for data centers. Here, to facilitate the development of novel carbon-free fuel cell data enters, this paper presents renewable power integrated with hydrogen infrastructures in four scenarios to provide reliable hydrogen supply from production to storage. Various paths were analyzed toward a hydrogen supply infrastructure by determining the proper component sizes and calculating the cost of meeting the server load. We used a microgrid modeling software, Hybrid Optimization of Multiple Energy Resources (HOMER), and studied the feasibility of fuel cell powered data centers employing renewable hydrogen. The modeling results show various renewable integration configurations to meet reliable and sustainable power requirement under four scenarios for a carbon-free data center.

08 HYDROGEN↗
DOE OSTI2023

Stakeholder analysis for designing an urban air quality data governance ecosystem in smart cities

Cities, the world over, are fuelling economic growth. At the same time, rapid urbanization is a root cause of serious environmental damage. Recent WHO global air pollution guidelines highlight air pollution as a critical environmental threat along with climate change. To address these threats, smart cities and clean air programs are on a rise. In smart cities, data and Information and Communication Technologies (ICT) are major drivers of city transformations. The 4th Industrial Revolution (4IR) technologies such as the Internet of Things (IoT), big data, artificial intelligence (AI), and cloud computing have the potential to accelerate these transformations toward urban resilience. However, the success of smart cities and clean air programs depends on cohesive multi-sector stakeholder contributions. This study conducted interdisciplinary participative stakeholder analysis to understand the data, and sectorial challenges, to outline the technological opportunities to facilitate clean air programs in Indian smart cities. The research highlights gaps due to siloed stakeholder operations, lack of data calibration, non-alignment of smart city and air quality management services, non-availability of health exposure data, and difficulty in translating scientific data into implementable actions. Stakeholders expressed potential ‘fit for the purpose’ use of IoT devices, satellites, smartphones, and mobility data augmented by AI methods in bridging these gaps. In conclusion, the analysis points toward a need to develop an easily accessible and ubiquitous urban data governance ecosystem enabling seamless cross-sector data exchanges to build trusting relationships among the stakeholders across the air quality management value chain.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

mzapy : An Open-Source Python Library Enabling Efficient Extraction and Processing of Ion Mobility Spectrometry-Mass Spectrometry Data in the MZA File Format

We have recently reported MZA, a new and simple mass spectrometry data structure based on the broadly supported HDF5 format and created to facilitate software development. While this format is inherently supportive of application development, the availability of a core library with standard mass spectrometry utilities greatly facilitates fast software development. Here, we present a Python library, mzapy, for efficient extraction and processing of mass spectrometry data in the MZA format. In addition to raw data extraction, mzapy contains supporting utilities enabling tasks including calibration, signal processing, peak finding, and generating plots. Being implemented in pure Python with minimal and largely standardized dependencies makes mzapy uniquely suited to application development in the multi-omics domain. The free and open source mzapy is built with extensibility in mind, and future development will support cloud computing and artificial intelligence/machine learning applications. The software source code is freely available at https://github.com/PNNL-m-q/mzapy.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

A scalable, open-source implementation of a large-scale mechanistic model for single cell proliferation and death signaling

Mechanistic models of how single cells respond to different perturbations can help integrate disparate big data sets or predict response to varied drug combinations. However, the construction and simulation of such models have proved challenging. Here, we developed a python-based model creation and simulation pipeline that converts a few structured text files into an SBML standard and is high-performance- and cloud-computing ready. We applied this pipeline to our large-scale, mechanistic pan-cancer signaling model (named SPARCED) and demonstrate it by adding an IFNγ pathway submodel. We then investigated whether a putative crosstalk mechanism could be consistent with experimental observations from the LINCS MCF10A Data Cube that IFNγ acts as an anti-proliferative factor. The analyses suggested this observation can be explained by IFNγ-induced SOCS1 sequestering activated EGF receptors. This work forms a foundational recipe for increased mechanistic model-based data integration on a single-cell level, an important building block for clinically-predictive mechanistic models.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Editorial: Themed issue on data-intensive water systems management and operation

The water sector increasingly looks at intensified instrumentation, data collection and automation as tools for daily use. Still, a massive opportunity remains to fully embrace emerging methods and technologies, such as artificial intelligence, data analytics, low-cost sensor hardware, and cloud computing. Indeed, sensing and automation technology has already infiltrated many facets of society. As such, the time is ripe to evaluate the role of novel technologies for systems monitoring, diagnostics, and automation of aquatic processes and large-scale water systems. When leveraged, the water sector will do more with less.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Using the ATLAS experiment software on heterogeneous resources

With the large dataset expected from 2030 onwards by the HL-LHC at CERN, the ATLAS experiment is reaching the limits of the current data processing model in terms of traditional CPU resources based on x86_64 architectures and an extensive program for software upgrades towards the HL-LHC has been set up. The ARM CPU architecture is becoming a competitive and energy efficient alternative. Accelerators like GPUs are available in any recent HPC. In the past years ATLAS has successfully ported its full data processing and simulation software framework Athena to ARM and has invested significant effort in porting parts of the reconstruction and simulation algorithms to GPUs. We report on the successful usage of the ATLAS experiment offline and online software framework Athena on ARM and GPUs through the PanDA workflow management system at various WLCG sites. Furthermore we report on performance optimizations of the builds for ARM CPUs and the GPU integration efforts. We will discuss performance comparisons of different ARM and x86_64 architectures on WLCG resources and Cloud compute providers like GCP and AWS using ATLAS productions workflows as used in the Hep-Score23 benchmark suite.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

VIBES: a workflow for annotating and visualizing viral sequences integrated into bacterial genomes

Abstract Bacteriophages are viruses that infect bacteria. Many bacteriophages integrate their genomes into the bacterial chromosome and become prophages. Prophages may substantially burden or benefit host bacteria fitness, acting in some cases as parasites and in others as mutualists. Some prophages have been demonstrated to increase host virulence. The increasing ease of bacterial genome sequencing provides an opportunity to deeply explore prophage prevalence and insertion sites. Here we present VIBES (Viral Integrations in Bacterial genomES), a workflow intended to automate prophage annotation in complete bacterial genome sequences. VIBES provides additional context to prophage annotations by annotating bacterial genes and viral proteins in user-provided bacterial and viral genomes. The VIBES pipeline is implemented as a Nextflow-driven workflow, providing a simple, unified interface for execution on local, cluster and cloud computing environments. For each step of the pipeline, a container including all necessary software dependencies is provided. VIBES produces results in simple tab-separated format and generates intuitive and interactive visualizations for data exploration. Despite VIBES’s primary emphasis on prophage annotation, its generic alignment-based design allows it to be deployed as a general-purpose sequence similarity search manager. We demonstrate the utility of the VIBES prophage annotation workflow by searching for 178 Pf phage genomes across 1072 Pseudomonas spp. genomes.

59 BASIC BIOLOGICAL SCIENCES↗