Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Cloud computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14

Nanosystems, Edge Computing, and the Next Generation Computing Systems

It is widely recognized that nanoscience and nanotechnology and their subfields, such as nanophotonics, nanoelectronics, and nanomechanics, have had a tremendous impact on recent advances in sensing, imaging, and communication, with notable developments, including novel transistors and processor architectures. For example, in addition to being supremely fast, optical and photonic components and devices are capable of operating across multiple orders of magnitude length, power, and spectral scales, encompassing the range from macroscopic device sizes and kW energies to atomic domains and single-photon energies. The extreme versatility of the associated electromagnetic phenomena and applications, both classical and quantum, are therefore highly appealing to the rapidly evolving computing and communication realms, where innovations in both hardware and software are necessary to meet the growing speed and memory requirements. Development of all-optical components, photonic chips, interconnects, and processors will bring the speed of light, photon coherence properties, field confinement and enhancement, information-carrying capacity, and the broad spectrum of light into the high-performance computing, the internet of things, and industries related to cloud, fog, and recently edge computing. Conversely, owing to their extraordinary properties, 0D, 1D, and 2D materials are being explored as a physical basis for the next generation of logic components and processors. Carbon nanotubes, for example, have been recently used to create a new processor beyond proof of principle. These developments, in conjunction with neuromorphic and quantum computing, are envisioned to maintain the growth of computing power beyond the projected plateau for silicon technology. We survey the qualitative figures of merit of technologies of current interest for the next generation computing with an emphasis on edge computing.

97 MATHEMATICS AND COMPUTING↗

torc (Torc Workflow Management System) [SWR-24-127]

This software package orchestrates execution of a workflow of jobs on distributed computing resources. It is optimized for use on HPCs with Slurm, but also can be used in the cloud and on local computers. Please refer to the documentation at https://nrel.github.io/torc

Thom, Daniel [National Renewable Energy Laboratory↗

Shifting institutional culture to develop climate solutions with Open Science

To address our climate emergency, “we must rapidly, radically reshape society”—Johnson & Wilkinson, All We Can Save. In science, reshaping requires formidable technical (cloud, coding, reproducibility) and cultural shifts (mindsets, hybrid collaboration, inclusion). We are a group of cross-government and academic scientists that are exploring better ways of working and not being too entrenched in our bureaucracies to do better science, support colleagues, and change the culture at our organizations. We share much-needed success stories and action for what we can all do to reshape science as part of the Open Science movement and 2023 Year of Open Science.

54 ENVIRONMENTAL SCIENCES↗

Shedding light on U.S. small and midsize data centers: Exploring insights from the CBECS survey

As demand for digital services accelerates, the energy and environmental footprint of data centers faces increasing scrutiny. While hyperscale cloud facilities have driven efficiency gains, small and midsize U.S. data centers remain a critical yet underexamined segment with significant untapped potential for energy savings. This study leverages data from the Commercial Buildings Energy Consumption Survey (CBECS) to analyze trends in server stocks, computing customers, cooling system adoption and efficiency, and geospatial distribution from 2012 to 2018. Findings reveal a sharp decline in small and midsize data centers, from 1.764 million to 1.398 million, with server counts dropping from 5.177 million to 4.262 million—aligning with the broader shift toward cloud computing. More than 40 % of servers in small data centers and 55 % in midsize data centers are housed in office buildings, and over half of all servers are concentrated in climate zones 5A (cold), 3A (mixed-humid), and 4A (mixed-humid), with the highest densities in metropolitan hubs. While direct expansion units remain the dominant cooling system, a clear transition toward more energy-efficient solutions, particularly air economizers, is evident. By integrating server and cooling system distributions, we estimate Power Usage Effectiveness (PUE) and Water Usage Effectiveness (WUE) for U.S. data centers by size and year. Results show that midsize data centers are more energy-efficient but more water-intensive due to the widespread use of water-cooled chillers. These findings highlight the trade-offs in cooling system selection and provide a critical foundation for policies aimed at enhancing efficiency in an evolving data center landscape.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Science Use Case Design Patterns for Autonomous Experiments

Connecting scientific instruments and robot-controlled laboratories with computing and data resources at the edge, the Cloud or the high-performance computing (HPC) center enables autonomous experiments, self-driving laboratories, smart manufacturing, and artificial intelligence (AI)-driven design, discovery and evaluation. The Self-driven Experiments for Science / Interconnected Science Ecosystem (INTERSECT) Open Architecture enables science breakthroughs using intelligent networked systems, instruments and facilities with a federated hardware/software architecture for the laboratory of the future. It relies on a novel approach, consisting of (1) science use case design patterns, (2) a system of systems architecture, and (3) a microservice architecture. This paper introduces the science use case design patterns of the INTERSECT Architecture. It describes the overall background, the involved terminology and concepts, and the pattern format and classification. It further offers an overview of the 12 defined patterns and 4 examples of patterns of 2 different pattern classes. It also provides insight into building solutions from these patterns. The target audience are computer, computational, instrument and domain science experts working in the field of autonomous experiments.

Engelmann, Christian↗

INTERSECT Architecture Specification: Use Case Design Patterns (V.0.9)

Connecting scientific instruments and robot-controlled laboratories with computing and data resources at the edge, the Cloud or the high-performance computing (HPC) center enables autonomous experiments, self-driving laboratories, smart manufacturing, and artificial intelligence (AI)-driven design, discovery and evaluation. The Self-driven Experiments for Science / Interconnected Science Ecosystem (INTERSECT) Open Architecture enables science breakthroughs using intelligent networked systems, instruments and facilities with a federated hardware/software architecture for the laboratory of the future. It relies on a novel approach, consisting of (1) science use case design patterns, (2) a system of systems architecture, and (3) a microservice architecture. This document introduces the science use case design patterns of the INTERSECT Architecture. It describes the overall background, the involved terminology and concepts, and the pattern format and classification. It further details the 12 defined patterns and provides insight into building solutions from these patterns. The document also describes the application of these patterns in the context of several INTERSECT autonomous laboratories. The target audience are computer, computational, instrument and domain science experts working in the field of autonomous experiments.

97 MATHEMATICS AND COMPUTING↗

Cloud forensics and incident response platform

A system, method, and device for cloud forensics and incident response is provided. In an embodiment, a computer-implemented method for performing cloud forensics and incident response includes intercepting, by a cloud incident response module (CIRM), communication between a virtual machine (VM) and a hypervisor. The method also includes extracting, by the CIRM, data from the communication between the VM and the hypervisor according to a forensic policy. Intercepting and extracting the data are transparent to the VM and to the hypervisor. Intercepting and extracting the data are independent of the VM and the hypervisor.

Urias, Vincent↗

US ATLAS and US CMS HPC and Cloud Blueprint

The Large Hadron Collider (LHC) at CERN houses two general purpose detectors - ATLAS and CMS - which conduct physics programs over multi-year runs to generate increasingly precise and extensive datasets. The efforts of the CMS and ATLAS collaborations lead to the discovery of the Higgs boson, a fundamental particle that gives mass to other particles, representing a monumental achievement in the field of particle physics that was recognized with the awarding of the Nobel Prize in Physics in 2013 to François Englert and Peter Higgs. These collaborations continue to analyze data from the LHC and are preparing for the high luminosity data taking phase at the end of the decade. The computing models of these detectors rely on a distributed processing grid hosted by more than 150 associated universities and laboratories worldwide. However, such new data will require a significant expansion of the existing computing infrastructure. To address this, both collaborations have been working for years on integrating High Performance Computers (HPC) and commercial cloud resources into their infrastructure and continue to assess the potential role of such resources in order to cope with the demands of the new high luminosity era. US ATLAS and US CMS computing management have charged the authors to provide a blueprint document looking at current and possibly future use of HPC and Cloud resources, outlining integration models, possibilities, challenges and costs. The document will address key questions such as the optimal use of resources for the experiments and funding agencies, the main obstacles that need to be overcome for resource adoption, and areas that require more attention.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

5G Energy FRAME: The Design and Implementation of Data, Model, and Use Case (Year 2 Report)

This report summarizes the Year 2 work of Pacific Northwest National Laboratory’s (PNNL’s) 5G Fabricated Resource and Asset Management Encompassment for energy infrastructure (Energy FRAME) project funded by the Department of Energy Office of Science’s Advanced Scientific Computing Research (ASCR) Program. In this report, newest 5G equipment testing results are presented, along with two 5G-enabled AI/ML examples for grid applications; in addition, the work flow of grid edge, cloud, and High Performance Computing (HPC) platform is introduced, to support and interface the cross-domain simulation for power system transmission, distribution, and communication networks. Last but not least, the outlook for Year 3 work and the overarching impact of 5G Energy FRAME work to a multitude of stakeholders are provided. Additional 5G performance data now is shared through the publicly available weblink, https://www.pnnl.gov/projects/5g-energy-frame/publications

24 POWER TRANSMISSION AND DISTRIBUTION↗

Foldy: An open-source web application for interactive protein structure analysis

Foldy is a cloud-based application that allows non-computational biologists to easily utilize advanced AI-based structural biology tools, including AlphaFold and DiffDock. With many deployment options, it can be employed by individuals, labs, universities, and companies in the cloud without requiring hardware resources, but it can also be configured to utilize locally available computers. Foldy enables scientists to predict the structure of proteins and complexes up to 6000 amino acids with AlphaFold, visualize Pfam annotations, and dock ligands with AutoDock Vina and DiffDock. In our manuscript, we detail Foldy’s interface design, deployment strategies, and optimization for various user scenarios. We demonstrate its application through case studies including rational enzyme design and analyzing proteins with domains of unknown function. Furthermore, we compare Foldy’s interface and management capabilities with other open and closed source tools in the field, illustrating its practicality in managing complex data and computation tasks. Our manuscript underlines the benefits of Foldy as a day-to-day tool for life science researchers, and shows how Foldy can make modern tools more accessible and efficient.

59 BASIC BIOLOGICAL SCIENCES↗

Expanding the computational frontier of multi-scale atmospheric simulation to advance understanding of low cloud / climate feedbacks. Final Report

This is an accomplishment final report that details the work that the co-PI Marat Khairoutdinov has done in support of the project goals and objectives. It described his work on optimizing the performance of UP-CAM model, tuning its parameters to improve results, and his work on a new approach to superparameterization using dynamically adjusting grid spacing that can potentially improve the global simulations of clouds.

58 GEOSCIENCES↗

The globus compute dataset: An open function-as-a-service dataset from the edge to the cloud

Here we present a unique function-as-a-service (FaaS) dataset capturing the use of the Globus Compute (previously funcX) platform. Globus Compute implements a federated model via which users may deploy endpoints on arbitrary remote computers, from the edge to high performance computing (HPC) cluster, and they may then invoke Python functions on those endpoints via a reliable cloud -hosted service. The dataset covers 31 weeks and includes 2121472 task submissions from 252 users executed on 580 remote computing endpoints. It includes 277386 registered functions. We describe the dataset and various observations, some that are similar to other FaaS datasets, for example, that 74% of tasks run for less than 1 s, and some that are unique to Globus Compute, for example, that endpoints are used in different ways and that the majority of functions are related to scientific computing and machine learning. To the best of our knowledge, this dataset represents the first federated FaaS dataset that includes user workloads, distributed computing endpoints, and analysis of registered function bodies. We expect the dataset to be useful for researching FaaS architectures, workload modeling, container warming, and other distributed computing architectures.

97 MATHEMATICS AND COMPUTING↗

PipeSight: A High-Performance Computing Platform for Pipeline Integrity Management

The Phase I feasibility study completed as part of this project has led to a number of innovative technologies being developed and has laid the foundation for a successful Phase II effort to commercialize a platform for managing the integrity of pipelines for the damage mechanisms of the new, hybrid-energy based economy. To ground the development efforts and direction of the project, an extensive market research and customer discovery effort was undertaken early in Phase I. Through this effort, a number of pipeline owners and operators were interviewed, and the following key findings were discovered about the pipeline industry: • Small pipeline operators do not have the central engineering groups necessary to perform their own independent analysis of inspection data, but instead rely on summarized tally sheets provided to them by inspection service providers. • The time it takes to go from an inspection to a completed engineering assessment, even for small segments of pipeline, can take anywhere from 30-120 days. During this delay, critical threats can (and have been known to) cause failures. • Uncertainty is often not accounted for in the assessment of pipeline integrity. The tally sheets provided by third-party service providers are almost always deterministic in nature, identifying threats that present a concern only to the current (not the future) integrity of the pipeline. • It is uncommon to apply the latest technologies to perform advanced assessments of damaged pipelines. There is a desire to use more advanced analysis capabilities to assess threats. Many pipeline operators indicated that they would often excavate a pipeline to perform an inspection and find that the damage was not as bad as they anticipated, thus using limited resources unnecessarily. Companies are not consistent in their use of inspection data to determine corrosion rates, and those that do only calculate deterministic corrosion rates. • The industry has prominently relied on time-based inspections but has recently started to transition to risk-based inspections. However, there appears to be no uniform guidance on how to do so while properly accounting for all sources of uncertainty. • Companies are not storing inspection data in a manner that allows for the ready determination of temporal trends. • Predictive maintenance principles and practices are beginning to be used by early adopters • Some pipelines are being re-purposed to transport different process fluids than they were designed for, e.g., H 2 and CO 2 rich process streams to serve the new hybrid-energy based economy, which are presenting new integrity concerns for the existing pipeline network that crisscrosses the United States. As a result of these discoveries, we were able to target the development efforts in Phase I to best serve the needs of the industry. In Phase I, we developed a way to correlate multiple large-scale scans of the pipeline to determine a probabilistic corrosion rate that accounts for all sources of error and uncertainty in the inspection process. This probabilistic corrosion rate can be used to predict the future thickness distribution of the pipe wall. We demonstrate how this analysis may be performed in an analytical fashion and has been implemented in such a manner that it can be readily distributed using GPU computing through integration of the Kokkos programming model. We also make a very novel extension of the analytical corrosion rate model to Bayesian Networks (an explainable AI technique) that can account for non-parametric distributions of corrosion rates. With the predictions made above for the probabilistic corrosion rate and corresponding future distribution of the pipe wall thickness, we can assess the integrity of the pipeline through the use of a probabilistic engineering assessment. We developed a novel screening data analysis approach that can rapidly identify ‘hotspots’ (local thin areas) where the integrity of the pipeline is a concern. Once more, we implemented this screening approach in C++ to leverage GPU computing via the Kokkos programming model. After the critical hotspots are identified, we developed a program that can automatically generate an advanced finite element model of the damaged regions. Since the number of damaged regions that require advanced analysis can number in the thousands, we integrated an open-source container-native workflow engine for orchestrating parallel jobs on the cloud. Initially, these advanced numerical models were only designed to account for loading due to internal pressure. However, in a slight pivot from the initial Phase I proposal, we developed a complete pipe stress analysis program (called Simflex) which can simulate the complete pipeline and its response to thermal expansion, pressure, thermal bowing, weight, wind, earthquake, support displacement, support friction and external forces. This pipe stress analysis program was written generically, to handle any piping system, but contains the features needed to model long pipelines (i.e., it incorporates a model for soil mechanics and can account for the nonlinear boundary conditions necessary to simulate long underground pipelines). This pipe stress analysis program can simulate any segment of the pipeline (simple or complex) under any set of conditions and loads, to determine the supplemental loads (axial forces and bending moments) at the location of damage. This enables the most accurate state of stress to be accounted for in the pipeline, which can prove critical when evaluating the integrity of a damaged region. In the process of developing the technologies to perform the integrity assessment of the pipeline, we also extended one of the industry standard approaches for performing the assessment of local thin areas that extend more in the circumferential direction than the longitudinal direction of the pipeline. This approach was presented to the API 579-1/AS ME FFS-1 steering committee in November 2021 for consideration in the next edition of the industry standard for Fitness-For-Service (expected to be released in 2023). To help pipeline operators make decisions with the results on any integrity assessment, we developed a new approach to the life-cycle management of pipelines which uses a Bayesian Decision Network. The network is designed to help pipeline operators plan and prioritize inspection activities and ultimately make smarter, more cost-effective decisions. The Bayesian approach accounts for all sources of uncertainty and carries them through to the final optimal decisions, providing a probabilistic framework for optimizing inspection intervals. The proof-of-concept networks developed in the feasibility study are complete, verified, and are focused on a subset of the pipeline. To expand this novel approach to the scale necessary for an entire network of pipelines in Phase II, we will leverage the DOE-funded Bengi solver for industrial-scale decision making with Bayesian Networks [22]. Once implemented, we will be able to provide the pipeline industry with a much-needed tool for optimal inspection planning using truly explainable artificial intelligence (XAI). To handle all of these advanced capabilities into a cloud-based platform, the architecture of the Equity Engineering Cloud (EEC) was extended to include Argo Workflows, a framework capable of distributing and managing a massive number of jobs that consume their own resources, such that thousands of serial finite element simulations can be run in parallel. As part of this substantial undertaking, we also integrated Argo Continuous Delivery (CD) into the EEC, to aid with the rapid prototyping and iterations that will be imperative to the success of the PipeSight platform’s Agile development process in Phase II. As part of the pipe stress analysis program, we also developed a custom visualizer that leverages the DOE-funded VTK visualization library. We added custom contouring capabilities and a means for interacting visually with both the inputs and outputs of the pipe stress analysis program. We also developed routines for automating the post-processing of the finite element simulations to determine if any failure criteria are met and to visualize the deformations, stresses and strains in ParaView using the exodus II file format (a subset of netCDF).

24 POWER TRANSMISSION AND DISTRIBUTION↗

A Privacy-Aware Federated Learning Framework for Distributed Energy Resource Analytics in Constrained Environments

To be resilient against extreme weather events, the rural communities in Puerto Rico are leveraging distributed energy resources (DER). However, computing frameworks sup-porting the grid in critical decision-making are still largely centralized. Sensitive consumer data are transmitted over the Internet or cellular networks to a secondary or tertiary node. It guarantees better situational awareness at the cost of a wider attack surface, jeopardizing user privacy, as more DER come online. Cloud, Edge, and Fog computing all require data aggregation at some level. This paper introduces a privacy-aware federated learning framework that leverages the Fog model by pushing analytics all the way to the DER and load assets. These local models train on individual asset data and transmit only learned parameters (such as weights) over secure communications to a global decision-maker. By abstracting personally identifiable consumer data without impacting decision optimality, this framework better aligns with distributed power generation paradigm.

Sundararajan, Aditya↗

Doppler Lidar (DL) Instrument Handbook

The Doppler lidar (DL) is an active remote sensing instrument that provides range- and time-resolved measurements of radial velocity and attenuated backscatter. The principle of operation is similar to radar in that pulses of energy are transmitted into the atmosphere; the energy scattered back to the transceiver is collected and measured as a time-resolved signal. From the time delay between each outgoing transmitted pulse and the backscattered signal, the distance to the scatterer is inferred. The radial or line-of-sight velocity of the scatterers is determined from the Doppler frequency shift of the backscattered radiation. The DL uses a heterodyne detection technique in which the return signal is mixed with a reference laser beam (i.e., local oscillator) of known frequency. An onboard signal processing computer then determines the Doppler frequency shift from the spectra of the heterodyne signal. The energy content of the Doppler spectra can also be used to determine attenuated backscatter.

99 GENERAL AND MISCELLANEOUS↗

Removing Numerical Pathologies in a Turbulence Parameterization Through Convergence Testing

Abstract Discretized numerical models of the atmosphere are usually intended to faithfully represent an underlying set of continuous equations, but this necessary condition is violated sometimes by subtle pathologies that have crept into the discretized equations. Such pathologies can introduce undesirable artifacts, such as sawtooth noise, into the model solutions. The presence of these pathologies can be detected by numerical convergence testing. This study employs convergence testing to verify the discretization of the Cloud Layers Unified By Binormals (CLUBB) model of clouds and turbulence. That convergence testing identifies two aspects of CLUBB's equation set that contribute to undesirable noise in the solutions. First, numerical limiters (i.e., clipping) used by CLUBB introduce discontinuities or slope discontinuities in model fields. Second, nonlinear artificial diffusion employed for improving numerical stability can introduce unintended small‐scale features into the solution of the model equations. Smoothing the limiters and using linear artificial diffusion reduces the noise and restores the expected first‐order convergence in CLUBB's solutions. These model reformulations enhance our confidence in the trustworthiness of solutions from CLUBB by eliminating the unphysical oscillations in high‐resolution simulations. The improvements in the results at coarser, near‐operational grid spacing and timestep are also seen in cumulus cloud and dry turbulence tests. In addition, convergence testing is proven to be a valuable tool for detecting pathologies, including unintended discontinuities and grid dependence, in the model equation set.

58 GEOSCIENCES↗