Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “distributed computation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Orchestration of materials science workflows for heterogeneous resources at large scale

In the era of big data, materials science workflows need to handle large-scale data distribution, storage, and computation. Any of these areas can become a performance bottleneck. We present a framework for analyzing internal material structures (e.g., cracks) to mitigate these bottlenecks. We demonstrate the effectiveness of our framework for a workflow performing synchrotron X-ray computed tomography reconstruction and segmentation of a silica-based structure. Our framework provides a cloud-based, cutting-edge solution to challenges such as growing intermediate and output data and heavy resource demands during image reconstruction and segmentation. Specifically, our framework efficiently manages data storage, scaling up compute resources on the cloud. The multi-layer software structure of our framework includes three layers. A top layer uses Jupyter notebooks and serves as the user interface. A middle layer uses Ansible for resource deployment and managing the execution environment. A low layer is dedicated to resource management and provides resource management and job scheduling on heterogeneous nodes (i.e., GPU and CPU). At the core of this layer, Kubernetes supports resource management, and Dask enables large-scale job scheduling for heterogeneous resources. The broader impact of our work is four-fold: through our framework, we hide the complexity of the cloud’s software stack to the user who otherwise is required to have expertise in cloud technologies; we manage job scheduling efficiently and in a scalable manner; we enable resource elasticity and workflow orchestration at a large scale; and we facilitate moving the study of nonporous structures, which has wide applications in engineering and scientific fields, to the cloud. While we demonstrate the capability of our framework for a specific materials science application, it can be adapted for other applications and domains because of its modular, multi-layer architecture.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Confidentiality-preserving machine learning algorithms for soft-failure detection in optical communication networks

Automated fault management is at the forefront of next-generation optical communication networks. The increase in complexity of modern networks has triggered the need for programmable and software-driven architectures to support the operation of agile and self-managed systems. In these scenarios, the European Telecommunications Standards Institute zero-touch network and service management approach is imperative. The need for machine learning algorithms to process the large volume of telemetry data brings safety concerns as distributed cloud-computing solutions become the preferred approach for deploying reliable communication network automation. This paper’s contribution is twofold. First, we propose a simple yet effective method to guarantee the confidentiality of the telemetry data based on feature scrambling. The method allows the operation of third-party computational services without direct access to the full content of the collected data. Additionally, the effectiveness of four unsupervised machine learning algorithms for soft-failure detection is evaluated when applied to the scrambled telemetry data. The methods are based on factor analysis, principal component analysis, nonlinear principal component analysis, and singular value decomposition. Most dimensionality reduction algorithms have the common property that they can maintain similar levels of fault classification performance while hiding the data structure from unauthorized access. Evaluations of the proposed algorithms demonstrate this capability.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

A New Capability of E4D For 3D Parallel Joint Inversion of DC Resistivity And Traveltime Data on Unstructured Mesh

A major challenge in interpreting geophysical data is how to derive consistent three-dimensional (3D) earth models of different physical properties from spatially and temporally limited measurements. Joint inversion with cross-gradient constraints is an approach to find such models by imposing structural similarities between different physical parameters. We have developed a parallel distributed-memory joint inversion code for direct-current (DC) resistivity and traveltime data using the cross-gradient constraint on unstructured mesh. The code utilizes existing E4D framework for parallel forward simulation, distributed storage and computation of the Jacobian matrix of forward operator, and parallel execution of matrix-vector multiplication during inversion. Besides, the joint inversion is solved by nonlinear conjugate gradient algorithm parallelized for DC resistivity and traveltime data. The joint inversion capability of E4D was tested using synthetic data from cross-borehole DC resistivity and traveltime data. The results indicate that the shape and size of the anomalies from the joint inversion are more reliable than those from separate inversions.

58 GEOSCIENCES↗
DOE OSTI2020

Platform for efficient large-scale storage and analysis of multi-omics data in plant and microbial systems (Final Technical Report)

Genomic variation at the sequence level fundamentally affects the phenotypic state of all organisms at all stages of development, while dynamic processes such as changes in the epigenome (e.g. DNA methylation state) and transcriptome regulate the specific phenotype expressed at any given state of development based upon that genomic variation. In plants, DNA methylation is a particularly important mechanism for both regulating transcriptomic expression and for management of genomic variations that could be deleterious to the organism due to the presence of active retrotransposons in plant genomes. While DNA methylation is heritable, it is also dynamic through a given plant’s development and life cycle, particularly during the development from seed to mature specimen suggesting variations in DNA methylation could be critical regulators of biologically and commercially important phenotypes such as time to flowering; in addition, plant DNA methylation is more complex than that of animals, with methylation of CHG and CHH trinucleotides evident in addition to the better-known CG methylation. The complexity of plant DNA methylation and its interplay with genomic sequence variation, transcriptomics and other epigenomic factors demand a storage and analysis framework that can cope with the complexity both within a single specimen and with analyses that span many individuals and even many species, such as attempts to extend models from model organisms to commercially relevant species. In addition to complexity, the rapid development and proliferation of sequencing technology has led to an explosion of data volume that conventional storage and analysis solutions will likely be unable to cope with in the long run. We proposed to study these with suitable distributed storage and computation and therefore for the application of cloud computing to biological analyses; integrate with existing data sources and compatible with virtually any interface use case, from fully automated shell scripts to notebooks and do all these at scale in this STTR grant.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

MPACT Theory Manual (V.4.3)

This manual presents the theory underlying the three-dimensional (3D) whole-core, pin-resolved neutron transport calculation methodologies employed in the MPACT code. MPACT’s primary goal is to provide accurate sub-pin power distributions in a computationally efficient manner. To accomplish this, MPACT offers several different transport method options. The 2D/1D method, in which 3D problems are decomposed into an axial stack of radial “slices,” is currently the most commonly used. In this option, two-dimensional (2D) planar solutions are provided by the method of characteristics (MOC), and axial solutions are provided via one-dimensional (1D) approximate diffusion, or P3 solutions. The radial and axial solutions are coupled by (i) axial and radial transverse leakages, and (ii) a global 3D coarse mesh finite difference (CMFD) solve, which provides both acceleration and stability to the solution iteration scheme. The subsequent chapters of this manual present a range of topics, including MOC, CMFD, axial nodal transport solvers, 2D/1D, self-shielding, depletion, thermal-hydraulics, and transient methods. Development of the underlying theory of the 2D/1D method is presented, diagrams are included to highlight important algorithmic flow, and important concepts are discussed as appropriate. This manual is intended to be self-sufficient, but references to published articles and other materials are included for further reading. MPACT is a relatively new code, with new capabilities and many computational methods that did not exist until recently. This manual is an even newer document, with chapters written by several different code contributors working under time constraints. For these reasons, the manual is not yet complete and finalized. Future versions of this manual will address current deficiencies.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Single Photon Detection with On-Chip Number Resolving Capability

Single photon detection (SPD) plays an important role in many forefront areas of fundamental science and advanced engineering applications. In recent years, rapid developments in superconducting quantum computation, quantum key distribution, and quantum sensing call for SPD in the microwave frequency range. We have explored in this LDRD project a new approach to SPD in an effort to provide deterministic photon-number-resolving capability by using topological Josephson junction structures. In this SAND report, we will present results from our experimental studies of microwave response and theoretical simulations of microwave photon number resolving detector in topological Dirac semimetal Cd 3 As 2 . These results are promising for SPD at the microwave frequencies using topological quantum materials.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

Tempus Project (Final Report)

This final technical report tracks the accomplishments of the Tempus Project to the statement of project objectives and resulting deliverables. The objective of the Tempus project was to develop and demonstrate the capabilities of a secure, modular, and customizable time synchronization platform that provides layers of protection from GPS spoofing attacks and other vulnerabilities. These vulnerabilities present a challenge to power system operators utilizing GPS-based time synchronization, as time synchronization increasingly plays a critical role in the efficient and reliable operation of power systems. The Tempus project addressed these concerns through the development of a secure, customizable time synchronization platform. The Tempus platform applies a layered approach to time security that protects from manipulation of timing systems used by power utilities. The Tempus platform consists of a component framework utilizing modularized time sources and customizable manipulation detection algorithms. With multiple time sources, and comparisons between sources and tracking sudden or subtle changes to the timing phase and time information, the Tempus system can detect and mitigate compromised timing sources.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

RTDP: Streaming Readout Real-Time Development and Testing Platform

The Thomas Jefferson National Accelerator Facility (JLab) has created and is currently working on various tools to facilitate streaming readout (SRO) for upcoming experiments. These include reconstruction frameworks with support for Artificial Intelligence/Machine Learning, distributed High Throughput Computing (HTC), and heterogeneous computing which all contribute significantly to swift data processing and analysis. Designing SRO systems that combine such components for new experiments would benefit from a platform that would combine both simulation and execution components for simulation, testing, and validation before large investments are made. The Real-Time Development Platform (RTDP) is being developed as part of an LDRD funded project at JLab. RTDP aims to establish a seamless connection between algorithms, facilitating the seamless processing of data from SRO to analysis, as well as enabling the execution of these algorithms in various configurations on compute and data centers. Individual software components simulating specific hardware can be replaced with actual hardware when it is available.

Gyurjyan, Vardan↗
DOE OSTI2024

Summary Report of the SOS26 Workshop held March 11-14, 2024

The SOS26 workshop was organized by Oak Ridge National Laboratory (ORNL) and held March 11-14, 2024, at Cocoa Beach, Florida. The SOS is a workshop organized annually, with a focus on distributed high performance computing (HPC). The technical program of the workshop was developed jointly by Sandia National Laboratories (SNL), ORNL and the Swiss National Supercomputing Center (CSCS). The 2024 SOS26 workshop theme was "Versatile HPC for the evolving and expanding needs of science" and had seven technical sessions covering HPC, data and machine learning (ML) topics. Each session consisted of four or five presentations, followed by a panel discussion. This report documents the workshop proceedings covering all the technical sessions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

X-Ray Emission of Ultraviolet Variable Active Galactic Nucleus Candidates

We analyze the X-ray properties of a sample of 23 high-probability active galactic nucleus (AGN) candidates with ultraviolet variability identified in Wasleske et al. Using data from the Chandra X-ray Observatory and XMM-Newton Observatory, we that find 11/23 nuclei are X-ray detected. We use spectral energy distribution modeling to compute star formation rates and show that the X-ray luminosities are typically in excess of the X-ray emission expected from star formation by at least an order of magnitude. Interestingly, this sample shows a diversity of optical spectroscopic properties. We explore possible reasons for why some objects lack optical spectroscopic signatures of black hole activity while still being UV-variable and X-ray bright. We find that host galaxy stellar emission and obscuration from gas and dust are potential factors. We study where this sample falls on relationships such as α OX - L 2500 and L X - L IR and find that some of the sample falls outside the typical scatter for these relations, indicating that they differ from the standard quasar population. With the diversity of optical spectroscopic signatures and varying impacts of dust and stellar emissions on our sample, these results emphasize the strength of variability in selecting the most complete set of AGN, regardless of other host galaxy properties.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Size-Resolved Chemical Composition of Particles Collected Using STAC at the Ground Site During the SAIL Campaign in Gunnison, Colorado

Aerosol particles were collected using a four-stage Size and Time-resolved Aerosol Collector (STAC) during the SAIL field campaign. Each stage of STAC separates particles into distinct aerodynamic size fractions with 50% cut-off diameters: Stage A: 2.27 µm Stage B: 0.615 µm Stage C: 0.421 µm Stage D: 0.119 µm Each stage provides both size- and time-resolved sampling, enabling investigation of particle composition across different atmospheric regimes. Only a subset of samples was selected for analysis based on prevailing meteorological conditions (e.g., temperature, humidity, and air-mass influence) to capture representative aerosol types under distinct weather patterns. Collected substrates were first examined under Scanning Electron Microscopy (SEM) to evaluate particle loading, morphology, and spatial distribution. Subsequently, Computer-Controlled Scanning Electron Microscopy with Energy-Dispersive X-ray Spectroscopy (CCSEM/EDX) was performed to obtain size-resolved elemental composition of individual particles. A rule-based classification scheme was applied to categorize particles into major compositional groups (e.g., biological, carbonaceous, dust, sulfate, Na-rich, and mixed types). This dataset provides high-resolution morphological and chemical information on atmospheric particles collected during the SAIL campaign, offering insights into the influence of meteorology on aerosol composition and mixing state.

Size and Time-resolved Aerosol Collector↗
DOE OSTI2022

Optimizing aircraft flows at airports using data driven predicted capabilities

A method for safe and efficient use of airport runway capacity includes receiving, at an air traffic control system at an airport, airport data related to movement areas of the airport, time data related to a time period, aircraft data related to a plurality of aircraft expected to operate into and out of the airport during the time period, and environmental data related to environmental conditions predicted for the airport during the time period. The method further includes computing a probability distribution for inter-aircraft spacing by applying the airport data, the time data, the aircraft data, and the environmental data to a trained Bayesian network, producing the probability distribution for the inter-aircraft spacing as an output observation of the trained Bayesian network, and, using the probability distribution and a confidence value, identifying an inter-aircraft spacing value for the plurality of aircraft expected to operate into and out of the airport during the time period.

Sweet, Douglas↗
DOE OSTI2025

EUREICA: Efficient UltRa Endpoint IoT-enabled Coordinated Architecture

The electricity grid has evolved from a physical system to a cyber-physical system with digital devices that perform measurement, control, communication, computation, and actuation. The increased penetration of distributed energy resources (DERs) that include renewable generation, flexible loads, and storage provides extraordinary opportunities for improvements in efficiency and sustainability. However, they can introduce new vulnerabilities in the form of cyberattacks, which can cause significant challenges in ensuring grid resilience. The purpose of this project was to develop a framework ((Efficient, Ultra-REsilient, IoT-Coordinated Assets, or EUREICA)for achieving grid resilience through suitably coordinated assets including a network of Internet of Things (IoT) devices, and a local electricity market (LEM) to identify trustable assets and carry out this coordination. Situational Awareness (SA) of locally available DERs with the ability to inject power or reduce consumption is enabled by the market, together with a monitoring procedure for their trustability and commitment. Experiments conducted during this project demonstrated that, with this SA, a variety of cyberattacks can be mitigated using local trustable resources without stressing the bulk grid. The demonstrations were carried out using a variety of high-fidelity co-simulation platforms, real-time hardware-in-the-loop validation, and a utility-friendly simulator.

14 SOLAR ENERGY↗
DOE OSTI2020

Modular performance prediction for scientific workflows using Machine Learning

Scientific workflows provide an opportunity for declarative computational experiment design in an intuitive and efficient way. A distributed workflow is typically executed on a variety of resources, and it uses a variety of computational algorithms or tools to achieve the desired outcomes. Such a variety imposes additional complexity in scheduling these workflows on large scale computers. As computation becomes more distributed, insights into expected workload that a workflow presents become critical for effective resource allocation. In this paper, we present a modular framework that leverages Machine Learning for creating precise performance predictions of a workflow. The central idea is to partition a workflow in such a way that makes the task of forecasting each atomic unit manageable and gives us a way to combine the individual predictions efficiently. We recognize a combination of an executable and a specific physical resource as a single module. This gives us a handle to characterize workload and machine power as a single unit of prediction. Overall, our modular technique of creating atomic modules and deployment of longest-path approach to estimate workflow performance, allows the framework to adapt to highly complex nested directed acyclic workflows and scale to new scenarios, since it does not make assumptions of underlying workflow structure. We present performance estimation results of independent workflow modules executed on the XSEDE SDSC Comet cluster using various Machine Learning algorithms. The results provide insights into the behavior and effectiveness of different algorithms in the context of scientific workflow performance prediction.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

The reproduction number and its probability distribution for stochastic viral dynamics

We consider stochastic models of individual infected cells. The reproduction number, R, is understood as a random variable representing the number of new cells infected by one initial infected cell in an otherwise susceptible (target cell) population. Variability in R results partly from heterogeneity in the viral burst size (the number of viral progeny generated from an infected cell during its lifetime), which depends on the distribution of cellular lifetimes and on the mechanism of virion release. We analyse viral dynamics models with an eclipse phase: the period of time after a cell is infected but before it is capable of releasing virions. The duration of the eclipse, or the subsequent infectious, phase is non-exponential, but composed of stages. We derive the probability distribution of the reproduction number for these viral dynamics models, and show it is a negative binomial distribution in the case of constant viral release from infectious cells, and under the assumption of an excess of target cells. In a deterministic model, the ultimate in-host establishment or extinction of the viral infection depends entirely on whether the mean reproduction number is greater than, or less than, one, respectively. Here, the probability of extinction is determined by the probability distribution of R, not simply its mean value. In particular, we show that in some cases the probability of infection is not an increasing function of the mean reproduction number.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

HTR-1.3 solver: Predicting electrified combustion using the hypersonic task-based research solver

Here this manuscript presents an updated open-source version of the Hypersonics Task-based Research (HTR) solver. The solver, whose main features are presented in Di Renzo et al. (2020) and Di Renzo & Pirozzoli (2021), is designed for direct numerical simulation of reacting flows at high Reynolds numbers. This new version extends the applications of the HTR solver to turbulent combustion in the presence of external electric fields. In particular, a new distributed Poisson solver compatible with heterogeneous architectures has been incorporated in the algorithm to compute the electric potential distribution in bi-periodic configurations. The drift fluxes of the electrically charged species are now included in the transport equations using a targeted essentially non-oscillatory scheme. A verification of these new features of the solver is provided using one-dimensional burner stabilized flames, whereas a three dimensional turbulent flame is utilized to discuss the scalability of the proposed numerical tool.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

A Review of Edge Computing Technology and Its Applications in Power Systems

Recent advancements in network-connected devices have led to a rapid increase in the deployment of smart devices and enhanced grid connectivity, resulting in a surge in data generation and expanded deployment to the edge of systems. Classic cloud computing infrastructures are increasingly challenged by the demands for large bandwidth, low latency, fast response speed, and strong security. Therefore, edge computing has emerged as a critical technology to address these challenges, gaining widespread adoption across various sectors. This paper introduces the advent and capabilities of edge computing, reviews its state-of-the-art architectural advancements, and explores its communication techniques. A comprehensive analysis of edge computing technologies is also presented. Furthermore, this paper highlights the transformative role of edge computing in various areas, particularly emphasizing its role in power systems. It summarizes edge computing applications in power systems that are oriented from the architectures, such as power system monitoring, smart meter management, data collection and analysis, resource management, etc. Additionally, the paper discusses the future opportunities of edge computing in enhancing power system applications.

24 POWER TRANSMISSION AND DISTRIBUTION↗