Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Distributed computation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

RuralAI in Tomato Farming: Integrated Sensor System, Distributed Computing, and Hierarchical Federated Learning for Crop Health Monitoring

Precision horticulture is evolving due to scalable sensor deployment and machine learning (ML) integration. These advancements boost the operational efficiency of individual farms, balancing the benefits of analytics with autonomy requirements. However, given concerns that affect wide geographic regions (e.g., climate change), there is a need to apply models that span farms. Federated learning (FL) has emerged as a potential solution. FL enables decentralized ML across different farms without sharing private data. Traditional FL assumes simple two-tier network topologies and, thus, falls short of operating on more complex networks found in real-world agricultural scenarios. Networks vary across crops and farms and encompass various sensor data modes, extending across jurisdictions. New hierarchical FL (HFL) approaches are needed for more efficient and context-sensitive model sharing, accommodating regulations across multiple jurisdictions. Here, we present the RuralAI architecture deployment for tomato crop monitoring, featuring sensor field units for soil, crop, and weather data collection. HFL with personalization is used to offer localized and adaptive insights. Model management, aggregation, and transfers are facilitated via a flexible approach, enabling seamless communication between local devices, edge nodes, and the cloud.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2012

Enabling event tracing at leadership-class scale through I/O forwarding middleware. In: HPDC '12 Proceedings of the 21st international symposium on High-Performance Parallel and Distributed Computing

Event tracing is an important tool for understanding the performance of parallel applications. As concurrency increases in leadership-class computing systems, the quantity of performance log data can overload the parallel file system, perturbing the application being observed. In this work we present a solution for event tracing at leadership scales. We enhance the I/O forwarding system software to aggregate and reorganize log data prior to writing to the storage system, significantly reducing the burden on the underlying file system for this type of traffic. Furthermore, we augment the I/O forwarding system with a write buffering capability to limit the impact of artificial perturbations from log data accesses on traced applications. To validate the approach, we modify the Vampir tracing toolset to take advantage of this new capability and show that the approach increases the maximum traced application size by a factor of 5x to more than 200,000 processes.

Ilsche, Thomas↗
DOE OSTI2017

Machine and Application Aware Partitioning for Adaptive Mesh Refinement Applications. In: HPDC '17 Proceedings of the 26th International Symposium on High-Performance Parallel and Distributed Computing

Load balancing and partitioning are critical when it comes to parallel computations. Popular partitioning strategies based on space filling curves focus on equally dividing work. The partitions produced are independent of the architecture or the application. Given the ever-increasing relative cost of data movement and increasing heterogeneity of our architectures, it is no longer sufficient to only consider an equal partitioning of work. Minimizing communication costs are equally if not more important. Our hypothesis is that an unequal partitioning that minimizes communication costs significantly can scale and perform better than conventional equal-work partitioning schemes. This tradeoff is dependent on the architecture as well as the application. We validate our hypothesis in the context of a finite-element computation utilizing adaptive mesh-refinement. Our central contribution is a new partitioning scheme that minimizes the overall runtime of subsequent computations by performing architecture and application-aware non-uniform work assignment in order to decrease time to solution, primarily by minimizing data-movement. We evaluate our algorithm by comparing it against standard space-filling curve based partitioning algorithms and observing time-to-solution as well as energy-to-solution for solving Finite Element computations on adaptively refined meshes. We demonstrate excellent scalability of our new partition algorithm up to $262,144$ cores on ORNL's Titan and demonstrate that the proposed partitioning scheme reduces overall energy as well as time-to-solution for application codes by up to 22.0%.

Fernando, Milinda↗
DOE OSTI2016

FEL Simulation Using Distributed Computing

While simulation tools are available and have been used regularly for simulating light sources, the increasing availability and lower cost of GPU-based processing opens up new opportunities. This poster highlights a method of how accelerating and parallelizing code processing through the use of COTS software interfaces.

43 PARTICLE ACCELERATORS↗
DOE OSTI2020

Harnessing the power of ab initio calculations, distributed computing and machine learning to efficiently locate extreme molecules for use in carbon-based solar cells (Final Technical Report)

The use of high-throughput virtual screening (HTVS) tools is a powerful tool to expedite the materials discovery of commercially relevant materials. In previous years, our group has developed a molecular discovery platform to generate libraries in order to obtain suitable candidates for different applications, starting from the Harvard Clean Energy Project. This platform is suitable to test in-silico on traditional supercomputing clusters and shared resources, for example, in the IBM World Community Grid. In this project, we used the molecular discovery platform to create and screen a library of candidates of organic photovoltaic (OPVs) molecules. Based on a set of candidates created with combinations of molecular moieties, we were able to filter, by conformation stability, the energy of electronic orbitals and approximated power conversion efficiencies (PCE). To improve the predictions of orbital energies calculated and the PCEs, we used Gaussian Process regression and two sets of molecules. These sets correspond to electronic structure calculations of a higher level of theory and experimental PCE values, respectively. Finally, we selected a subset of the best candidates (molecules with a PCE higher than 10%) to understand its absorbance properties with TD-DFT. This project has demonstrated the capabilities of our molecular discovery platform for HTVS. Finally, machine learning can help us to introduce more complex effects included in bulk conditions and computational intensive calculations on models.

14 SOLAR ENERGY↗
DOE OSTI2020

Distributed heterogeneous compute infrastructure for the study of additive manufacturing systems

We present the current status of a large-scale computing framework to address the need of the multidisciplinary effort to study chemical dynamics. Specifically, we are enabling scientists to process and store experimental data, run large-scale computationally expensive high-fidelity physical simulation, and analyze these results using the state-of-the-art data analytics tools, machine learning, and uncertainty quantification methods using heterogeneous computing resources, such as CPU and GPU cluster. The framework can integrate or abstract out multiple domains based on roles. In order to develop this framework, we have leveraged an existing framework coupled with in-house heterogeneous computing resources. We present the results of using this framework on a single metadata triggered workflow to accelerate an additive manufacturing use case.

heterogeneous computing, hpc, workflows, reproduci↗
DOE OSTI2016

Distributing an executable job load file to compute nodes in a parallel computer

Distributing an executable job load file to compute nodes in a parallel computer, the parallel computer comprising a plurality of compute nodes, including: determining, by a compute node in the parallel computer, whether the compute node is participating in a job; determining, by the compute node in the parallel computer, whether a descendant compute node is participating in the job; responsive to determining that the compute node is participating in the job or that the descendant compute node is participating in the job, communicating, by the compute node to a parent compute node, an identification of a data communications link over which the compute node receives data from the parent compute node; constructing a class route for the job, wherein the class route identifies all compute nodes participating in the job; and broadcasting the executable load file for the job along the class route for the job.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Distributing an executable job load file to compute nodes in a parallel computer

Distributing an executable job load file to compute nodes in a parallel computer, the parallel computer comprising a plurality of compute nodes, including: determining, by a compute node in the parallel computer, whether the compute node is participating in a job; determining, by the compute node in the parallel computer, whether a descendant compute node is participating in the job; responsive to determining that the compute node is participating in the job or that the descendant compute node is participating in the job, communicating, by the compute node to a parent compute node, an identification of a data communications link over which the compute node receives data from the parent compute node; constructing a class route for the job, wherein the class route identifies all compute nodes participating in the job; and broadcasting the executable load file for the job along the class route for the job.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

S-QGPU: Shared quantum gate processing unit for distributed quantum computing

We propose a distributed quantum computing (DQC) architecture in which individual small-sized quantum computers are connected to a shared quantum gate processing unit (S-QGPU). The S-QGPU comprises a collection of hybrid two-qubit gate modules for remote gate operations. In contrast to conventional DQC systems, where each quantum computer is equipped with dedicated communication qubits, S-QGPU effectively pools the resources (e.g., the communication qubits) together for remote gate operations, and, thus, significantly reduces the cost of not only the local quantum computers but also the overall distributed system. Our preliminary analysis and simulation show that S-QGPU's shared resources for remote gate operations enable efficient resource utilization. When not all computing qubits (also called data qubits) in the system require simultaneous remote gate operations, S-QGPU-based DQC architecture demands fewer communication qubits, further decreasing the overall cost. Alternatively, with the same number of communication qubits, it can support a larger number of simultaneous remote gate operations more efficiently, especially when these operations occur in a burst mode.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗