Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “hardware efficiency”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25

Hybrid Electric Drivetrain Testing and Design (CRADA CRD-17-00699 Final Report)

Hybrid electric drivetrains have recently become of a great interest in the medium and heavy-duty vehicle market as it enables substantial reduction of petroleum use, vehicle level fuel use and criteria pollutant emissions. These vehicle performance improvements are not only economically beneficial for business operations relying on large fleets of vehicles, they are also paramount for curbing energy use and the negative effects of vehicle operations on the environment. Efficient Drivetrains, Inc. (EDI), acquired in July 2018 by Cummins, Inc., is a small company focused on development of medium and heavy-duty hybrid electric drivetrains. EDI has already developed the general hardware architecture of their drivetrain system, but there is still a significant effort to be done on optimizing the system in terms of control strategies and component sizing in order to maximize the benefits of the hybrid drivetrain. Historically, hybrid electric systems have proven to deliver better fuel economy in certain applications than their conventional counterparts. Particular areas of advantageous applications are vocations with kinetically intensive transient duty cycles and operations requiring some sort of power take off whose power demand is not well matched to the size of the vehicle’s main engine, thus forcing it to operate in extremely inefficient operating modes. Hybrid drivetrains can be at a disadvantage when pressed into duty cycle operation consisting of extensive steady state highway cruise due to various design compromises optimized for more transient operation. This can also lead to increase in vehicle emissions if the system is not optimized properly. Ample opportunity for extensive optimization is needed to overcome these obstacles.

33 ADVANCED PROPULSION SYSTEMS↗

LDRD Abbreviated report: High-Order General-Discrete-Ordinates Method Enabling Efficient Deterministic Transport in Hydrodynamic Simulations

Deterministic transport simulations for national-security and energy applications often operate in high-dimensional phase-space, where accuracy and cost both become major challenges. A common numerical artifact in such problems is the “ray-effect,” which appears as unphysical streaks. Beyond misinterpretation, these artifacts can contaminate tightly coupled physics, such as fluid dynamics, radiation-hydrodynamics, and laser-plasma interactions, eroding the predictive capability of entire multiphysics workflows. Our objective was to make high-dimension studies practical on modern hardware while mitigating the ray-effect without relying on prohibitively expensive sampling approaches such as Monte Carlo methods. We developed the Generic Discretization Library (GenDiL), a Graphics Processing Unit (GPU)-first framework that uses high-order Discontinuous Galerkin (DG) methods and matrix-free algorithms to reduce memory usage and improve computational efficiency, critical for phase-space simulations. GenDiL supports phase-space adaptivity in both mesh size and polynomial order (hp-adaptivity) to place resolution only where it is needed. A central capability is Local Dimensional Refinement (LDR), which couples lower-dimension continuum models to higher-dimension kinetic models through stable and conservative interfaces, so that high-fidelity physics is applied only in regions where it is essential. Building on the GenDiL framework, we developed the General SN (GSN) family of algorithms as a true generalization of the polar SN approach (discrete ordinates, often denoted SN). Rather than tying discrete ordinates to a specific polar change of coordinates, GSN formulates transport on an arbitrary change of coordinates chosen to reduce ray-effect. We studied two complementary variants: an analytic variant, where the coordinate map is prescribed in advance by a closed-form function; and a data-driven variant, where a quantity of interest, such as the net flux, guides the coordinate system. GenDiL provides the library infrastructure for efficient GPU execution, but the GSN concept is algorithmic and independent of any one library. Across representative high-dimension tests, including non-symmetric solutions, both variants delivered strong ray-effect mitigation at practical cost, moving four- to six-dimensional analysis toward repeatable, routine studies.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Hybrid Electric Drivetrain Testing and Design (CRADA CRD-17-00699 Final Report)

Hybrid electric drivetrains have recently become of a great interest in the medium and heavy-duty vehicle market as it enables substantial reduction of petroleum use, vehicle level fuel use and criteria pollutant emissions. These vehicle performance improvements are not only economically beneficial for business operations relying on large fleets of vehicles, they are also paramount for curbing energy use and the negative effects of vehicle operations on the environment. Efficient Drivetrains, Inc. (EDI), acquired in July 2018 by Cummins, Inc., is a small company focused on development of medium and heavy-duty hybrid electric drivetrains. EDI has already developed the general hardware architecture of their drivetrain system, but there is still a significant effort to be done on optimizing the system in terms of control strategies and component sizing in order to maximize the benefits of the hybrid drivetrain. Historically, hybrid electric systems have proven to deliver better fuel economy in certain applications than their conventional counterparts. Particular areas of advantageous applications are vocations with kinetically intensive transient duty cycles and operations requiring some sort of power take off whose power demand is not well matched to the size of the vehicle’s main engine, thus forcing it to operate in extremely inefficient operating modes. Hybrid drivetrains can be at a disadvantage when pressed into duty cycle operation consisting of extensive steady state highway cruise due to various design compromises optimized for more transient operation. This can also lead to increase in vehicle emissions if the system is not optimized properly. Ample opportunity for extensive optimization is needed to overcome these obstacles.

33 ADVANCED PROPULSION SYSTEMS↗

Early Evaluation of Fugaku A64FX Architecture Using Climate Workloads

The Energy Exascale Earth System Model (E3SM) Project is an ongoing, state-of-the-science Earth system modeling, simulation, and prediction project that targets efficient utilization of U.S. Department of Energy’s (DOE) supercomputers to meet the science needs of the nation and the mission needs of DOE. This work focuses on our early evaluation of the A64FX architecture on Fugaku supercomputer using E3SM benchmarks. We will present results that track hardware trends, facilitate architecture comparison and the specific impact on our workload using an atmospheric model benchmark. We have two variants of the code written in Fortran and C++/Kokkos respectively which were used to collect data on a variety of CPU and GPU platforms. Furthermore, we have conducted a comparative evaluation of the compilers on the A64FX architecture and found GNU to be the best performer for our workload. Our experience so far indicates that Fugaku/A64FX shows promising energy efficiency (performance/Watt) with further performance gains possible through architecture-aware optimization efforts.

Sreepathi, Sarat↗

hls4ml: An Open-Source Codesign Workflow to Empower Scientific Low-Power Machine Learning Devices

Accessible machine learning algorithms, software, and diagnostic tools for energy-efficient devices and systems are extremely valuable across a broad range of application domains. In scientific domains, real-time near-sensor processing can drastically improve experimental design and accelerate scientific discoveries. To support domain scientists, we have developed hls4ml, an open-source software-hardware codesign workflow to interpret and translate machine learning algorithms for implementation with both FPGA and ASIC technologies. We expand on previous hls4ml work by extending capabilities and techniques towards low-power implementations and increased usability: new Python APIs, quantization-aware pruning, end-to-end FPGA workflows, long pipeline kernels for low power, and new device backends include an ASIC workflow. Taken together, these and continued efforts in hls4ml will arm a new generation of domain scientists with accessible, efficient, and powerful tools for machine-learning-accelerated discovery.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Microchannel-based Membrane-less Extraction of Li from Unconventional Lithium Sources & the Separation of REE

This final report provides an overview of the Project's entire duration, covering July 1, 2021 to December 31, 2023. It primarily focuses on the achievements, technological developments, and unique challenges the team faced while working on separating and extracting Lithium from produced waters. The project's primary aim was to create an integrated, high-throughput, membrane-less, and modular microfluidic platform that could extract Lithium from unconventional sources. We have successfully met all goals and milestones envisioned in the SOPO document. The most critical primary milestones, including the Go-No-Go milestone (refer to the Gantt chart in the Appendices), were successfully accomplished. We demonstrated phase separation (>90%) and extraction (>85%) performance in the MPSE using synthetic, and representative produced water composition feed at 50 ml/min total flow through MPSE 36. We have also performed a parametric study of the MPSE operations, beyond the scope of SOPO, exploring operating conditions of current and broader interest. The extended investigation of operational parameters is concurrent with our efforts to seek further development of the MPSE technology beyond the scope of the Project. Along these lines of development, we have made efforts to be responsive to DOE calls for technological developments of other types of resources (beyond PW) for the recovery of Critical Materials and higher TRL development (beyond TRL 4). During the work on this Project, we developed and implemented three innovative technical approaches that emerged from our efforts to successfully meet the Project milestones. The innovative & original technical approaches developed and implemented in this Project are now the contributions to process engineering that could be clearly credited to the Project. First, Convergent Design Approach is a comprehensive feedforward & feedback loop of four design phases: i) design for functionality, ii) design for manufacturing, iii) design for sustainability, and iv) design for market. Next was Process Intensification. A major aim of this Project was to create an innovative phase separation & extraction microscale-based technology for Li separation – thus the words microchannel-based in the Project title. A microscale-based technology is intrinsically in the center of the Process Intensification domain as defined by its unique principles. Therefore, Process Intensification was implicitly envisioned in the Project’s SOPO. Lastly, Time Scale Analysis is a novel tool for discovering the needs and directions of Process Intensification implementations in any process technology. This Project is fully credited for developing and implementing the three novel technical approaches mentioned above. These are general contributions to process engineering that emerged from this Project. Beyond the original SOPO scope, the OSU-U.Pitt research group utilized a Convergent Design methodology, integrating first-principles mathematical modeling with experimental validation on the Minimum Development Vehicle. By creating these Digital Twins, the team rapidly assessed manufacturing iterations to support TEA analysis. This framework further enabled the development of advanced Surface Modification Techniques, where hydrophobic and oleophobic coating strategies were optimized via Digital Twin tools and validated through rigorous 100-hour longevity testing. TEA Analysis: The closing efforts of this Project were focused on the TEA analysis. TEA analysis had two primary functions: i) enabling critical assessments of design variations withing 10 the Concurrent Design Approach, thus enabling evolution of the MPSE design to reach faster- better-cheaper alternatives; and ii) to create a bridge between the accomplishments of this Project and future projects of higher TRL, beyond TRL 6 level. It is important to note that the TEA model created in the Project stirred the technological solutions for the recovery of critical materials toward a vision of a very profitable modular plant that has unique zero-waste water discharge signature. More importantly, thanks to our experimental performance data and conservative assumptions, the TEA model predicts minimal technological and investment risks. Low cost of a modular unit of a nominal capacity of [1000 tons of Li 2 CO 3 /year] positions the MPSE based technology within the reach of community investors, thus offering a paradigm shift in the development of critical technologies. The project successfully navigated two primary challenges: solvent selection and manufacturing adaptation. Restricted by the SOPO to existing literature for lithium recovery, the team identified a critical need for a "material excellence program" to develop next-generation solvents, eventually concluding with a preliminary investigation into promising Ionic Liquids (ILs). Simultaneously, COVID-19 supply chain disruptions forced a pivot from traditional manufacturing to advanced additive methods at ATAMI-OSU. By transitioning from stainless steel to 3D-printed polymer substrates, the team achieved a transformative three-order-of- magnitude reduction in manufacturing costs and compressed prototyping timelines from several months to just two days. The MPSE technology offers significant energy, environmental, and economic advantages by overcoming the traditional bottlenecks of phase-separation hardware and contactor size. Unlike conventional mixer-settlers or membrane-based systems, MPSE operates without moving parts or fouling-prone membranes, achieving robust performance even with challenging, viscous, or particulate-heavy feeds. Key performance metrics include an energy intensity reduction of 5–50x (3–40 kJ/m 3 ) compared to incumbent technologies and a dramatic reduction of processing time to under 60 seconds, which drastically reduces the physical plant footprint. These technical efficiencies translate into superior economic outcomes; for a 100 t/year Li 2 CO 3 facility, implementing MPSE is projected to nearly halve contactor CAPEX (from $\$$6.08M to $\$$3.01M) and significantly increase the project's Net Present Value (NPV), derisking new investment and enabling distributed critical-mineral processing configurations. The commercialization of MPSE technology is being spearheaded by Vigsur Dynamics Inc., which has adopted a structured, parallel approach to technical and business development since its formation in January 2026. Following extensive customer discovery and engagement with the Oregon State University accelerator, Vigsur Dynamics is working to establish a business model that transitions from pilot demonstrations to modular hardware sales, ultimately aiming for a "build-own-operate" service strategy. Current technical milestones—including 100 hours of continuous operation, superior energy efficiency, and successful 6-unit modular scale-up— provide a foundation for this transition. Backed by ongoing IP licensing and a growing network of industrial and venture advisors, the company is actively de-risking the platform to replace conventional mixer-settler systems in the critical minerals market.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Early experiences evaluating the HPE/Cray ecosystem for AMD GPUs

Summary The Oak Ridge Leadership Computing Facility (OLCF) has a long history of supporting and promoting GPU‐accelerated computing starting with the deployment of the Titan supercomputer in 2021 and continuing with the Summit supercomputer which has a theoretical peak performance of approximately 200 petaflops. Because the majority of Summit's computational power comes from its 27,972 GPUs, users must port their applications to one of the supported programming models in order to make efficient use of the system. To prepare the transition to Frontier, the OLCF's exascale supercomputer, users will need to adapt to an entirely new ecosystem which will include new hardware and software technologies. First, users will need to familiarize themselves with the AMD Radeon GPU architecture. Furthermore, users who have been previously relying on CUDA will need to transition to the Heterogeneous‐Computing Interface for Portability (HIP) or one of the other supported programming models (e.g., OpenMP, OpenACC). In this work, we describe our initial experiences and lessons learned in porting three applications or proxy apps currently running on Summit to the HPE/Cray ecosystem to leverage the compute power from AMD GPUs: minisweep, GenASiS, and Sparkler. Each one is representative of current production workloads utilized at the OLCF, different programming languages, and different programming models.

Melesse Vergara, Verónica G.↗

Robotics for HVAC applications: A critical review and future perspectives

Recent advances in artificial intelligence (AI), enhanced computational capabilities, and innovations in sensors and hardware have driven the increasing development and application of robots in heating, ventilation, and air conditioning (HVAC) systems. We selected and reviewed 101 studies published between 2005 and 2025, sourced from IEEE Xplore, Scopus, Web of Science, and the ACM Digital Library. To analyze these works, we developed a five-dimensional analytical framework (morphology, sensing, navigation, task execution, and system integration), inspired by the Springer Handbook of Robotics and tailored specifically for robotic applications in HVAC. Based on the reviewed studies, six distinct tasks spanning the entire HVAC lifecycle have been identified. Among the six tasks, inspection and maintenance dominate (59 %), followed by indoor monitoring and auditing (21 %), whereas leakage detection, comfort support, and installation/retrofit remain less explored. To address the identified gaps, this review proposes future research directions including investigating robot-aware HVAC design principles, developing multimodal HVAC sensing and data fusion techniques, enhancing robot training and hardware capabilities, and expanding robotic applications beyond Maintenance and Operations (M&O). The findings from this review inform future robotics research for HVAC applications and ultimately enhance system affordability, energy efficiency, resilience or reliability, and occupant environmental comfort. Moreover, it seeks to inspire researchers to explore the intersections of robotics, computer science, building science, and HVAC engineering fostering advancements in this multidisciplinary field.

AI↗

Towards Automatic and Agile AI/ML Accelerator Design with End-to-End Synthesis

Domain-specific designs offer greater energy efficiency and performance gain than general-purpose processors. For this reason, modern system-on-chips have a significant portion of their silicon area with custom accelerators. However, designing hardware by hand is laborious and time-consuming, given the large design space and the performance, power, and area constraints that are not realized in the software. Moreover, domain-specific algorithms (e.g., machine learning models) are evolving quickly, challenging the accelerator design further. To address these issues, this paper presents SODA Synthesizer, an automated open-source high-level ML framework to Verilog modular compiler targeting AI/ML Application-Specific Integrated Circuits (ASICs) accelerators. SODA tightly couples the Multi- Level Intermediate Representation (MLIR) compiler infrastructure [24] and open-source HLS approaches. Thus, SODA can support various ML frameworks and algorithms and can perform optimizations that combine specialized architecture templates and conventional HLS to generate the hardware modules. In addition, SODA’s closed-loop design space exploration (DSE) engine allows developers to perform end-to-end design space explorations on different metrics and technology nodes.

Zhang, Jeff↗

Development of a water source heat pump hardware-in-the-loop (HIL) testing facility for smart building applications

Over the last decade, the global fight against climate change through electrification has led to an increase in research on building heating, ventilation, and air conditioning (HVAC) systems that utilize intelligent control algorithms to provide demand-side grid service while maintaining the thermal comfort of building occupants. As the pivotalpoint between building electricity consumption and indoor thermal comfort, high-efficiency electrical heatpumps are at the center of these emerging studies, and various grid-interactive and occupant-comfort control algorithms have been developed for them. The impact of these algorithms on the heatpump operation andperformance under different weather, building load, and grid requests calls for investigation and verification via experimental tests with actual heat pumps integrated with real-time building and grid responses. This study presents a Water-Source Heat Pump Hardware-in-The-Loop (HIL) Test Facility developed with the capability to perform such tests. The hardware configuration for this testfacility introduces a hydronic system that emulates the conditions for the heat pump water-side, and a duct system that emulates conditions for the heat pump airside. Both data acquisition and emulator control are implemented through the National Instruments (NI) LabVIEW software running on an NI PXIplatform. The HIL mechanism based on the hardware-software integration that allows the testbed to communicate with a generic simulation environment is also discussed. Currently, the test facility setup includes a single heatpump and virtual building model in EnergyPlus coupled with an occupant behavioral model in MATLAB. Preliminary test results of the current setup demonstrate the building load emulator's ability to track the simulated gone temperature with a Root Mean Square Deviation (RSME) below 0.12°C (0.216°F). An uncertainty analysis based on sensor accuracies shows that the heat pump coefficient of performance (COP) can be measured with a relative uncertainty of 10.4% in cooling and 3.7% in heating. Apartfrom the current testing on a single heat pump, the test facility also provides the flexibility to include additional heat pumps to form a heat pump cluster, as well as coupling the heat pump with active thermal storage to provide enhanced demandflexibility.

Calfa, Caleb↗

IoT-Based Comfort Control and Fault Diagnostics System for Energy-Efficient Homes

This project studies an Internet of Things (IoT)-based comfort control and fault diagnostics system (referred as iComfort in this report) for energy-efficient homes. The system delivers an occupant-comfort-oriented thermal environment adaptive to fault scenarios and achieves HVAC energy savings in a cost-effective and straightforward way. This smart iComfort home system consists of the following key features. 1) Cost-effectiveness and scalability of the entire hardware and software system: The system includes low-cost temperature, humidity, and airflow sensors, and a Raspberry Pi-based local hub that interfaces with the cloud and IoT-enabled devices. The cost is low, not only for sensors, but also the costs associated with sensor installation, system setup and commissioning, data communication and storage, and data analytics (e.g., the development of automated fault detection and diagnosis (AFDD), as well as adaptive control strategies that are both computationally efficient and practical to implement). 2) Energy performance and user satisfaction: The system delivers user satisfaction and energy savings. This includes a) ease of use, b) optimal occupant thermal comfort, and c) accurate system feedback (e.g., low false alarm of AFDD strategies). 3) Favorable demonstrated prototype performance: The prototype tested at the Pacific Northwest National Laboratory (PNNL) Lab Homes demonstrates the accuracy of fault detections and diagnoses and shows thermal comfort improvement and energy savings through adaptive and optimal HVAC operations.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Synthetic-domain computing and neural networks using lithium niobate integrated nonlinear phononics

Analogue computing uses the physical behaviours of devices to provide energy-efficient arithmetic operations. However, scaling up analogue computing platforms by simply increasing the number of devices leads to challenges such as device-to-device variation. Here, in this study, we report scalable analogue computing and neural networks in the synthetic frequency domain using an integrated nonlinear phononic platform on lithium niobate. This synthetic-domain computing is robust to device variations, as vectors and matrices are concurrently encoded at different frequencies within a single device, achieving a high throughput per area. Leveraging inherent nonlinearities, our device-aware neural network can perform a four-class classification task with an accuracy of 98.2%. The nonlinear phononic computing hardware also maintains consistent performance over a wide operational temperature range (characterized up to 192 °C). Our synthetic-domain computing combines single-device parallelism, inherent nonlinearity and environmental stability, and could be of use in edge computing applications in which power efficiency and environmental resilience are crucial.

Ji, Jun [Virginia Polytechnic Inst. and State Univ↗

Tunable stochastic memristors for energy-efficient encryption and computing

Information security and computing, two critical technological challenges for post-digital computation, pose opposing requirements – security (encryption) requires a source of unpredictability, while computing generally requires predictability. Each of these contrasting requirements presently necessitates distinct conventional Si-based hardware units with power-hungry overheads. This work demonstrates Cu 0.3 Te 0.7 /HfO 2 (‘CuTeHO’) ion-migration-driven memristors that satisfy the contrasting requirements. Under specific operating biases, CuTeHO memristors generate truly random and physically unclonable functions, while under other biases, they perform universal Boolean logic. Using these computing primitives, this work experimentally demonstrates a single system that performs cryptographic key generation, universal Boolean logic operations, and encryption/decryption. Circuit-based calculations reveal the energy and latency advantages of the CuTeHO memristors in these operations. This work illustrates the functional flexibility of memristors in implementing operations with varying component-level requirements.

97 MATHEMATICS AND COMPUTING↗

Emerging applications: Neuromorphic computing and reservoir computing

The emergence of doped hafnium oxide (HfO 2 )-based ferroelectric films has enabled highly scalable and silicon-compatible ferroelectric devices, opening new frontiers in neuromorphic and reservoir computing. Among these, ferroelectric field-effect transistors (FeFETs) are particularly promising due to their analog memory characteristics and unique polarization dynamics. These properties make FeFETs ideal candidates for artificial synapses in neuromorphic architectures, supporting deep neural networks and spiking neural networks based on leaky-integrate-and-fire (LIF) mechanisms. Beyond neuromorphic computing, FeFETs also play a crucial role in physical reservoir computing, leveraging their intrinsic nonlinear and history-dependent behavior for efficient real-time learning. This approach offers significant advantages for time-series processing and edge artificial intelligence (AI) applications, addressing the growing need for energy-efficient computing. As a result, this article explores the principles, key demonstrations, and future potential of FeFET-based neuromorphic and reservoir computing, highlighting their impact on next-generation AI hardware.

36 MATERIALS SCIENCE↗

How to Build a Quantum Supercomputer: Scaling from Hundreds to Millions of Qubits

In the span of four decades, quantum computation has evolved from an intellectual curiosity to a potentially realizable technology. Today, small-scale demonstrations have become possible for quantum algorithmic primitives on hundreds of physical qubits and proof-of-principle error-correction on a single logical qubit. Nevertheless, despite significant progress and excitement, the path toward a full-stack scalable technology is largely unknown. There are significant outstanding quantum hardware, fabrication, software architecture, and algorithmic challenges that are either unresolved or overlooked. These issues could seriously undermine the arrival of utility-scale quantum computers for the foreseeable future. Here, we provide a comprehensive review of these scaling challenges. We show how the road to scaling could be paved by adopting existing semiconductor technology to build much higher-quality qubits, employing system engineering approaches, and performing distributed quantum computation within heterogeneous high-performance computing infrastructures. These opportunities for research and development could unlock certain promising applications, in particular, efficient quantum simulation/learning of quantum data generated by natural or engineered quantum systems. To estimate the true cost of such promises, we provide a detailed resource and sensitivity analysis for classically hard quantum chemistry calculations on surface-code error-corrected quantum computers given current, target, and desired hardware specifications based on superconducting qubits, accounting for a realistic distribution of errors. Furthermore, we argue that, to tackle industry-scale classical optimization and machine learning problems in a cost-effective manner, heterogeneous quantum-probabilistic computing with custom-designed accelerators should be considered as a complementary path toward scalability.

Mohseni, Masoud↗

Automated, reliable, and efficient continental-scale replication of 7.3 petabytes of computational simulation data: A case study

We report on our experiences replicating 7.3 petabytes (PB) of Earth System Grid Federation (ESGF) computational simulation data from Lawrence Livermore National Laboratory (LLNL) in California to Argonne National Laboratory (ANL) in Illinois and Oak Ridge National Laboratory (ORNL) in Tennessee—a task motivated by a need for increased reliability, capacity, and performance. This task presented significant challenges: the need to move 29 million files twice under time pressure from aging storage hardware; a source file system bottleneck limiting throughput to 1.5 GB/s; frequent site maintenance windows; and the need for complete reliability at scale. We addressed these challenges using a simple replication tool that invoked Globus to transfer large bundles of files while tracking progress in a database, dynamically rerouting transfers to work around maintenance periods and file system limitations. Under the covers, Globus organized transfers to make efficient use of the high-speed Energy Sciences network (ESnet) and the data transfer nodes deployed at participating sites, and also addressed security, integrity checking, and recovery from a variety of transient failures. This success demonstrates the considerable benefits that can accrue from the adoption of performant data replication infrastructure. The replication tool is available at https://github.com/esgf2-us/data-replication-tools.

Globus↗

A machine learning approach to galaxy properties: joint redshift–stellar mass probability distributions with Random Forest

We demonstrate that highly accurate joint redshift–stellar mass probability distribution functions (PDFs) can be obtained using the Random Forest (RF) machine learning (ML) algorithm, even with few photometric bands available. As an example, we use the Dark Energy Survey (DES), combined with the COSMOS2015 catalogue for redshifts and stellar masses. We build two ML models: one containing deep photometry in the griz bands, and the second reflecting the photometric scatter present in the main DES survey, with carefully constructed representative training data in each case. We validate our joint PDFs for 10 699 test galaxies by utilizing the copula probability integral transform and the Kendall distribution function, and their univariate counterparts to validate the marginals. Benchmarked against a basic set-up of the template-fitting code bagpipes, our ML-based method outperforms template fitting on all of our predefined performance metrics. In addition to accuracy, the RF is extremely fast, able to compute joint PDFs for a million galaxies in just under 6 min with consumer computer hardware. Such speed enables PDFs to be derived in real time within analysis codes, solving potential storage issues. As part of this work we have developed galpro 1, a highly intuitive and efficient python package to rapidly generate multivariate PDFs on-the-fly. galpro is documented and available for researchers to use in their cosmology and galaxy evolution studies.

79 ASTRONOMY AND ASTROPHYSICS↗

Dual Channel Dual Staging: Hierarchical and Portable Staging for GPU-Based In-Situ Workflow

In-situ workflows have emerged as an attractive approach for addressing data movement challenges at very large scales. Since GPU-based architectures dominate the HPC landscapes, porting these in-situ workflows, and, specifically, the inter-application data exchange, to GPU-based systems can be challenging. Technologies such as GPUDirect RDMA (GDR), which is typically used for I/O in GPU applications as an optimization that circumvents the CPU overhead, can be leveraged to support bulk data exchanges between GPU applications. However, current GDR design often lacks performance portability across HPC clusters built with different hardware configurations. Furthermore, the local CPU may also be effectively used as an auxiliary communication mechanism to offload data exchanges. In this paper, we present a dual channel dual staging approach for efficient, scalable, and performance-portable inter-application data exchange for in-situ workflows. This approach exploits the data access pattern within in-situ workflows along with the inherent execution asynchrony to accelerate data exchanges and, at the same time, improve performance portability. Specifically, the dual channel dual staging method leverages both the local CPU and the remote data staging server to build a hierarchical joint staging area and uses this staging area to transform blocking inter-application bulk data exchanges into best-effort local data movements between GPU and CPU. The dual channel dual staging is implemented as a portability extension of the Dataspaces-GPU staging framework. We present an experimental evaluation of its performance, portability, and scalability using this implementation on three leadership GPU clusters. The evaluation results demonstrate that the dual channel dual staging method saves up to 75% in data-exchange time compared to host-based, GDR, and alternate portable designs, while maintaining scalability (up to 512 GPUs) and performance portability across the three platforms.

Zhang, Bo [University of Utah]↗