Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “scalable computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,441 records · Page 80
All sourcesNASA NTRSDOE OSTI
DOE OSTI2016

Scalable Analysis Methods and In Situ Infrastructure for Extreme Scale Knowledge Discovery

The primary challenge motivating this team’s work is the widening gap between the ability to compute information and to store it for subsequent analysis. This gap adversely impacts science code teams, who are able to perform analysis only on a small fraction of the data they compute, resulting in the very real likelihood of lost or missed science, when results are computed but not analyzed. Our approach is to perform as much analysis or visualization processing on data while it is still resident in memory, an approach that is known as in situ processing. The idea in situ processing was not new at the time of the start of this effort in 2014, but efforts in that space were largely ad hoc, and there was no concerted effort within the research community that aimed to foster production-quality software tools suitable for use by DOE science projects. In large, our objective was produce and enable use of production-quality in situ methods and infrastructure, at scale, on DOE HPC facilities, though we expected to have impact beyond DOE due to the widespread nature of the challenges, which affect virtually all large-scale computational science efforts. To achieve that objective, we assembled a unique team of researchers consisting of representatives from DOE national laboratories, academia, and industry, and engaged in software technology R&D, as well as engaged in close partnerships with DOE science code teams, to produce software technologies that were shown to run effectively at scale on DOE HPC platforms.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Scalable Heterogeneous Execution of a Coupled-Cluster Model with Perturbative Triples

The CCSD(T) coupled-cluster model with perturbative triples is considered a gold standard for computational modeling of the correlated behavior of electrons in molecular systems. A fundamental constraint is the relatively small global-memory capacity in GPUs compared to the main-memory capacity on host nodes, necessitating relatively smaller tile sizes for high-dimensional tensor contractions in NWChem's GPU-accelerated implementation of the CCSD(T) method. A coordinated redesign is described to address this limitation and associated data movement overheads, including a novel fused GPU kernel for a set of tensor contractions, along with inter-node communication optimization and data caching. The new implementation of GPU-accelerated CCSD(T) improves overall performance by 3.4x. Finally, we discuss the trade-offs in using this fused algorithm on current and future supercomputing platforms.

Kim, Jinsung↗
DOE OSTI2012

Warewulf 3.0

Warewulf is a scalable systems management suite originally developed to manage large high-performance Linux clusters. Focused on general scalable systems management, it includes a framework for system configuration, management, provisioning/installation, monitoring, event notification, and more via a modular plugin architecture. Install the components and features you need or leverage the existing system configuration stored within Warewulf to create custom solutions to meet your performance needs. One of the goals of the project is to grow the project by allowing other organizations and developers to contribute back to the project under an open source BSD license. We have been successful with gaining support within the open source community already and have contributors and support in not only the government and academic space but also the commercial sector (e.g. Intel). Being both open source and vendor neutral puts Warewulf in a unique position to be supported and utilized as an open standard in High Performance Computing (HPC). Warewulf is an active member of various HPC initiatives and groups and is considered one of the most utilized, extensible and scalable clustering toolkits available.

Kurtzer, GregoryM.↗
NASA NTRS1996

An Open Computing Infrastructure that Facilitates Integrated Product and Process Development from a Decision-Based Perspective

Computer applications for design have evolved rapidly over the past several decades, and significant payoffs are being achieved by organizations through reductions in design cycle times. These applications are overwhelmed by the requirements imposed during complex, open engineering systems design. Organizations are faced with a number of different methodologies, numerous legacy disciplinary tools, and a very large amount of data. Yet they are also faced with few interdisciplinary tools for design collaboration or methods for achieving the revolutionary product designs required to maintain a competitive advantage in the future. These organizations are looking for a software infrastructure that integrates current corporate design practices with newer simulation and solution techniques. Such an infrastructure must be robust to changes in both corporate needs and enabling technologies. In addition, this infrastructure must be user-friendly, modular and scalable. This need is the motivation for the research described in this dissertation. The research is focused on the development of an open computing infrastructure that facilitates product and process design. In addition, this research explicitly deals with human interactions during design through a model that focuses on the role of a designer as that of decision-maker. The research perspective here is taken from that of design as a discipline with a focus on Decision-Based Design, Theory of Languages, Information Science, and Integration Technology. Given this background, a Model of IPPD is developed and implemented along the lines of a traditional experimental procedure: with the steps of establishing context, formalizing a theory, building an apparatus, conducting an experiment, reviewing results, and providing recommendations. Based on this Model, Design Processes and Specification can be explored in a structured and implementable architecture. An architecture for exploring design called DREAMS (Developing Robust Engineering Analysis Models and Specifications) has been developed which supports the activities of both meta-design and actual design execution. This is accomplished through a systematic process which is comprised of the stages of Formulation, Translation, and Evaluation. During this process, elements from a Design Specification are integrated into Design Processes. In addition, a software infrastructure was developed and is called IMAGE (Intelligent Multidisciplinary Aircraft Generation Environment). This represents a virtual apparatus in the Design Experiment conducted in this research. IMAGE is an innovative architecture because it explicitly supports design-related activities. This is accomplished through a GUI driven and Agent-based implementation of DREAMS. A HSCT design has been adopted from the Framework for Interdisciplinary Design Optimization (FIDO) and is implemented in IMAGE. This problem shows how Design Processes and Specification interact in a design system. In addition, the problem utilizes two different solution models concurrently: optimal and satisfying. The satisfying model allows for more design flexibility and allows a designer to maintain design freedom. As a result of following this experimental procedure, this infrastructure is an open system that it is robust to changes in both corporate needs and computer technologies. The development of this infrastructure leads to a number of significant intellectual contributions: 1) A new approach to implementing IPPD with the aid of a computer; 2) A formal Design Experiment; 3) A combined Process and Specification architecture that is language-based; 4) An infrastructure for exploring design; 5) An integration strategy for implementing computer resources; and 6) A seamless modeling language. The need for these contributions is emphasized by the demand by industry and government agencies for the development of these technologies.

Hale, Mark A.↗
NASA NTRS2000

Testing New Programming Paradigms with NAS Parallel Benchmarks

Over the past decade, high performance computing has evolved rapidly, not only in hardware architectures but also with increasing complexity of real applications. Technologies have been developing to aim at scaling up to thousands of processors on both distributed and shared memory systems. Development of parallel programs on these computers is always a challenging task. Today, writing parallel programs with message passing (e.g. MPI) is the most popular way of achieving scalability and high performance. However, writing message passing programs is difficult and error prone. Recent years new effort has been made in defining new parallel programming paradigms. The best examples are: HPF (based on data parallelism) and OpenMP (based on shared memory parallelism). Both provide simple and clear extensions to sequential programs, thus greatly simplify the tedious tasks encountered in writing message passing programs. HPF is independent of memory hierarchy, however, due to the immaturity of compiler technology its performance is still questionable. Although use of parallel compiler directives is not new, OpenMP offers a portable solution in the shared-memory domain. Another important development involves the tremendous progress in the internet and its associated technology. Although still in its infancy, Java promisses portability in a heterogeneous environment and offers possibility to "compile once and run anywhere." In light of testing these new technologies, we implemented new parallel versions of the NAS Parallel Benchmarks (NPBs) with HPF and OpenMP directives, and extended the work with Java and Java-threads. The purpose of this study is to examine the effectiveness of alternative programming paradigms. NPBs consist of five kernels and three simulated applications that mimic the computation and data movement of large scale computational fluid dynamics (CFD) applications. We started with the serial version included in NPB2.3. Optimization of memory and cache usage was applied to several benchmarks, noticeably BT and SP, resulting in better sequential performance. In order to overcome the lack of an HPF performance model and guide the development of the HPF codes, we employed an empirical performance model for several primitives found in the benchmarks. We encountered a few limitations of HPF, such as lack of supporting the "REDISTRIBUTION" directive and no easy way to handle irregular computation. The parallelization with OpenMP directives was done at the outer-most loop level to achieve the largest granularity. The performance of six HPF and OpenMP benchmarks is compared with their MPI counterparts for the Class-A problem size in the figure in next page. These results were obtained on an SGI Origin2000 (195MHz) with MIPSpro-f77 compiler 7.2.1 for OpenMP and MPI codes and PGI pghpf-2.4.3 compiler with MPI interface for HPF programs.

Jin, H.↗
DOE OSTI2024

Intro to HPC Bootcamp: Engaging New Communities Through Energy Justice Projects

The U.S. Department of Energy (DOE) is a long-standing leader in research and development of high-performance computing (HPC) in the pursuit of science. However, we face daunting challenges in fostering a robust and diverse HPC workforce. Basic HPC is not typically taught at early stages of students' academic careers, and the capacity and knowledge of HPC at many institutions are limited. Even so, such topics are prerequisites for advanced training programs, internships, graduate school, and ultimately for careers in HPC. To help address this challenge, as part of the DOE Exascale Computing Project's Broadening Participation Initiative, we recently launched the Introduction to HPC Training and Workforce Pipeline Program to provide accessible introductory material on HPC, scalable AI, and analytics. We describe the Intro to HPC Bootcamp, an immersive program designed to engage students from underrepresented groups as they learn foundational HPC skills. Here, the program takes a novel approach to HPC training by turning the traditional curriculum upside down. Instead of focusing on technology and its applications, the bootcamp focuses on energy justice to motivate the training of HPC skills through project-based pedagogy and real-life science stories. Additionally, the bootcamp prepares students for internships and future careers at DOE labs. The first bootcamp, hosted by the advanced computing facilities at Argonne, Lawrence Berkeley, and Oak Ridge National Labs and organized by Sustainable Horizons Institute, took place in August 2023.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2013

A 2nd Generation Parallel Advancing Front Grid Generator

A scalable, parallel advancing grid generation technique has been developed for complex geometries and meshes with large size variations. The key innovation compared to previous techniques is the use of a domain-defining grid that has the same fine surface triangulation as the final mesh desired, but a much coarser interior mesh. In this way, the domain to be gridded is uniquely defined, overcoming a shortcoming of previous approaches. This domain-defining grid is then partitioned according to the estimated number of elements to be generated, allowing for a balanced distribution of work among the processors. The domain defining grid is also used to redistribute the elements and points after grid generation, and during the subsequent mesh improvement. Timings show that the proposed approach is scalable and able to produce large grids of high quality in a modest amount of clocktime. With the proposed parallel grid generator, a major impediment to a completely scalable simulation pipeline (grid generation, solvers, post-processing) has been removed, opening the way for truly large-scale computations using unstructured, body-fitted grids.

Grid Generation↗
DOE OSTI2023

Insights From Dayflow: A Historical Streamflow Reanalysis Dataset for the Conterminous United States

Abstract Reconstructed historical streamflow time series can supplement limited streamflow gauge observations. However, there are common challenges of typical modeling approaches: process‐based hydrologic models can be data/computation‐intensive, and statistics‐based models can be region/stream‐specific. Here we present a nationally scalable modeling framework integrating the simulated runoff from the Variable Infiltration Capacity (VIC) model with the Routing Application for Parallel computatIon of Discharge (RAPID) routing model leveraging high‐performance computing. We demonstrate an efficient method of assimilating streamflow at US Geological Survey (USGS) streamflow monitoring sites using a simple hierarchical approach in the VIC‐RAPID framework. The result is a reconstructed 36‐year (1980–2015) daily and monthly streamflow dataset (Dayflow) at ∼2.7 million NHDPlusV2 stream reaches in the conterminous US (CONUS). We perform a comprehensive evaluation at 7,526 USGS sites and characterize their error statistics. The results demonstrate that 49% of the USGS sites demonstrate Kling–Gupta Efficiency (KGE) > 0.5 and 58% of the sites show percentage bias within ±20% for the daily naturalized streamflow. Streamflow data assimilation across CONUS shows an overall improvement over naturalized streamflow, notably in the western semiarid‐to‐arid regions. Comparison to other national and global streamflow reanalysis datasets such as the National Water Model and Global Reach‐scale A priori Discharge Estimates for SWOT demonstrates improved KGE, reduced bias, and directions for Dayflow improvements. Investigations of error statistics with key hydrologic, hydroclimatic, and geomorphologic basin characteristics reveal region‐specific patterns which may help improve future framework applications. Overall, Dayflow may enable a better understanding of hydrologic conditions in a changing environment, especially in locations currently not represented by streamflow monitoring networks.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

RIViT-seq enables systematic identification of regulons of transcriptional machineries

Abstract Transcriptional regulation is a critical process to ensure expression of genes necessary for growth and survival in diverse environments. Transcription is mediated by multiple transcription factors including activators, repressors and sigma factors. Accurate computational prediction of the regulon of target genes for transcription factors is difficult and experimental identification is laborious and not scalable. Here, we demonstrate regulon identification by in vitro transcription-sequencing (RIViT-seq) that enables systematic identification of regulons of transcription factors by combining an in vitro transcription assay and RNA-sequencing. Using this technology, target genes of 11 sigma factors were identified in Streptomyces coelicolor A3(2). The RIViT-seq data expands the transcriptional regulatory network in this bacterium, discovering regulatory cascades and crosstalk between sigma factors. Implementation of RIViT-seq with other transcription factors and in other organisms will improve our understanding of transcriptional regulatory networks across biology.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2014

Coupled-cluster representation of Green function employing modified spectral resolutions of similarity transformed Hamiltonians

In this paper we discuss a new formalism for producing an analytic coupled-cluster (CC) Green's function for an N-electron system by shifting the poles of similarity transformed Hamiltonians represented in N − 1 and N + 1 electron Hilbert spaces. Simple criteria are derived for the states in N − 1 and N + 1 electron spaces that are then corrected in the spectral resolution of the corresponding matrix representations of the similarity transformed Hamiltonian. The accurate description of excited state processes within a Green's function formalism would be of significant importance to a number of scientific communities ranging from physics and chemistry to engineering and the biological sciences. This is because the Green's function methodology provides a direct path for not only calculating properties whose underlying origins come from coupled many-body interactions but also provides a straightforward path for calculating electron transport, response, and correlation functions that allows for a direct link with experiment. As a special case of this general formulation, we discuss the application of this technique for Green's function defined by the CC with singles and doubles representation of the ground-state wave function.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Evaluating probabilistic deep learning methods for uncertainty quantification of temperature downscaling

Deep learning (DL) has emerged as a promising tool for downscaling coarse-resolution climate data to high-resolution outputs, enabling improved regional climate predictions. A critical aspect of DL-based downscaling is the incorporation of uncertainty quantification (UQ), which enhances the interpretability and reliability of predictions—key factors for climate risk assessment and decision-making. This study develops a DL model to downscale 2 m temperature across the contiguous United States using reanalysis datasets. We systematically evaluate three epistemic UQ methods—deep ensembles (DEns), Monte Carlo dropout (MCD), and Flipout—based on their probabilistic accuracy, downscaling performance, sensitivity to geographical features, and computational efficiency. Results indicate that MCD generally outperforms Flipout and DEns in terms of calibration and downscaling accuracy. However, DEns demonstrate lower calibration errors in coastal regions, indicating its higher confidence within these areas. Flipout, in contrast, is more sensitive to elevation gradients and exhibits higher calibration errors in mountainous regions. Hence, the choice of UQ method for this task depends on the specific requirements of the application. For applications that prioritize overall calibration, downscaling accuracy, and computational efficiency, MCD is a strong candidate. These findings highlight the importance of selecting UQ methods based on application-specific requirements, such as geographical context and computational constraints. By addressing the trade-offs between UQ methods, this study provides actionable insights for improving the reliability, scalability, and utility of DL-based downscaling in climate science.

Environmental sciences↗
DOE OSTI2020

A Generic High Bandwidth Data Acquisition Card for Physics Experiments

In high energy physics and nuclear physics experiments particularly the ones based on particle accelerator, the data rate from the detector is usually in the order of Terabytes per second. This high throughput data from detector front-end electronics need to be transmitted to the back-end computing farm for high level event selection and building. A Data Acquisition (DAQ) system with features of high-density, scalable, easily upgradeable is crucial to simplify the readout architecture of whole experiment. This paper will introduce the design of a generic high bandwidth PCIe card which can be used as the important input output card in a scalable DAQ system. It can factorize front-end electronics from data handling, and reduce amount of custom hardware in favor of scalable detectorindependent commercial hardware and software. Besides the 48 channels of bidirectional high speed fiber optical links with frontends, it also supports to synchronize with the experiment timing system, and to fanout the clock and trigger information with a fixed latency to the front-end electronics.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2017

Optimal net-load balancing in smart grids with high PV penetration

Mitigating mismatches in supply demand curve is critical for smooth power grid operation. Traditionally, load curtailment techniques such as Demand Response (DR) have been used for this purpose. However, smart grids with high PV penetration sometimes exhibit supply surplus causing over-voltages. Thus, load curtailment techniques cannot be the only component of a net-load balancing framework. Supply curtailment techniques such as Volt-Var Optimizations are complex and computationally expensive. This increases the complexity of net-load balancing systems used by the grid operator and limits their scalability. Recently, new technologies have been developed that enable the rapid and selective connection of PV modules of an installation to the grid. Taking advantage of these advancements, we develop a unified optimal net-load balancing framework which performs both load and solar curtailment. We show that when the available curtailment values are discrete, this problem is NP-hard and develop bounded approximation algorithms for minimizing the curtailment cost. Our algorithms produce fast solutions, given the tight timing constraints required for grid operation. We also incorporate the notion of fairness to ensure that curtailment is evenly distributed among all the nodes. Finally, we develop an online algorithm which performs net-load balancing using only data available for the current interval. Using both theoretical analysis and practical evaluations, we show that our net-load balancing algorithms provide solutions which are close to optimal in a small amount of time.

14 SOLAR ENERGY↗
DOE OSTI2025

Data Readiness for Scientific AI at Scale

This paper examines how Data Readiness for AI (DRAI) principles apply to leadership-scale scientific datasets used to train foundation models. We analyze archetypal workflows across four representative domains—climate, nuclear fusion, bio/health, and materials—to identify common preprocessing patterns and domain-specific constraints. We introduce a two-dimensional readiness framework that combines canonical preprocessing patterns with a five-level operational readiness scale, both tailored to high-performance computing (HPC) environments. This framework helps outline key challenges in transforming large-scale scientific data into formats suitable for scalable AI training. Together, these dimensions form a conceptual maturity matrix that characterizes scientific data readiness and guides infrastructure development toward standardized, cross-domain support for scalable and reproducible AI for science.

Brewer, Wes [ORNL] (ORCID:0000000236393956)↗
DOE OSTI2023

Dayflow: CONUS Daily Streamflow Reanalysis, Version 2 (DayflowV2)

The DayflowV2 dataset provides multiple meteorologic forcings driven hourly streamflow information for approximately 2.7 million NHDPlusV2 stream reaches in the conterminous US (CONUS). DaymetV4, Stage-IV, and Analysis of Period of Record for Calibration (AORC) forcings and their corresponding hybrids drive a nationally scalable modeling framework integrating the simulated runoff from the Variable Infiltration Capacity (VIC) model with the Routing Application for Parallel computatIon of Discharge (RAPID) routing model. Streamflow with (Assimilated) and without (Naturalized) streamflow assimilation at US Geological Survey (USGS) streamflow monitoring sites are included in DayflowV2. A comprehensive evaluation of streamflow at 7,526 USGS gauges is performed for both streamflow types. The resulting key evaluation metrics are also included in the Dayflow dataset. The reanalysis data are available for variable periods; 36 years (1980-2015) for DaymetV4 (DayflowV1), 18 years (2002-2019) for Stage-IV and its hybrids, and 40 years (1980-2019) for AORC and its hybrids.

13 HYDRO ENERGY↗
DOE OSTI2016

Extreme-scale Algorithms and Solver Resilience

A widening gap exists between the peak performance of high-performance computers and the performance achieved by complex applications running on these platforms. Over the next decade, extreme-scale systems will present major new challenges to algorithm development that could amplify this mismatch in such a way that it prevents the productive use of future DOE Leadership computers due to the following; Extreme levels of parallelism due to multicore processors; An increase in system fault rates requiring algorithms to be resilient beyond just checkpoint/restart; Complex memory hierarchies and costly data movement in both energy and performance; Heterogeneous system architectures (mixing CPUs, GPUs, etc.); and Conflicting goals of performance, resilience, and power requirements.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Advancing Sustainable Ammonia Synthesis through Plasma-Assisted Catalysis (Final Report)

The objective of this project was to perform systematic plasma catalysis experiments supported by computational models that capture the molecular-scale physics and chemistry to use plasma-assisted catalysis as an alternative, scalable means to achieve the sustainable synthesis of ammonia. We carefully controlled the plasma properties and coupled the reactive plasma environment with appropriate catalysts to generate non-thermal intermediates and open surface kinetic pathways at low temperature (< 200°C) and ambient pressure to facilitate high ammonia production rates.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗