SEARCH · Engineering Papers

Results for “Computational pipelines”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17

All sources NASA NTRS DOE OSTI

DOE OSTI2020

Supercomputing Pipelines Search for Therapeutics Against COVID-19

The urgent search for drugs to combat SARS-CoV-2 has included the use of supercomputers. The use of general-purpose graphical processing units (GPUs), massive parallelism, and new software for high-performance computing (HPC) has allowed researchers to search the vast chemical space of potential drugs faster than ever before. We developed a new drug discovery pipeline using the Summit supercomputer at Oak Ridge National Laboratory to help pioneer this effort, with new platforms that incorporate GPU-accelerated simulation and allow for the virtual screening of billions of potential drug compounds in days compared to weeks or months for their ability to inhibit SARS-COV-2 proteins. Here, this effort will accelerate the process of developing drugs to combat the current COVID-19 pandemic and other diseases.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2024

An Active Learning-Based Streaming Pipeline for Reduced Data Training of Structure Finding Models in Neutron Diffractometry

Structure determination workloads in neutron diffractometry are computationally expensive and routinely require several hours to many days to determine the structure of a material from its neutron diffraction patterns. The potential for machine learning models trained on simulated neutron scattering patterns to significantly speed up these tasks have been reported recently. However, the amount of simulated data needed to train these models grows exponentially with the number of structural parameters to be predicted and poses a significant computational challenge. To overcome this challenge, we introduce a novel batch-mode active learning (AL) policy that uses uncertainty sampling to simulate training data drawn from a probability distribution that prefers labelled examples about which the model is least certain. We confirm its efficacy in training the same models with ∼ 75% less training data while improving the accuracy. We then discuss the design of an efficient stream-based training workflow that uses this AL policy and present a performance study on two heterogeneous platforms to demonstrate that, compared with a conventional training workflow, the streaming workflow delivers ∼ 20% shorter training time without any loss of accuracy.

Wang, Tianle [Brookhaven National Laboratory (BNL)↗

DOE OSTI2021

CryoFold: Determining protein structures and data-guided ensembles from cryo-EM density maps

Cryoelectron microscopy requires molecular modeling for refinement of structures. Ensemble models arrive at low free-energy molecular structures, but are computationally expensive and limited to resolving only small proteins. Here, we introduce CryoFold, a pipeline of molecular dynamics simulations that determines ensembles of protein structures by integrating density data of varying sparsity at 3–5 Å resolution with sequence information and coarse-grained topological knowledge of the protein folds. We present six examples, folding proteins between 72 and 2,000 residues, including large membrane and multi-domain systems, and results from two Electron Microscopy Data Bank (EMDB) competitions. Driven by data from a single state, CryoFold discovers ensembles of common low-energy models together with rare low-probability structures that capture the equilibrium distribution of proteins constrained by the density maps. Many of these conformations are experimentally validated and functionally relevant. We arrive at a set of best practices for data-guided protein folding that are controlled using a Python graphical user interface (GUI).

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2026

Improving 3D reconstruction quality for root phenotyping: assessing the impact of camera calibration and imaging parameters

Arate 3D reconstruction is essential for high-throughput plant phenotyping, particularly for studying complex structures such as root systems. While photogrammetry and Structure from Motion (SfM) techniques have become widely used for 3D root imaging, the camera settings used are often underreported in studies, and the impact of camera calibration on model accuracyccu remains largely underexplored in plant science. In this study, we systematically evaluate the effects of focus, aperture, exposure time, and gain settings on the quality of 3D root models made with a multi-camera scanning system. We show through a series of experiments that calibration significantly improves model quality, with focus misalignment and shallow depth of field (DoF) being the most important factors affecting reconstruction accuracy. Our results further show that proper calibration has a greater effect on reducing noise than filtering it during post-processing, emphasizing the importance of optimizing image acquisition rather than relying solely on computational corrections. This work improves the repeatability and accuracy of 3D root imaging for phenotyping pipelines by giving useful calibration guidelines. This leads to better trait quantification for use in crop research and plant breeding in downstream analysis.

3D reconstruction↗

DOE OSTI2020

Planck intermediate results

We present the NPIPE processing pipeline, which produces calibrated frequency maps in temperature and polarization from data from the Planck Low Frequency Instrument (LFI) and High Frequency Instrument (HFI) using high-performance computers. NPIPE represents a natural evolution of previous Planck analysis efforts, and combines some of the most powerful features of the separate LFI and HFI analysis pipelines. For example, following the LFI 2018 processing procedure, NPIPE uses foreground polarization priors during the calibration stage in order to break scanning-induced degeneracies. Similarly, NPIPE employs the HFI 2018 time-domain processing methodology to correct for bandpass mismatch at all frequencies. In addition, NPIPE introduces several improvements, including, but not limited to: inclusion of the 8% of data collected during repointing manoeuvres; smoothing of the LFI reference load data streams; in-flight estimation of detector polarization parameters; and construction of maximally independent detector-set split maps. For component-separation purposes, important improvements include: maps that retain the CMB Solar dipole, allowing for high-precision relative calibration in higher-level analyses; well-defined single-detector maps, allowing for robust CO extraction; and HFI temperature maps between 217 and 857 GHz that are binned into 0'.9 pixels (Nside = 4096), ensuring that the full angular information in the data is represented in the maps even at the highest Planck resolutions. The net effect of these improvements is lower levels of noise and systematics in both frequency and component maps at essentially all angular scales, as well as notably improved internal consistency between the various frequency channels. Based on the NPIPE maps, we present the first estimate of the Solar dipole determined through component separation across all nine Planck frequencies. The amplitude is (3366.6 ± 2.7) μK, consistent with, albeit slightly higher than, earlier estimates. From the large-scale polarization data, we derive an updated estimate of the optical depth of reionization of τ = 0.051 ± 0.006, which appears robust with respect to data and sky cuts. There are 600 complete signal, noise and systematics simulations of the full-frequency and detector-set maps. As a Planck first, these simulations include full time-domain processing of the beam-convolved CMB anisotropies. The release of NPIPE maps and simulations is accompanied with a complete suite of raw and processed time-ordered data and the software, scripts, auxiliary data, and parameter files needed to improve further on the analysis and to run matching simulations.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2022

Creating ground truth for nanocrystal morphology: a fully automated pipeline for unbiased transmission electron microscopy analysis

Control over colloidal nanocrystal morphology (size, size distribution, and shape) is important for tailoring the functionality of individual nanocrystals and their ensemble behavior. Despite this, traditional methods to quantify nanocrystal morphology are laborious. New developments in automated morphology classification will accelerate these analyses but the assessment of machine learning models is limited by human accuracy for ground truth, causing even unsupervised machine learning models to have inherent bias. Herein, we introduce synthetic image rendering to solve the ground truth problem of nanocrystal morphology classification. By simulating 2D images of nanocrystal shapes via a function of high-dimensional parameter space, we trained a convolutional neural network to link unique morphologies to their simulated parameters, defining nanocrystal morphology quantitatively rather than qualitatively. An automated pipeline then processes, quantitatively defines, and classifies nanocrystal morphology from experimental transmission electron microscopy (TEM) images. Using improved computer vision techniques, 42,650 nanocrystals were identified, assessed, and labeled with quantitative parameters, offering a 600-fold improvement in efficiency over best-practice manual measurements. Further, a classification algorithm was trained with a prediction accuracy of 99.5%, which can successfully analyze a range of concave, convex, and irregular nanocrystal shapes. The resulting pipeline was applied to differentiating two syntheses of nominally cuboidal CsPbBr 3 nanocrystals and uniquely classifying binary nickel sulfide nanocrystal phase based on morphology. This pipeline provides a simple, efficient, and unbiased method to quantify nanocrystal morphology and represents a practical route to construct large datasets with an absolute ground truth for training unbiased morphology-based machine learning algorithms.

77 NANOSCIENCE AND NANOTECHNOLOGY↗

DOE OSTI2023

Operational Intelligence in the ATLAS Continuous Integration System

Describes the role of the ATLAS Continuous Integration (CI) System in the ATLAS offline software development infrastructure • Outlines the CI system components and processes • Details Operational Intelligence techniques to accelerate CI jobs and lower operating costs • Explains the Directed Acyclic Graph (DAG) approach in CI pipelines • Reports achieved improvements

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

FPDeep: Scalable Acceleration of CNN Training on Deeply-Pipelined FPGA Clusters

In this paper, we propose a framework called FPDeep, which uses a hybrid of model and layer paral- lelism to configure distributed reconfigurable clusters to train DNNs. This approach has numerous benefits. First, the design does not suffer from batch size growth. Second, novel workload and weight partitioning leads to balanced loads of both among nodes. And third, the entire system is fine-grained pipeline. This leads to high parallelism and utilization and also minimizes the time features need to be cached while waiting for back-propagation.

Wang, Tianqi↗

NASA NTRSDate not supplied

NASA GeneLab RNASeq Consensus Pipeline: A Nextflow Implementation

The NASA GeneLab project (genelab.nasa.gov) seeks to accelerate space biology research through cataloging and democratizing omics data. Since raw omics data is largely inaccessible to non-bioinformaticians, GeneLab works with the scientific community to develop standard processing pipelines to generate and publish processed data. Unlike raw data, processed data has greater immediate value to a wide range of users with varying technical backgrounds and computational capabilities. Standardizing processing workflows is essential to match the pace of raw data generation, ensure reproducibility, and enable standardized processed data for comparison across datasets. Previously, GeneLab developed a standardized pipeline for processing RNAseq data, referred to as the ‘GeneLab RNAseq Consensus Pipeline (RCP)’, in collaboration with GeneLab’s Analysis Working Groups. The work presented here is a Nextflow implementation of GeneLab’s RCP that automates and accelerates data processing of RNASeq datasets hosted on GeneLab. In addition to the core data processing, the workflow also includes staging of GeneLab raw data and a robust verification and validation (V&V) program that runs after each processing step to identify errors in real-time, stop additional downstream computation, and preserve computational resources. The workflow, including the staging and V&V functionality, is open source for others to reuse and modify at https://github.com/nasa/GeneLab_Data_Processing/tree/master/RNAseq.

Jonathan Dejesus Oribello↗

NASA NTRSDate not supplied

NASA GeneLab RNASeq Consensus Pipeline: A Nextflow Implementation

Jonathan D Oribello↗

DOE OSTI2024

Intro to HPC Bootcamp: Engaging New Communities Through Energy Justice Projects

The U.S. Department of Energy (DOE) is a long-standing leader in research and development of high-performance computing (HPC) in the pursuit of science. However, we face daunting challenges in fostering a robust and diverse HPC workforce. Basic HPC is not typically taught at early stages of students' academic careers, and the capacity and knowledge of HPC at many institutions are limited. Even so, such topics are prerequisites for advanced training programs, internships, graduate school, and ultimately for careers in HPC. To help address this challenge, as part of the DOE Exascale Computing Project's Broadening Participation Initiative, we recently launched the Introduction to HPC Training and Workforce Pipeline Program to provide accessible introductory material on HPC, scalable AI, and analytics. We describe the Intro to HPC Bootcamp, an immersive program designed to engage students from underrepresented groups as they learn foundational HPC skills. Here, the program takes a novel approach to HPC training by turning the traditional curriculum upside down. Instead of focusing on technology and its applications, the bootcamp focuses on energy justice to motivate the training of HPC skills through project-based pedagogy and real-life science stories. Additionally, the bootcamp prepares students for internships and future careers at DOE labs. The first bootcamp, hosted by the advanced computing facilities at Argonne, Lawrence Berkeley, and Oak Ridge National Labs and organized by Sustainable Horizons Institute, took place in August 2023.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

DOE OSTI2024

Comparative Study of Large Language Model Architectures on Frontier

Large language models (LLMs) have garnered significant attention in both the AI community and beyond. Among these, the Generative Pre-trained Transformer (GPT) has emerged as the dominant architecture, spawning numerous variants. However, these variants have undergone pre-training under diverse conditions, including variations in input data, data preprocessing, and training methodologies, resulting in a lack of controlled comparative studies. Here we meticulously examine two prominent open-sourced GPT architectures, GPT-NeoX and LLaMA, leveraging the computational power of Frontier, the world’s first Exascale supercomputer. Employing the same materials science text corpus and a comprehensive end-to-end pipeline, we conduct a comparative analysis of their training and downstream performance. Our efforts culminate in achieving state-of-the-art performance on a challenging materials science benchmark. Furthermore, we investigate the computation and energy efficiency, and propose a computationally efficient method for architecture design. To our knowledge, these pre-trained models represent the largest available for materials science. Our findings provide practical guidance for building LLMs on HPC platforms.

Yin, Junqi↗

NASA NTRS1975

Hypermatrix scheme for finite element systems on CDC STAR-100 computer

A study is made of the adaptation of the hypermatrix (block matrix) scheme for solving large systems of finite element equations to the CDC STAR-100 computer. Discussion is focused on the organization of the hypermatrix computation using Cholesky decomposition and the mode of storage of the different submatrices to take advantage of the STAR pipeline (streaming) capability. Consideration is also given to the associated data handling problems and the means of balancing the I/Q and cpu times in the solution process. Numerical examples are presented showing anticipated gain in cpu speed over the CDC 6600 to be obtained by using the proposed algorithms on the STAR computer.

Noor, A. K.↗

DOE OSTI2022

BOSS DR12 full-shape cosmology: Λ CDM constraints from the large-scale galaxy power spectrum and bispectrum monopole

Here, we present a full Λ CDM analysis of the BOSS DR12 dataset, including information from the power spectrum multipoles, the real-space power spectrum, the reconstructed power spectrum and the bispectrum monopole. This is the first analysis to feature a complete treatment of the galaxy bispectrum, including a consistent theoretical model and without large-scale cuts. Unlike previous works, the statistics are measured using window-free estimators: this greatly reduces computational costs by removing the need to window-convolve the theory model. Our pipeline is tested using a suite of high-resolution mocks and shown to be robust and precise, with systematic errors far below the statistical thresholds. Inclusion of the bispectrum yields consistent parameter constraints and shrinks the σ 8 posterior by 13% to reach < 5 % precision; less conservative analysis choices would reduce the error bars further. Our constraints are broadly consistent with Planck: in particular, we find H 0 = 69.6$^{+1.1}_{–1.3}$ km s –1 Mpc –1 , σ 8 = 0.692$^{+0.035}_{–0.041}$ and n s = 0.870$^{+0.067}_{–0.064}$, including a BBN prior on the baryon density. When n s is set by Planck, we find H 0 = 68.31$^{+0.83}_{–0.86}$ km s –1 Mpc –1 and σ 8 = 0.722$^{+0.032}_{–0.036}$. Our S 8 posterior, 0.751 ± 0.039, is consistent with weak lensing studies, but lower than Planck. Constraints on the higher-order bias parameters are significantly strengthened from the inclusion of the bispectrum, and we find no evidence for deviation from the dark matter halo bias relations. These results represent the most complete full-shape analysis of BOSS DR12 to-date, and the corresponding spectra will enable a variety of beyond-Λ CDM analyses, probing phenomena such as the neutrino mass and primordial non-Gaussianity.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2024

ICED: An Integrated CGRA Framework Enabling DFVS-Aware Acceleration

oarse-grained reconfigurable arrays (CGRAs) are a promising solution to enable energy-efficient acceleration of applications from different domains. By leveraging reconfiguration at the functional level, they can adapt to significantly different computational patterns. Existing CGRA mapping approaches extract instruction-level parallelism, exploit loop-pipelining opportunities, guarantee the data dependency, and target high throughput of a given loop. However, the recurrence data-dependency in the DFG and the mismatch between required and available computing/communication resources complicate the mapping, and might lead to significant unbalances in the utilization of the CGRA's tiles. This results in wasted power for tiles with low utilization. Applying dynamic voltage and frequency scaling (DVFS) can potentially solve this challenge and improve energy efficiency by adjusting voltage and frequency of different tiles independently. CGRAs have also been successful in accelerating data-dependent streaming applications. However, in these applications, the execution time of each kernel in the pipeline might dynamically vary depending on the characteristics of the input. This also leads to under-utilization of resources for the dynamically changing kernels that do not limit the application throughput. DVFS can also improve energy efficiency for these applications by dynamically changing the voltage and frequency levels of tiles that host non performance-constraining kernels. This paper proposes ICEDTEA -- an integrated DVFS-aware framework to map applications on CGRAs that support power islands. ICEDTEA proposes a CGRA architecture supporting DVFS islands at varying granularity (from a single tile to a group of tiles) and the related DVFS-aware compilation and mapping toolchain. ICEDTEA is the first work that introduces DVFS support for spatio-temporal CGRAs at power-island levels. The experimental evaluation shows that ICEDTEA improves average utilization by 2.3$\times$ and energy-efficiency by 1.32$\times$ over a conventional CGRA. With streaming applications, ICEDTEA improves energy efficiency by 1.12$\times$ over a state-of-the-art CGRA that introduces partial dynamic reconfiguration to adapt to variations in kernels' throughput.

Tan, Cheng↗

DOE OSTI2025

Accelerating Advanced Light Source Science Through Multi-Facility HPC Workflows

Synchrotron light sources support a wide array of techniques to investigate materials, often producing complex, high-volume data that challenge traditional workflows. At the Advanced Light Source (ALS), we developed infrastructure to move microtomography data over ESnet to ALCF and NERSC, where CPU- and GPU-based algorithms generate 3D reconstructed volumes of experimental samples. We employ two data movement and reconstruction models: real-time processing as data streams directly to NERSC compute nodes, and automated file transfer to NERSC and ALCF file systems. The streaming pipeline provides users with feedback in under ten seconds, while the file-based workflow produces high-quality reconstructions suitable for deeper analysis in 20-30 minutes. This infrastructure enables users to utilize HPC resources without direct access to backend systems. We plan to extend this architecture to more endstations, supporting our beamline scientists and users.

Abramov, David↗

DOE OSTI2025

Tensorized Interior Radiative Heat Transfer for a Scalable and Calibrated Building Energy Simulator

Building energy simulation is a critical tool for developing and testing advanced control strategies, such as Reinforcement Learning (RL), to provide demand flexibility and affordable energy costs. The recently introduced Smart Buildings Control Suite (sbsim) provides a lightweight, scalable, and data-calibrated simulation environment based on a 2D finite-difference model. However, the initial model primarily focused on conductive and convective heat transfer, neglecting the significant impact of long-wave radiative heat exchange between interior surfaces. This paper presents a significant extension to the sbsim framework by incorporating a physically-grounded model for interior radiative heat transfer. Our primary contribution is the development and integration of a fully tensorized radiative heat transfer module, which preserves the computational efficiency and scalability of the original simulator. This was achieved by developing a pipeline for view factor calculation, including an algorithm to identify directly seeing surfaces within complex floor plans, and formulating the net radiation equations for efficient execution on modern hardware accelerators. We validate the numerical accuracy of our tensorized implementation by comparing its results against a traditional iterative approach, demonstrating identical outcomes. This enhancement increases the physical fidelity of sbsim, enabling more accurate training of RL agents for building energy optimization.

Ham, Sang woo↗

DOE OSTI2024

Liquid Piston with Spray Cooling Near-Isothermal Compressor

The goal of this project was to prototype and characterize the performance of a liquid-piston spray-cooled gas compressor. The working principle of the compressor enables optimized high-efficiency operation over a very wide range of operating conditions, unlike conventional compressors that are optimized for a narrow range of operating conditions. The compressor technology is suitable for many applications, such as gas pipeline transport, gas storage, and commercial and residential heat pumps. Both physical testing and computational fluid dynamics (CFD) modeling of the processes using the Oak Ridge National Laboratory high-performance computing center were completed. The experimental and CFD studies focused on a near-isothermal liquid-piston compressor (LPC) that uses propylene glycol to compress CO 2 . The first prototype demonstrated isothermal operation during several sequentially executed cycles of CO 2 compression and raised the temperature of the compressed CO 2 by only 2 K, compared with approximately 6 K when the gas was compressed non-isothermally. Isothermal operation was demonstrated at CO 2 flow rates of up to 2 L/min. The second prototype was designed with two compression chambers to allow continuous flow of high-pressure CO 2 . However, the design of the valve train to direct flow between the compression chambers was not sufficient to allow demonstration of CO 2 compression. Numerical simulations of the LPC in which the compression chamber was filled with propylene glycol injected from the bottom inlet were performed using large eddy simulation (LES) with the wall-adapting local eddy-viscosity subgrid-scale model coupled with the multiphase volume of fluid (VOF) model to simulate the transient interface between gas and liquid and to capture the heat and mass transfers within the compression chamber. In this effort, the effects of boundary conditions applied to the LES-VOF calculations (i.e., no wall, an adiabatic wall, and a wall with a heat flux subscribed) on the overall pressure and temperature of the CO 2 gas as well as the transient evolution of flow and heat transfer within the compression chamber were investigated and are discussed in this report. The LES calculation with no wall showed no dynamical flow patterns, and the volume-averaged temperature of CO 2 increased from 305 to 392.7 K, whereas LES calculations with a constant wall temperature or a wall heat flux had similar increases of CO 2 temperatures. The results of the LES simulation using a wall heat flux showed different stages in the compression process and revealed dynamical formation and interaction of CO 2 gas layers and circulation flow patterns within the chamber that contributed to the overall heat transfer between the solid wall, gas, and liquid surface in the compressor. Though an industrial partnership for commercializing the compressor was not secured, the technology was attractive for an industrial partner to use in two research proposals in response to US Department of Energy funding opportunity announcements.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗