Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Computational pipelines”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Are light curve classification metrics good proxies for SN Ia cosmological constraining power?

Context. When selecting a light curve classifier for use as part of a photometric supernova Ia (SN Ia) cosmological analysis, it is common to make decisions based on metrics of classification performance, such as the contamination within the photometrically classified SN Ia sample, rather than a measure of cosmological constraining power. If the former is an appropriate proxy for the latter, this practice would eliminate the computational expense of a full cosmology forecast in the analysis pipeline design process. Aims. This study tests the assumption that light curve classification metrics are an appropriate proxy for cosmology metrics. Methods. We emulated photometric SN Ia cosmology light curve samples with controlled contamination rates of individual contaminant classes and evaluated each of them under a set of classification metrics. We then derived cosmological parameter constraints from all samples under two common analysis approaches and quantified the impact of contamination by each contaminant class on the resulting cosmological parameter estimates. Results. We observe that cosmology metrics are sensitive to both the contamination rate and the class of the contaminating population, whereas the classification metrics are shown to be insensitive to the latter. Conclusions. Based on these findings, we discourage any exclusive reliance on light curve classification-based metrics for analysis design decisions, which (counterintuitively) include but are not limited to the classifier choice. Instead, we recommend optimising science analysis pipeline design choices using a metric of the information gained about the physical parameters of interest.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Processing Full-Scale Square Kilometre Array Data on the Summit Supercomputer

This work presents a workflow for simulating and processing the full-scale low-frequency telescope data of the Square Kilometre Array (SKA) Phase 1. The SKA project will enter the construction phase soon, and once completed, it will be the world’s largest radio telescope and one of the world’s largest data generators. The authors used Summit to mimic an endto-end SKA workflow, simulating a dataset of a typical 6 hour observation and then processing that dataset with an imaging pipeline. This workflow was deployed and run on 4,560 compute nodes, and used 27,360 GPUs to generate 2.6 PB of data. This was the first time that radio astronomical data were processed at this scale. Results show that the workflow has the capability to process one of the key SKA science cases, an Epoch of Reionization observation. This analysis also helps reveal critical design factors for the next-generation radio telescopes and the required dedicated processing facilities.

Wang, Ruonan↗
NASA NTRS2004

Packaging data products using data grid middleware for Deep Space Mission Systems

We are investigating a distributed computing infrastructure for releasing science products out of the mission pipeline. We are creating a DSMS Product Service and DSMS Data Packaging Service based on the OODT middleware, a Data Grid middleware technology that has proven successful in providing access to heterogeneous, disparate data, and federating that data using a common data model.

Deep Space Mission Systems↗
DOE OSTI2024

YOU-ONLY-LOOK-ONCE (YOLO) FOR RADIO FREQUENCY SIGNAL CLASSIFICATION

We propose a method that uses deep learning (DL) to identify and frame various signals that are present in an environment. This DL framework is based on the You Only Look Once (YOLO) object detection pipeline Our work demonstrates a specific application of high performance computing and computer vision to the field of telecommunications.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1988

A pipeline design of a fast prime factor DFT on a finite field

A conventional prime factor discrete Fourier transform (DFT) algorithm is used to realize a discrete Fourier-like transform on the finite field, GF(q sub n). This algorithm is developed to compute cyclic convolutions of complex numbers and to decode Reed-Solomon codes. Such a pipeline fast prime factor DFT algorithm over GF(q sub n) is regular, simple, expandable, and naturally suitable for VLSI implementation. An example illustrating the pipeline aspect of a 30-point transform over GF(q sub n) is presented.

Truong, T. K.↗
DOE OSTI2022

Moving small files in a networked environment

Globally distributed computing infrastructures, such as clouds and supercomputers, are currently used to manage data that is generated with an unprecedented speed from a variety of resources. Coping with this trend, the volume of data exchanged across distant sites increases substantially. To accelerate data transfer, high-speed networks are provided to connect remote sites. Most existing data movement solutions are optimized for moving large files. However, it is still challenging to transfer a large number of small files across networks. This disadvantage not only lowers data transfer performance, but also decreases overall system utilization. Here, we identify that moving small files is mainly constrained by degraded file system throughput, not just network performance as might be suspected. We have built a data transfer pipeline model to analyze the impact of small network I/O and storage I/O on data movement. Extending one of the widely used open source data movement solutions, GridFTP, we demonstrate several appropriate engineering approaches that mitigate the bottleneck and increase data transfer efficiency. We show optimizations that improve data transfer performance more than 5 times. In comparison to existing solutions, our approaches can save a significant amount of system resources for moving lots of small files.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Reinforcement Learning as a Parsimonious Alternative to Prediction Cascades: A Case Study on Image Segmentation

Deep learning architectures have achieved state-of-the-art (SOTA) performance on computer vision tasks such as object detection and image segmentation. This may be attributed to the use of over-parameterized, monolithic deep learning architectures executed on large datasets. Although such large architectures lead to increased accuracy, this is usually accompanied by a larger increase in computation and memory requirements during inference. While this is a non-issue in traditional machine learning (ML) pipelines, the recent confluence of machine learning and fields like the Internet of Things (IoT) has rendered such large architectures infeasible for execution in low-resource settings. For some datasets, large monolithic pipelines may be overkill for simpler inputs. To address this problem, previous efforts have proposed decision cascades where inputs are passed through models of increasing complexity until the desired performance is achieved. However, we argue that cascaded prediction leads to sub-optimal throughput and increased computational cost due to wasteful intermediate computations. To address this, we propose PaSeR (Parsimonious Segmentation with Reinforcement Learning) a non-cascading, cost-aware learning pipeline as an efficient alternative to cascaded decision architectures. Through experimental evaluation on both real-world and standard datasets, we demonstrate that PaSeR achieves better accuracy while minimizing computational cost relative to cascaded models. Further, we introduce a new metric IoU/GigaFlop to evaluate the balance between cost and performance. On the real-world task of battery material phase segmentation, PaSeR yields 179% improvement over SOTA MatPhase model and a 196% improvement over IDK Cascades under the IoU/GigaFlop metric. We also demonstrate PaSeR’s adaptability to complementary models trained on a noisy MNIST dataset, where it outperforms all baselines on IoU/GigaFlop by an average of 44%.

Srikshan, Bharat↗
DOE OSTI2021

Machine Learning-enabled Scalable Performance Prediction of Scientific Codes

Hardware architectures become increasingly complex as the compute capabilities grow to exascale. Here, we present the Analytical Memory Model with Pipelines (AMMP) of the Performance Prediction Toolkit (PPT). PPT-AMMP takes high-level source code and hardware architecture parameters as input and predicts runtime of that code on the target hardware platform, which is defined in the input parameters. PPT-AMMP transforms the code to an (architecture-independent) intermediate representation, then (i) analyzes the basic block structure of the code, (ii) processes architecture-independent virtual memory access patterns that it uses to build memory reuse distance distribution models for each basic block, and (iii) runs detailed basic-block level simulations to determine hardware pipeline usage. PPT-AMMP uses machine learning and regression techniques to build the prediction models based on small instances of the input code, then integrates into a higher-order discrete-event simulation model of PPT running on Simian PDES engine. We validate PPT-AMMP on four standard computational physics benchmarks and present a use case of hardware parameter sensitivity analysis to identify bottleneck hardware resources on different code inputs. We further extend PPT-AMMP to predict the performance of a scientific application code, namely, the radiation transport mini-app SNAP. To this end, we analyze multi-variate regression models that accurately predict the reuse profiles and the basic block counts. We validate predicted SNAP runtimes against actual measured times.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Real-Time GPU-Accelerated OFDR With an Integrated Auxiliary Interferometer

A GPU-accelerated optical frequency domain reflectometry (OFDR) system with an improved integrated auxiliary interferometer is proposed. Unlike conventional approaches that require separate auxiliary interferometers and multiple detection channels, the proposed OFDR system embeds this functionality directly into the signal via an intentional beat component. This enables self-calibration of laser nonlinearity while maintaining a cost-effective hardware configuration. Building on this simplified configuration, the system leverages GPU acceleration with an NVIDIA RTX 4070 Ti to achieve real-time performance, delivering high-throughput signal processing for continuous OFDR interrogation. The signal processing pipeline comprises signal capture, resampling for nonlinearity compensation, and frequency shift computation, all optimized for parallel execution. Hardware benchmarking demonstrates substantial acceleration over CPU implementations, achieving up to a 45× speedup for resampling and frequency shift computations and enabling processing latencies below 30 ms. Thermal response validation is conducted under two complementary scenarios: localized heating using a water bath and cryogenic-temperature conditions using liquid nitrogen. Under localized heating, the system achieves an accuracy of 0.249 °C with a thermal sensitivity of 5.971 GHz/°C, while cryogenic-temperature validation demonstrates a frequency shift response with a sensitivity of 2.383 GHz/°C and an accuracy of 2.04 °C. The high acceleration of the proposed GPU-accelerated OFDR system and its accuracy are achieved by exploiting CUDA-based stride indexing, enabling efficient parallel segmentation and processing of large datasets without additional memory copies. The benchmarking results confirm the robustness, accuracy, and deployability of the proposed OFDR system across a wide temperature range, establishing it as a practical platform for real-time distributed fiber sensing in structurally dynamic environments.

Harb, Salah [Lawrence Berkeley National Laboratory↗
DOE OSTI2021

Mitigating Catastrophic Forgetting in Deep Learning in a Streaming Setting Using Historical Summary

Recent advancements in scientific equipment and the adaptation of electronics and the Internet of Things (IoT) in our everyday lives resulted in large and complex data production at a high rate. Making meaningful and timely knowledge discovery at a modest cost from this big data is difficult for computing power and storage limitations. Training deep learning models incrementally in a streaming setting can help us with overcoming these limitations. However, in a well-known phenomenon named catastrophic forgetting, incrementally trained models increasingly perform poorly on the past data. To mitigate catastrophic forgetting in training in a streaming setting, we propose constructing a historical summary over time and use the summary with newly arrived data during incremental training. We propose various data summarization techniques such as random sampling, micro clustering, coreset computation, and Auto Encoders to counteract catastrophic forgetting. We built a pipeline for incremental training with a historical summary for training deep learning models for streaming data. We demonstrate the effectiveness of historical summary in mitigating catastrophic forgetting using three case studies involving three different deep learning applications: an Artificial Neural Network (ANN) for classification task on MNIST dataset, a language model (RNN-LM) on the WikiText2 dataset, and a Convolutional Neural Network (CNN), ResNet50 to classify the ImageNet dataset. Through the training of the models, we observe that catastrophic forgetting is evident in ANN and CNN but not in an RNN. For the first task, our method recovers up to 47.9% lost accuracy due to catastrophic forgetting. For the third task, the historical summary recovers classification accuracy by up to 25%. For the second task, though there is not proof of catastrophic forgetting, the training performance (PPL) improves by up to 26% with historical summary.

Dash, Sajal↗
DOE OSTI2021

Numerical Codes for the DESC-LSST Analysis Pipeline: Core Cosmology Library Standard Modules and Beyond wCDM Modules (Final Technical Report)

The overall objective of the project is to investigate and develop specific software modules and analysis components for the software pipeline of LSST Dark Energy Science Collaboration (DESC). Following the key projects of DESC Science Roadmap (SRM), we will write and test computer codes for the Core Cosmology Library (CCL) in order to complete its modules, functionalities, and interface to work with the analysis pipelines from the five science probes of DESC (parts of SRM deliverables CX4.2TJP, CX6.2CS). We will also code modules for CCL to test models beyond w-Cold-Dark-Matter (wCDM) and modification to gravity (MG). Interfaces for MG models will also be developed for the TJPCOSMO software which is the main pipeline of the Theory and Joint Probe (TJP) working group (deliverable TJP2.3). In order to use the full power of LSST data to constrain MG models, we will also work on constraints from nonlinear regime by running and analyzing MG N-Body simulations using a Parameterized-Post-Friedmann framework into the Gadget-2 simulation package (deliverable TJP2.2). Preliminary results for the simulations were obtained in the past. In collaboration with other DESC groups, we plan to make these simulations feedable to cosmic emulators that are practical for likelihood analyzes (deliverable TJP2.2, parts of CX6.2CS). We will also modify and integrate our current codes for consistency tests between data sets and probes into the pipeline (parts of deliverables CX8.2TJP, TJP2.3). We understand that other groups will contribute to some of these objectives but our team will focus and collaborate with others on the particular part of testing MG and models beyond wCDM and refine the DESC pipeline for this purpose. PI has been coordinating his work with the TJP and CS working groups and the DESC management team. PI is a full member of DESC since June 2013. He and his students have been contributing to LSST-DESC activities and work including TJP telecons, collaboration meetings, hack-weeks, and workshops. PI chaired or co-chaired sessions at collaboration meetings and hack-weeks about testing gravity and models beyond wCDM using LSST. He is coordinating the TJP2 projects for testing models beyond wCDM including the writing of DESC-research-note, development of code for pipeline, and N-Body simulations for MG and beyond wCDM models testable with LSST analyses. As stressed in the DESC white paper, SRM, and P5 report, one of the important questions in understanding cosmic acceleration and dark energy is to be able to distinguish whether the acceleration is due to a dark energy component in the universe or a modification to gravity. Answering these questions will have a significant impact on the question of cosmic acceleration and dark energy. The methods that we will use include analytical work, numerical code, and N-Body simulations. A first approach that that we will use consists of using growth rate parameters that enter the perturbed dynamics equations. These parameters take distinctive values for distinct gravity theories and have potential to distinguish between Dark Energy and Modified Gravity. The second method is to look for inconsistencies in Dark Energy parameter spaces using specific combinations of cosmological data sets. Our investigation addresses the Dark Energy problem that is relevant to the mission of the HEP program to understand how our universe works at its most fundamental level. It will allow us to make progress on the HEP mission to explore the nature of Dark Energy and the basic nature of space and time using future surveys such as LSST. The investigation supports the DOE HEP program Cosmic Frontier as it will contribute to the study and understanding of dark energy and fundamental properties of the universe. The investigation contributes directly to LSST-DESC key projects and their deliverables as described in the Science Road-map document to build analysis pipeline and to test dark energy and beyond wCDM models using LSST.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

Pipe-in-Pipe by Rapid, Continuous, Smart Alloy Coating

Our team has developed a comprehensive solution for performing in-situ structural repairs on metal piping and pipelines. This is based on a multifaceted and integrated (experimental and computational) approach, that uses a slurry-based coating and our patented high temperature sintering process, to build new metal in a pipe-in-pipe configuration. This can be utilized in either a complete pipe-in-pipe embodiment, or for patch repairs.

42 ENGINEERING↗
NASA NTRSDate not supplied

Science Target Prioritization Framework for Remote Sensing

Behind the scenes of a remote sensing mission there are complex decision making and planning operations. Streamlining these operations, with a quantitative scientific value framework, aids efficient and optimized science data collection.While there have been previous efforts to quantify the science value for specific science scenarios, our work aims to develop a general framework which can be applied across different scenarios. We describe a pipeline of processes which combines model forecast and observation data, in computational forms, as dictated by the mission objectives set forth by subject matter experts. The framework is described with use cases involving the monitoring of nitrogen dioxide (NO2) concentrations over the Gulf of Mexico and methane concentrations over interior Alaska.

Remote Sensing↗
DOE OSTI2020

Enabling Highly Efficient Capsule Networks Processing Through A PIM-Based Architecture Design

In recent years, the CNNs have achieved great successes in the image processing tasks, e.g., image recognition and object detection. Unfortunately, traditional CNN's classication is found to be easily misled by increasingly complex image features due to the usage of pooling operations, hence unable to preserve accurate position and pose information of the objects. To address this challenge, a novel neural network structure called Capsule Network has been proposed, which introduces equivariance through capsules to signicantly enhance the learning ability for image segmentation and object detection. Due to its requirement of performing a high volume of matrix operations, CapsNets have been generally accelerated on modern GPU platforms that provide highly optimized software library for common deep learning tasks. However, based on our performance characterization on modern GPUs, CapsNets exhibit low effciency due to the special program and execution features of their routing procedure, including massive unshareable intermediate variables and intensive syn- chronizations, which are very dicult to optimize at software level. To address these challenges, we propose a hybrid computing architecture design named PIM-CapsNet. It preserves GPU's on-chip computing capability for accelerating CNN types of layers in CapsNet, while pipelining with an off-chip in-memory acceleration solution that effectively tackles routing procedure's ineffciency by leveraging the processing-in-memory capability of today's 3D stacked memory. Using routing procedure's inherent parallellization feature, our design enables hierarchical improvements on CapsNet inference effciency through minimizing data movement and maximizing parallel processing in memory. Evaluation results demonstrate that our proposed design can achieve substantial improvement on both performance and energy savings for CapsNet inference, with almost zero accuracy loss. The results also suggest good performance scalability in optimizing the routing procedure with increasing network size.

Zhang, Xingyao↗
DOE OSTI2021

GENESPACE R Package (GENESPACE) v1.0

In short, the GENESPACE pipeline conducts analysis of orthology networks, constrained within syntenic regions. Since analyses are limited to local tests conducted within syntenic blocks, GENESPACE is agnostic to ploidy, duplicated regions, inversions or other whole-genome chromosomal complexities that are common across many evolutionary lineages. This advantage allows for evolutionary tests in polyploids (e.g. switchgrass, manuscript in review), species with ancient, but retained whole-genome duplications (e.g. pecan, manuscript in prep), high levels of tandem array proliferation (e.g. eukalypts, manuscript in review) and many other factors that can confound comparative genomic analyses. The major advances of GENESPACE are three-fold: First, this is the first R package to integrate visualization and analysis of large-scale comparative genomics. R, which offers a high-level environment for graphical and statistical exploration of data, is often speed- and memory-limited and not used for computationally intensive tasks such as comparative genomics. The highly efficient C++ scripts used in GENESPACE (via data.table) permit a much faster and computationally lightweight implementation of comparative genomics than is currently available. Second, the pipeline itself is novel. To the best of our knowledge, no other program accomplishes synteny-constrained and ploidy-agnostic comparative genomics. Since nearly all plants and many animals have a history of whole-genome duplications, this is a major and necessary advance to the field. Third, GENESPACE offers high-level and intuitive multi-genome graphical outputs. The dotplots and 'riparian' plots produced herein, which are produced entirely through original R code, are publication-ready and easily customizable.

Schmutz, Jeremy↗
NASA NTRS1997

A Parallel Pipelined Renderer for the Time-Varying Volume Data

This paper presents a strategy for efficiently rendering time-varying volume data sets on a distributed-memory parallel computer. Time-varying volume data take large storage space and visualizing them requires reading large files continuously or periodically throughout the course of the visualization process. Instead of using all the processors to collectively render one volume at a time, a pipelined rendering process is formed by partitioning processors into groups to render multiple volumes concurrently. In this way, the overall rendering time may be greatly reduced because the pipelined rendering tasks are overlapped with the I/O required to load each volume into a group of processors; moreover, parallelization overhead may be reduced as a result of partitioning the processors. We modify an existing parallel volume renderer to exploit various levels of rendering parallelism and to study how the partitioning of processors may lead to optimal rendering performance. Two factors which are important to the overall execution time are re-source utilization efficiency and pipeline startup latency. The optimal partitioning configuration is the one that balances these two factors. Tests on Intel Paragon computers show that in general optimal partitionings do exist for a given rendering task and result in 40-50% saving in overall rendering time.

Chiueh, Tzi-Cker↗
DOE OSTI2020

Optimal economic operation of liquid petroleum products pipeline systems

The majority of overland transport needs for crude petroleum and refined petroleum products are met using pipelines. Numerous studies have developed optimization methods for design of these systems in order to minimize construction costs while meeting capacity requirements. Here, we formulate problems to optimize the operations of existing single liquid commodity pipeline systems subject to physical flow and pump engineering constraints. The objectives are to maximize the economic value created for users of the system and to minimize operating costs. We present a general computational method for this class of continuous, non-convex nonlinear programs, and examine the use of pump operating settings and flow allocations as decision variables. Furthermore, the approach is applied to compute optimal operating regimes and perform engineering economic sensitivity analyses for a case study of a crude oil pipeline developed using publicly available data.

02 PETROLEUM↗