Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Computational pipelines”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31
All sourcesNASA NTRSDOE OSTI
NASA NTRS2008

Software for Verifying Image-Correlation Tie Points

A computer program enables assessment of the quality of tie points in the image-correlation processes of the software described in the immediately preceding article. Tie points are computed in mappings between corresponding pixels in the left and right images of a stereoscopic pair. The mappings are sometimes not perfect because image data can be noisy and parallax can cause some points to appear in one image but not the other. The present computer program relies on the availability of a left- right correlation map in addition to the usual right left correlation map. The additional map must be generated, which doubles the processing time. Such increased time can now be afforded in the data-processing pipeline, since the time for map generation is now reduced from about 60 to 3 minutes by the parallelization discussed in the previous article. Parallel cluster processing time, therefore, enabled this better science result. The first mapping is typically from a point (denoted by coordinates x,y) in the left image to a point (x',y') in the right image. The second mapping is from (x',y') in the right image to some point (x",y") in the left image. If (x,y) and(x",y") are identical, then the mapping is considered perfect. The perfect-match criterion can be relaxed by introducing an error window that admits of round-off error and a small amount of noise. The mapping procedure can be repeated until all points in each image not connected to points in the other image are eliminated, so that what remains are verified correlation data.

Klimeck, Gerhard↗
DOE OSTI2021

Real-Time Omnidirectional Stereo Rendering: Generating 360° Surround-View Panoramic Images for Comfortable Immersive Viewing

Surround-view panoramic images and videos have become a popular form of media for interactive viewing on mobile devices and virtual reality headsets. Viewing such media provides a sense of immersion by allowing users to control their view direction and experience an entire environment. When using a virtual reality headset, the level of immersion can be improved by leveraging stereoscopic capabilities. Stereoscopic images are generated in pairs, one for the left eye and one for the right eye, and result in providing an important depth cue for the human visual system. For computer generated imagery, rendering proper stereo pairs is well known for a fixed view. However, it is much more difficult to create omnidirectional stereo pairs for a surround-view projection that work well when looking in any direction. One major drawback of traditional omnidirectional stereo images is that they suffer from binocular misalignment in the peripheral vision as a user's view direction approaches the zenith / nadir (north / south pole) of the projection sphere. This paper presents a real-time geometry-based approach for omnidirectional stereo rendering that fits into the standard rendering pipeline. Our approach includes tunable parameters that enable pole merging - a reduction in the stereo effect near the poles that can minimize binocular misalignment. Results from a user study indicate that pole merging reduces visual fatigue and discomfort associated with binocular misalignment without inhibiting depth perception.

360° panorama↗
DOE OSTI2021

Improving reproducibility in synchrotron tomography using implementation-adapted filters

For reconstructing large tomographic datasets fast, filtered backprojection-type or Fourier-based algorithms are still the method of choice, as they have been for decades. These robust and computationally efficient algorithms have been integrated in a broad range of software packages. The continuous mathematical formulas used for image reconstruction in such algorithms are unambiguous. However, variations in discretization and interpolation result in quantitative differences between reconstructed images, and corresponding segmentations, obtained from different software. This hinders reproducibility of experimental results, making it difficult to ensure that results and conclusions from experiments can be reproduced at different facilities or using different software. In this paper, a way to reduce such differences by optimizing the filter used in analytical algorithms is proposed. These filters can be computed using a wrapper routine around a black-box implementation of a reconstruction algorithm, and lead to quantitatively similar reconstructions. Use cases for this approach are demonstrated by computing implementation-adapted filters for several open-source implementations and applying them to simulated phantoms and real-world data acquired at the synchrotron. Our contribution to a reproducible reconstruction step forms a building block towards a fully reproducible synchrotron tomography data processing pipeline.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

FECM/NETL CO 2 Transport Cost Model (2024): Description and User’s Manual

This is the user's manual for the 2024 version of the FECM/NETL CO 2 Transport Cost Model (CO2_T_COM). CO2_T_COM is an Excel-based tool that estimates revenues and capital, operating, and financing costs for transporting liquid phase CO 2 by pipeline. It is assumed that the CO 2 delivered to the pipeline meets pipeline specifications for purity. Costs are estimated for a single point-to-point pipeline, which may have pumps along the pipeline to boost the pressure. The model can be accessed here: https://www.netl.doe.gov/energy-analysis/details?id=42e3c409-b88f-467f-bff0-fadb92a68676 .

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2021

Improving the Accessibility and Usability of Geothermal Information with Data Lakes and Data Pipelines on the Geothermal Data Repository: Preprint

The Geothermal Data Repository (GDR) provides universal access to data and information resulting from research and development activities funded by the Department of Energy (DOE). The GDR has extended this universal access to big data through integration with data lakes developed by the Open Energy Data Initiative (OEDI). Previously, large datasets such as seismic waveform or distributed acoustic sensing (DAS) data could only be accessed by institutions with high performance data storage and compute capabilities, effectively limiting the accessibility of big data to national labs, larger universities, and major corporations. Moreover, the time and resources needed to transport big data and configure them can produce additional barriers to use. Many of the standard formats used for structured data models (also known as content models) are incapable of handling big data and can introduce additional usability problems, often requiring data to be reformatted prior to use. This paper will explore how recent integrations between the GDR and the OEDI data lake have improved the accessibility and usability of geothermal data in a big way, making the data available to a broader audience, and enabling collaborative analysis and innovation across the greater geothermal industry.

access↗
DOE OSTI2024

Automated Approach to Accurate, Precise, and Fast Detector Simulation and Reconstruction

Detector simulation and reconstruction are a significant computational bottleneck in particle physics. Here, we develop particle-flow neural-assisted simulations (parnassus) to address this challenge. Our deep learning model takes as input a point cloud (particles impinging on a detector) and produces a point cloud (reconstructed particles). By combining detector simulations and reconstruction into one step, we aim to minimize resource utilization and enable fast surrogate models suitable for application both inside and outside large collaborations. We demonstrate this approach using a publicly available dataset of jets passed through the full simulation and reconstruction pipeline of the Compact Muon Solenoid (CMS) experiment. We show that parnassus accurately mimics the CMS particle flow algorithm on the (statistically) same events it was trained on and can generalize to jet momentum and type outside of the training distribution.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2026

Two-stage formation-energy correction (NbZr, TaZr, VZr)

This bundle contains the scripts, the raw and corrected per-structure data, and the manuscript plots for the NbZr / TaZr / VZr BCC binary formation energies and the associated RMSDs. Why a two-stage correction is necessary: The "raw" formation energy of every relaxed VASP configuration is computed in the usual way, FE_raw(c) = E_alloy(c) - sum_i x_i * E_pure_i , where E_pure_i are the per-atom total energies of the pure-element reference structures (Nb, Ta, V, Zr in the same BCC supercell, with identical INCAR / KPOINTS / PAW choices). With perfectly consistent reference runs the raw FE should vanish at the two pure-element endpoints (x = 0 and x = 1) by construction. In practice this does not hold for two reasons that are present in our dataset: 1. Reference-energy inconsistency (composition-dependent bias). Even with identical input parameters, the pure-element runs (stored in `corrected_DFT_pure_element_runs/`) differ slightly from the values that would be implied by the alloy runs at near-pure compositions (a few meV/atom). This bias is approximately linear in concentration, because the residual error in E_pure_Nb (or E_pure_Ta / E_pure_V) propagates into FE_raw(c) as (1 - x) * dE_pure_1, and the corresponding error in E_pure_Zr propagates as x * dE_pure_2. Left uncorrected, this produces a non-physical "tilt" of FE_raw(x) and shifts the entire FE-vs-x cloud away from zero at the endpoints. 2. Endpoint anchoring against the audited true endpoints. The strict endpoint values (FE_x0_meVatom, FE_x1_meVatom in `corrected_fe_strict_endpoints_20260518/strict_endpoint_check_20260518.csv`) were re-derived from an independent cross-check of the pure-element runs. After stage 1 removes the linear bias, the near-pure compositions in the alloy dataset still extrapolate to values that differ slightly from these audited endpoints — because stage 1 is fit from a few near-end alloy bins, not from the audited pure-element references themselves. The README.txt file discusses how these issues are addressed by the two-stage correction, and describes folder layout, pipeline summary, and how to re-run.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Boosting RDataFrame performance with transparent bulk event processing

RDataFrame is ROOT’s high-level interface for Python and C++ data analysis. Since it first became available, RDataFrame adoption has grown steadily and it is now poised to be a major component of analysis software pipelines for LHC Run 3 and beyond. Thanks to its design inspired by declarative programming principles, RDataFrame enables the development of highperformance, highly parallel analyses without requiring expert knowledge of multi-threading and I/O: user logic is expressed in terms of self-contained, small computation kernels tied together by a high-level API. This design completely decouples analysis logic from its actual execution, and opens several interesting avenues for workflow optimization. In particular, in this work we explore the benefits of moving internal data processing from an event-by-event to a bulkby-bulk loop. This refactoring dramatically reduces the framework’s runtime overheads; in collaboration with the I/O layer it improves data access patterns; it exposes information that optimizing compilers might use to auto-vectorize the invocation of user-defined computations; finally, while existing user-facing interfaces remain unaffected, it becomes possible to additionally offer interfaces that explicitly expose bulks of events, useful e.g. for the injection of GPU kernels into the analysis workflow. In order to inform similar future R&D, design challenges will be presented, as well as an investigation of the relevant timememory trade-off backed by novel performance benchmarks.

Guiraud, Enrico↗
DOE OSTI2025

PyCMG-based Simulation of Volumetric Concrete Microstructure

Concrete is a complex, heterogeneous material with a microstructure composed of aggregates, cement paste, and pores spanning multiple length scales. Understanding this microstructure is critical for advancing the performance, durability, and modeling of concrete-based systems. While experimental imaging such as X-ray computed tomography (XCT) provides valuable insights, generating large datasets with detailed ground truth annotations is both costly and labor-intensive due to challenges in segmenting similar phases, such as aggregates and cement paste, that often share similar attenuation properties. To address this, we developed a pipeline to simulate realistic 3D concrete microstructures using the open-source Python package PyCMG. This simulation effort focuses on generating high-fidelity, annotated microstructures that can serve as training or benchmarking datasets for image analysis, segmentation algorithms, and machine learning models, particularly in scenarios where experimental data is scarce.

Ziabari, Amir [Oak Ridge National Laboratory; ORNL↗
DOE OSTI2026

Chemical Recommender System: Replacement Suggestions for Small Molecules

The Chemical Recommender System (CRS) is an open-source, high-performance toolkit that enables real-time similarity searches across the complete PubChem database (over 50 million molecules) using commodity hardware. The CRS addresses critical limitations in existing chemical informatics platforms through a novel vector database infrastructure, extensible model integration capabilities, and complete algorithmic transparency. The system implements a vector database deployment with partitioned indexing that achieves a ~60x speedup over traditional approaches. A containerized model integration framework allows researchers to seamlessly incorporate custom predictive models into the full-scale search and scoring pipeline, while complete configurability of search parameters, filtering logic, and scoring functions provides capabilities not available in existing black-box solutions. Beyond structural similarity, the CRS integrates OPERA QSAR models for thermophysical and toxicity predictions, RDKit synthetic accessibility scoring, and user-defined models to compute weighted final replacement scores. The complete system is accessible through an interactive web application supporting real-time progress monitoring, post-processing score re-weighting, automated PDF reporting, and batch processing capabilities.

Nair, Parthiv Anand [Sandia National Laboratories ↗
NASA NTRSDate not supplied

GeoNEX: A Geostationary Earth Observatory

The latest generation of geostationary satellites (Himawari 8/9, GOES-16/17, FY-4, GK-2A) carries sensors that closely mimic the spatial and spectral characteristics of widely used polar-orbiting, global monitoring sensors such as MODIS and VIIRS. When combined, data from various currently operating/planned geostationary platforms provide a geo-ring of hyper-temporal (5-10 minutes), multispectral observations at spatial resolutions as high as 500 m. These high frequency observations offer exciting new possibilities for monitoring our planet, including better retrievals of geophysical variables by overcoming cloud cover, enabling studies of diurnally varying phenomena in the atmosphere, land, and the oceans, and support operational decision-making in agriculture, hydrology and disaster management. The NASA Earth Exchange (NEX) team, in collaboration with scientists from JAXA, KARI, NOAA and other international institutions, created the GeoNEX (www.nasa.gov/geonex) pipeline to integrate data from all available geostationary platforms and produce and distribute spatially, temporally, and radiometrically consistent data for the earth science community. We envision various institutions adapting the Geo component (e.g., GeoNOAA, GeoKARI, GeoChiba, GeoJAXA, GeoCMA) and customizing the pipeline and downstream products to serve the local/regional research and applied science communities. To facilitate collaborative work among the partners, we have established the OpenNEX platform on the public cloud. OpenNEX provides researchers, developers, educators, and ordinary users with easy access to an integrated Earth science computational and data platform, enabling citizen scientists and application developers to realize the full value of GeoNEX data assets and software tools.

GeoNEX↗
DOE OSTI2022

In situ feature analysis for large-scale multiphase flow simulations

The study of multiphase flow is essential for designing chemical reactors such as fluidized bed reactors (FBR), as a detailed understanding of hydrodynamics is critical for optimizing reactor performance and stability. An FBR allows scientists to conduct different types of chemical reactions involving multiphase materials, especially interaction between gas and solids. During such complex chemical processes, the formation of void regions in the reactor, generally termed as bubbles, is an important phenomenon. The study of these bubbles has a deep implication in predicting the reactor’s overall efficiency. But physical experiments needed to understand bubble dynamics are costly and non-trivial due to the technical difficulties involved and harsh working conditions of the reactors. Therefore, to study such chemical processes and bubble dynamics, a state-of-the-art computational simulation MFIX-Exa is being developed. Despite the proven accuracy of MFIX-Exa in modeling bubbling phenomena, the large-scale output data prohibits the use of traditional post hoc analysis capabilities in both storage and I/O time. Herein, to address these issues and allow the application scientists to explore the bubble dynamics in an efficient and timely manner, we have developed an end-to-end analytics pipeline that enables in situ detection of bubbles, followed by a flexible post hoc visual exploration methodology of bubble dynamics. The proposed method enables interactive analysis of bubbles, along with quantification of several bubble characteristics, enabling experts to understand the bubble interactions in detail. Positive feedback from the experts has indicated the efficacy of the proposed approach for exploring bubble dynamics in very-large-scale multiphase flow simulations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Subject-specific modeling framework for particle deposition using computational fluid dynamics

Quantifying particle deposition and dose in the respiratory tract requires a physiologically realistic representation and reproducible computational workflows. However, existing modeling frameworks, such as the International Commission on Radiological Protection (ICRP) compartmental models and the Multiple Path Particle Dosimetry (MPPD) tool, lack detailed deposition profiles and subject-specific capabilities. The combination of advances in computer vision algorithms applied to the respiratory tract and Computational Fluid and Particle Dynamics (CFPD) allows high-fidelity simulations of particle behavior in anatomically accurate geometries derived from individual CT scans. The segmentation, preprocessing, and file preparation task for a CFPD simulation was often time-consuming, and no prior studies to-date have yet presented a fully automated framework. This work presents a fully automated workflow to obtain individualized particle deposition profiles in the human respiratory tract. The pipeline starts with segmenting upper and lower airway geometries using morphological and deep learning-based methods, generating three-dimensional (3D) models from CT imaging data. Next, a series of algorithms are presented to quality check and prepare the 3D geometry for a CFD or CFPD simulation. The preprocessing step includes correcting geometric artifacts, enforcing a physically consistent mesh, and automatically identifying and capping multiple outlets, which is required for CFD/CFPD simulations. These processed models are then input into open-source (OpenFOAM) or commercial (StarCCM+) CFD solvers, where flow and transient particle transport equations — including turbulence and particle–wall interactions are solved under realistic breathing conditions. Finally, the resulting particle deposition profiles can be integrated with Monte Carlo radiation transport codes and state-of-the-art computational phantoms to assess organ-specific absorbed doses in scenarios of radioactive aerosol inhalation. The presented work streamlines respiratory tract segmentation, preprocessing for CFD/CFPD simulations, and integration with dose assessment workflows, reducing manual intervention and improving access to high-fidelity, subject-specific modeling. The high precision in predicted particle deposition and dose distributions can improve personalized treatment strategies in respiratory medicine and refine dose estimates for radiation protection.

AI↗
NASA NTRS2018

Generalizing a Data Analysis Pipeline in the Cloud to Handle Diverse Use Cases in NASA's EOSDIS

NASA's Earth Observing System Data and Information System (EOSDIS) is tasked with archiving and distributing Earth Observation data across a range of disciplines, including atmospheric science, oceanography, land processes, natural hazards, solar radiance and even socioeconomic aspects relating to the environment. Driven by rapidly rising data volumes, EOSDIS is migrating to a cloud computing based archive over the next few years. Although this simplifies data management somewhat, the main aim is to provide the data in an environment where end users can bring their analysis to the data rather than attempting to download and manage ever-increasing volumes. To that end, a cloud-based analysis platform is being constructed to enable data transformations, analyses and visualization without egressing the data from the cloud. In this endeavor, we expect a wide variety of users, algorithms and use cases. Consequently, the architecture of this cloud analytics platform is expressly designed to be based on open services, thus fostering an ecosystem that enables the efficient combination of common components with data-specific or analysis-specific components. Reviewed and approved by Andrew Mitchell, ESDIS project manager.

Cloud computing↗
DOE OSTI2026

Make the Fastest Faster: Importance Mask Synthesis for Interactive Volume Visualization using Reconstruction Neural Networks

Visualizing a large-scale volumetric dataset with high resolution is challenging due to the substantial computational time and space complexity. Recent deep learning-based image inpainting methods significantly improve rendering latency by reconstructing a high-resolution image for visualization in constant time on GPU from a partially rendered image where only a portion of pixels go through the expensive rendering pipeline. However, existing solutions need to render every pixel of either a predefined regular sampling pattern or an irregular sample pattern predicted from a low-resolution image rendering. Both methods require a significant amount of expensive pixel-level rendering. In this work, we provide Importance Mask Learning (IML) and Synthesis (IMS) networks, which are the first attempts to directly synthesize important regions of the regular sampling pattern from the user’s view parameters, to further minimize the number of pixels to render by jointly considering the dataset, user behavior, and the downstream reconstruction neural network. Our solution is a unified framework to handle various types of inpainting methods through the proposed differentiable compaction/decompaction layers. Experiments show our method can further improve the overall rendering latency of state-of-the-art volume visualization methods using reconstruction neural network for free when rendering scientific volumetric datasets. Our method can also directly optimize the off-the-shelf pre-trained reconstruction neural networks without elongated retraining.

Large-scale data↗
NASA NTRS1998

Algorithms and Libraries

This exploratory study initiated our inquiry into algorithms and applications that would benefit by latency tolerant approach to algorithm building, including the construction of new algorithms where appropriate. In a multithreaded execution, when a processor reaches a point where remote memory access is necessary, the request is sent out on the network and a context--switch occurs to a new thread of computation. This effectively masks a long and unpredictable latency due to remote loads, thereby providing tolerance to remote access latency. We began to develop standards to profile various algorithm and application parameters, such as the degree of parallelism, granularity, precision, instruction set mix, interprocessor communication, latency etc. These tools will continue to develop and evolve as the Information Power Grid environment matures. To provide a richer context for this research, the project also focused on issues of fault-tolerance and computation migration of numerical algorithms and software. During the initial phase we tried to increase our understanding of the bottlenecks in single processor performance. Our work began by developing an approach for the automatic generation and optimization of numerical software for processors with deep memory hierarchies and pipelined functional units. Based on the results we achieved in this study we are planning to study other architectures of interest, including development of cost models, and developing code generators appropriate to these architectures.

Dongarra, Jack↗
DOE OSTI2021

Integrated System and Application Continuous Performance Monitoring and Analysis Capability (Final)

Scientific applications run on high-performance computing (HPC) systems are critical for many national security missions within Sandia and the NNSA complex. However, these applications often face performance degradation and even failures that are challenging to diagnose. To provide unprecedented insight into these issues, the HPC Development, HPC Systems, Computational Science, and Plasma Theory & Simulation departments at Sandia crafted and completed their FY21 ASC Level 2 milestone entitled "Integrated System and Application Continuous Performance Monitoring and Analysis Capability." The milestone created a novel integrated HPC system and application monitoring and analysis capability by extending Sandia’s Kokkos application portability framework, Lightweight Distributed Metric Service (LDMS) monitoring tool, and scalable storage, analysis, and visualization pipeline. The extensions to Kokkos and LDMS enable collection and storage of application data during run time, as it is generated, with negligible overhead. This data is combined with HPC system data within the extended analysis pipeline to present relevant visualizations of derived system and application metrics that can be viewed at run time or post run. This new capability was evaluated using several week-long, 290-node runs of Sandia’s ElectroMagnetic Plasma In Realistic Environments (EMPIRE) modeling and design tool and resulted in 1TB of application data and 50TB of system data. EMPIRE developers remarked this capability was incredibly helpful for quickly assessing application health and performance alongside system state. In short, this milestone work built the foundation for expansive HPC system and application data collection, storage, analysis, visualization, and feedback framework that will increase total scientific output of Sandia’s HPC users.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRS1989

Optical backplane interconnect switch for data processors and computers

An optoelectronic integrated device design is reported which can be used to implement an all-optical backplane interconnect switch. The switch is sized to accommodate an array of processors and memories suitable for direct replacement into the basic avionic multiprocessor backplane. The optical backplane interconnect switch is also suitable for direct replacement of the PI bus traffic switch and at the same time, suitable for supporting pipelining of the processor and memory. The 32 bidirectional switchable interconnects are configured with broadcast capability for controls, reconfiguration, and messages. The approach described here can handle a serial interconnection of data processors or a line-to-link interconnection of data processors. An optical fiber demonstration of this approach is presented.

Hendricks, Herbert D.↗