Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Computational pipelines”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Meta2DB: Curated Shotgun Metagenomic Feature Sets and Metadata for Health State Prediction

Meta2DB is a curated metagenomic and metadata database that provides structurally consistent microbiome taxonomy feature count tables for 13 897 samples across 84 studies, 23 disease states, and 34 geographical locations. All samples were uniformly processed using a streamlined metagenomic classification pipeline that employs a unique and comprehensive reference database indexed to contain all sequences across all kingdoms of life that were present in the NCBI Nucleotide (nt) database retrieved on 4 January 2023. This pipeline leverages high-performance computing (HPC) resources at Lawrence Livermore National Laboratory and was used to process 50TB of publicly available raw metagenomic sequence data. Extensive metadata curation was carried out through a combination of manual curation and automated parsing, producing a consistent inter-study metadata table specifically structured to facilitate training of ML models for prediction of human health.

Kok, C [Lawrence Livermore National Laboratory (LL↗
DOE OSTI2020

PtychoShelves , a versatile high-level framework for high-performance analysis of ptychographic data

Over the past decade, ptychography has been proven to be a robust tool for non-destructive high-resolution quantitative electron, X-ray and optical microscopy. It allows for quantitative reconstruction of the specimen's transmissivity, as well as recovery of the illuminating wavefront. Additionally, various algorithms have been developed to account for systematic errors and improved convergence. With fast ptychographic microscopes and more advanced algorithms, both the complexity of the reconstruction task and the data volume increase significantly. PtychoShelves is a software package which combines high-level modularity for easy and fast changes to the data-processing pipeline, and high-performance computing on CPUs and GPUs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Unified Nanotechnology Format: One Way to Store Them All

The domains of DNA and RNA nanotechnology are steadily gaining in popularity while proving their value with various successful results, including biosensing robots and drug delivery cages. Nowadays, the nanotechnology design pipeline usually relies on computer-based design (CAD) approaches to design and simulate the desired structure before the wet lab assembly. To aid with these tasks, various software tools exist and are often used in conjunction. However, their interoperability is hindered by a lack of a common file format that is fully descriptive of the many design paradigms. Therefore, in this paper, we propose a Unified Nanotechnology Format (UNF) designed specifically for the biomimetic nanotechnology field. UNF allows storage of both design and simulation data in a single file, including free-form and lattice-based DNA structures. By defining a logical and versatile format, we hope it will become a widely accepted and used file format for the nucleic acid nanotechnology community, facilitating the future work of researchers and software developers. Together with the format description and publicly available documentation, we provide a set of converters from existing file formats to simplify the transition. Finally, we present several use cases visualizing example structures stored in UNF, showcasing the various types of data UNF can handle.

77 NANOSCIENCE AND NANOTECHNOLOGY↗
DOE OSTI2021

Large-scale Gravitational Lens Modeling with Bayesian Neural Networks for Accurate and Precise Inference of the Hubble Constant

We investigate the use of approximate Bayesian neural networks (BNNs) in modeling hundreds of time delay gravitational lenses for Hubble constant (H 0 ) determination. Here, our BNN was trained on synthetic Hubble Space Telescope quality images of strongly lensed active galactic nuclei with lens galaxy light included. The BNN can accurately characterize the posterior probability density functions (PDFs) of model parameters governing the elliptical power-law mass profile in an external shear field. We then propagate the BNN-inferred posterior PDFs into an ensemble H 0 inference, using simulated time delay measurements from a plausible dedicated monitoring campaign. Assuming well-measured time delays and a reasonable set of priors on the environment of the lens, we achieve a median precision of 9.3% per lens in the inferred H 0 . A simple combination of a set of 200 test lenses results in a precision of 0.5 km s –1 Mpc –1 (0.7%), with no detectable bias in this H 0 recovery test. The computation time for the entire pipeline—including the generation of the training set, BNN training and H0 inference—translates to 9 minutes per lens on average for 200 lenses and converges to 6 minutes per lens as the sample size is increased. Being fully automated and efficient, our pipeline is a promising tool for exploring ensemble-level systematics in lens modeling for H 0 inference.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Simulation of gas mixture dynamics in a pipeline network using explicit staggered-grid discretization

Here we develop an explicit staggered finite difference discretization scheme for simulating the transport of highly heterogeneous gas mixtures through pipeline networks. This study is motivated by the proposed blending of hydrogen into natural gas pipelines to reduce end use carbon emissions while using existing pipeline systems throughout their planned lifetimes. Our computational method accommodates an arbitrary number of constituent gases with very different physical properties that may be injected into a network with significant spatiotemporal variation. In this setting, the gas flow physics are highly location- and time- dependent, so that local composition and nodal mixing must be accounted for. The resulting conservation laws are formulated in terms of pressure, partial densities and flows, and volumetric and mass fractions of the constituents. We include non-ideal equations of state that employ linear approximations of gas compressibility factors, so that the pressure dynamics propagate locally according to a variable wave speed that depends on mixture composition and density. We derive compatibility relationships for network edge boundary values that are more complex than for a homogeneous gas. The simulation method is evaluated on initial boundary value problems for a single pipe and a small network, is cross-validated with a lumped element simulation, and used to demonstrate a local monitoring and control policy for maintaining allowable concentration levels.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

VIBES: a workflow for annotating and visualizing viral sequences integrated into bacterial genomes

Abstract Bacteriophages are viruses that infect bacteria. Many bacteriophages integrate their genomes into the bacterial chromosome and become prophages. Prophages may substantially burden or benefit host bacteria fitness, acting in some cases as parasites and in others as mutualists. Some prophages have been demonstrated to increase host virulence. The increasing ease of bacterial genome sequencing provides an opportunity to deeply explore prophage prevalence and insertion sites. Here we present VIBES (Viral Integrations in Bacterial genomES), a workflow intended to automate prophage annotation in complete bacterial genome sequences. VIBES provides additional context to prophage annotations by annotating bacterial genes and viral proteins in user-provided bacterial and viral genomes. The VIBES pipeline is implemented as a Nextflow-driven workflow, providing a simple, unified interface for execution on local, cluster and cloud computing environments. For each step of the pipeline, a container including all necessary software dependencies is provided. VIBES produces results in simple tab-separated format and generates intuitive and interactive visualizations for data exploration. Despite VIBES’s primary emphasis on prophage annotation, its generic alignment-based design allows it to be deployed as a general-purpose sequence similarity search manager. We demonstrate the utility of the VIBES prophage annotation workflow by searching for 178 Pf phage genomes across 1072 Pseudomonas spp. genomes.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Pipeline Structural Stability Under Nuclear Blast Pressure Loading

This report documents a set of simplified models to predict pipeline collapse under nuclear pressure loading. After a review of pipeline design literature, a set of simple expressions have been selected to represent an approximation of the threshold pressure for failure from cross- sectional yielding, cross-section buckling, and longitudinal buckling. These expressions provide a first-order approximation on load levels needed to achieve collapse. As a demonstration, the collapse pressure for a set of representative pipelines are calculated. Estimated pressure fields are also computed for a set of nuclear detonations, providing estimates of the ground range limit for pipeline collapse.

42 ENGINEERING↗
DOE OSTI2021

A Semi-Analytical Solution Approach for Solving Constant-Coefficient First-Order Partial Differential Equations

Simulation and control of many dynamic systems involve solving partial differential equations (PDE). This letter proposes a semi-analytical solution (SAS) approach for fast and high-quality solution of first-order PDEs. The region of interest of the studied PDE is divided into a grid, and an SAS is derived for each grid cell in the form of the multivariate polynomials, of which the coefficients are identified using initial value and boundary value conditions. The solutions are solved in a “time-stepping” manner, i.e. within one time step, the coefficients of the SAS are identified and the initial value of the next time step is evaluated. This approach achieves a significantly larger grid cell than the widely used finite difference method, and thus enhances the computational efficiency significantly. Furthermore, the simulation result on the natural gas pipeline model demonstrates the advantages of SAS in accuracy and computational efficiency.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

An FPGA-based hardware accelerator supporting sensitive sequence homology filtering with profile hidden Markov models

Abstract Background Sequence alignment lies at the heart of genome sequence annotation. While the BLAST suite of alignment tools has long held an important role in alignment-based sequence database search, greater sensitivity is achieved through the use of profile hidden Markov models (pHMMs). Here, we describe an FPGA hardware accelerator, called HAVAC, that targets a key bottleneck step (SSV) in the analysis pipeline of the popular pHMM alignment tool, HMMER. Results The HAVAC kernel calculates the SSV matrix at 1739 GCUPS on a $$\sim$$ ∼ $3000 Xilinx Alveo U50 FPGA accelerator card, $$\sim$$ ∼ 227× faster than the optimized SSV implementation in nhmmer . Accounting for PCI-e data transfer data processing, HAVAC is 65× faster than nhmmer’s SSV with one thread and 35× faster than nhmmer with four threads, and uses $$\sim$$ ∼ 31% the energy of a traditional high end Intel CPU. Conclusions HAVAC demonstrates the potential offered by FPGA hardware accelerators to produce dramatic speed gains in sequence annotation and related bioinformatics applications. Because these computations are performed on a co-processor, the host CPU remains free to simultaneously compute other aspects of the analysis pipeline.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Operation of Natural Gas Pipeline Networks With Storage Under Transient Flow Conditions

Here, we formulate a nonlinear optimal control problem for intraday operation of a natural gas pipeline network that includes storage reservoirs. The dynamics of compressible gas flow through pipes, compressors, reservoirs, and wells are considered. In particular, a reservoir is modeled as a rigid, hollow container that stores gas under isothermal conditions and uniform density, and a well is modeled as a vertical pipe. For each pipe, flow dynamics are described by a coupled partial differential equation (PDE) system in density and mass flux variables, with momentum dissipation modeled using the Darcy–Wiesbach friction approximation. Compressors are modeled as scaling up the pressure of gas between the inlet and outlet. The governing equations for all network components are spatially discretized and assembled into a nonlinear differential-algebraic equation (DAE) system, which synthesizes above-ground pipeline and subsurface reservoir dynamics into a single reduced-order model. We seek to maximize an objective function that quantifies economic profit and network efficiency subject to the flow equations and inequalities that represent operating limitations. The problem is solved using a primal–dual interior point solver, and the solutions are validated in computational experiments and simulations on several pipeline test networks to demonstrate the effectiveness of the proposed methodology.

03 NATURAL GAS↗
DOE OSTI2025

Data for "Discovery, Characterization, and Application of Chromosomal Integration Sites in the Hyperthermophilic Archaeon Sulfolobus islandicus"

Sulfolobus islandicus , an emerging archaeal model organism, offers unique advantages for metabolic engineering and synthetic biology applications owing to its ability to thrive in extreme environments. Although several genetic tools have been established for this organism, the lack of well-characterized chromosomal integration sites has limited its potential as a cellular factory. Here, we systematically identified and characterized 13 artificial CRISPR RNAs targeting eight integration sites in S. islandicus using the CRISPR-COPIES pipeline and a multi-omics-informed computational workflow. We leveraged the endogenous CRISPR-Cas system to integrate the reporter gene lacS and validated heterologous expression through a β-galactosidase assay, revealing significant positional effects. As a proof of concept, we utilized these sites to genetically manipulate lipid ether composition by overexpressing glycerol dibiphytanyl glycerol tetraether (GDGT) ring synthase B (GrsB). This study expands the genetic toolbox for S. islandicus and advances its potential as a robust platform for archaeal synthetic biology and industrial biotechnology.

AI/ML↗
DOE OSTI2020

Accelerated cryo-EM-guided determination of three-dimensional RNA-only structures

The discovery and design of biologically important RNA molecules is outpacing three-dimensional structural characterization. In this work, we demonstrate that cryo-electron microscopy can routinely resolve maps of RNA-only systems and that these maps enable subnanometer-resolution coordinate estimation when complemented with multidimensional chemical mapping and Rosetta DRRAFTER computational modeling. Furthermore ,this hybrid ‘Ribosolve’ pipeline detects and falsifies homologies and conformational rearrangements in 11 previously unknown 119- to 338-nucleotide protein-free RNA structures: full-length Tetrahymena ribozyme, hc16 ligase with and without substrate, full-length Vibrio cholerae and Fusobacterium nucleatum glycine riboswitch aptamers with and without glycine, Mycobacterium SAM-IV riboswitch with and without S-adenosylmethionine, and the computer-designed ATP-TTR-3 aptamer with and without AMP. Simulation benchmarks, blind challenges, compensatory mutagenesis, cross-RNA homologies and internal controls demonstrate that Ribosolve can accurately resolve the global architectures of RNA molecules but does not resolve atomic details. These tests offer guidelines for making inferences in future RNA structural studies with similarly accelerated throughput.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Pipeline Hydrogen Decarbonization and Repurposing Analyzers (P-HyDRAs)

The Pipeline Hydrogen Decarbonization and Repurposing Analyzers (P-HyDRAs) are a set of prototype computational tools for simulating and optimizing midstream natural gas pipeline system operations subject to location and time-dependent hydrogen blending. The models can accurately resolve dynamic gas flows through large-scale pipeline networks using non-ideal gas equations of state. The codes can be used as decision support for planning and design decisions involving intra-day energy flow schedules as well as spatiotemporal economic values of natural gas, hydrogen, and net energy delivered to consumers while ensuring that pipeline hydraulic limitations, gas compressor station constraints, operational factors, and pre-existing shipping contracts are satisfied. The inputs to the codes are a model of the pipeline system as well as time-series data that specify boundary conditions on the network. For optimization, the code module requires price and quantity offers for natural gas and hydrogen and price and quantity bids for energy, which are used as time-dependent constraints in an optimal control problem. The outputs are time-series data that provide a predictive simulation of gas flows, mass fractions, and pressures, or with additional degrees of freedom give an approximately optimal solution for gas injections/withdrawals, compressor settings, and sensitivities to the objective function that provide locational values of energy.

Zlotnik, Anatoly↗
DOE OSTI2025

Critical Simulation Pipeline for COG Suites [Poster]

The CRItical Simulation Pipeline (CRISP) is a Python package for automating validation of reactor criticality benchmarks. CRISP supplies COG—a multi-particle radiation transport code maintained by the Nuclear Criticality Safety Division—with a pipeline to calculate k eff performance for 400+ benchmark experiments with 3,400+ configurations from the International Criticality Safety Benchmark Evaluation Project (ICSBEP). The pipeline includes four stages: materials configuration, input card templating, cluster submission, and results analysis. CRISP includes a command-line interface to facilitate user interaction.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Drugsniffer: An Open Source Workflow for Virtually Screening Billions of Molecules for Binding Affinity to Protein Targets

The SARS-CoV2 pandemic has highlighted the importance of efficient and effective methods for identification of therapeutic drugs, and in particular has laid bare the need for methods that allow exploration of the full diversity of synthesizable small molecules. While classical high-throughput screening methods may consider up to millions of molecules, virtual screening methods hold the promise of enabling appraisal of billions of candidate molecules, thus expanding the search space while concurrently reducing costs and speeding discovery. Here, we describe a new screening pipeline, called drugsniffer, that is capable of rapidly exploring drug candidates from a library of billions of molecules, and is designed to support distributed computation on cluster and cloud resources. As an example of performance, our pipeline required ~40,000 total compute hours to screen for potential drugs targeting three SARS-CoV2 proteins among a library of ~3.7 billion candidate molecules.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Fast Algorithms for Scientific Data Compression

Many scientific simulations and experiments generate terabytes to petabytes of data daily, necessitating data compression techniques. Unlike video and image compression, scientists require methods that accurately preserve primary data (PD) and derived quantities of interest (QoIs). In our previous work, we demonstrated the effectiveness of hybrid compression techniques that combine machine learning with traditional approaches. This paper presents innovative computational techniques aimed at expediting the compression pipeline. Our experiments, conducted on two distinct platforms with a large-scale XGC-based fusion simulation, demonstrate that the overhead incurred by these new approaches is less than one percent of the computational resources needed for the simulation.

Banerjee, Tania↗
DOE OSTI2024

A goldilocks computational protocol for inhibitor discovery targeting DNA damage responses including replication-repair functions

While many researchers can design knockdown and knockout methodologies to remove a gene product, this is mainly untrue for new chemical inhibitor designs that empower multifunctional DNA Damage Response (DDR) networks. Here, we present a robust Goldilocks (GL) computational discovery protocol to efficiently innovate inhibitor tools and preclinical drug candidates for cellular and structural biologists without requiring extensive virtual screen (VS) and chemical synthesis expertise. By computationally targeting DDR replication and repair proteins, we exemplify the identification of DDR target sites and compounds to probe cancer biology. Our GL pipeline integrates experimental and predicted structures to efficiently discover leads, allowing early-structure and early-testing (ESET) experiments by many laboratories. By employing an efficient VS protocol to examine protein-protein interfaces (PPIs) and allosteric interactions, we identify ligand binding sites beyond active sites, leveraging in silico advances for molecular docking and modeling to screen PPIs and multiple targets. A diverse 3,174 compound ESET library combines Diamond Light Source DSI-poised, Protein Data Bank fragments, and FDA-approved drugs to span relevant chemotypes and facilitate downstream hit evaluation efficiency for academic laboratories. Two VS per library and multiple ranked ligand binding poses enable target testing for several DDR targets. This GL library and protocol can thus strategically probe multiple DDR network targets and identify readily available compounds for early structural and activity testing to overcome bottlenecks that can limit timely breakthrough drug discoveries. By testing accessible compounds to dissect multi-functional DDRs and suggesting inhibitor mechanisms from initial docking, the GL approach may enable more groups to help accelerate discovery, suggest new sites and compounds for challenging targets including emerging biothreats and advance cancer biology for future precision medicine clinical trials.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Scalable Pattern Matching in Metadata Graphs via Constraint Checking

Pattern matching is a fundamental tool for answering complex graph queries. Unfortunately, existing solutions have limited capabilities: They do not scale to process large graphs and/or support only a restricted set of search templates or usage scenarios. Moreover, the algorithms at the core of the existing techniques are not suitable for today’s graph processing infrastructures relying on horizontal scalability and shared-nothing clusters, as most of these algorithms are inherently sequential and difficult to parallelize. In this article we present an algorithmic pipeline that bases pattern matching on constraint checking. The key intuition is that each vertex and edge participating in a match has to meet a set of constraints implicitly specified by the search template. These constraints can be verified independently and typically are less expensive to compute than searching the full template. The pipeline we propose generates these constraints and iterates over them to eliminate all the vertices and edges that do not participate in any match, thus reducing the background graph to a subgraph that is the union of all template matches—the complete set of all vertices and edges that participate in at least one match. Additional analysis can be performed on this annotated, reduced graph, such as full match enumeration, match counting, or computing vertex/edge centrality. Furthermore, a vertex-centric formulation for constraint checking algorithms exists, and this makes it possible to harness existing high-performance, vertex-centric graph processing frameworks. This technique (i) enables highly scalable pattern matching in metadata (labeled) graphs; (ii) supports arbitrary patterns with 100% precision; (iii) enables tradeoffs between precision and time-to-solution, while always selects all vertices and edges that participate in matches, thus offering 100% recall; and (iv) supports a set of popular data analytics scenarios. We implement our approach on top of HavoqGT, an open-source asynchronous graph processing framework, and demonstrate its advantages through strong and weak scaling experiments on massive scale real-world (up to 257 billion edges) and synthetic (up to 4.4 trillion edges) labeled graphs, respectively, and at scales (1,024 nodes / 36,864 cores), orders of magnitude larger than used in the past for similar problems. This article serves two purposes: First, it synthesises the knowledge accumulated during a long-term project. Second, it presents new system features, usage scenarios, optimizations, and comparisons with related work that strengthen the confidence that pattern matching based on iterative pruning via constraint checking is an effective and scalable approach in practice. The new contributions include the following: (i) We demonstrate the ability of the constraint checking approach to efficiently support two additional search scenarios that often emerge in practice, interactive incremental search and exploratory search. (ii) We empirically compare our solution with two additional state-of-the-art systems, Arabsque and TriAD. (iii) We show the ability of our solution to accommodate a more diverse range of datasets with varying properties, e.g., scale, skewness, label distribution, and match frequency. (iv) We introduce or extend a number of system features (e.g., work aggregation, load balancing, and the ability to cap the generated traffic) and design optimizations and demonstrate their advantages with respect to improving performance and scalability. (v) We present bottleneck analysis and insights into artifacts that influence performance. (vi) We present a theoretical complexity argument that motivates the performance gains we observe.

97 MATHEMATICS AND COMPUTING↗