Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “scalable computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2018

Stochastic Simulation of Complex Fluid Flows (Progress Report for period 07/01/2016 - 06/30/2018)

At a microscopic scale, fluids are composed of molecules whose positions and velocities are random. This gives rise to thermal fluctuations that span the whole range of scales from the microscopic through the mesoscopic, and even the macroscopic. The inclusion of thermal fluctuations is crucial in multi-scale models, which are an important theme in the research program of the DOE Office of Science, and in particular the ASCR Applied Mathematics program's priority focus area on modeling of complex systems involving processes that span vastly different time and/or length scales. In this five-year Early Career project, the PI Aleksandar Donev and collaborators developed computational algorithms for modeling complex fluid mixtures at small scales using a formulation based on fluctuating hydrodynamics. Novel computational methods were developed to model complex fluids with increasing physical complexity, starting from binary miscible and immiscible mixtures, going through multispecies non-reactive and reactive mixtures, and culminating with reactive electrolytes mixtures of neutral molecules and ions. In close collaboration with the group of John Bell at Lawrence Berkeley National Laboratory, the methods were implemented in a scalable computational framework suitable for modern parallel supercomputers, and made publicly available on github. A number of physical examples in which giant nonequilibrium fluctuations are improtant were studied, with a special focus on instabilities at a liquid-liquid interface driven by gravity, diffusion, reactions, and/or electric fields. The methods and codes developed in this project are expected to enable other novel applications in the DOE Basic Energy Sciences program, and engineering sciences more broadly.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

PlasmoData.jl — A Julia framework for modeling and analyzing complex data as graphs

Datasets encountered in scientific and engineering applications appear in complex formats (e.g., images, multivariate time series, molecules, video, text strings, networks). Graph theory provides a unifying framework to model such datasets and enables the use of powerful tools that can help analyze, visualize, and extract value from data. In this work, we present PlasmoData.jl, an open-source, Julia framework that uses concepts of graph theory to facilitate the modeling and analysis of complex datasets. The core of our framework is a general data modeling abstraction, which we call a DataGraph. We show how the abstraction and software implementation can be used to represent diverse data objects as graphs and to enable the use of tools from topology, graph theory, and machine learning (e.g., graph neural networks) to conduct a variety of tasks. We illustrate the versatility of the framework by using real datasets: (i) an image classification problem using topological data analysis to extract features from the graph model to train machine learning models; (ii) a disease outbreak problem where we model multivariate time series as graphs to detect abnormal events; and (iii) a technology pathway analysis problem where we highlight how we can use graphs to navigate connectivity. Further, our discussion also highlights how PlasmoData.jl leverages native Julia capabilities to enable compact syntax, scalable computations, and interfaces with diverse packages. Overall, we show that the DataGraph abstraction and PlasmoData.jl Julia package are able to model data within graphs and enable useful analysis.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2014

Multiscale Simulation Framework for Coupled Fluid Flow and Mechanical Deformation

A multiscale linear-solver framework for the pressure equation associated with flow in highly heterogeneous porous formations was developed. The multiscale based approach is cast in a general algebraic form, which facilitates integration of the new scalable linear solver in existing flow simulators. The Algebraic Multiscale Solver (AMS) is employed as a preconditioner within a multi-stage strategy. The formulations investigated include the standard MultiScale Finite-Element (MSFE) andMultiScale Finite-Volume (MSFV) methods. The local-stage solvers include incomplete factorization and the so-called Correction Functions (CF) associated with the MSFV approach. Extensive testing of AMS, as an iterative linear solver, indicate excellent convergence rates and computational scalability. AMS compares favorably with advanced Algebraic MultiGrid (AMG) solvers for highly detailed three-dimensional heterogeneous models. Moreover, AMS is expected to be especially beneficial in solving time-dependent problems of coupled multiphase flow and transport in large-scale subsurface formations.

02 PETROLEUM↗
DOE OSTI2023

Robust scalable initialization for Bayesian variational inference with multi-modal Laplace approximations

Predictive modeling typically relies on Bayesian model calibration to provide uncertainty quantification. Variational inference utilizing fully independent (“mean-field”) Gaussian distributions are often used as approximate probability density functions. This simplification is attractive since the number of variational parameters grows only linearly with the number of unknown model parameters. However, the resulting diagonal covariance structure and unimodal behavior can be too restrictive to provide useful approximations of intractable Bayesian posteriors that exhibit highly non-Gaussian behavior, including multimodality. High-fidelity surrogate posteriors for these problems can be obtained by considering the family of Gaussian mixtures. Gaussian mixtures are capable of capturing multiple modes and approximating any distribution to an arbitrary degree of accuracy, while maintaining some analytical tractability. Unfortunately, variational inference using Gaussian mixtures with full-covariance structures suffers from a quadratic growth in variational parameters with the number of model parameters. The existence of multiple local minima due to strong nonconvex trends in the loss functions often associated with variational inference present additional complications, These challenges motivate the need for robust initialization procedures to improve the performance and computational scalability of variational inference with mixture models. In this work, we propose a method for constructing an initial Gaussian mixture model approximation that can be used to warm-start the iterative solvers for variational inference. The procedure begins with a global optimization stage in model parameter space. In this step, local gradient-based optimization, globalized through multistart, is used to determine a set of local maxima, which we take to approximate the mixture component centers. Around each mode, a local Gaussian approximation is constructed via the Laplace approximation. Finally, the mixture weights are determined through constrained least squares regression. The robustness and scalability of the proposed methodology is demonstrated through application to an ensemble of synthetic tests using high-dimensional, multimodal probability density functions. Here, the practical aspects of the approach are demonstrated with inversion problems in structural dynamics.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Assimilation of Multiscale Data into Multifidelity Biogeochemical Models (Final Report)

Quantitative predictions of subsurface processes rely on computational models that capture, with different degrees of fidelity, complex interactions between hydrologic and biogeochemical processes. Molecular- and pore-scale models provide a high-fidelity representation of these processes but are impractical at the field scale. Reduced complexity (e.g., field-scale or data-driven) models sacrifice some degree of fidelity in favor of computational efficiency. Uncertainty and assimilation of data into model predictions, pose a question of model selection: Given a significant difference in computational cost, can a lower-fidelity model be preferable to its higher-fidelity counterpart? Since a predictive model must be computable in reasonable time it has to operate at the field scale, with molecular- and pore-scale data (obtained either from simulations or measurements) determining both the model's structure and parameters. Availability of such multi-resolution data raises a question hitherto undressed in hydrogeology: Can a coarse model dynamically ``learn'' its own structure (e.g., adjust the reaction pathways in its transport module) as more fine-scale data become available during simulations? Our results in multifidelity simulations, data assimilation, and machine learning led us to hypothesize that the answer to these questions is ``yes''. The overarching goal of this project was to confirm this hypothesis by developing scalable, computationally efficient tools for assimilation of data into multiscale models, in which fine-scale data and simulations dynamically inform and autonomously modify coarse-scale models. Our research led to nine manuscripts, three of which have been published and the other six are currently under review.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2018

Toward Transparent Data Management in Multi-layer Storage Hierarchy for HPC Systems

Upcoming exascale high performance computing (HPC) systems are expected to comprise multi-tier storage hierarchy, and thus will necessitate innovative storage and I/O mechanisms. Traditional disk and block-based interfaces and file systems face severe challenges in utilizing capabilities of storage hierarchies due to the lack of hierarchy support and semantic interfaces. Object-based and semantically-rich data abstractions for scientific data management on large scale systems offer a sustainable solution to these challenges. Such data abstractions can also simplify users involvement in data movement. Here, we take the first steps of realizing such an object abstraction and explore storage mechanisms for these objects to enhance I/O performance, especially for scientific applications. We explore how an object-based interface can facilitate next generation scalable computing systems by presenting the mapping of data I/O from two real world HPC scientific use cases: a plasma physics simulation code (VPIC) and a cosmology simulation code (HACC). Our storage model stores data objects in different physical organizations to support data movement across layers of memory/storage hierarchy. Our implementation sclaes well to 16K parallel processes, and compared to the state of the art, such as MPI-IO and HDF5, our object-based data abstractions and data placement strategy in multi-level storage hierarchy achieves up to 7 X I/O performance improvement for scientific data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

A Graphical Model for Fusing Diverse Microbiome Data

This paper develops a Bayesian graphical model for fusing disparate types of count data. The motivating application is the study of bacterial communities from diverse high-dimensional features, in this case, transcripts, collected from different treatments. In such datasets, there are no explicit correspondences between the communities and each corresponds to different factors, making data fusion challenging. We introduce a flexible multinomial-Gaussian generative model for jointly modeling such count data. This latent variable model jointly characterizes the observed data through a common multivariate Gaussian latent space that parameterizes the set of multinomial probabilities of the transcriptome counts. The covariance matrix of the latent variables induces a covariance matrix of co-dependencies between all the transcripts, effectively fusing multiple data sources. We present a computationally scalable variational Expectation-Maximization (EM) algorithm for inferring the latent variables and the parameters of the model. Here, the inferred latent variables provide a common dimensionality reduction for visualizing the data and the inferred parameters provide a predictive posterior distribution. In addition to simulation studies that demonstrate the variational EM procedure, we apply our model to a bacterial microbiome dataset.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2019

A Bayesian Approach to Real-Time Dynamic Parameter Estimation Using Phasor Measurement Unit Measurement

Here, we develop a polynomial-chaos-expansion (PCE)-based approach for decentralized dynamic parameter estimation through Bayesian inference. Using this approach, the non-Gaussian distribution of the inverted parameters is obtained. More specifically, we first represent the decentralized generator model with the PCE-based surrogate. This surrogate allows us to efficiently evaluate the time-consuming dynamic solver at parameter values through Metropolis-Hastings (M-H)-based Markov chain Monte Carlo (MCMC). Then, we propose a two-stage hybrid Markov chain Monte Carlo (MCMC) to recover a posteriori distribution of the decentralized generator model parameters. In the first stage, we use the gradient-enhanced Langevin MCMC algorithm to characterize an intermediate posterior parameter distribution. This algorithm is computationally scalable to the high-dimensional parameter space. Based on the intermediate posterior distribution, during the second stage, we use the adaptive MCMC algorithm to fine-tune the strong correlations between the parameters. Finally, the fully recovered a posterior distribution is obtained in the end. The simulation results show that the proposed PCE-based hybrid MCMC algorithm can accurately and efficiently estimate the high-dimensional generator dynamic model parameters with full probabilistic distribution provided.

42 ENGINEERING↗
DOE OSTI2017

A lightweight network anomaly detection technique

While the network anomaly detection is essential in network operations and management, it becomes further challenging to perform the first line of detection against the exponentially increasing volume of network traffic. In this paper, we develop a technique for the first line of online anomaly detection with two important considerations: (i) availability of traffic attributes during the monitoring time, and (ii) computational scalability for streaming data. The presented learning technique is lightweight and highly scalable with the beauty of approximation based on the grid partitioning of the given dimensional space. With the public traffic traces of KDD Cup 1999 and NSL-KDD, we show that our technique yields 98.5% and 83% of detection accuracy, respectively, only with a couple of readily available traffic attributes that can be obtained without the help of post-processing. Finally, the results are at least comparable with the classical learning methods including decision tree and random forest, with approximately two orders of magnitude faster learning performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

CAEBAT OAS/VIBE Production Release v1.1

The objective of the project is to develop a mathematical and computational infrastructure, and modeling framework that will enable seamless multi-scale and multi-physics simulations of battery performance and safety. The modeling framework will transfer the information between models in a physically consistent and mathematically rigorous fashion for both spatial and temporal variations. The end result will be a verified, computationally scalable, portable, and flexible (extensible and easily-modified) framework that can integrate models from the other CAEBAT tasks and industrial partners. The framework will be used to validate models and modeling approaches against experiments and to support rapid prototyping of advanced battery concepts. Fig. 2 shows different parts of CAEBAT VIBE simulation environment that work together to provide user with flexibility in the problem setup, solution formulation and simulation launch. Each of the parts is discussed in subsequent sections with corresponding examples.

25 ENERGY STORAGE↗
DOE OSTI2026

Transplatformer: translating toxicogenomic profiles between generations of platforms

Background Transcriptomic profiling technologies have advanced the analysis of biological and toxicological responses. However, substantial differences in probe design, dynamic range, gene coverage, and preprocessing pipelines across platforms introduce artifacts that limit cross-study integration and hinder the reuse of historical datasets. We aim to develop computational methods for accurate cross-platform translation to maximize the value of legacy resources. Results We present TransPlatformer a deep learning framework for translating gene expression profiles across heterogeneous toxicogenomics platforms. TransPlatformer employs a novel attention-based architecture to map high-dimensional fold-change vectors from legacy microarray technologies to current platforms. Models are trained and evaluated using DrugMatrix, spanning three technological generations. We investigate mixed-tissue, single-tissue, and cross-tissue training paradigms and benchmark performance against multilayer perceptron and matrix-completion baselines. In mixed-tissue training, TransPlatformer achieves a greater than 50% reduction in mean absolute error (0.043 vs. 0.09) and nearly doubles Pearson correlation ( ≈ 0.71 vs. 0.37) relative to baseline methods. Importantly, TransPlatformer preserves rare but biologically meaningful over- and under-expressed signals, with mean absolute error below 0.22. Single-tissue models yield further improvements for well-represented organs, such as a 10% reduction in liver mean absolute error, while underscoring the need for data augmentation strategies in low-sample tissues.ra Conclusions TransPlatformer provides an effective and scalable computational solution for cross-platform transcriptomic translation. By enabling biologically faithful harmonization of gene expression data, the proposed approach facilitates the reuse of legacy toxicogenomics datasets, enhances downstream biomarker discovery, and supports more reproducible predictive modeling in toxicology.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

A Level Set Model for Structured Mesh Representation of Fibrous Preforms used in Composites Manufacturing

Here, a model for implicit representation of fibrous and woven preforms used for composites manufacturing is described. The method is based on a level set function defined on a structured mesh to implicitly capture the complex fiber and weave geometries. Since most software packages used to model woven preforms are based on unstructured mesh, a numerical model for transforming a discretized surface to a structured mesh representation and vice-a-versa is presented. Specific emphasis is on scalable computation of a level set function for large and complex 3D woven structures from a triangulated surface of the smallest unit (yarn). The model is applied to capture a periodic 3D layered 5-harness satin (5HS) weave geometry on a structured mesh. Geometric transformations needed to efficiently compute the level set function for the 3D structure on a high resolution mesh are described. Effects of mesh resolution on the geometric features such as number of resolved fibers, effective fiber size, and the surface area to volume ratio of the iso-surface are analyzed. Using the distance property of the level set function, an approach to evaluate the structure function resulting from transient evolving topology representing densification due to matrix phase depositions is also presented. The approach captures transition of the geometric characteristic from fiber governed at early stages of densification to yarn governed at the terminal stage.

36 MATERIALS SCIENCE↗
DOE OSTI2022

General, Rigorous Approach for the Treatment of Interfragment Covalent Bonds

Here, a generalized, projection-based transformation of the method-agnostic Fock operator in various ab initio fragment-based quantum chemistry methods has been developed for the treatment of interfragment covalent bonds. This transformation freezes the relevant localized molecular orbital associated with each interfragment bond, thereby restricting the variational subspace of the fragment wave functions, in order to maintain the proper physical characteristics of the involved covalent bonds. In addition, sets of orbitals that would lead to multiple occupancy of certain orbitals are explicitly removed from the variational space. The transformation is developed for the specific case of mutually orthonormal frozen and unfrozen orbitals within each fragment. The newly developed approach is then used to study model systems with two popular ab initio fragment-based methods, and the results of these calculations are compared to those obtained by existing methodologies. Analysis is focused on both quantitative and qualitative accuracy as well as computational scalability and stability. Other methods for which the developed formalisms are appropriate are outlined, and future extensions of the methods are discussed.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Genome-scale metabolic reconstruction of 7,302 human microorganisms for personalized medicine

The human microbiome influences the efficacy and safety of a wide variety of commonly prescribed drugs. Designing precision medicine approaches that incorporate microbial metabolism would require strain- and molecule-resolved, scalable computational modeling. Here, we extend our previous resource of genome-scale metabolic reconstructions of human gut microorganisms with a greatly expanded version. AGORA2 (assembly of gut organisms through reconstruction and analysis, version 2) accounts for 7,302 strains, includes strain-resolved drug degradation and biotransformation capabilities for 98 drugs, and was extensively curated based on comparative genomics and literature searches. The microbial reconstructions performed very well against three independently assembled experimental datasets with an accuracy of 0.72 to 0.84, surpassing other reconstruction resources and predicted known microbial drug transformations with an accuracy of 0.81. We demonstrate that AGORA2 enables personalized, strain-resolved modeling by predicting the drug conversion potential of the gut microbiomes from 616 patients with colorectal cancer and controls, which greatly varied between individuals and correlated with age, sex, body mass index and disease stages. AGORA2 serves as a knowledge base for the human microbiome and paves the way to personalized, predictive analysis of host–microbiome metabolic interactions.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2018

Communication: Calculation of interatomic forces and optimization of molecular geometry with auxiliary-field quantum Monte Carlo

We introduce an algorithm for accurate, systematic, and scalable computation of interatomic forces within the auxiliary-field quantum Monte Carlo (AFQMC) method. The algorithm relies on the Hellmann-Feynman theorem and incorporates Pulay corrections in the presence of atomic orbital basis sets. We benchmark the technique for small molecules by comparing the computed forces with the derivatives of the AFQMC potential energy surface and by direct comparison with other quantum chemistry methods. Furthermore, we perform geometry optimizations using the steepest descent algorithm in larger molecules. With realistic basis sets, we obtain equilibrium geometries in agreement, within statistical error bars, with experimental values. The increase in computational cost for computing forces in this approach is only a small prefactor over that of calculating the total energy. This paves the way for a general and efficient method for geometry optimization and molecular dynamics within AFQMC.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2020

Material coherence from trajectories via Burau eigenanalysis of braids

In this paper, we provide a numerical tool to study a material’s coherence from a set of 2D Lagrangian trajectories sampling a dynamical system, i.e., from the motion of passive tracers. We show that eigenvectors of the Burau representation of a topological braid derived from the trajectories have levelsets corresponding to components of the Nielsen–Thurston decomposition of the dynamical system. One can thus detect and identify clusters of space–time trajectories corresponding to coherent regions of the dynamical system by solving an eigenvalue problem. Unlike previous methods, the scalable computational complexity of our braid-based approach allows the analysis of large amounts of trajectories.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Structured illumination with thermal imaging (SI-TI): A dynamically reconfigurable metrology for parallelized thermal transport characterization

The recent push for the “materials by design” paradigm requires synergistic integration of scalable computation, synthesis, and characterization. Among these, techniques for efficient measurement of thermal transport can be a bottleneck limiting the experimental database size, especially for diverse materials with a range of roughness, porosity, and anisotropy. Traditional contact thermal measurements have challenges with throughput and the lack of spatially resolvable property mapping, while non-contact pump-probe laser methods generally need mirror smooth sample surfaces and also require serial raster scanning to achieve property mapping. Here, we present structured illumination with thermal imaging (SI-TI), a new thermal characterization tool based on parallelized all-optical heating and thermometry. Experiments on representative dense and porous bulk materials as well as a 3D printed thermoelectric thick film (~50 μm) demonstrate that SI-TI (1) enables paralleled measurement of multiple regions and samples without raster scanning; (2) can dynamically adjust the heating pattern purely in software, to optimize the measurement sensitivity in different directions for anisotropic materials; and (3) can tolerate rough (~3 μm) and scratched sample surfaces. Here, this work highlights a new avenue in adaptivity and throughput for thermal characterization of diverse materials.

42 ENGINEERING↗