Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “heterogeneous networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Towards a robust out-of-the-box neural network model for genomic data

The accurate prediction of biological features from genomic data is paramount for precision medicine and sustainable agriculture. For decades, neural network models have been widely popular in fields like computer vision, astrophysics and targeted marketing given their prediction accuracy and their robust performance under big data settings. Yet neural network models have not made a successful transition into the medical and biological world due to the ubiquitous characteristics of biological data such as modest sample sizes, sparsity, and extreme heterogeneity. Here, we investigate the robustness, generalization potential and prediction accuracy of widely used convolutional neural network and natural language processing models with a variety of heterogeneous genomic datasets. Mainly, recurrent neural network models outperform convolutional neural network models in terms of prediction accuracy, overfitting and transferability across the datasets under study. While the perspective of a robust out-of-the-box neural network model is out of reach, we identify certain model characteristics that translate well across datasets and could serve as a baseline model for translational researchers.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Heterogenous catalysis for oxygen tolerant photoredox atom transfer radical polymerization and small-molecule dehalogenation

Heterogeneous photocatalysts (PCs) have garnered attention for their sustainability and cost-effectiveness. Despite the existence of various types of these PCs, their synthesis often involves complex, multi-step procedures and laborious purification. Herein, we propose a simple method for attaching small-molecule photocatalytic species onto crosslinked 3-D polymer networks as insoluble scaffolds to create robust heterogeneous PCs. The highly swellable poly(ethylene glycol)-based ChemMatrix (CM) resin, known for its amphiphilic properties and high functional group loading, facilitated the covalent immobilization of the photoredox dye Eosin Y (EY), but also streamlined functionalization with Ir( III ) complexes. The resulting heterogeneous CM-EY demonstrated efficient photocatalytic performance in open-to-air dual photoredox catalysis of atom transfer radical polymerization (photo-ATRP) under green light. This was confirmed by the well-controlled synthesis of polymers with molecular masses ranging from 20 kDa to 300 kDa and low dispersities. Furthermore, CM-EY exhibited excellent photostability and recyclability over multiple cycles of ATRP. The heterogeneous catalysis of photo-ATRP provided high temporal control and enabled benign conditions for synthesizing protein-polymer hybrids (PPH). When combined with the initiator-modified CM (CM-BIB), CM-EY facilitated the solid-phase synthesis of homopolymers and block copolymers with recyclable performance. However, the coordinatively bound Ir@CM showed decreased catalytic activity and efficiency toward photoredox dehalogenation due to the leaching of active species during recycling. This study highlights the advantages of the covalent linking of catalysts to solid supports over non-covalent interactions, underscoring the potential of functionalized polymer resin as a promising scaffold. Such an approach offers customization and tunability, presenting opportunities for innovation in green chemistry.

Kapil, Kriti↗
DOE OSTI2022

Opinion dynamics in financial markets via random networks

We investigate financial market dynamics by introducing a heterogeneous agent-based opinion formation model. In this work, we organize individuals in a financial market according to their trading strategy, namely, whether they are noise traders or fundamentalists. The opinion of a local majority compels the market exchanging behavior of noise traders, whereas the global behavior of the market influences the decisions of fundamentalist agents. We introduce a noise parameter, q , to represent the level of anxiety and perceived uncertainty regarding market behavior, enabling the possibility of adrift financial action. We place individuals as nodes in an Erdös-Rényi random graph, where the links represent their social interactions. At any given time, individuals assume one of two possible opinion states ±1 regarding buying or selling an asset. The model exhibits fundamental qualitative and quantitative real-world market features such as the distribution of logarithmic returns with fat tails, clustered volatility, and the long-term correlation of returns. We use Student’s t distributions to fit the histograms of logarithmic returns, showing a gradual shift from a leptokurtic to a mesokurtic regime depending on the fraction of fundamentalist agents. Furthermore, we compare our results with those concerning the distribution of the logarithmic returns of several real-world financial indices.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Training neural networks using Metropolis Monte Carlo and an adaptive variant

Abstract We examine the zero-temperature Metropolis Monte Carlo (MC) algorithm as a tool for training a neural network by minimizing a loss function. We find that, as expected on theoretical grounds and shown empirically by other authors, Metropolis MC can train a neural net with an accuracy comparable to that of gradient descent (GD), if not necessarily as quickly. The Metropolis algorithm does not fail automatically when the number of parameters of a neural network is large. It can fail when a neural network’s structure or neuron activations are strongly heterogenous, and we introduce an adaptive Monte Carlo algorithm (aMC) to overcome these limitations. The intrinsic stochasticity and numerical stability of the MC method allow aMC to train deep neural networks and recurrent neural networks in which the gradient is too small or too large to allow training by GD. MC methods offer a complement to gradient-based methods for training neural networks, allowing access to a distinct set of network architectures and principles.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Benchmarking Operators in Deep Neural Networks for Improving Performance Portability of SYCL

SYCL is a portable programming model for heterogeneous computing, so it is important to obtain reasonable performance portability of SYCL. Towards the goal of better understanding and improving performance portability of SYCL for machine learning workloads, we have been developing benchmarks for basic operators in deep neural networks (DNNs). These operators could be offloaded to heterogeneous computing devices such as graphics processing units (GPUs) to speed up computation. In this paper, we introduce the benchmarks, evaluate the performance of the operators on GPU-based systems, and describe the causes of the performance gap between the SYCL and Compute Unified Device Architecture (CUDA) kernels. We find that the causes are related to the utilization of the texture cache for read-only data, optimization of the memory accesses with strength reduction, use of local memory, and register usage per thread. We hope that the efforts of developing benchmarks for studying performance portability will stimulate discussion and interactions within the community.

Jin, Zheming [ORNL] (ORCID:000000027197780X)↗
DOE OSTI2024

Evaluating Operators in Deep Neural Networks for Improving Performance Portability of SYCL

SYCL is a portable programming model for heterogeneous computing, so it is important to obtain reasonable performance portability of SYCL. Towards the goal of better understanding and improving performance portability of SYCL for machine learning workloads, we have been developing benchmarks for basic operators in deep neural networks (DNNs). These operators could be offloaded to heterogeneous computing devices such as graphics processing units (GPUs) to speed up computation. In this work, we introduce the benchmarks, evaluate the performance of the operators on GPU-based systems, and describe the causes of the performance gap between the SYCL and Compute Unified Device Architecture (CUDA) kernels. We find that the causes are related to the utilization of the texture cache for read-only data, optimization of the memory accesses with strength reduction, shared local memory accesses, and register usage per thread. We hope that the efforts of developing benchmarks for studying performance portability will stimulate discussion and interactions within the community.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Neural Network Sampling of the Free Energy Landscape for Nitrogen Dissociation on Ruthenium

In heterogeneous catalysis, free energy profiles of reactions govern the mechanisms, rates, and equilibria. Energetics are conventionally computed using the harmonic approximation (HA), which requires determination of critical states a priori. Here, we use neural networks to efficiently sample and directly calculate the free energy surface (FES) of a prototypical heterogeneous catalysis reaction—the dissociation of molecular nitrogen on ruthenium—at density-functional-theory-level accuracy. We find that the vibrational entropy of surface atoms, often neglected in HA for transition metal catalysts, contributes significantly to the reaction barrier. The minimum free energy path for dissociation reveals an “on-top” adsorbed molecular state prior to the transition state. While a previously reported flat-lying molecular metastable state can be identified in the potential energy surface, it is absent in the FES at relevant reaction temperatures. Finally, these findings demonstrate the importance of identifying critical points self-consistently on the FES for reactions that involve considerable entropic effects.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Clarifying trust of materials property predictions using neural networks with distribution-specific uncertainty quantification

It is critical that machine learning (ML) model predictions be trustworthy for high-throughput catalyst discovery approaches. Uncertainty quantification (UQ) methods allow estimation of the trustworthiness of an ML model, but these methods have not been well explored in the field of heterogeneous catalysis. Herein, we investigate different UQ methods applied to a crystal graph convolutional neural network to predict adsorption energies of molecules on alloys from the Open Catalyst 2020 dataset, the largest existing heterogeneous catalyst dataset. We apply three UQ methods to the adsorption energy predictions, namely k-fold ensembling, Monte Carlo dropout, and evidential regression. The effectiveness of each UQ method is assessed based on accuracy, sharpness, dispersion, calibration, and tightness. Evidential regression is demonstrated to be a powerful approach for rapidly obtaining tunable, competitively trustworthy UQ estimates for heterogeneous catalysis applications when using neural networks. Recalibration of model uncertainties is shown to be essential in practical screening applications of catalysts using uncertainties.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Deep Learning Prediction of Interspecies Interactions from Self-organized Spatiotemporal Patterns of Co-evolving Organisms

Microorganisms colonizing natural habits such as soils co-evolve to form specific spatial patterns through interspecies interactions. These self-organized patterns are a key ecological phenotype, which provides critical information on their interaction mechanisms. However, conventional network inference techniques that analyze species population data in bulk have yet to be extended to account for such spatial heterogeneity. Here we proposed supervised deep learning as a new network inference tool for predicting interspecies interactions from spatiotemporal patterns of microbial evolution. Due to lack of biological imaging data that can be used for training deep learning networks, we used in silico data generated from high-fidelity agent-based models to determine model structure and parameters. Even though networks were trained under simple configurations where interaction coefficients are assumed to be spatially invariant, we demonstrated that the resulting model can be utilized to successfully predict spatial variation of interactions in more complex domains (i.e., configured with a context-dependent mixture of interaction coefficients) as well as in simple domains without further training. In the further test against real biological data obtained through imaging experiments of a binary consortium (Pseudomonas fluorescens and a mutant of Escherichia coli), our model also predicted the dramatic shifts in interactions of the two organisms across different environmental contexts. Through various successful demonstrations in this work, the combined use of the agent-based model and machine learning algorithm provides a means to use new type of data - microscopic images - for extracting microbial interactions, therefore presenting itself as a useful tool for the analysis of more complex microbial community interactions.

Lee, Joon-Yong↗
NASA NTRS1984

Measurements of mantle wave velocities and inversion for lateral heterogeneity and anisotropy. II - Analysis by the single-station method

In the present investigation, the single-station method reported by Brune et al. (1960) is utilized for an analysis of long-period Love(G) and Rayleigh(R) waves recorded on digital seismic networks. The analysis was conducted to study the lateral heterogeneity of surface wave velocities. The data set is examined, and a description is presented of the single-station method. Attention is given to an error analysis for velocity measurements, the estimation of geographical distribution of surface wave velocities, the global distribution of surface wave velocities, and the correlation between the surface wave velocities and the heat flow on the geoid. The conducted measurements and inversions of surface wave velocities are used as a basis to derive certain conclusions. It is found that the application of the single-station method to long-period surface waves recorded on digital networks makes it possible to reach an accuracy level comparable to great circle velocity measurements.

Nakanishi, I.↗
DOE OSTI2025

Model Development and Analysis of a High-Fidelity Neutron Transport Sensor: The Quadrupole Detector Concept for Measurement of the Neutron Flux Gradient

Accurate reconstruction of the neutron flux distribution within a reactor core is essential for safe and efficient reactor operation. Traditional power shape synthesis in Light Water Reactors relies on hundreds of in-core detectors. However, this approach becomes impractical for Advanced Reactors and Microreactors due to limited space and harsh environments. To address this challenge, we propose a data-driven methodology that combines high-fidelity modeling with real-time ex-core sensor measurements, enabling the reconstruction of core power distribution while minimizing the reliance on intrusive in-core instrumentation. This project began in FY24 and achieved two initial milestones: (1) the definition of a three-year development plan for a Digital Twin framework and (2) the development of high-fidelity neutronics models of the Purdue University Reactor One (PUR-1) using both MCNP6 and OpenMC. The PUR-1 reactor, a zero-power facility, was selected due to its suitability for neutronics-focused modeling and the availability of experimental data for validation. Both models were benchmarked using neutron flux measurements obtained from irradiated gold foils, which were strategically placed within the core during a dedicated campaign in July 2024. This report marks the continuation and completion of those foundational tasks. The OpenMC model has been refined (improved geometric accuracy, expanded cross-section libraries, and refined sampling) and validated using additional experimental data. An updated sensor design—based on quadrupole configuration—was designed to measure both ex-core flux and its spatial gradient. These measurements will serve as inputs to a neural network-based reconstruction algorithm. Finally, the methodology was demonstrated on a two-dimensional test case representative of the heterogeneous material composition of the PUR-1 reactor core. A neural network implementation of the Kirchhoff-Helmholtz integral equation was employed to solve the boundary value problem using peripheral sensor measurements. The preliminary results confirm the strong potential of the proposed approach for accurate and minimally invasive neutron flux reconstruction.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

Anticipating Technical Expertise and Capability Evolution in Research Communities Using Dynamic Graph Transformers

The ability to anticipate global technical expertise and capability evolution trends is essential for national and global security, especially in safety-critical domains such as nuclear nonproliferation (NN) and rapidly emerging fields like artificial intelligence (AI). Here, in this work, we extend traditional statistical relational learning approaches (e.g., link prediction in collaboration networks) and formulate a problem of anticipating technical expertise and capability evolution using dynamic heterogeneous graph representations. We develop novel capabilities to forecast collaboration patterns, authorship behavior, and technical capability evolution at different granularities (e.g., scientist and institution levels) in two distinct research fields. We implement a dynamic graph transformer (DGT) neural architecture, which pushes the state-of-the-art graph neural network models by: 1) forecasting heterogeneous (rather than homogeneous) nodes and edges; and 2) relying on both discrete- and continuous-time inputs. We demonstrate that our DGT models predict collaboration, partnership, and expertise patterns with 0.26, 0.73, and 0.53 mean reciprocal rank values for AI and 0.48, 0.93, and 0.22 for NN domains. DGT model performance exceeds the best-performing static graph baseline models by 30%–80% across AI and NN domains. Our findings demonstrate that DGT models boost inductive task performance when previously unseen nodes appear in the test data for the domains with emerging collaboration patterns (e.g., AI). Specifically, models accurately predict which established scientists will collaborate with early career scientists and vice versa in the AI domain.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Solute clustering in polycrystals: Unveiling the interplay of grain boundary junction and long-range solute attraction effects

Spectral analysis of local atomic environments has become a powerful tool for studying solute atom segregation and interactions at grain boundaries in nanocrystalline alloys. When applied to individual grain boundaries, the spectral analysis has shown that solute-solute interaction can be either attractive or repulsive, with long-range relative attraction enhancing the likelihood for solute atoms to begin clustering. In this article, we combine this analysis with a new grain-boundary structure descriptor based on grain-boundary atom coordination, to investigate the impact of grain-boundary junctions on solute atom segregation in polycrystals. Specifically, we systematically characterize the tendency of solute clusters to begin forming at various types of ordinary grain boundaries, triple junctions, and high-order junctions in Ag polycrystals containing either Ni or Cu solute atoms. Our findings demonstrate that the formation of solute clusters at grain boundaries is primarily driven by long-range relative solute attraction, rather than short-range solute-solute interactions. Furthermore, this effect is most pronounced near grain-boundary junctions. Our study highlights the multiscale nature of solute segregation at crystalline interfaces and provides new insights into the complex phenomena governing heterogeneous solute segregation in grain-boundary networks.

Grain-boundary junction↗
DOE OSTI2020

Rapid Mineral Precipitation During Shear Fracturing of Carbonate-Rich Shales

Target subsurface reservoirs for emerging low-carbon energy technologies and geologic carbon sequestration typically have low permeability and thus rely heavily on fluid transport through natural and induced fracture networks. Sustainable development of these systems requires deeper understanding of how geochemically mediated deformation impacts fracture microstructure and permeability evolution, particularly with respect to geochemical reactions between pore fluids and the host rock. Here, a series of triaxial direct shear experiments was designed to evaluate how fractures generated at subsurface conditions respond to penetration of reactive fluids with a focus on the role of mineral precipitation. Calcite-rich shale cores were directly sheared under 3.5 MPa confining pressure using BaCl 2 -rich solutions as a working fluid. Experiments were conducted within an X-ray computed tomography (xCT) scanner to capture 4-D evolution of fracture geometry and precipitate growth. Three shear tests evidenced nonuniform precipitation of barium carbonates (BaCO 3 ) along through-going fractures, where the extent of precipitation increased with increasing calcite content. Precipitates were strongly localized within fracture networks due to mineral, geochemical, and structural heterogeneities and generally concentrated in smaller apertures where rock:water ratios were highest. The combination of elevated fluid saturation and reactive surface area created in freshly activated fractures drove near-immediate mineral precipitation that led to an 80% permeability reduction and significant flow obstruction in the most reactive core. While most previous studies have focused on mixing-induced precipitation, this work demonstrates that fluid–rock interactions can trigger precipitation-induced permeability alterations that can initiate or mitigate risks associated with subsurface energy systems.

58 GEOSCIENCES↗
DOE OSTI2023

Physics-embedded inverse analysis with algorithmic differentiation for the earth’s subsurface

Abstract Inverse analysis has been utilized to understand unknown underground geological properties by matching the observational data with simulators. To overcome the underconstrained nature of inverse problems and achieve good performance, an approach is presented with embedded physics and a technique known as algorithmic differentiation. We use a physics-embedded generative model, which takes statistically simple parameters as input and outputs subsurface properties (e.g., permeability or P-wave velocity), that embeds physical knowledge of the subsurface properties into inverse analysis and improves its performance. We tested the application of this approach on four geologic problems: two heterogeneous hydraulic conductivity fields, a hydraulic fracture network, and a seismic inversion for P-wave velocity. This physics-embedded inverse analysis approach consistently characterizes these geological problems accurately. Furthermore, the excellent performance in matching the observational data demonstrates the reliability of the proposed method. Moreover, the application of algorithmic differentiation makes this an easy and fast approach to inverse analysis when dealing with complicated geological structures.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Extending Rucio with modern cloud storage support

Rucio is a software framework designed to facilitate scientific collaborations in efficiently organising, managing, and accessing extensive volumes of data through customizable policies. The framework enables data distribution across globally distributed locations and heterogeneous data centres, integrating various storage and network technologies into a unified federated entity. Rucio offers advanced features like distributed data recovery and adaptive replication, and it exhibits high scalability, modularity, and extensibility. Originally developed to meet the requirements of the high-energy physics experiment ATLAS, Rucio has been continuously expanded to support LHC experiments and diverse scientific communities. Recent R&D projects within these communities have evaluated the integration of both private and commercially-provided cloud storage systems, leading to the development of additional functionalities for seamless integration within Rucio. Furthermore, the underlying systems, FTS and GFAL/Davix, have been extended to cater to specific use cases. This contribution focuses on the technical aspects of this work, particularly the challenges encountered in building a generic interface for self-hosted cloud storage, such as MinIO or CEPH S3 Gateway, and established providers like Google Cloud Storage and Amazon Simple Storage Service. Additionally, the integration of decentralised clouds like SEAL is explored. Key aspects, including authentication and authorisation, direct and remote access, throughput and cost estimation, are highlighted, along with shared experiences in daily operations.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS1990

The role of data management in discipline-independent data visualization

The common data format (CDF) is described in terms of its support applications for the database management of visualization systems. The CDF is a self-describing data abstraction technique for the storage and manipulation of multidimensional data that are based on block structures. The discipline-independent approach is designed to manage, manipulate, archive, display, and analyze data, and can be applied to heterogeneous equipment communicating different data structures over networks. An improved CDF version incorporates a hyperplane access allowing random aggregate access to subdimensional blocks within a multidimensional variable. The visualization pipeline is also discussed, which controls the flow of data and permits the visualization of different classes of data representation techniques. The system is found to accommodate a large variety of scientific data structures and large disk-based data sets.

Treinish, Lloyd A.↗