Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “residual network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13

Physics-Informed Neural Network Method for Forward and Backward Advection-Dispersion Equations

Advection-dispersion equations (ADEs) are commonly used to describe transport phenomena in porous media. Even though mature discretization-based numerical methods for ADEs exist, some challenges still remain, especially when it comes to solving advection-dominated forward ADEs and diffusion-dominated backward ADEs. The latter problem usually arises in the source identification context and leads to numerically unstable grid-based solutions that require a form of regularization or should be treated as an inverse problem that is computationally more expensive because it requires solving the forward problem multiple times. In this study, we propose a discretization-free approach based on the physics-informed neural network (PINN) method for solving coupled ADE and Darcy flow equations with space-dependent hydraulic conductivity. In this approach, the hydraulic conductivity, hydraulic head, and concentration fields are approximated with deep neural networks (DNNs). We assume that the conductivity field is given by its values on a grid, and we use these values to train the conductivity DNN. The head and concentration DNNs are trained by minimizing the residuals of the flow equation and ADE and using the initial and boundary conditions as additional constraints. The PINN method is applied to one- and two-dimensional forward ADE problems, where its performance for various P\'{e}clet numbers ($Pe$) is compared with the analytical and numerical solutions. We find that the PINN method is accurate with errors of less than 1\% and outperforms some conventional discretization-based methods for $Pe$ larger that 100. Next, we demonstrate that the PINN method remains accurate for the backward ADEs, with the relative errors in most cases staying under 5\% compared to the reference concentration field. Finally, we show that when available, the concentration measurements can be easily incorporated in the PINN method and significantly improve (by more than 50\% in the considered cases) the accuracy of the PINN solution of the backward ADE.

He, Qizhi↗

Comparative Assessment of U-Net-Based Deep Learning Models for Segmenting Microfractures and Pore Spaces in Digital Rocks

Segmentation of high-resolution X-ray microcomputed tomography (µCT) images is crucial in digital rock physics (DRP), affecting the characterization and analysis of microscale phenomena in the porous media. The complexity of geological structures and nonideal scanning conditions pose significant challenges to conventional image segmentation approaches. Motivated by the recent increasing popularity of deep learning (DL) techniques in image processing, this work undertakes a comparative study of DL models, specifically U-Net and its variants, for segmenting multiple targets with distinguished features in digital rocks, including discrete fracture networks (DFNs), pore spaces, and solid rock. Particularly, DFNs have a smaller volumetric fraction over others, bringing in a substantial challenge of imbalanced segmentation. The primary focus is to evaluate the architecture and feature enhancement strategies of various DL models, including U-Net, attention U-Net, residual U-Net, U-Net++, and residual U-Net++. The models were designed as 2.5D, utilizing a central 2D image and its two adjacent upper and lower 2D images as input to provide a pseudo-3D context. In addition, because the ground truth of segmentation was unknown for real-world digital rocks, we created a benchmark data set following the inverse operations of segmentation. The data synthesis started from the label images (i.e., solid rock, pore spaces, and DFNs), followed by simulating partial volume blurring, adding random background noise, and introducing ring artifacts to mimic real raw X-ray µCT images. The data set, which included various rock types (i.e., sandstone and artificial data), scanning resolution, and magnitudes of noise and artifacts, was divided into training and testing data sets with a 90% and 10% ratio, respectively. Moreover, in addition to the conventional pixel-wise evaluation metrics, the physics-based metric of the lattice-Boltzmann method (LBM) simulated permeability provided more comprehensive assessments. The results demonstrated that the residual connections, nested architectures, and redesigned skip connections contribute to the model performance and give the residual U-Net++ the highest accuracy. The improvements were mainly on the boundaries and small targets, especially the DFNs, which dominate the interconnectivity and therefore affect the permeability greatly. This study also rigorously evaluated the efficiency and generalization of each model, demonstrating that the sophisticated architectures achieved excellent practicability and maintained robust performance on completely unseen data, ensuring their suitability for diverse and challenging DRP applications.

58 GEOSCIENCES↗

Physics-informed neural networks for solving forward and inverse flow problems via the Boltzmann-BGK formulation

We report the Boltzmann equation with the Bhatnagar-Gross-Krook collision model (Boltzmann-BGK equation) has been employed to describe multiscale flows, i.e., from the hydrodynamic limit to free molecular flow. In this study we employ physics-informed neural networks (PINNs) to solve forward and inverse problems via the Boltzmann-BGK formulation (PINN-BGK), enabling PINNs to model flows in both the continuum and rarefied regimes. In particular, the PINN-BGK is composed of three sub-networks, i.e., the first for approximating the equilibrium distribution function, the second for approximating the non-equilibrium distribution function, and the third one for encoding the Boltzmann-BGK equation as well as the corresponding boundary/initial conditions. By minimizing the residuals of the governing equations and the mismatch between the predicted and provided boundary/initial conditions, we can approximate the Boltzmann-BGK equation for both continuous and rarefied flows. For forward problems, the PINN-BGK is utilized to solve various benchmark flows given boundary/initial conditions, e.g. Kovasznay flow, Taylor-Green flow, cavity flow, and micro Couette flow for Knudsen number up to 5. For inverse problems, we focus on rarefied flows in which accurate boundary conditions are difficult to obtain. We employ the PINN-BGK to infer the flow field in the entire computational domain given a limited number of interior scattered measurements on the velocity without using the (unknown) boundary conditions. Results for the two-dimensional micro Couette and micro cavity flows with Knudsen numbers ranging from 0.1 to 10 indicate that the PINN-BGK can infer the velocity field in the entire domain with good accuracy. Finally, we also present some results on using transfer learning to accelerate the training process. Specifically, we can obtain a three-fold speedup comparted to the standard training process (e.g., Adam plus L-BFG-B) for the two-dimensional flow problems considered in our work.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

CoordNet: Data Generation and Visualization Generation for Time-Varying Volumes via a Coordinate-Based Neural Network

Although deep learning has demonstrated its capability in solving diverse scientific visualization problems, it still lacks generalization power across different tasks. To address this challenge, we propose CoordNet, a single coordinate-based framework that tackles various tasks relevant to time-varying volumetric data visualization without modifying the network architecture. The core idea of our approach is to decompose diverse task inputs and outputs into a unified representation (i.e., coordinates and values) and learn a function from coordinates to their corresponding values. We achieve this goal using a residual block-based implicit neural representation architecture with periodic activation functions. We evaluate CoordNet on data generation (i.e., temporal super-resolution and spatial super-resolution) and visualization generation (i.e., view synthesis and ambient occlusion prediction) tasks using time-varying volumetric data sets of various characteristics. Here, the experimental results indicate that CoordNet achieves better quantitative and qualitative results than the state-of-the-art approaches across all the evaluated tasks. Source code and pre-trained models are available at https://github.com/stevenhan1991/CoordNet.

97 MATHEMATICS AND COMPUTING↗

Hybrid Improved Empirical Mode Decomposition and Artificial Neural Network Model for the Prediction of Critical Heat Flux (CHF)

Three Hybrid artificial neural network (ANN) models namely radial basis function (RBF), generalized regression neural networks (GRNN), and multi-layer perceptron (MLP) combined with empirical mode decomposition (EMD) are developed for CHF predictive modelling using CHF experimental databases.First, the original experimental inputs data series are decomposed into several intrinsic mode functions (IMFs) and one residual by EMD, whose components are divided into high, medium and low components. The performance parameters of the hybrid models indicates that the root mean square error (RMSE) are 0.8831, 0.6522, and 0.4149; the mean absolute error (MAE) are 0.6697, 0.4636, and 0.1935. The values of the R-square of the developed prediction approach utilizing EMD-RBF, EMD-GRNN, and EMD-MLP models are 0.8553, 0.9302, and 0.9818, and the index of agreement are 0.9464, 0.9700, and 0.9894., The value of the R-square and the index of agreement of the proposed models are much higher than those of the simple models .The Pearson's test results show that the association strength between the measured and the predicted values of the proposed model EMD-MLP is the strongest. These results show the following: (a) compared with other related, recent studies, the prediction accuracy of the hybrid model EMD- MLP proposed in this research is the best hybrid model; (b) the proposed hybrid model (EMD-MLP) attains superior performance compared with simple models.

Djeddou, Messaoud↗

Structural dynamics in the water and proton channels of photosystem II during the S2 to S3 transition

Abstract Light-driven oxidation of water to molecular oxygen is catalyzed by the oxygen-evolving complex (OEC) in Photosystem II (PS II). This multi-electron, multi-proton catalysis requires the transport of two water molecules to and four protons from the OEC. A high-resolution 1.89 Å structure obtained by averaging all the S states and refining the data of various time points during the S 2 to S 3 transition has provided better visualization of the potential pathways for substrate water insertion and proton release. Our results indicate that the O1 channel is the likely water intake pathway, and the Cl1 channel is the likely proton release pathway based on the structural rearrangements of water molecules and amino acid side chains along these channels. In particular in the Cl1 channel, we suggest that residue D1-E65 serves as a gate for proton transport by minimizing the back reaction. The results show that the water oxidation reaction at the OEC is well coordinated with the amino acid side chains and the H-bonding network over the entire length of the channels, which is essential in shuttling substrate waters and protons.

59 BASIC BIOLOGICAL SCIENCES↗

The use of digital thread for reconstruction of local fiber orientation in a compression molded pin bracket via deep learning

A deep convolutional neural network (DCNN) was used for microstructure reconstruction using artificial intelligence (MR-AI) by predicting local average fiber orientation distributions (FOD) in a 3D prepreg platelet molded composite (PPMC) pin bracket. To train the MR-AI model, surface strain fields from residual stresses simulated in PPMC plates were used as the input to the DCNN. A training dataset included PPMC plates with various degrees of global fiber alignment, based on the information obtained from high-fidelity flow simulation of a pin bracket. Further, the MR-AI model was then deployed to analyze FOD in the 3D pin bracket by conducting thermo-elastic residual stress analysis. Initially, the MR-AI model was established entirely on the synthetic simulation data. Then, a μCT scan of a physically molded pin bracket was used to create a finite element model that provided data for additional validation of the DCNN model. For the μCT scan finite element pin bracket the MR-AI model predicted the distribution of fiber orientation tensor components with MAE of 0.10 indicating a global prediction error of 10%. For the flow simulated pin bracket, the MR-AI model predicted the distribution of fiber orientation tensor components with a global prediction error of 11%. The MR-AI model showed the ability to predict regions of varying alignment in the base and flange of the pin bracket. The proposed MR-AI methodology allows for rapid prediction of FOD in geometrically complex parts and offers a promising path to detecting unique fiber orientation states in molded components.

42 ENGINEERING↗

The mechanisms of catalysis and ligand binding for the SARS-CoV-2 NSP3 macrodomain from neutron and x-ray diffraction at room temperature

The nonstructural protein 3 (NSP3) macrodomain of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) (Mac1) removes adenosine diphosphate (ADP) ribosylation posttranslational modifications, playing a key role in the immune evasion capabilities of the virus responsible for the coronavirus disease 2019 pandemic. Here, we determined neutron and x-ray crystal structures of the SARS-CoV-2 NSP3 macrodomain using multiple crystal forms, temperatures, and pHs, across the apo and ADP-ribose–bound states. We characterize extensive solvation in the Mac1 active site and visualize how water networks reorganize upon binding of ADP-ribose and non-native ligands, inspiring strategies for displacing waters to increase the potency of Mac1 inhibitors. Determining the precise orientations of active site water molecules and the protonation states of key catalytic site residues by neutron crystallography suggests a catalytic mechanism for coronavirus macrodomains distinct from the substrate-assisted mechanism proposed for human MacroD2. These data provoke a reevaluation of macrodomain catalytic mechanisms and will guide the optimization of Mac1 inhibitors.

60 APPLIED LIFE SCIENCES↗

A hereditary integral, transient network approach to modeling permanent set and viscoelastic response in polymers

An efficient numerical framework is presented for modeling viscoelasticity and permanent set of polymers. It is based on the hereditary integral form of transient network theory, in which polymer chains belong to distinct networks each with different natural equilibrium states. Chains continually detach from previously formed networks and reattach to new networks in a state of zero stress. The free energy of these networks is given in terms of the deformation gradient relative to the configuration at which the network was born . A decomposition of the kernel for various free energies allows for a recurrence relationship to be established, bypassing the need to integrate over all time history. The technique is established for both highly compressible and nearly incompressible materials through the use of neo-Hookean, Blatz–Ko, Yeoh, and Ogden-Hill material models. Multiple examples are presented showing the ability to handle rate-dependent response and residual strains under complex loading histories.

Finite element method↗

Evaluating Image Classification Deep Convolutional Neural Network Architectures for Remaining Useful Life Estimation of Turbofan Engines

Accurate estimation of the remaining useful life (RUL) is a key component of condition-based maintenance (CBM) and prognosis and health management (PHM). Data-based models for the estimation of RUL are of particular interest because expert knowledge of systems is not always available, and physical modeling is often not feasible. Additionally, using data-based models, which make decisions based on raw sensor data, allow features to be learned instead of manually determined. In this work, deep convolutional neural network (CNN) architectures are investigated for their ability to estimate the RUL of turbofan engines. To improve the accuracy of the models, CNN architectures, which have proven successful in image classification, are implemented and tested. Specifically, the blocks used in the Visual Geometry Group (VGG) architecture, inception modules used in the GoogLeNet architecture, and residual blocks used in the ResNet architecture are incorporated. To account for varying flight lengths, the input to the models is a window of time series data collected from the engine under test. Window locations at the climb, cruise, and descent stages are considered. To further improve the RUL estimations, multiple overlapping windows at each location are used. This increases the amount of training data available and is found to increase the accuracy of the resulting RUL estimations by averaging the estimates from all overlapping segments. The model is trained and tested using the new Commercial Modular Aero-Propulsion System Simulation (N-CMAPSS) data set, and high prognosis accuracy was achieved. Furthermore, this work expands on the model developed and used in the 2021 PHM Society Data Challenge, which received second place.

convolutional neural networks↗

Novel artificial neural network model for instantaneous power losses and operational efficiency mapping of MW-scale vanadium redox flow battery for improved technoeconomic analysis

A novel data-driven, machine-learning-based method for modeling the instantaneous power losses of a distribution-sited 2 MW/8MWh vanadium redox flow battery (VRFB), a grid-scale electrochemical storage technology, is introduced and compared against benchmark empirical modeling approaches, including symmetric and asymmetric models, as well as a recent convex hull modeling approach. The novel loss modeling method introduces several advantages over the benchmark models and over simplistic efficiency estimates, the most significant of which is that the model can accurately reflect the stepwise and non-linear parasitic losses associated with the duty cycles of mechanical auxiliary systems like pump motor drives and blower fans. Residuals of the models are compared; the proposed data driven model features significantly improved accuracy over the benchmark models. The model's coefficient of determination is also improved relative to that of the benchmark models. Furthermore, a novel method for visualization of operational efficiency of the grid-scale storage technology is introduced. To demonstrate the benefits of the novel data-driven method for modeling the VRFB, the benchmark models and the proposed models are embedded into an Open DSS distribution network model to study two applications of the grid-scale electrical storage system: load leveling for grid support and energy arbitrage. This article demonstrates that the accuracy of the instantaneous power loss model significantly impacts the understanding of the state of charge of the VRFB. In turn, the accuracy of the efficiency modeling of the VRFB impacts the understanding of the potential economic value and technical benefits to the distribution network operators. In conclusion, the presented power loss modeling approach is, therefore, highly relevant for utility-stakeholders, battery asset owners, system engineers, system designers, and financial planners interested in evaluating or optimizing the operation of grid-scale VRFBs.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Deconvoluting Structures of Component Plant Biopolymers Using Deuterium Labeled Brassica oleracea Stems

Several Brassica species are cultivated globally for the production of seed oils as food, lubricants, and increasingly biofuels. The stem and leaf residues of these herbaceous dicotyledonous crops constitute another feedstock for biofuels and other products. Plant cell walls are complex, multipolymeric structures that consist primarily of polysaccharides and lignin. Cellulose chains coalesce to form crystalline microfibrils while the amorphous biopolymers, hemicellulose, and lignin form a network structure and fill the interstitial space. Neutron scattering has been used for the structural study of the assembly and deconstruction of the plant cell walls. However, similar neutron sensitivity of the different amorphous biopolymers has made structural association to individual component biopolymers nontrivial and ambiguous. To improve the association of structural features to specific biopolymer components, partial deuteration of the plant cell wall can be employed to increase the difference in the neutron scattering length density between amorphous carbohydrate and lignin plant polymers. Vegetative stems from partially deuterated Brassica oleracea acephala (kale) plants were obtained commercially, and the plant cell wall structures were studied by contrast-variation small-angle neutron scattering (CV-SANS) and Fourier-transform infrared spectroscopy (FTIR). FTIR results indicated that deuterium substitution for hydrogen in the carbohydrates was higher than that in lignin. By combining CV-SANS and FTIR results, the neutron scattering length density (nSLD) of the polysaccharides and lignin was determined to match nSLD of 65%:35% and 48%:52% D 2 O/H 2 O solvent mixtures, respectively. These nSLD values were higher than the nSLD values of polysaccharides and lignin for H 2 O grown biopolymers. The nSLD increase correlates to replacing about 42.5% of hydrogens present as both C–H and O–H groups in cellulose with deuterium atoms, while only 21% for lignin. In conclusion, this study lays the foundation to use partially deuterated plants to deconvolute structural features of the different component biopolymers, especially cellulose, hemicellulose, and lignin, of the plant cell wall without introducing unintended structural modifications due to the pretreatment extraction processes.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Hutchinson Trace Estimation for high-dimensional and high-order Physics-Informed Neural Networks

Physics-Informed Neural Networks (PINNs) have proven effective in solving partial differential equations (PDEs), especially when some data are available by seamlessly blending data and physics. However, extending PINNs to high-dimensional and even high-order PDEs encounters significant challenges due to the computational cost associated with automatic differentiation in the residual loss function calculation. Herein, we address the limitations of PINNs in handling high-dimensional and high-order PDEs by introducing the Hutchinson Trace Estimation (HTE) method. Starting with the second-order high-dimensional PDEs, which are ubiquitous in scientific computing, HTE is applied to transform the calculation of the entire Hessian matrix into a Hessian vector product (HVP). This approach not only alleviates the computational bottleneck via Taylor-mode automatic differentiation but also significantly reduces memory consumption from the Hessian matrix to an HVP’s scalar output. We further showcase HTE’s convergence to the original PINN loss and its unbiased behavior under specific conditions. Comparisons with the Stochastic Dimension Gradient Descent (SDGD) highlight the distinct advantages of HTE, particularly in scenarios with significant variability and variance among dimensions. We further extend the application of HTE to higher-order and higher-dimensional PDEs, specifically addressing the biharmonic equation. By employing tensor-vector products (TVP), HTE efficiently computes the colossal tensor associated with the fourth-order high-dimensional biharmonic equation, saving memory and enabling rapid computation. The effectiveness of HTE is illustrated through experimental setups, demonstrating comparable convergence rates with SDGD under memory and speed constraints. Additionally, HTE proves valuable in accelerating the Gradient-Enhanced PINN (gPINN) version as well as the Biharmonic equation. Overall, HTE opens up a new capability in scientific machine learning for tackling high-order and high-dimensional PDEs.

Curse of dimensionality↗

Improving deep learning model performance under parametric constraints for materials informatics applications

Abstract Modern machine learning (ML) and deep learning (DL) techniques using high-dimensional data representations have helped accelerate the materials discovery process by efficiently detecting hidden patterns in existing datasets and linking input representations to output properties for a better understanding of the scientific phenomenon. While a deep neural network comprised of fully connected layers has been widely used for materials property prediction, simply creating a deeper model with a large number of layers often faces with vanishing gradient problem, causing a degradation in the performance, thereby limiting usage. In this paper, we study and propose architectural principles to address the question of improving the performance of model training and inference under fixed parametric constraints. Here, we present a general deep-learning framework based on branched residual learning (BRNet) with fully connected layers that can work with any numerical vector-based representation as input to build accurate models to predict materials properties. We perform model training for materials properties using numerical vectors representing different composition-based attributes of the respective materials and compare the performance of the proposed models against traditional ML and existing DL architectures. We find that the proposed models are significantly more accurate than the ML/DL models for all data sizes by using different composition-based attributes as input. Further, branched learning requires fewer parameters and results in faster model training due to better convergence during the training phase than existing neural networks, thereby efficiently building accurate models for predicting materials properties.

36 MATERIALS SCIENCE↗

Spatial variation of the low-temperature micro-photoluminescence of THM-grown CdZnTeSe

The quaternary compound CdZnTeSe (CZTS) has emerged as a next-generation detector material. Unlike CdZnTe (CZT), CZTS has the distinct advantages of little or no sub-grain boundary networks and a much lower concentration of Te inclusions, plus better compositional homogeneity. Thus, the material is expected to offer better spatial charge-transport homogeneity compared to CZT. However, an inhomogeneous distribution of point defects in both CZT and CZTS, such as Cd vacancies and residual impurities, can impose additional spatial inhomogeneity of the resistivity, carrier mobilities, and carrier lifetimes of the material. To investigate the spatial distribution of such defects in CZTS, low-temperature photoluminescence (PL) spectroscopic studies were performed at different positions along a single crystalline CZTS sample surface, which was grown by the traveling heater method (THM). In conclusion, the intensity variation of the PL emission of excitons bound to a neutral acceptor defect (A 0 , X), which is likely dominated by an acceptor like Cu-related level, with respect to the PL emission line from a neutral donor-bound exciton (D 0 , X) exhibited significant spatial variation, while the peak energy positions were approximately uniform due to the high compositional and bandgap homogeneity of the material.

II-VI semiconductor↗

Fast tensor disentangling algorithm

Many recent tensor network algorithms apply unitary operators to parts of a tensor network in order to reduce entanglement. However, many of the previously used iterative algorithms to minimize entanglement can be slow. We introduce an approximate, fast, and simple algorithm to optimize disentangling unitary tensors. Our algorithm is asymptotically faster than previous iterative algorithms and often results in a residual entanglement entropy that is within 10 to 40% of the minimum. For certain input tensors, our algorithm returns an optimal solution. When disentangling order-4 tensors with equal bond dimensions, our algorithm achieves an entanglement spectrum where nearly half of the singular values are zero. We further validate our algorithm by showing that it can efficiently disentangle random 1D states of qubits.

Slagle, Kevin↗

Atomically Dispersed Iron–Nitrogen Sites on Hierarchically Mesoporous Carbon Nanotube and Graphene Nanoribbon Networks for CO 2 Reduction

Atomically dispersed metal and nitrogen co-doped carbon (M-N/C) catalysts hold great promise for electrochemical CO 2 conversion. However, there is a lack of cost-effective synthesis approaches to meet the goal of economic mass production of single-atom M-N/C with desirable carbon support architecture for efficient CO 2 reduction. Herein, we report facile transformation of commercial carbon nanotube (CNT) into isolated Fe-N 4 sites anchored on carbon nanotube and graphene nanoribbon (GNR) networks (Fe-N/CNT@GNR). The oxidization-induced partial unzipping of CNT results in the generation of GNR nanolayers attached to the remaining fibrous CNT frameworks, which reticulates a hierarchically mesoporous complex and thus enables a high electrochemical active surface area and smooth mass transport. The Fe residues originating from CNT growth seeds serve as Fe sources to form isolated Fe-N-4 moieties located at the CNT and GNR basal plane and edges with high intrinsic capability of activating CO 2 and suppressing hydrogen evolution. The Fe-N/CNT@GNR delivers a stable CO Faradaic efficiency of 96% with a partial current density of 22.6 mA cm -2 at a low overpotential of 650 mV, making it one of the most active M-N/C catalysts reported. This work presents an effective strategy to fabricate advanced atomistic catalysts and highlights the key roles of support architecture in single-atom electrocatalysis.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗