Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “multiscale neural networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Multiscale Neural Networks for Approximating Green’s Functions

Neural networks (NNs) have been widely used to solve partial differential equations (PDEs) in the applications of physics, biology, and engineering. One effective approach for solving PDEs with a fixed differential operator is learning Green’s functions. However, Green’s functions are notoriously difficult to learn due to their poor regularity, which typically requires larger NNs and longer training times. In this work, we address these challenges by leveraging multiscale NNs to learn Green’s functions. Through theoretical analysis using multiscale Barron space methods and experimental validation, we show that the multiscale approach significantly reduces the necessary NN size and accelerates training.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Computationally Efficient Multiscale Neural Networks Applied to Fluid Flow in Complex 3D Porous Media

Abstract The permeability of complex porous materials is of interest to many engineering disciplines. This quantity can be obtained via direct flow simulation, which provides the most accurate results, but is very computationally expensive. In particular, the simulation convergence time scales poorly as the simulation domains become less porous or more heterogeneous. Semi-analytical models that rely on averaged structural properties (i.e., porosity and tortuosity) have been proposed, but these features only partly summarize the domain, resulting in limited applicability. On the other hand, data-driven machine learning approaches have shown great promise for building more general models by virtue of accounting for the spatial arrangement of the domains’ solid boundaries. However, prior approaches building on the convolutional neural network (ConvNet) literature concerning 2D image recognition problems do not scale well to the large 3D domains required to obtain a representative elementary volume (REV). As such, most prior work focused on homogeneous samples, where a small REV entails that the global nature of fluid flow could be mostly neglected, and accordingly, the memory bottleneck of addressing 3D domains with ConvNets was side-stepped. Therefore, important geometries such as fractures and vuggy domains could not be modeled properly. In this work, we address this limitation with a general multiscale deep learning model that is able to learn from porous media simulation data. By using a coupled set of neural networks that view the domain on different scales, we enable the evaluation of large ( $$>512^3$$ > 512 3 ) images in approximately one second on a single graphics processing unit. This model architecture opens up the possibility of modeling domain sizes that would not be feasible using traditional direct simulation tools on a desktop computer. We validate our method with a laminar fluid flow case using vuggy samples and fractures. As a result of viewing the entire domain at once, our model is able to perform accurate prediction on domains exhibiting a large degree of heterogeneity. We expect the methodology to be applicable to many other transport problems where complex geometries play a central role.

36 MATERIALS SCIENCE↗
DOE OSTI2022

NH-PINN: Neural homogenization-based physics-informed neural network for multiscale problems

Physics-informed neural network (PINN) is a data-driven approach to solving equations. It is successful in many applications; however, the accuracy of the PINN is not satisfactory when it is used to solve multiscale equations. Homogenization approximates a multiscale equation by a homogenized equation without multiscale property; it includes solving cell problems and the homogenized equation. The cell problems are periodic, and we propose an oversampling strategy that significantly improves the PINN accuracy on periodic problems. The homogenized equation has a constant or slow dependency coefficient and can also be solved by PINN accurately. We hence proposed a 3-step method, neural homogenization based PINN (NH-PINN), to improve the PINN accuracy for solving multiscale problems with the help of homogenization.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

A seamless multiscale operator neural network for inferring bubble dynamics

Modelling multiscale systems from nanoscale to macroscale requires the use of atomistic and continuum methods and, correspondingly, different computer codes. Here, we develop a seamless method based on DeepONet, which is a composite deep neural network (a branch and a trunk network) for regressing operators. In particular, we consider bubble growth dynamics, and we model tiny bubbles of initial size from 100 nm to 10 $\mathrm {\mu }\textrm {m}$ , modelled by the Rayleigh–Plesset equation in the continuum regime above 1 $\mathrm {\mu }\textrm {m}$ and the dissipative particle dynamics method for bubbles below 1 $\mathrm {\mu }\textrm {m}$ in the atomistic regime. After an offline training based on data from both regimes, DeepONet can make accurate predictions of bubble growth on-the-fly (within a fraction of a second) across four orders of magnitude difference in spatial scales and two orders of magnitude in temporal scales. The framework of DeepONet is general and can be used for unifying physical models of different scales in diverse multiscale applications.

Mechanics↗
DOE OSTI2023

CEGANN: Crystal Edge Graph Attention Neural Network for multiscale classification of materials environment

Abstract We introduce Crystal Edge Graph Attention Neural Network (CEGANN) workflow that uses graph attention-based architecture to learn unique feature representations and perform classification of materials across multiple scales (from atomic to mesoscale) and diverse classes ranging from metals, oxides, non-metals to hierarchical materials such as zeolites and semi-ordered mesophases. CEGANN can classify based on a global, structure-level representation such as space group and dimensionality (e.g., bulk, 2D, clusters, etc.). Using representative materials such as polycrystals and zeolites, we demonstrate its transferability in performing local atom-level classification tasks, such as grain boundary identification and other heterointerfaces. CEGANN classifies in (thermal) noisy dynamical environments as demonstrated for representative zeolite nucleation and growth from an amorphous mixture. Finally, we use CEGANN to classify multicomponent systems with thermal noise and compositional diversity. Overall, our approach is material agnostic and allows for multiscale feature classification ranging from atomic-scale crystals to heterointerfaces to microscale grain boundaries.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Towards computational polar-topotronics: Multiscale neural-network quantum molecular dynamics simulations of polar vortex states in SrTiO3/PbTiO3 nanowires

Recent discoveries of polar topological structures ( e.g ., skyrmions and merons) in ferroelectric/paraelectric heterostructures have opened a new field of polar topotronics. However, how complex interplay of photoexcitation, electric field and mechanical strain controls these topological structures remains elusive. To address this challenge, we have developed a computational approach at the nexus of machine learning and first-principles simulations. Our multiscale neural-network quantum molecular dynamics molecular mechanics approach achieves orders-of-magnitude faster computation, while maintaining quantum-mechanical accuracy for atoms within the region of interest. This approach has enabled us to investigate the dynamics of vortex states formed in PbTiO 3 nanowires embedded in SrTiO 3 . We find topological switching of these vortex states to topologically trivial, uniformly polarized states using electric field and trivial domain-wall states using shear strain. These results, along with our earlier results on optical control of polar topology, suggest an exciting new avenue toward opto-electro-mechanical control of ultrafast, ultralow-power polar topotronic devices.

Linker, Thomas↗
DOE OSTI2023

A multiscale recurrent neural network model for predicting energy production from geothermal reservoirs

Optimization of energy production from geothermal reservoirs requires reliable prediction of energy production performance under alternative operation and development scenarios. Traditionally, reservoir simulation models are used for the evaluation and screening of alternative production and development plans. However, simulation models require extensive data collection and modeling efforts and are time-consuming to build, run, and update. Data-driven predictive models, on the other hand, can serve as efficient prediction tools that can be used for decision support and management of daily operations and surveillance activities. Data-driven models become particularly attractive when a reservoir simulation model for a field does not exist and/or is difficult to build. Machine learning (ML)-based data-driven models that have recently become popular in several fields exploit statistical patterns and relations in training data to generate predictions. As such, they tend to perform better in interpolation problems (that is, prediction within the training data range) than when they are used to extrapolate beyond the training data. Production data from geothermal reservoirs tend to exhibit short-term variabilities as well as long-term trends, such as monotonically declining production temperatures. Capturing both short-term features and long-term trends with ML-based models is not trivial. We evaluate the use of recurrent neural networks (RNN) for the prediction of energy production from geothermal reservoirs. RNN is a class of ML architectures that are used to represent and predict sequential/dynamic data. Thus, it can be challenging to apply RNN to problems where long-term trends must be captured and extrapolation beyond the training data range is needed. We introduce the multiscale RNN architecture to extend the application of RNN to detect and predict both short-term variabilities and long-term trends in geothermal data. The developed architecture consists of a long-term component to only capture low-frequency data patterns, and a short-term component to detect features with higher frequency and more nonlinearity. The final prediction is obtained by combining the long-term and short-term predictions. Both synthetic and field data are used to evaluate the presented multiscale RNN model. The prediction performance of the multiscale RNN is compared against those obtained from the regular RNN and the autoregressive (AR) model. The results suggest that the multiscale architecture improves the long-term prediction performance of the regular RNN and enhances its robustness against noise.

15 GEOTHERMAL ENERGY↗
DOE OSTI2025

Mesh-based super-resolution of fluid flows with multiscale graph neural networks

A graph neural network (GNN) approach is introduced in this work which enables mesh-based three-dimensional super-resolution of fluid flows. In this framework, the GNN is designed to operate not on the full mesh-based field at once, but on localized meshes of elements (or cells) directly. To facilitate mesh-based GNN representations in a manner similar to spectral (or finite) element discretizations, a baseline GNN layer (termed a message passing layer, which updates local node properties) is modified to account for synchronization of coincident graph nodes, rendering compatibility with commonly used element-based mesh connectivities. Furthermore, the architecture is multiscale in nature, and is comprised of a combination of coarse-scale and fine-scale message passing layer sequences (termed processors) separated by a graph unpooling layer. The coarse-scale processor embeds a query element (alongside a set number of neighboring coarse elements) into a single latent graph representation using coarse-scale synchronized message passing over the element neighborhood, and the fine-scale processor leverages additional message passing operations on this latent graph to correct for interpolation errors. Demonstration studies are performed using hexahedral mesh-based data from Taylor–Green Vortex and backward-facing step flow simulations at Reynolds numbers of 1600 and 3200. Through analysis of both global and local errors, the results ultimately show how the GNN is able to produce accurate super-resolved fields compared to targets in both coarse-scale and multiscale model configurations. Reconstruction errors for fixed architectures were found to increase in proportion to the Reynolds number. Geometry extrapolation studies on a separate cavity flow configuration show promising cross-mesh capabilities of the super-resolution strategy.

Backward-facing step↗
DOE OSTI2023

Multiscale graph neural network autoencoders for interpretable scientific machine learning

The goal of this work is to address two limitations in autoencoder-based models: latent space interpretability and compatibility with unstructured meshes. This is accomplished here with the development of a novel graph neural network (GNN) autoencoding architecture with demonstrations on complex fluid flow applications. To address the first goal of interpretability, the GNN autoencoder achieves reduction in the number nodes in the encoding stage through an adaptive graph reduction procedure. Further, this reduction procedure essentially amounts to flowfieldconditioned node sampling and sensor identification, and produces interpretable latent graph representations tailored to the flowfield reconstruction task in the form of so-called masked fields. These masked fields allow the user to (a) visualize where in physical space a given latent graph is active, and (b) interpret the time-evolution of the latent graph connectivity in accordance with the time-evolution of unsteady flow features (e.g. recirculation zones, shear layers) in the domain. To address the goal of unstructured mesh compatibility, the autoencoding architecture utilizes a series of multi-scale message passing (MMP) layers, each of which models information exchange among node neighborhoods at various lengthscales. The MMP layer, which augments standard single-scale message passing with learnable coarsening operations, allows the decoder to more efficiently reconstruct the flowfield from the identified regions in the masked fields. Analysis of latent graphs produced by the autoencoder for various model settings are conducted using unstructured snapshot data sourced from large-eddy simulations in a backward-facing step (BFS) flow configuration with an OpenFOAM-based flow solver at high Reynolds numbers.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Application of First-Principles-Based Artificial Neural Network Potentials to Multiscale-Shock Dynamics Simulations on Solid Materials

The use of artificial neural network (ANN) potentials trained with first-principles calculations has emerged as a promising approach for molecular dynamics (MD) simulations encompassing large space and time scales while retaining first-principles accuracy. To date, however, the application of ANN-MD has been limited to near-equilibrium processes. Here we combine first-principles-trained ANN-MD with multiscale shock theory (MSST) to successfully describe far-from-equilibrium shock phenomena. Overall, our ANN-MSST-MD approach describes shock-wave propagation in solids with first-principles accuracy but a 5000 times shorter computing time. Accordingly, ANN-MD-MSST was able to resolve fine, long-time elastic deformation at low shock speed, which was impossible with first-principles MD because of the high computational cost. This work thus lays a foundation of ANN-MD simulation to study a wide range of far-from-equilibrium processes.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Modeling the 4D discharge of lithium-ion batteries with a multiscale time-dependent deep learning framework

The lithium-ion battery (LIB) field is moving towards the direction of investigating spatially resolved physical phenomena in the 3D porous microstructure of electrodes. These pore-scale simulations give new insights into the local dynamics of lithiation/de-lithiation and charge transport, Nevertheless, the computational time of these simulations limits the integration of these models in optimization workflows of cycling conditions or electrode manufacturing processes. Machine learning models present a way of assessing in real-time the performance of materials. While several successful techniques for replicating simulations with machine learning have been proposed, this case study presents a more demanding problem, due to the necessity of understanding the behavior of heterogeneous 3D local data, as it evolves in time: this poses both a scientific and a technical challenge. To this end, we propose an autoregressive multiscale convolutional neural network model to predict relevant quantities at the pore-scale in the solid phase: the lithium concentration (in the active material) and potential (in the active material and carbon binder). Here, these are ultimately used to reconstruct the battery discharge curve. 3D images of the electrode microstructures are the input to the network, trained with a dataset of finite element method simulations to predict the discharge behavior of the cathode side in lithium ion batteries. We propose this machine learning model as a proof-of-concept of the applicability of multiscale networks for time-dependent physics problems. The trained model exhibits very high accuracy (with errors lower than 2 %) in forecasting the discharge behavior of new unseen cathodes.

25 ENERGY STORAGE↗
DOE OSTI2021

Development of a Filtered CFD-DEM Drag Model with Multiscale Markers Using an Artificial Neural Network and Nonlinear Regression

Here, the accuracy of coarse-grained Euler-Lagrangian simulations of fluidized beds heavily depends on the mesoscale drag models to account for the influences of the unresolved sub-grid structures. Traditional filtered drag models are regressed with mesoscale markers such as voidage and slip velocities. In this research, a filtered drag was regressed with both mesoscale and macro-scale markers using fine grid Computational Fluid Dynamics - Discrete Element Method (CFD-DEM) simulations. The traditional non-linear regression method was compared with machine learning regression using an Artificial Neural Network (ANN) implemented in PyTorch and coupled with MFiX. The new drag showed higher accuracy than the Wen-Yu drag and another filtered drag derived from the two-fluid model. The nonlinear regression shows slightly better results than ANN regression in cases with similar R 2 values. The utilization of the gas inlet velocity as an additional macro-scale marker reduced the errors by up to 55.3% in the tested cases.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Exploring far-from-equilibrium ultrafast polarization control in ferroelectric oxides with excited-state neural network quantum molecular dynamics

Ferroelectric materials exhibit a rich range of complex polar topologies, but their study under far-from-equilibrium optical excitation has been largely unexplored because of the difficulty in modeling the multiple spatiotemporal scales involved quantum-mechanically. To study optical excitation at spatiotemporal scales where these topologies emerge, we have performed multiscale excited-state neural network quantum molecular dynamics simulations that integrate quantum-mechanical description of electronic excitation and billion-atom machine learning molecular dynamics to describe ultrafast polarization control in an archetypal ferroelectric oxide, lead titanate. Far-from-equilibrium quantum simulations reveal a marked photo-induced change in the electronic energy landscape and resulting cross-over from ferroelectric to octahedral tilting topological dynamics within picoseconds. The coupling and frustration of these dynamics, in turn, create topological defects in the form of polar strings. The demonstrated nexus of multiscale quantum simulation and machine learning will boost not only the emerging field of ferroelectric topotronics but also broader optoelectronic applications.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Data assimilation empowered neural network parametrizations for subgrid processes in geophysical flows

In the past couple of years, there has been a proliferation in the use of machine learning approaches to represent subgrid-scale processes in geophysical flows with an aim to improve the forecasting capability and to accelerate numerical simulations of these flows. Despite its success for different types of flow, the online deployment of a data-driven closure model can cause instabilities and biases in modeling the overall effect of subgrid-scale processes, which in turn leads to inaccurate prediction. To tackle this issue, we exploit the data assimilation technique to correct the physics-based model coupled with the neural network as a surrogate for unresolved flow dynamics in multiscale systems. In particular, we use a set of neural network architectures to learn the correlation between resolved flow variables and the parametrizations of unresolved flow dynamics and formulate a data assimilation approach to correct the hybrid model during their online deployment. We illustrate our framework in a set of applications of the multiscale Lorenz 96 system for which the parametrization model for unresolved scales is exactly known, and the two-dimensional Kraichnan turbulence system for which the parametrization model for unresolved scales is not known a priori. Our analysis, therefore, comprises a predictive dynamical core empowered by (i) a data-driven closure model for subgrid-scale processes, (ii) a data assimilation approach for forecast error correction, and (iii) both data-driven closure and data assimilation procedures. We show significant improvement in the long-term prediction of the underlying chaotic dynamics with our framework compared to using only neural network parametrizations for future prediction. Moreover, we demonstrate that these data-driven parametrization models can handle the non-Gaussian statistics of subgrid-scale processes, and effectively improve the accuracy of outer data assimilation workflow loops in a modular nonintrusive way.

42 ENGINEERING↗
DOE OSTI2025

High‐Resolution National‐Scale Water Modeling Is Enhanced by Multiscale Differentiable Physics‐Informed Machine Learning

Abstract The National Water Model (NWM) is a key tool for flood forecasting, planning, and water management. Key challenges facing the NWM include calibration and parameter regionalization when confronted with big data. We present two novel versions of high‐resolution (∼37 km 2 ) differentiable models (a type of hybrid model): one with implicit, unit‐hydrograph‐style routing and another with explicit Muskingum‐Cunge routing in the river network. The former predicts streamflow at basin outlets whereas the latter presents a discretized product that seamlessly covers rivers in the conterminous United States (CONUS). Both versions use neural networks to provide a multiscale parameterization and process‐based equations to provide a structural backbone, which were trained simultaneously (“end‐to‐end”) on 2,807 basins across the CONUS and evaluated on 4,997 basins. Both versions show great potential to elevate future NWM performance for extensively calibrated as well as ungauged sites: the median daily Nash‐Sutcliffe efficiency of all 4,997 basins is improved to around 0.68 from 0.48 of NWM3.0. As they resolve spatial heterogeneity, both versions greatly improved simulations in the western CONUS and also in the Prairie Pothole Region, a long‐standing modeling challenge. The Muskingum‐Cunge version further improved performance for basins >10,000 km 2 . Overall, our results show how neural‐network‐based parameterizations can improve NWM performance for providing operational flood predictions while maintaining interpretability and multivariate outputs. The modeling system supports the Basic Model Interface (BMI), which allows seamless integration with the next‐generation NWM. We also provide a CONUS‐scale hydrologic data set for further evaluation and use.

Song, Yalan [Civil and Environmental Engineering T↗