SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
This study combines Graph Neural Networks (GNNs) and Large Language Models (LLMs) to improve material property predictions. By leveraging both embeddings, this hybrid approach achieves up to a 25% improvement over GNN-only model in accuracy.
Abstract Deciphering the sequence-function relationship encoded in enhancers holds the key to interpreting non-coding variants and understanding mechanisms of transcriptomic variation. Several quantitative models exist for predicting enhancer function and underlying mechanisms; however, there has been no systematic comparison of these models characterizing their relative strengths and shortcomings. Here, we interrogated a rich data set of neuroectodermal enhancers in Drosophila, representing cis- and trans- sources of expression variation, with a suite of biophysical and machine learning models. We performed rigorous comparisons of thermodynamics-based models implementing different mechanisms of activation, repression and cooperativity. Moreover, we developed a convolutional neural network (CNN) model, called CoNSEPT, that learns enhancer ‘grammar’ in an unbiased manner. CoNSEPT is the first general-purpose CNN tool for predicting enhancer function in varying conditions, such as different cell types and experimental conditions, and we show that such complex models can suggest interpretable mechanisms. We found model-based evidence for mechanisms previously established for the studied system, including cooperative activation and short-range repression. The data also favored one hypothesized activation mechanism over another and suggested an intriguing role for a direct, distance-independent repression mechanism. Our modeling shows that while fundamentally different models can yield similar fits to data, they vary in their utility for mechanistic inference. CoNSEPT is freely available at: https://github.com/PayamDiba/CoNSEPT.
Modeling and prediction of the dynamic behavior of thermal systems operating under intermittent energy input and variable load requirements represent one of the greatest challenges in the development of efficient and reliable renewable-based power generation technologies. In this work, a data-driven machine learning modeling framework was developed based on a modified version of the Deep Operator Network architecture where the time coordinate in the trunk net is replaced with historical data of the predicting quantity. The modeling framework can be used to accurately predict the performance of renewable-based energy conversion technologies including wind- and solar-based power plants. This novel framework was applied on a solar-thermal system that consists of a solar collection loop using a flat plate collector, a power generation loop comprising an Organic Rankine Cycle, and a thermal energy storage tank connecting both loops. Variable solar irradiance, air temperature, and power load profiles were used by the Deep Operator Network to predict the State-of-Charge and the efficiency of the thermal system for several days. The results were compared with the State-of-Charge and efficiency functions calculated using a physics-based model. For a simple operation scenario, characterized by a clear sky solar irradiance profile and constant load, the standard deviation in the State-of-Charge prediction by Deep Operator Network is below 0.9% during a seven-day prediction time horizon. For the most realistic operation scenario that considers real solar irradiance and a rough load profile, the maximum standard deviation in the predictions for the State-of-Charge and efficiency are below 6.8% and 2.5%, respectively. A comparison between Deep Operator Network and Long Short Term Memory network was also performed. In general, both networks predict very well the State-of-Charge for different data density conditions; however, a higher accuracy, with a standard deviation below 2.0%, is obtained by the Deep Operator Network during three and half days using sparser training data of 20-minute points. The same accuracy for the State-of-Charge prediction with the Long Short Term Memory network is achieved only for 14 h. Average standard deviations for the State-of-Charge prediction of 1.1% with the Deep Operator Network and 1.5% with the Long Short Term Memory network are obtained for a four-day prediction time using a denser training data of 5-minute points.
Abstract Terrestrial ring current dynamics are a critical part of the near‐space environment, in that they directly drive geomagnetic field variations that control particle drifts, and define geomagnetic storms. The present study aims to specify a global and time‐varying distribution of ring current proton using geomagnetic indices and solar wind parameters with their history as input. We train an artificial neural network (ANN) model to reproduce proton fluxes measured by the Radiation Belt Storm Probes Ion Composition Experiment instrument onboard Van Allen Probes. By choosing optimal feature parameters and their history length, the model results show a high correlation and a small error between model specifications and satellite measurements. The modeled results well capture energy‐dependent proton dynamics in association with geomagnetic storms, including inward radial diffusion, acceleration and decay. Our ANN model produces proton fluxes with their corresponding 3D spatiotemporal variations, capturing the latitudinal distribution and local time asymmetry that are consistent with observations and that can further inform theory.
Abstract In this paper, we present a physically informed neural network (NN) representation of the effective interactions associated with coupled-cluster downfolding models to describe chemical systems and processes. The NN representation not only allows us to evaluate the effective interactions efficiently for various geometrical configurations of chemical systems corresponding to various levels of complexity of the underlying wave functions, but also reveals that the bare and effective interactions are related by a tangent function of some latent variables. We refer to this characterization of the effective interaction as a tangent model. We discuss the connection between this tangent model for the effective interaction with the previously developed theoretical analysis that examines the difference between the bare and effective Hamiltonians in the corresponding active spaces.
Surrogate model development is a critical step for uncertainty quantification or other sample-intensive tasks for complex computational models. Here, in this work, we develop a multi-output surrogate form using a class of neural networks (NNs) that employ shortcut connections, namely Residual NNs (ResNets). ResNets are known to regularize the surrogate learning problem and improve the efficiency and accuracy of the resulting surrogate. Inspired by the continuous, Neural ODE analogy, we augment ResNets with weight parameterization strategy with respect to ResNet depth. Weight-parameterized ResNets regularize the NN surrogate learning problem and allow better generalization with a drastically reduced number of learnable parameters. We demonstrate that weight-parameterized ResNets are more accurate and efficient than conventional feed-forward multi-layer perceptron networks. We also compare various options for parameterization of the weights as functions of ResNet depth. We demonstrate the results on both synthetic examples and a large scale earth system model of interest.
Abstract Many problems in science and engineering require making predictions based on few observations. To build a robust predictive model, these sparse data may need to be augmented with simulated data, especially when the design space is multi-dimensional. Simulations, however, often suffer from an inherent bias. Estimation of this bias may be poorly constrained not only because of data sparsity, but also because traditional predictive models fit only one type of observed outputs, such as scalars or images, instead of all available output data modalities, which might have been acquired and simulated at great cost. To break this limitation and open up the path for multi-modal calibration, we propose to combine a novel, transfer learning technique for suppressing the bias with recent developments in deep learning, which allow building predictive models with multi-modal outputs. First, we train an initial neural network model on simulated data to learn important correlations between different output modalities and between simulation inputs and outputs. Then, the model is partially retrained, or transfer learned, to fit the experiments; a method that has never been implemented in this type of architecture. Using fewer than 10 inertial confinement fusion experiments for training, transfer learning systematically improves the simulation predictions while a simple output calibration, which we design as a baseline, makes the predictions worse. We also offer extensive cross-validation with real and carefully designed synthetic data. The method described in this paper can be applied to a wide range of problems that require transferring knowledge from simulations to the domain of experiments.
Abstract Shearing of fractures and faults is important because it can result in permeability change or even induce seismicity—both are keys for efficient and safe energy recovery and storage in Earth systems. Quantitative analysis of shearing of intersecting fractures and faults is challenging because it can involve dynamic frictional contacts that are complicated by deformation of the rock matrix. To predict the shearing of intersecting fractures/faults, we attempt to answer the question of how intersections impact the shearing of a fracture network and whether we can simplify the description as compared to classical discrete fracture network (DFN) models. To answer these questions, we conducted a series of numerical simulations on scenarios for variable numbers of intersecting fractures. All these examples yield consistent results: the results of using DFNs are consistent with those of using hypothetical major paths. This leads to a new model, which we name simplified discrete fracture network model , to analyze shearing of intersecting fractures/faults using major path(s). We found that the intersections of fractures do not fundamentally change the shearing of two intersecting fractures if the intersecting angles are small. Furthermore, increasing the number of fractures/faults may relax the stress as more fractures/faults become available for shearing and distributing the stress. The simplified DFN model, which can capture efficiently the shearing behavior of each major paths from a large number of intersecting fractures/faults, will be a promising conceptual model that is complementary to existing equivalent continuum and discrete fracture models to analyze shearing of intersecting fractures/faults.
Inverse problems associated with stochastic models constitute a significant portion of scientific and engineering applications. In such cases the unknown quantities are distributions. The applicability of traditional methods is limited because of their demanding assumptions or prohibitive computational consumption; for example, maximum likelihood methods require closed-form density functions, and Markov Chain Monte Carlo needs a large number of simulations. We propose a new method that estimates the unknown distribution by matching the statistical properties between observed and simulated random processes. We leverage the expressive power of neural networks to approximate the unknown distribution and use a discriminative neural network for computing the statistical discrepancies between the observed and simulated random processes. Here we demonstrated numerically that the proposed methods can estimate both the model parameters and learn complicated unknown distributions.
In this work, a composite continuum-pore network formulation is presented to model single and two-phase transport thin porous layers, such as gas diffusion layers in polymer electrolyte fuel cells (PEFCs) and active electrodes in redox flow batteries (RFBs). The formulation can be integrated into CFD codes, thus combining the ease of implementation of continuum-based modeling and the computational power of pore-network modeling. The composite model includes a control volume (CV) mesh at the layer scale, which embeds a cubic pore network [1,2]. The pore-network model is used to determine analytically local anisotropic effective transport properties (local effective diffusivity and permeability), which are mapped onto the CV mesh to simulate transport in the porous transport layer [3,4]. Good agreement is found between the predicted global effective transport properties (global effective diffusivity and permeability) under dry and wet conditions and previous experimental data reported in the literature for Toray TGP-H series carbon paper. Water saturation distributions are also compared with results obtained using X-ray computed tomography [4]. [1] P.A. García-Salaberri, I.V. Zenyuk, J.T. Gostick, A.Z. Weber, Modeling Gas Diffusion Layer in Polymer Electrolyte Fuel Cells Using a Continuum-Based Pore-Network Formulation, ECS Trans. 97 (2020) 615. [2] P.A. García-Salaberri, Modeling diffusion and convection in thin porous transport layers using a composite continuum-network model: Application to gas diffusion layers in polymer electrolyte fuel cells, Int J. Heat Mass Transf. (2020), submitted. [3] P.A. García-Salaberri, J.T. Gostick, G. Hwang, A.Z. Weber, M. Vera, Effective diffusivity in partially-saturated carbon-fiber gas diffusion layers: Effect of local saturation and application to macroscopic continuum models, J. Power Sources 296 (2015) 440–453. [4] P.A. García-Salaberri, G. Hwang, M. Vera, A.Z. Weber, J.T. Gostick, Effective diffusivity in partially-saturated carbon-fiber gas diffusion layers: Effect of through-plane saturation distribution, Int. J. Heat Mass Transf. 86 (2015) 319–333.
In this work, we investigate the applicability and relative merit of discrete and continuous versions of physics-informed neural network (PINN) methods for learning unknown governing equations or constitutive relations in a nonlinear dynamical system. In the case of unknown dynamics, entire right-hand-side (RHS) equations of the ordinary differential equations are unknown. In the case of unknown constitutive relations, however, the RHS equations are known up to the specification of constitutive relations (that may depend on the state of the system). We use a deep neural network to model unknown governing equations or constitutive relations. The discrete PINN approach combines classical multistep discretization methods for dynamical systems with neural-network-based machine learning methods. On the other hand, the continuous versions utilize deep neural networks to minimize the residual function for the continuous governing equations. We use the case of a fedbatch bioreactor system to study the effectiveness of these approaches and discuss conditions for their applicability. Our results indicate that the accuracy of the trained neural network models is much higher for the cases where we only have to learn a constitutive relation instead of all dynamics. This finding corroborates the well-known fact from scientific computing that building as much structural information as is available into an algorithm can enhance its efficiency and/or accuracy.
This dataset includes reports and a slide presentation on discrete fracture network (DFN) generation and hydraulic fracture modeling at the Utah FORGE site. It details the characterization of natural fractures using well log and core data, as well as stochastic modeling techniques. The reports describe simulations of hydraulic fracture propagation, fluid-mechanical interactions, and induced microseismicity. The dataset also includes history-matching of net pressure and analyses of fracture growth in naturally fractured geothermal reservoirs. The slides summarize key findings and future research directions.
Deep learning has shown great success in settings with massive amounts of data but has struggled when data is limited. Few-shot learning algorithms, which seek to address this limitation, are designed to generalize well to new tasks with limited data. Typically, models are evaluated on unseen classes and datasets that are defined by the same fundamental task as they are trained for (e.g. category membership). One can also ask how well a model can generalize to fundamentally different tasks within a fixed dataset (for example: moving from category membership to tasks that involve detecting object orientation or quantity). To formalize this kind of shift we define a notion of “independence of tasks” and identify three new sets of labels for established computer vision datasets that test a model's ability to generalize to tasks which draw on orthogonal attributes in the data. We use these datasets to investigate the failure modes of metric-based few-shot models. Based on our findings, we introduce a new few-shot model called Fuzzy Simplicial Networks (FSN) which leverages a construction from topology to more flexibly represent each class from limited data. In particular, FSN models can not only form multiple representations for a given class but can also begin to capture the low-dimensional structure which characterizes class manifolds in the encoded space of deep networks. We show that FSN outperforms state-of-the-art models on the challenging tasks we introduce in this paper while remaining competitive on standard few-shot benchmarks.
Real-world networks evolve over time via the addition or removal of vertices and edges. In current network evolution models, vertex degree varies or grows arbitrarily. A recently introduced degree-preserving network growth (DPG) family of models preserves vertex degree, resulting in structures significantly different from and more diverse than previous models. Despite its degree preserving property, the DPG model is able to replicate the output of several well-known real-world network growth models. Simulations showed that many real-world networks can also be constructed from small seed graphs via the DPG process. Here, we start the development of a rigorous mathematical theory underlying the DPG family of network growth models. We prove that the degree sequence of the output of some of the well-known, real-world network growth models can be reconstructed via the DPG process, using proper parametrization. We also show that the general problem of deciding whether a simple graph can be obtained via the DPG process from a small seed (DPG feasibility) is, however, NP-complete. In conclusion, it is an intriguing open problem to uncover whether there is a structural reason behind the DPG-constructability of real-world networks.
In this paper, we introduce MARS, a new scheduling system for HPC-cloud infrastructures based on a cost-aware, flexible reinforcement learning approach, which serves as an intermediate layer for next generation HPC-cloud resource manager. MARS ensembles the pre-trained models from heuristic workloads and decides on the most cost-effective strategy for optimization. A whole workflow application would be split into several optimizable dependent sub-tasks, then based on the pre- defined resource management plan, a reward will be generated after executing a scheduled task. Lastly, MARS updates the Deep Neural Network (DNN) model based on the reward. MARS is designed to optimize the existing models through reinforcement mechanisms. MARS adapts to the dynamics of workflow applications, selects the most cost-effective scheduling solution among pre-built scheduling strategies (backfilling, SJF, etc.) and self- learning deep neural network model at run-time. We evaluate MARS with different real-world workflow traces. MARS can achieve 5%-60% increased performance compare to state-of-the- art approaches.
Not provided.
Accuracy-optimized convolutional neural networks (CNNs) have emerged as highly effective models at predicting neural responses in brain areas along the primate ventral stream, but it is largely unknown whether they effectively model neurons in the complementary primate dorsal stream. We explored how well CNNs model the optic flow tuning properties of neurons in dorsal area MSTd and we compared our results with the Non-Negative Matrix Factorization (NNMF) model, which successfully models many tuning properties of MSTd neurons. To better understand the role of computational properties in the NNMF model that give rise to optic flow tuning that resembles that of MSTd neurons, we created additional CNN model variants that implement key NNMF constraints – non-negative weights and sparse coding of optic flow. While the CNNs and NNMF models both accurately estimate the observer's self-motion from purely translational or rotational optic flow, NNMF and the CNNs with nonnegative weights yield substantially less accurate estimates than the other CNNs when tested on more complex optic flow that combines observer translation and rotation. Despite its poor accuracy, NNMF gives rise to tuning properties that align more closely with those observed in primate MSTd than any of the accuracy-optimized CNNs. This work offers a step toward a deeper understanding of the computational properties and constraints that describe the optic flow tuning of primate area MSTd.
We show that explainable neural networks can identify regions of oceanic variability that contribute predictability on decadal timescales in a fully coupled Earth-system model. The neural networks learn to use sea-surface temperature anomalies to predict future continental surface temperature anomalies. We then use a neural-network explainability method called layerwise relevance propagation to infer which oceanic patterns lead to accurate predictions made by the neural networks. In particular, regions within the North Atlantic Ocean and North Pacific Ocean lend the most predictability for surface temperature across continental North America. We apply the proposed methodology to decadal variability, although the concept is generalizable to other timescales of predictability. Furthermore, while our approach focuses on predictable patterns of internal variability within climate models, it should be generalizable to observational data as well. Our study contributes to the growing evidence that explainable neural networks are important tools for advancing geoscientific knowledge.