SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
BUTTER ( An Empirical Deep Learning Experimental Framework) is a system with which empirical deep learning experiments can be conducted and their results, including training and model performance characteristics, can be accumulated for further analysis.
This dataset has results and the model associated with the publication Ciulla et al., (2024). It contains a U-Net semantic segmentation model (unet_model.h5) and associated code implemented in tensorflow 2.0 for the model training and identification of oil and gas well symbols in USGS historical topographic maps (HTMC). Given a quadrangle map (7.5 minutes), downloadable at this url: https://ngmdb.usgs.gov/topoview/, and a list of coordinates of the documented wells present in the area, the model returns the coordinates of oil and gas symbols in the HTMC maps. For reproducibility of our workflow, we provide a sample map in California and the documented well locations for the entire State of California (CalGEM_AllWells_20231128.csv) downloaded from https://www.conservation.ca.gov/calgem/maps/Pages/GISMapping2.aspx. Additionally, the locations of 1,301 potential undocumented orphaned wells identified using our deep learning framework or the counties of Los Angeles and Kern in California, and Osage and Oklahoma in Oklahoma are provided in the file found_potential_UOWs.zip. The results of the visual inspection of satellite imagery in Osage County is in the file visible_potential_UOWs.zip. The dataset also includes a custom tool to validate the detected symbols in the HTMC maps (vetting_tool.py). More details about the methodology can be found in the associated paper: Ciulla, F., Santos, A., Jordan, P., Kneafsey, T., Biraud, S.C., and Varadharajan, C. (2024) A Deep Learning Based Framework to Identify Undocumented Orphaned Oil and Gas Wells from Historical Maps: a Case Study for California and Oklahoma. Accepted for publication in Environmental Science and Technology. The geographical coordinates provided correspond to the locations of potential undocumented orphaned oil and gas wells (UOWs) extracted from historical maps. The actual presence of wells need to be confirmed with on-the-ground investigations. For your safety, do not attempt to visit or investigate these sites without appropriate safety training, proper equipment, and authorization from local authorities. Approaching these well sites without proper personal protective equipment (PPE) may pose significant health and safety risks. Oil and gas wells can emit hazardous gasses including methane, which is flammable, odorless and colorless, as well as hydrogen sulfide, which can be fatal even at low concentrations. Additionally, there may be unstable ground near the wellhead that may collapse around the wellbore. This dataset was prepared as an account of work sponsored by the United States Government. While this document is believed to contain correct information, neither the United States Government nor any agency thereof, nor the Regents of the University of California, nor any of their employees, makes any warranty, express or implied, or assumes any legal responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by its trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof, or the Regents of the University of California. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof or the Regents of the University of California.
Securing satellite groundstations against cyber-attacks is vital to national security missions. However, these cyber threats are constantly evolving. As vulnerabilities are discovered and patched, new vulnerabilities are discovered and exploited. In order to automate the process of discovering existing vulnerabilities and the means to exploit them, a reinforcement learning framework is presented in this report. We demonstrate that this framework can learn to successfully navigate an unknown network and detect nodes of interest despite the presence of a moving target defense. The agent then exfiltrates a file of interest from the node as quickly as possible. This framework also incorporates a defensive software agent that learns to impede the attacking agents progress. This setup allows for the agents to work against each other and improve their abilities. We anticipate that this capability will help uncover unforeseen vulnerabilities and the means to mitigate them. The modular nature of the framework enables users to swap out learning algorithms and modify the reward functions in order to adapt the learning tasks to various use cases and environments. Several algorithms, viz., tabular Q learning, deep Q networks, proximal policy optimization, advantage actor-critic, generative adversarial imitation learning, are explored for the agents and the results highlighted. The agent learns to solve the tasks in a light-weight abstract environment. Once the agent learns to perform sufficiently well, it can be deployed in a minimega virtual machine environment (or a real network) with wrappers that map abstract actions to software commands. The agent also uses a local representation of the actions called a ‘slot-mechanism’. This allows the agent to learn in a certain network and generalize it to different networks. The defensive agent learns to predict the actions taken by an offensive agent and uses that information to anticipate the threat. This information can then either be used to raise an alarm or to take actions to thwart the attack. We believe that with the appropriate reward design, a representative environment, and action set, this framework can be generalized to tackle other cybersecurity tasks. By sufficiently training these agents, we can anticipate vulnerabilities leading to robust future designs. We can also deploy automated defensive agents that can help secure satellite groundstation and their vital national security missions.
We developed a biomechanics-informed online learning framework to learn the dynamics with ground truth generated with multiscale modeling simulation. It was built on Summit-like supercomputers, which were also used to benchmark and validate our framework on one physiologically significant modeling of deformable biological cells. We generalized the century-old equation of Jeffery orbits to a new equation of motion with additional parameters to account for the flow conditions and the cell deformability. Using simulation data at particle-based resolutions for flowing cells and the learned parameters from our framework, we validated the new equation by the motions, mostly rotations, of a human platelet in shear blood flow at various shear stresses and platelet deformability. Our online framework, which surrogates redundant computations in the conventional multiscale modeling by solutions of our learned equation, accelerates the conventional modeling by three orders of magnitude without visible loss of accuracy.
Ligand unbinding is mediated by its free energy change, which has intertwined contributions from both energy and entropy. It is important, but not easy, to quantify their individual contributions to the free energy profile. We model hydrophobic ligand unbinding for two systems, a methane particle and a C 60 fullerene, both unbinding from hydrophobic pockets in all-atom water. Using a modified deep learning framework, we learn a thermodynamically optimized reaction coordinate to describe the hydrophobic ligand dissociation for both systems. Interpretation of these reaction coordinates reveals the roles of entropic and enthalpic forces as the ligand and pocket sizes change. In both cases, we observe that the free-energy barrier to unbinding is dominated by entropy considerations. Furthermore, the process of methane unbinding is driven by methane solvation, while fullerene unbinding is driven first by pocket wetting and then fullerene wetting. For both solutes, the direct importance of the distance from the binding pocket to the learned reaction coordinate is present, but low. Furthermore, our framework and subsequent feature important analysis thus give useful thermodynamic insight into hydrophobic ligand dissociation problems that are otherwise difficult to glean.
Partial differential equations (PDEs) play a central role in the mathematical analysis and modeling of complex dynamic processes across all corners of science and engineering. Their solution often requires laborious analytical or computational tools, associated with a cost that is markedly amplified when different scenarios need to be investigated, for example, corresponding to different initial or boundary conditions, different inputs, etc. In this work, we introduce physics-informed DeepONets, a deep learning framework for learning the solution operator of arbitrary PDEs, even in the absence of any paired input-output training data. We illustrate the effectiveness of the proposed framework in rapidly predicting the solution of various types of parametric PDEs up to three orders of magnitude faster compared to conventional PDE solvers, setting a previously unexplored paradigm for modeling and simulation of nonlinear and nonequilibrium processes in science and engineering.
Transfer learning enables the transfer of knowledge gained while learning to perform one task (source) to a related but different task (target), hence addressing the expense of data acquisition and labelling, potential computational power limitations and dataset distribution mismatches. Here, we propose a new transfer learning framework for task-specific learning (functional regression in partial differential equations) under conditional shift based on the deep operator network (DeepONet). Task-specific operator learning is accomplished by fine-tuning task-specific layers of the target DeepONet using a hybrid loss function that allows for the matching of individual target samples while also preserving the global properties of the conditional distribution of the target data. Inspired by conditional embedding operator theory, we minimize the statistical distance between labelled target data and the surrogate prediction on unlabelled target data by embedding conditional distributions onto a reproducing kernel Hilbert space. We demonstrate the advantages of our approach for various transfer learning scenarios involving nonlinear partial differential equations under diverse conditions due to shifts in the geometric domain and model dynamics. Our transfer learning framework enables fast and efficient learning of heterogeneous tasks despite considerable differences between the source and target domains.
We report that the accurate disease diagnosis and prognosis based on omics data rely on the effective identification of robust prognostic and diagnostic markers that reflect the states of the biological processes underlying the disease pathogenesis and progression. In this article, we present GCNCC, a Graph Convolutional Network-based approach for Clustering and Classification, that can identify highly effective and robust network-based disease markers. Based on a geometric deep learning framework, GCNCC learns deep network representations by integrating gene expression data with protein interaction data to identify highly reproducible markers with consistently accurate prediction performance across independent datasets possibly from different platforms. GCNCC identifies these markers by clustering the nodes in the protein interaction network based on latent similarity measures learned by the deep architecture of a graph convolutional network, followed by a supervised feature selection procedure that extracts clusters that are highly predictive of the disease state. By benchmarking GCNCC based on independent datasets from different diseases (psychiatric disorder and cancer) and different platforms (microarray and RNA-seq), we show that GCNCC outperforms other state-of-the-art methods in terms of accuracy and reproducibility.
Data assimilation is concerned with sequentially estimating a temporally evolving state. This task, which arises in a wide range of scientific and engineering applications, is particularly challenging when the state is high-dimensional and the state-space dynamics are unknown. This paper introduces a machine learning framework for learning dynamical systems in data assimilation. Here, our auto-differentiable ensemble Kalman filters (AD-EnKFs) blend ensemble Kalman filters for state recovery with machine learning tools for learning the dynamics. In doing so, AD-EnKFs leverage the ability of ensemble Kalman filters to scale to high-dimensional states and the power of automatic differentiation to train high-dimensional surrogate models for the dynamics. Numerical results using the Lorenz-96 model show that AD-EnKFs outperform existing methods that use expectation-maximization or particle filters to merge data assimilation and machine learning. In addition, AD-EnKFs are easy to implement and require minimal tuning.
Understanding spatial relations of objects is critical in many robotic applications such as grasping, manipulation, and obstacle avoidance. Humans can simply reason object's spatial relations from a glimpse of a scene based on prior knowledge of spatial constraints. The proposed method enables a robot to comprehend spatial relationships among objects from RGB-D data. This paper proposed a neural-logic learning framework to learn and reason spatial relations from raw data by following logic rules on spatial constraints. The neural-logic network consists of three blocks: grounding block, spatial logic block, and inference block. The grounding block extracts high-level features from the raw sensory data. The spatial logic blocks can predicate fundamental spatial relations by training a neural network with spatial constraints. The inference block can infer complex spatial relations based on the predicated fundamental spatial relations. Simulations and robotic experiments evaluated the performance of the proposed method.
To curb the energy consumption of buildings and their related CO2 emissions, Oak Ridge National Laboratory (ORNL) has developed the thermally anisotropic building envelope (TABE) —a multi-layer design comprising insulation materials and metal foils connected to thermal loops. In this study, a machine learning-assisted framework was developed to control the TABE in residential buildings to reduce the computation load for future optimal rule-based control and application. First, a 2D finite element model was established in COMSOL to calculate the hourly heat flux through exterior walls installed with the TABE. Then, TABE wall heat fluxes were simulated for various given indoor and outdoor boundary conditions, as well as thermal loops fluid temperatures and flow rates. Since the finite element simulations are computationally expensive, an artificial neural network (ANN) was then trained to use as a proxy of the finite element (COMSOL) modeling. Finally, the trained ANN model was coupled with the EnergyPlus model to predict the energy consumption of a US Department of Energy prototype single-family house installed with the TABE. An optimal simple rule-based control was determined from predefined rules for a case study. The results demonstrate that the developed machine learning–assisted framework can reduce 99.9% of the computation time while efficiently managing residential building energy for installed TABE walls.
This is the conference paper accompanying an oral presentation “Unveiling the Potential of MeshGraphNets for Predicting Subsurface Evolution in Carbon Storage Projects” at the 17th International Conference on Greenhouse Gas Control Technologies GHGT-17 held in Calgary, Canada, October 20-24 , 2024. Carbon capture and storage (CCS) technology is critical for mitigating climate change but requires effective subsurface reservoir management to ensure safe containment of injected CO2. Accurate predictions of reservoir pressure and saturation are essential for assessing long-term CCS performance. Traditional numerical simulations, while effective, are computationally intensive, time-consuming, and constrained by data discretization. Previous work has shown the effectiveness of MeshGraphNets (MGN), a graph-based machine learning framework, as an innovative alternative for predicting reservoir behavior. MGN leverages graph neural networks (GNNs) and mesh representations to model complex geological formations, offering superior adaptability across different discretizations and reservoir configurations. Classic MGN implementations utilize an autoregressive technique to predict future behavior based on current predictions, but this technique is hampered by error accumulation over time. To enhance the model accuracy in time-series predictions, this study implemented a multi-step rollout strategy that integrates autoregressive predictions during training to stabilize prediction of saturation over time. Using the Illinois Basin – Decatur Project (IBDP) dataset, comprising 100 simulations of CO2 injection, pressure, and saturation changes, the framework demonstrated its ability to learn spatial dependencies and temporal dynamics. With inputs including permeabilities, porosities, and injection rates, MGN accurately predicted CO2 plume evolution over time, even with limited training data. Moreover, the addition of a multi-step rollout procedure during training improved the ability of MGN to predict stably over time by ~15%. This research positions MGN, enhanced with multi-step rollout capabilities, as a robust and efficient tool for CCS applications. It advances the field by enabling precise, computationally efficient predictions of reservoir behavior, providing a foundation for the broader adoption of machine learning frameworks in CCS and other geoscience domains.
Joint inversion has drawn considerable attention due to the availability of multiple geophysical data sets, ever-increasing computational resources, the development of advanced algorithms, and its ability to reduce inversion uncertainty. A key issue of joint inversion is to develop effective strategies to link different geophysical data in a unified mathematical framework, in which the information obtained from different models can complement each other. We have developed a deep learning-enhanced joint inversion framework to simultaneously reconstruct different physical models by fusing different types of geophysical data. Traditionally, structure similarity constraints are pursued by joint inversion algorithms using manually crafted formulations (e.g., cross gradient). The constraint is constructed by a deep neural network (DNN) during the learning process. The framework is designed to combine the DNN and a traditional independent inversion workflow and improve the joint inversion result iteratively. The network can be easily extended to incorporate multiphysics without structural changes. Numerical experiments on the joint inversion of 2D DC resistivity data and seismic traveltime are used to validate our method. In addition, this learning-based framework demonstrates excellent generalization abilities when tested on data sets using different geologic structures. It also can handle different sensing configurations and nonconforming discretization.
The critical heat flux (CHF) corresponding to the departure from nucleate boiling (DNB) crisis is a regulatory limit for the licensing of pressurized water reactors (PWRs) worldwide. Despite the abundance of predictive tools available to the reactor thermal-hydraulics community, the path for an accurate CHF model remains elusive. This work approaches the prediction of DNB through a physics-informed machine learning-aided framework (PIMLAF) with the objective of achieving superior predictive capabilities for a rod bundle. In view of the limitations in existing macro-scale physics-driven tools, an improved mechanistic model is first proposed, leveraging key concepts in the liquid sublayer dryout and bubble crowding mechanisms. Furthermore, the proposed mechanistic model is able to predict DNB in different heater geometries for a broad range of flow conditions without the need for recalibration. This model is then incorporated as the physics-informed component of the hybrid framework PIMLAF, which takes advantage of established understanding in the field (i.e., domain knowledge [DK]) and uses machine learning (ML) to capture undiscovered information from the mismatch between the actual and DK-predicted output. Two bundle-related case studies using the PWR subchannel and bundle tests (PSBT) database are carried out to illustrate the PIMLAF’s improved performance over traditional approaches for both interpolation and extrapolation purposes. In light of the PIMLAF’s promising potential to reduce prediction error, reactor vendors are encouraged to leverage their in-house experimental efforts and apply the hybrid framework to potentially achieve margin reductions in the minimum DNB ratio (MDNBR) for the designs of interest.
This project aimed to develop and demonstrate a Data Integration, Assimilation, and Learning framework for geologic carbon sequestration projects (DIAL-GCS). DIAL-GCS is an intelligence monitoring system (IMS) for automating GCS closed-loop management by leveraging recent developments in machine learning technologies, complex event processing (CEP), and reduced-order modeling. The safe and efficient operation of GCS repositories requires integrated monitoring to track the injected CO¬2 as it moves within a storage reservoir. GCS projects are data intensive, as a result of proliferation of digital instrumentation and smart-sensing technologies. GCS projects are also resource intensive, often requiring multidisciplinary teams performing different monitoring, verification, accounting (MVA) tasks throughout the lifecycle of a project to ensure secure containment of injected CO2. The success of GCS thus depends in a large part on our ability to access, assimilate, and analyze heterogeneous data and information sources in a timely manner. This project included a number of meaningful and necessary tasks to transform the human domain knowledge into machine-interpretable rules for automating knowledge extraction and discovery in GCS. The specific technical objectives of the proposed DIAL-GCS project were to develop an ontology-driven GCS data management module for storing, querying, and exchanging GCS data (both historic and live sensor data) from multiple sources and in heterogeneous formats. Incorporate a CEP engine for detecting abnormal situations by seamlessly combining expert knowledge, rule-based reasoning, and machine learning. Enable uncertainty quantification and predictive analytics using a combination of coupled-process modeling, AI/ML methods, and reduced-order modeling, and integrate and demonstrate the system’s capabilities with both real and simulated data. As far as we know, this is one of the first projects aimed to develop intelligent monitoring systems (IMS) targeting the GCS. Under this project, the team had developed a large number of web applications and scientific algorithms that contribute the main theme of intelligent monitoring. The team has published more than a dozen peer reviewed papers and disseminated the research results at multiple technical meetings.
Pore-scale modeling has limited applicability at large scales due to its high computational cost. One common approach to upscale pore-scale models is the use of effective medium theories, which homogenize small-scale features in a porous structure and characterize the medium by macroscale properties (e.g., permeability) and equations (e.g., Darcy’s law). However, there are classes of physical processes for which effective medium approximations may become inaccurate, e.g., mineral precipitation and clogging during reactive transport. We have developed a deep learning upscaling framework, in which pore-scale modeling is directly employed in macroscale systems, without relying on effective medium approximations. The upscaling framework is first developed for general multiscale systems and then applied to modeling reactive transport with mineral precipitation in the altered layer in fracture-matrix structures. Solute transport from the fractures to the matrix is modeled as a wall boundary condition for the fractures, which, in turn, is predicted by recurrent neural networks using the concentration histories at the fracture-matrix boundary. Specifically, we consider a meter-scale fracture network embedded in sandstones, where the smallest feature is at the micron scale. Here the proposed framework allows us to span five orders of magnitude in length scales by capturing mineral precipitation in the altered layer of the rock matrix at the pore scale across the entire meter-scale fracture network.
We present a machine learning framework bridging manifold learning, neural networks, Gaussian processes, and Equation-Free multiscale approach, for the construction of different types of effective reduced order models from detailed agent-based simulators and the systematic multiscale numerical analysis of their emergent dynamics. The specific tasks of interest here include the detection of tipping points, and the uncertainty quantification of rare events near them. Our illustrative examples are an event-driven, stochastic financial market model describing the mimetic behavior of traders, and a compartmental stochastic epidemic model on an Erdös-Rényi network. We contrast the pros and cons of the different types of surrogate models and the effort involved in learning them. Importantly, the proposed framework reveals that, around the tipping points, the emergent dynamics of both benchmark examples can be effectively described by a one-dimensional stochastic differential equation, thus revealing the intrinsic dimensionality of the normal form of the specific type of the tipping point. This allows a significant reduction in the computational cost of the tasks of interest.
Traditional methods for solvability region analysis can only have inner approximations with inconclusive conservatism and handle limited types of power flow models. In this letter, we propose a deep active learning framework for solvability prediction in power systems. Compared with passive learning where the training is performed after all instances are labeled, active learning selects most informative instances to be labeled and therefore significantly reduces the size of the labeled dataset for training. In the active learning framework, the acquisition functions, which correspond to different sampling strategies, are defined in terms of the on-the-fly posterior probability from the classifier. First, the IEEE 39-bus system is employed to validate the proposed framework, where a two-dimensional case is illustrated to visualize the effectiveness of the sampling method followed by the high-dimensional numerical experiments. Then, the Northeast Power Coordinating Council (NPCC) 140-bus system is used to validate the performance on large-scale power systems.