SEARCH · Engineering Papers
Results for “Learning Framework”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
STAGED DDCM: A Framework for Closure Modeling through Data Assimilation and Machine Learning
Presentation video for ML/DL workshop
X-BASE: the first terrestrial carbon and water flux products from an extended data-driven scaling framework, FLUXCOM-X
Mapping in situ eddy covariance measurements of terrestrial land–atmosphere fluxes to the globe is a key method for diagnosing the Earth system from a data-driven perspective. We describe the first global products (called X-BASE) from a newly implemented upscaling framework, FLUXCOM-X, representing an advancement from the previous generation of FLUXCOM products in terms of flexibility and technical capabilities. The X-BASE products are comprised of estimates of CO 2 net ecosystem exchange (NEE), gross primary productivity (GPP), evapotranspiration (ET), and for the first time a novel, fully data-driven global transpiration product (ETT), at high spatial (0.05°) and temporal (hourly) resolution. X-BASE estimates the global NEE at −5.75 ± 0.33 Pg C yr −1 for the period 2001–2020, showing a much higher consistency with independent atmospheric carbon cycle constraints compared to the previous versions of FLUXCOM. The improvement of global NEE was likely only possible thanks to the international effort to increase the precision and consistency of eddy covariance collection and processing pipelines, as well as to the extension of the measurements to more site years resulting in a wider coverage of bioclimatic conditions. However, X-BASE global net ecosystem exchange shows a very low interannual variability, which is common to state-of-the-art data-driven flux products and remains a scientific challenge. With 125 ± 2.1 Pg C yr −1 for the same period, X-BASE GPP is slightly higher than previous FLUXCOM estimates, mostly in temperate and boreal areas. X-BASE evapotranspiration amounts to 74.7×10 3 ± 0.9×10 3 km 3 globally for the years 2001–2020 but exceeds precipitation in many dry areas, likely indicating overestimation in these regions. On average 57 % of evapotranspiration is estimated to be transpiration, in good agreement with isotope-based approaches, but higher than estimates from many land surface models. Despite considerable improvements to the previous upscaling products, many further opportunities for development exist. Pathways of exploration include methodological choices in the selection and processing of eddy covariance and satellite observations, their ingestion into the framework, and the configuration of machine learning methods. For this, the new FLUXCOM-X framework was specifically designed to have the necessary flexibility to experiment, diagnose, and converge to more accurate global flux estimates.
CoDL: A Framework for Studying Cross-Component Interference in Deep Learning Training Pipelines
Explore the source record for details and available documents.
GOOML - Finding Optimization Opportunities for Geothermal Operations: Preprint
Geothermal Operational Optimization with Machine Learning (GOOML) is a transferable and extensible component-based geothermal asset modeling framework that considers complex steamfield relationships and identifies optimization prospects using a data-driven approach. We have used this framework to develop digital twins that provide steamfield operators with an operational environment to analyze and understand historical and forecasted power production, explore new steamfield configuration possibilities, and seek optimal asset management for real world applications. The GOOML modeling software is built on a generic component-based systems framework that allows for both historical and forecast analysis. A GOOML model can perform historical data-assimilation using first-principal thermodynamics to create a meaningful data model. Historical production data can then be coupled with a forecast framework to train machine-learning models of steamfield components to predict future outputs. This modeling environment enables digital exploration of steamfield design configurations and operational scenarios. GOOML digital twins have been developed for steamfields in New Zealand and the United States representing differing power generation and field conditions. These digital twins have been validated by comparing hindcast predictions against historical production data. Reinforcement learning experiments were conducted to demonstrate the ability to programmatically explore the operations space using machine learning agents. Our initial results are compelling; two to five percent increases in annual energy production were demonstrated by the GOOML models with no additional infrastructure build required. GOOML offers a new approach to geothermal operations by applying state-of-the-art machine learning algorithms, comprehensive data analytics, and interaction with digital twins. Through application of these tools, operators will realize greater availability and higher net generation which will increase the cost effectiveness of geothermal energy projects.
Multi-head physics-informed neural networks for learning functional priors and uncertainty quantification
In numerous applications, the integration of prior knowledge and historical information is essential, particularly for tasks requiring the solution of ordinary or partial differential equations (ODEs/PDEs) in data-sparse or noisy environments. For instance, achieving accurate solutions to time-dependent PDEs with limited initial condition measurements necessitates an effective strategy for embedding prior knowledge. Hard-parameter sharing architectures in neural networks (NNs) have demonstrated success in both traditional and scientific machine learning domains, facilitating the learning of informative representations. Here, in this study, we introduce a novel, yet efficient, method to enhance physics-informed neural networks (PINNs) by incorporating a multi-head structure that enables the learning of functional priors from both empirical data and governing physical laws. This prior information can then be used to address data sparsity and high-level noise in solving ODE/PDE problems with uncertainty quantification (UQ). The approach, termed Multi-Head PINN (MH-PINN), consists of a shared body NN and multiple head NNs, each corresponding to an individual PINN instance. Our framework for functional prior learning is carried out in two stages: (1) training the MH-PINNs to develop a shared body NN alongside multiple head NNs, and (2) employing these trained head NNs to estimate a prior distribution through a normalizing flow-based density estimator. The learned functional prior can then be applied as a regularization mechanism in deterministic contexts or as an informative prior within a Bayesian inference framework, aiding in the resolution of subsequent ODE/PDE tasks. We evaluate the efficacy of MH-PINNs across five benchmark problems, including a high-dimensional parametric PDE, all characterized by data sparsity or substantial noise levels. Our findings reveal that MH-PINNs deliver accurate solutions and robust UQ, demonstrating adaptability across a range of complex and challenging scenarios.
Learning Grain-Boundary Segregation: From First Principles to Polycrystals
The segregation of solute atoms at grain boundaries (GBs) can strongly impact the structural and functional properties of polycrystals. Yet, due to the limited availability of simulation tools to study polycrystals at the atomistic scale (i.e., interatomic potentials), there is a minimal understanding of the variation of solute segregation tendencies across the very complex space of GB microenvironments and the large range of alloys in which it can occur. Here, we develop an algorithmic framework that can directly learn the full spectrum of segregation energies for a metal solute atom in a metal polycrystal from ab initio methods, bypassing the need for alloy interatomic potentials. This framework offers a pathway to a comprehensive catalog of GB solute segregation with quantum accuracy, for the entire alloy space. As an initial demonstration in this pursuit, we build an extensive GB segregation database for aluminum-based alloys across the periodic table, including dozens of alloys for which there are substantially no prior data.
Modular performance prediction for scientific workflows using Machine Learning
Scientific workflows provide an opportunity for declarative computational experiment design in an intuitive and efficient way. A distributed workflow is typically executed on a variety of resources, and it uses a variety of computational algorithms or tools to achieve the desired outcomes. Such a variety imposes additional complexity in scheduling these workflows on large scale computers. As computation becomes more distributed, insights into expected workload that a workflow presents become critical for effective resource allocation. In this paper, we present a modular framework that leverages Machine Learning for creating precise performance predictions of a workflow. The central idea is to partition a workflow in such a way that makes the task of forecasting each atomic unit manageable and gives us a way to combine the individual predictions efficiently. We recognize a combination of an executable and a specific physical resource as a single module. This gives us a handle to characterize workload and machine power as a single unit of prediction. Overall, our modular technique of creating atomic modules and deployment of longest-path approach to estimate workflow performance, allows the framework to adapt to highly complex nested directed acyclic workflows and scale to new scenarios, since it does not make assumptions of underlying workflow structure. We present performance estimation results of independent workflow modules executed on the XSEDE SDSC Comet cluster using various Machine Learning algorithms. The results provide insights into the behavior and effectiveness of different algorithms in the context of scientific workflow performance prediction.
Energy storage planning for enhanced resilience of power systems against wildfires and heatwaves
Extreme weather events pose significant risks to power grid stability due to their severe consequences and potential for widespread failures. Energy storage systems hold great potential for enhancing grid resilience against such events by providing reliable power during peak demand periods. However, accurately quantifying the size, location, and investment costs of new energy storage assets is a complex task, as energy storage planning decisions depend on the investment choices of other generation technologies and the integration of new transmission projects. Here, this paper presents a novel capacity expansion planning framework that simultaneously optimizes investments in energy storage, generation, and transmission, determining their optimal size, location, and type, while incorporating extreme weather events into long-term planning. More specifically, our stress-event-informed planning framework integrates the impact of heatwaves and wildfires into the planning process, identifying least-cost investment solutions that comply with policy goals and enhance grid resilience. The proposed framework employs machine-learning-based modeling to project heatwave-induced loads and performance-based risk assessment to evaluate wildfire-driven transmission line derates. Using industry-standard datasets to accurately represent the transmission topology of the Western Interconnection (WI) system, the proposed framework is applied to the WI 40-zone system, with investment decisions reported for the years 2030, 2035, and 2040. Simulation results reveal that with just a 10% increase in investment costs, resilience against extreme events can be significantly improved, with investment decisions heavily favoring energy storage, particularly 4-hour energy storage systems.
Extreme-scale stochastic optimization and simulation via learning-enhanced decomposition and parallelization (Final Technical Report)
Stochastic optimization and simulation models ubiquitously arise in designing and operating complex service/engineering systems. They can be extreme in scale due to high-dimensional data and decisions, and can also involve decisions made sequentially in response to newly revealed data, both causing significant computational challenge. The objective of this research is to explore a unified framework that integrates machine learning with discrete optimization and risk-averse modeling, to improve the efficiency of decomposition paradigms for stochastic optimization and simulations at extreme scale. The models we consider represent a broad class of complex decision-making problems, where 0-1 or continuous decisions are made before and/or after knowing multiple sources of uncertainties that could be correlated. We will employ machine learning methods to dynamically decide and prioritize computational procedures, including cut generation, branching, and bounding of the optimal objective. Furthermore, the research will shed new lights on the traditional decomposition algorithms for extreme-scale computing. Deliverables of the research include new modeling and computational methods for advancing the state-of-the-art research in optimization and simulation, bringing many relevant risk-averse, data-driven optimization problems in practice within the range of tractability. Examples include distributed computing server scheduling and sensor deployment for monitoring critical infrastructures. Success in this effort will enable progress in solving multiple extreme-scale problems in the complex system design and operations arising from DoE missions in energy, environment, and national security.
Deep learning-enhanced design for functionally graded auxetic lattices
Auxetic materials with counterintuitive negative Poisson’s ratio have been of significant interest due to potential applications across diverse engineering fields. Functionally grading such auxetics further enables customization of the structural response and harnesses the potential for multi-functional applications. However, a critical challenge in designing functionally graded lattices is to efficiently determine the spatial variation of the functional gradient and the corresponding geometric designs to achieve the desired response. In this paper, a highly efficient deep learning-based inverse design framework for functionally graded tetra-petal auxetics with spatially tailored properties is presented. This framework significantly improves the efficiency of tailoring functionally graded auxetics where many unit cells need to be tailor-designed. The graded tetra-petal auxetics obtained from the inverse design framework are additively manufactured and subjected to impact tests. The results show superior impact performance compared with uniform designs, demonstrating the effectiveness of the proposed inverse design framework, which can be inspirable to promote advanced structures/materials with enhanced impact resistance.
Ca X ML: Chemistry‐informed machine learning explains mutual changes between protein conformations and calcium ions in calcium‐binding proteins using structural and topological features
Proteins' flexibility is a feature in communicating changes in cell signaling instigated by binding with secondary messengers, such as calcium ions, associated with the coordination of muscle contraction, neurotransmitter release, and gene expression. When binding with the disordered parts of a protein, calcium ions must balance their charge states with the shape of calcium-binding proteins and their versatile pool of partners depending on the circumstances they transmit. Accurately determining the ionic charges of those ions is essential for understanding their role in such processes. However, it is unclear whether the limited experimental data available can be effectively used to train models to accurately predict the charges of calcium-binding protein variants. Here, we developed a chemistry-informed, machine-learning algorithm that implements a game theoretic approach to explain the output of a machine-learning model without the prerequisite of an excessively large database for high-performance prediction of atomic charges. We used the ab initio electronic structure data representing calcium ions and the structures of the disordered segments of calcium-binding peptides with surrounding water molecules to train several explainable models. Network theory was used to extract the topological features of atomic interactions in the structurally complex data dictated by the coordination chemistry of a calcium ion, a potent indicator of its charge state in protein. Our design created a computational tool of Ca X ML, which provided a framework of explainable machine learning model to annotate ionic charges of calcium ions in calcium-binding proteins in response to the chemical changes in an environment. Our framework will provide new insights into protein design for engineering functionality based on the limited size of scientific data in a genome space.
Network Anomaly Detection in Distributed Edge Computing Infrastructure
As networks continue to grow in complexity and scale, detecting anomalies has become increasingly challenging, particularly in diverse and geographically dispersed environments. Traditional approaches often struggle with managing the computational burden associated with analyzing large-scale network traffic to identify anomalies. This paper introduces a distributed edge computing framework that integrates federated learning with Apache Spark and Kubernetes to address these challenges. We hypothesize that our approach, which enables collaborative model training across distributed nodes, significantly enhances the detection accuracy of network anomalies across different network types. We show that by leveraging distributed computing and containerization technologies, our framework not only improves scalability and fault tolerance but also achieves superior detection performance compared to state-of-the-art methods. Extensive experiments on the UNSW-NB15 and ROAD datasets validate the effectiveness of our approach, demonstrating statistically significant improvements in detection accuracy and training efficiency over baseline models, as confirmed by MannWhitney U and Kolmogorov-Smirnov tests (p<0.05).
Developing a complete AI-accelerated workflow for superconductor discovery
The quest to identify new superconducting materials with enhanced properties is hindered by the prohibitive cost of computing electron-phonon spectral functions, severely limiting the materials space that can be explored. Here, we introduce a Bootstrapped Ensemble of Equivariant Graph Neural Networks (BEE-NET), a machine-learning model trained to predict the Eliashberg spectral function and superconducting critical temperature with a mean-absolute-error of 0.87 K relative to DFT-based Allen-Dynes calculations. Intriguingly, BEE-NET achieves a true-negative-rate of 99.4%, enabling highly efficient screening for the rare property of superconductivity. Integrated into a multi-stage, AI-accelerated discovery pipeline that incorporates elemental-substitution strategies and machine-learned interatomic potentials, our workflow reduced over 1.3 million candidate structures to 741 dynamically and thermodynamically stable compounds with DFT-confirmed T c > 5 K. We report the successful synthesis and experimental confirmation of superconductivity in two of these previously unreported compounds. This study establishes a data-driven framework that integrates machine learning, quantum calculations, and experiments to systematically accelerate superconductor discovery.
Scientific exploration with expert knowledge (SEEK) in autonomous scanning probe microscopy with active learning
This work introduces a knowledge-informed framework that integrates human expertise and prior interest into active learning-driven autonomous experimentation, which enhances the exploration efficiency through more targeted experimentation.
End-to-End Automated Segmentation Framework for Four-Dimensional Scanning Transmission Electron Microscopy Data
Four-dimensional scanning transmission electron microscopy (4D-STEM) is powerful for rapidly characterizing arrays of nanoparticles produced via high-throughput synthesis. However, such 4D-STEM datasets typically contain thousands of nanoparticles, each characterized by thousands of diffraction patterns spatially distributed across the nanoparticle, necessitating efficient and comprehensive analysis. We propose an end-to-end segmentation framework to automatically segment each nanoparticle into regions with distinct composition/orientation of crystal grains, using only the 4D-STEM data. Bragg disk information is extracted in a physics-informed manner from the diffraction patterns at each spatial location and combined with the real space coordinates to form feature vectors. These feature vectors are then used as inputs to a Gaussian mixture model (GMM) to segment the nanoparticle into distinct regions. We also develop two visualization tools based on the GMM outputs to infer the interface transition and the degree of superposition. Our framework comprehensively integrates machine learning tools and physics knowledge, and provides a basis for substantially compressing enormous 4D-STEM datasets, e.g., by replacing the full 4D-STEM dataset for each nanoparticle with only a single set of Bragg disk features for each distinct crystal grain identified in the nanoparticle. In this article, we demonstrate the power of our framework by presenting results for real, complex datasets.
Fast and Accurate Machine Learning Strategy for Calculating Partial Atomic Charges in Metal–Organic Frameworks
Computational high-throughput screening using molecular simulations is a powerful tool for identifying top-performing metal–organic frameworks (MOFs) for gas storage and separation applications. Accurate partial atomic charges are often required to model the electrostatic interactions between the MOF and the adsorbate, especially when the adsorption involves molecules with dipole or quadrupole moments such as water and CO 2 . Although ab initio methods can be used to calculate accurate partial atomic charges, these methods are impractical for screening large material databases because of the high computational cost. We developed a random forest machine learning model to predict the partial atomic charges in MOFs using a small yet meaningful set of features that represent both the elemental properties and the local environment of each atom. The model was trained and tested on a collection of about 320 000 density-derived electrostatic and chemical (DDEC) atomic charges calculated on a subset of the Computation-Ready Experimental Metal–Organic Framework (CoRE MOF-2019) database and separately on charge model 5 (CM5) charges. The model predicts accurate atomic charges for MOFs at a fraction of the computational cost of periodic density functional theory (DFT) and is found to be transferable to other porous molecular crystals and zeolites. In conclusion, a strong correlation is observed between the partial atomic charge and the average electronegativity difference between the central atom and its bonded neighbors.