Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Scientific Machine Learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 901 records · Page 50

Unsupervised discovery of extreme weather events using universal representations of emergent organization

Spontaneous self-organization is ubiquitous in systems far from thermodynamic equilibrium. While organized structures that emerge dominate transport properties, universal representations that identify and describe these key objects remain elusive. Here, we introduce a theoretically grounded framework for describing emergent organization that, via data-driven algorithms, is constructive in practice. Its building blocks are spacetime lightcones that embody how information propagates across a system through local interactions. We show that predictive equivalence classes of lightcones—local causal states—capture organized behaviors in complex spatiotemporal systems. Employing an unsupervised physics-informed machine learning algorithm and a high-performance computing implementation, we demonstrate automatically discovering organized structures in two real-world domain science problems. We show that local causal states identify vortices and track their power-law decay behavior in two-dimensional fluid turbulence. We then show how to detect and track familiar extreme weather events—hurricanes and atmospheric rivers—and discover other novel structures associated with precipitation extremes in high-resolution climate data at the grid-cell level.

Rupe, Adam [Pacific Northwest National Laboratory ↗

AI-based design of a nuclear reactor core

The authors developed an artificial intelligence (AI)-based algorithm for the design and optimization of a nuclear reactor core based on a flexible geometry and demonstrated a 3× improvement in the selected performance metric: temperature peaking factor. The rapid development of advanced, and specifically, additive manufacturing (3-D printing) and its introduction into advanced nuclear core design through the Transformational Challenge Reactor program have presented the opportunity to explore the arbitrary geometry design of nuclear-heated structures. The primary challenge is that the arbitrary geometry design space is vast and requires the computational evaluation of many candidate designs, and the multiphysics simulation of nuclear systems is very time-intensive. Therefore, the authors developed a machine learning-based multiphysics emulator and evaluated thousands of candidate geometries on Summit, Oak Ridge National Laboratory’s leadership class supercomputer. The results presented in this work demonstrate temperature distribution smoothing in a nuclear reactor core through the manipulation of the geometry, which is traditionally achieved in light water reactors through variable assembly loading in the axial direction and fuel shuffling during refueling in the radial direction. The conclusions discuss the future implications for nuclear systems design with arbitrary geometry and the potential for AI-based autonomous design algorithms.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

An Integrated Framework for Risk Assessment of Safety-related Digital Instrumentation and Control Systems in Nuclear Power Plants: Methodology Advancement and Application

This report documents activities performed by Idaho National Laboratory (INL) during fiscal year (FY) 2024 for the U.S. Department of Energy (DOE) Light Water Reactor Sustainability (LWRS) Program, Risk Informed Systems Analysis (RISA) Pathway, Digital Instrumentation and Control (DI&C) Risk Assessment project. The goal of the RISA Pathway is to optimize safety margins and minimize uncertainties to achieve economic efficiencies while maintaining high levels of safety. This is accomplished by providing scientific basis to better represent safety margins and factors that contribute to cost and safety, and by developing new technologies that reduce operating costs. The research efforts for FY 2024 encompass methodology refinement and exploration. The efforts include: (1) The implementation of a natural language processing tool to expedite key aspects of the reliability analysis methods developed by INL; (2) advances to support intersystem CCF analysis by providing guidance for and identification of coupling mechanisms that may contribute to CCF; (3) the investigation of how generative artificial intelligence tools can aid in hazard analysis and diversity and defense in depth (i.e., D3) assessments; (4) Industry collaboration, allowing the demonstration of and INL's risk assessment tools to support risk assessment of DI&C systems at early and late stages of development; (4) a roadmap for the development of a software for each of INL's risk assessment tools; (5) The development of a theory and methodology manual for a risk quantification methodology; (6) the development of a reliability analysis for machine learning (ML)-integrated control systems.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Accelerating science: The usage of commercial clouds in ATLAS Distributed Computing

The ATLAS experiment at CERN is one of the largest scientific machines built to date and will have ever growing computing needs as the Large Hadron Collider collects an increasingly larger volume of data over the next 20 years. ATLAS is conducting R&D projects on Amazon Web Services and Google Cloud as complementary resources for distributed computing, focusing on some of the key features of commercial clouds: lightweight operation, elasticity and availability of multiple chip architectures. The proof of concept phases have concluded with the cloud-native, vendoragnostic integration with the experiment’s data and workload management frameworks. Google Cloud has been used to evaluate elastic batch computing, ramping up ephemeral clusters of up to O(100k) cores to process tasks requiring quick turnaround. Amazon Web Services has been exploited for the successful physics validation of the Athena simulation software on ARM processors. We have also set up an interactive facility for physics analysis allowing endusers to spin up private, on-demand clusters for parallel computing with up to 4 000 cores, or run GPU enabled notebooks and jobs for machine learning applications. The success of the proof of concept phases has led to the extension of the Google Cloud project, where ATLAS will study the total cost of ownership of a production cloud site during 15 months with 10k cores on average, fully integrated with distributed grid computing resources and continue the R&D projects.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

A Software Defined Storage Approach to Exascale Storage Services (Final Technical Report)

The Mochi project began in 2015 in recognition of a set of changes that were set to alter the landscape of storage and I/O for DOE science. On the one hand, an influx of new classes of applications including an initial set of data intensive and machine learning applications were operating with new types of data and in new ways, as compared to traditional computational science simulation codes. Simultaneously, new memory, storage, and networking technologies were driving a need to adapt software designs to a rapidly changing design space. The goal of the three-year Mochi project was to develop and demonstrate an approach for composing storage services which provides new levels of functionality, performance, and reliability for science applications at extreme scale. The team, including a mix of Laboratory, university, and industry collaborators, focused on systems and applications of interest to DOE Office of Science, tracking developments at NERSC and the LCFs as well as application trends within ECP and SciDAC. Technical work focused on three themes: composition (understand the building blocks needed to provide storage services that meet DOE mission needs and how these components are flexibly composed), application interfacing (what service compositions are most effective for common DOE application use cases), and consistency and synchronization (taking advantage of knowledge about, and from, the system and scientific applications to provide better service than would otherwise be possible). The project was highly successful, producing 13 publications and tens of thousands of lines of open source software that is now being used by multiple ECP teams, HPC software vendors, and international collaborators. The uptake of Mochi technologies speaks to both the value of the service composition model and the quality of architecture and software developed by the team. Through continued funding under ASCR and ECP, the Mochi team and its technologies are on track to further contribute to DOE science successes into the exascale timeframe.

97 MATHEMATICS AND COMPUTING↗

Beyond Optimization: Exploring Novelty Discovery in Autonomous Experiments

Autonomous experiments (AEs) are transforming how scientific research is conducted by integrating artificial intelligence with automated experimental platforms. Current AEs primarily focus on the optimization of a predefined target; while accelerating this goal, such an approach limits the discovery of unexpected or unknown physical phenomena. Here, we introduce a novel framework, INS 2 ANE (Integrated Novelty Score−Strategic Autonomous Non-Smooth Exploration), to enhance the discovery of novel phenomena in autonomous microscopy experimentation. Our method integrates two key components: (1) a novelty scoring system that evaluates the uniqueness of experimental results and (2) a strategic sampling mechanism that promotes exploration of under-sampled regions even if they appear less promising by conventional criteria. We validate this approach on a preacquired data set with a known ground truth comprising of image−spectral pairs. We further implement the process on autonomous scanning probe microscopy experiments. INS 2 ANE significantly increases the diversity of explored phenomena in comparison to conventional optimization routines, enhancing the likelihood of discovering previously unobserved phenomena. These results demonstrate the potential for autonomous microscopy experiments to enhance the scientific discovery by navigating complex experimental spaces to uncover novel phenomena.

Materials↗

On Performance Prediction of Big Data Transfer in High-performance Networks

Big data generated by large-scale scientific and industrial applications need to be transferred between different geographical locations for remote storage, processing, and analysis. High-speed dedicated connections provisioned in High-performance Networks (HPNs) are increasingly utilized to carry out such big data transfer. HPN management highly relies on an important capability of performance (mainly throughput) prediction to reserve sufficient bandwidth and meanwhile avoid over-provisioning that may result in unnecessary resource waste. This capability is critical to improving the resource (mainly bandwidth) utilization of dedicated connections and meeting various user requests for data transfer. Conventional methods conduct performance prediction by fitting prior observed transfer history with predefined loss functions, without considering unobservable latent factors such as competing loads on end hosts. Such latent factors also have a significant impact on the application-level data transfer performance, which may result in an inaccurate prediction model. In this paper, we first investigate the impact of latent factors and propose a clustering-based method to eliminate their negative impact on performance prediction. We then develop a robust machine learning-based performance predictor by: i) incorporating the proposed latent factor elimination method into data preprocessing, and ii) adopting a customized domain guided loss function. Extensive experimental results show that our predictor achieves significantly higher prediction accuracy than several other state-of-the-art methods.

Liu, Wuji↗

The Zwicky Transient Facility: Data Processing, Products, and Archive

The Zwicky Transient Facility (ZTF) is a new robotic time-domain survey currently in progress using the Palomar 48-inch Schmidt Telescope. ZTF uses a 47 square degree field with a 600 megapixel camera to scan the entire northern visible sky at rates of ∼3760 square degrees/hour to median depths of g ~ 20.8 and r ~ 20.6 mag (AB, 5σ in 30 sec). We describe the Science Data System that is housed at IPAC, Caltech. This comprises the data-processing pipelines, alert production system, data archive, and user interfaces for accessing and analyzing the products. The real-time pipeline employs a novel image-differencing algorithm, optimized for the detection of point-source transient events. These events are vetted for reliability using a machine-learned classifier and combined with contextual information to generate data-rich alert packets. The packets become available for distribution typically within 13 minutes (95th percentile) of observation. Detected events are also linked to generate candidate moving-object tracks using a novel algorithm. Objects that move fast enough to streak in the individual exposures are also extracted and vetted. We present some preliminary results of the calibration performance delivered by the real-time pipeline. The reconstructed astrometric accuracy per science image with respect to Gaia DR1 is typically 45 to 85 milliarcsec. This is the RMS per-axis on the sky for sources extracted with photometric S/N ≥10 and hence corresponds to the typical astrometric uncertainty down to this limit. The derived photometric precision (repeatability) at bright unsaturated fluxes varies between 8 and 25 millimag. The high end of these ranges corresponds to an airmass approaching ∼2—the limit of the public survey. Photometric calibration accuracy with respect to Pan-STARRS1 is generally better than 2%. The products support a broad range of scientific applications: fast and young supernovae; rare flux transients; variable stars; eclipsing binaries; variability from active galactic nuclei; counterparts to gravitational wave sources; a more complete census of Type Ia supernovae; and solar-system objects.

Frank J. Masci↗

On the Need to Align Intent and Implementation in Uncertainty Quantification for Machine Learning

Quantifying uncertainties for machine learning (ML) models is a foundational challenge in modern data analysis. This challenge is compounded by at least two key aspects of the field: (a) inconsistent terminology surrounding uncertainty and estimation across disciplines, and (b) the varying technical requirements for establishing trustworthy uncertainties in diverse problem contexts. In this position paper, we aim to clarify the depth of these challenges by identifying these inconsistencies and articulating how different contexts impose distinct epistemic demands. We examine the current landscape of estimation targets (e.g., prediction, inference, simulation-based inference), uncertainty constructs (e.g., frequentist, Bayesian, fiducial), and the approaches used to map between them. Drawing on the literature, we highlight and explain examples of problematic mappings. To help address these issues, we advocate for standards that promote alignment between the \textit{intent} and \textit{implementation} of uncertainty quantification (UQ) approaches. We discuss several axes of trustworthiness that are necessary (if not sufficient) for reliable UQ in ML models, and show how these axes can inform the design and evaluation of uncertainty-aware ML systems. Our practical recommendations focus on scientific ML, offering illustrative cases and use scenarios, particularly in the context of simulation-based inference (SBI).

Trivedi, Shubhendu [MIT] (ORCID:0000000312374301)↗

Green AI: Insights Into Deep Learning's Looming Energy Efficiency Crisis

As demands grow to integrate artificial intelligence into every aspect of industry, commerce, and life, deep learning's exploding energy cost has become a looming crisis, making AI systems a salient energy-efficiency challenge. One might expect that doubling a neural network's size would halve its error rate, or at least allow it to achieve greater performance given the same amount of time and energy. I will present clear and substantial scientific evidence which indicates that not only is this intuition wildly wrong, but that neural networks scale so poorly that to increase deep learning performance by only a small fraction can easily require an order of magnitude or more increase in computational resources and energy. Further, the marginal trade-off price of to increase model performance rapidly explodes as performance targets are increased. To address this challenge, I will provide a toolkit of techniques that can be applied today to mitigate the inefficiency of modern deep learning. And, I will conclude by illuminating a practical path forward towards efficient, Green AI.

artificial intelligence↗

Portable, heterogeneous ensemble workflows at scale using libEnsemble

libEnsemble is a Python-based toolkit for running dynamic ensembles, developed as part of the DOE Exascale Computing Project. The toolkit utilizes a unique generator–simulator–allocator paradigm, where generators produce input for simulators, simulators evaluate those inputs, and allocators decide whether and when a simulator or generator should be called. The generator steers the ensemble based on simulation results. Generators may, for example, apply methods for numerical optimization, machine learning, or statistical calibration. libEnsemble communicates between a manager and workers. Flexibility is provided through multiple manager–worker communication substrates each of which has different benefits. These include Python’s multiprocessing, mpi4py, and TCP. Multisite ensembles are supported using Balsam or Globus Compute. We overview the unique characteristics of libEnsemble as well as current and potential interoperability with other packages in the workflow ecosystem. We highlight libEnsemble’s dynamic resource features: libEnsemble can detect system resources, such as available nodes, cores, and GPUs, and assign these in a portable way. These features allow users to specify the number of processors and GPUs required for each simulation; and resources will be automatically assigned on a wide range of systems, including Frontier, Aurora, and Perlmutter. Such ensembles can include multiple simulation types, some using GPUs and others using only CPUs, sharing nodes for maximum efficiency. We also describe the benefits of libEnsemble’s generator–simulator coupling, which easily exposes to the user the ability to cancel, and portably kill, running simulations based on models that are updated with intermediate simulation output. We demonstrate libEnsemble’s capabilities, scalability, and scientific impact via a Gaussian process surrogate training problem for the longitudinal density profile at the exit of a plasma accelerator stage. In conclusion, the study uses gpCAM for the surrogate model and employs either Wake-T or WarpX simulations, highlighting efficient use of resources that can easily extend to exascale.

Dynamic ensembles↗

Estimation of RTT and Loss Rate of Wide-Area Connections Using MPI Measurements

Scientific computations are expected to be increasingly distributed across wide-area networks, and the Message Passing Interface (MPI) has been shown to scale to support their communications over long distances. These computations should account for certain network parameters to ensure an effective execution, for example, by avoiding highly congested and long connections. The execution times of MPI basic operations reflect the connection parameters, including the Round Trip Time (RTT) and loss rate. We describe five machine leaning methods to estimate the connection RTT and loss rate using execution times of MPI basic operations. We utilize execution time measurements of MPI Sendrecv operations collected over emulated 10 Gbps connections with 0-366 ms round-trip times, wherein the longest connection spans the globe, under up to 20% periodic losses. These methods provide disparate, namely, linear and non-linear, and smooth and non-smooth, estimates of RTT and loss rate. Our results show that accurate estimates can be generated at low loss rates but they become inaccurate at loss rates 10% and higher. Overall, these results constitute a case study of the strengths and limitations of machine learning methods in inferring network-level parameters using application-level measurements.

Rao, Nageswara S.↗

Open Science for Life in Space: Data Sharing and Tools for Knowledge Discovery

The next era in human space exploration is rapidly approaching and will require the use of countermeasures to deep space health hazards. The development of countermeasures (or, the re-purposing of existing agents) will be highly dependent on our understanding of basic biological responses to space stressors (e.g. ionizing radiation, altered gravitational fields, altered day-night cycles, confinement, isolation, hostile-closed environments, distance-duration from Earth, exposure to celestial regolith, etc.). The fast-growing array of space biological data, which in the past was simply archived after minimal analysis, holds great potential if it can be reorganized and formatted for Open Science. Organizing the data for such analysis is a challenge because of its diverse nature (molecular, cellular, tissue, imaging, whole organism and behavior). We will discuss here several strategies that NASA’s Biological and Physical Science Division has put in place to maximize the return on investment for spaceflight bioscience data. Open Science, as a scientific philosophy, is the concept that the more people who have access to the data, the more knowledge will be gained from it. This guiding principle led NASA to develop GeneLab in 2015. GeneLab houses spaceflight and relevant ground-based multi-omics data, and has grown to ~400 transcriptomic, proteomic, metabolomic and epigenomic datasets from plant, rodent, small animal, and microbial space experiments. GeneLab provides users with various tools for data analysis and a visualization portal that allows users to interact with gene expression data from space-related ‘omics experiments. Open Science is also about building scientific communities, and with this spirit in mind, GeneLab has spawned several Analysis Working Groups (AWGs), comprised of more than 200 volunteer scientists. The AWGs initially provided feedback on the processing pipeline and metadata ‘omics standards for GeneLab. Over the last few years, they have become a community-driven science enterprise, engaging in large meta-analysis of GeneLab datasets, resulting in 10 publications (beyond the originally submitted research). Overall, the Open Science nature of GeneLab has resulted in a high degree of data re-use, resulting in 38 additional publications derived from the original 67 publication over the past four years. The enormous success and knowledge gained from GeneLab has led to a collection of sister NASA “Open Science Data Repositories (OSDR)” and research support groups. These include the NASA Ames Life Sciences Data Archive (ALSDA), the NASA Biological Institutional Scientific Collection (NBISC), and the Biospecimen Sharing Program (BSP). All are adopting the GeneLab data architecture system to maximize open-access, find-ability, accessibility, interoperability, and reusability (FAIR). ALSDA collects and curates phenotypic-physiological bioimaging-behavioral data from space and space-relevant non-human experiments, oftentimes coming from the same omics-associated experimental datasets found in GeneLab. Since 2021, a community of ~100 researchers have rallied around ALSDA, to provide feedback in a new ALSDA AWG focused on phenotypic-physiological investigation-sample-assay metadata standards (e.g., Micro-Computed Tomography, Light/Fluorescence Microscopy, Western Blot, Flow Cytometry, Novel Object Recognition, Elevated Plus Maze, etc. of ~50 assays collected). These standards are part of a new single point-of-entry data submission portal for all non-human Space Biology and Human Research Program principal investigators, to submit, curate, and share their research data. With open-access space biological data now collected and curated together with rich metadata, and with the potential for linkage to “big data” from the international biological and medical communities (NIH, EBI, etc.), the artificial intelligence and machine learning (AI/ML) era has started for Space Biology. Several other talks will cover these topics in this conference.

life sciences↗

Open Science for Life in Space: Data Sharing and Tools for Knowledge Discovery

The next era in human space exploration is rapidly approaching and will require the use of countermeasures to deep space health hazards. The development of countermeasures (or, there-purposing of existing agents) will be highly dependent on our understanding of basic biological responses to space stressors (e.g. ionizing radiation, altered gravitational fields, altered day-night cycles, confinement, isolation, hostile-closed environments, distance-duration from Earth, exposure to celestial regolith, etc.). The fast-growing array of space biological data, which in the past was simply archived after minimal analysis, holds great potential if it can be reorganized and formatted for Open Science. Organizing the data for such analysis is a challenge because of its diverse nature (molecular, cellular, tissue, imaging, whole organism and behavior). We will discuss here several strategies that NASA's Biological and Physical Science Division has put in place to maximize the return on investment for spaceflight bioscience data. Open Science, as a scientific philosophy, is the concept that the more people who have access to the data, the more knowledge will be gained from it. This guiding principle led NASA to develop GeneLab in 2015. GeneLab houses spaceflight and relevant ground-based multi-omics data, and has grown to ~400 transcriptomatic, proteomic, metabolomic and epigenomic datasets from plant, rodent, small animal, and microbial space experiments. GeneLab provides users with various tools for data analysis and a visualization portal that allows users to interact with gene expression data from space-related 'omics experiments. Open Science is also about building scientific communities, and with this spirit in mind, GeneLab has spawned several Analysis Working Groups (AWGs), comprised of more than 200 volunteer scientists. The AWGs initially provided feedback on the processing pipeline and metadata 'omics standards for GeneLab. Over the last few years, they have become a community-driven science enterprise, engaging in large meta-analysis of GeneLab datasets, resulting in 10 publications (beyond the originally submitted research). Overall, the Open Science nature of GeneLab has resulted in a high degree of data-use, resulting in 40 enabled publications by open data. The enormous success and knowledge gained from GeneLab has led to a collection of sister NASA "Open Science Data Repositories (OSDR)" and research support groups. These include the NASA Ames Life Sciences Data Archive (ALSDA), the NASA Biological Institutional Scientific Collection (NBISC), and the Biospecimen Sharing Program (BSP). All are adopting the GeneLab data architecture system to maximize open-access, find-ability, accessibility, interoperability, and reusability (FAIR). ALSDA collects and curates phenotypic-physiological bioimaging-behavioral data from space and space-relevant non-human experiments, oftentimes coming from the same omics-associated experimental datasets found in GeneLab. Since 2021, a community of ~100 researchers have rallied around ALSDA, to provide feedback in a new ALSDA AWG focused on phenotypic-physiological investigation-sample-assay metadata standards (e.g., Micro-Computed Tomography, Light/Flourescence Microscopy, Western Blot, Flow Cytometry, Novel Object Recognition, Elevated Plus Maze, etc. of ~50 assays collected). These standards are part of a new single point-of-entry data submission portal for all non-human Space Biology and Human Research Program principal investigators, to submit, curate, and share their research data. With open-access space biological data now collected and curated together with rich metadata, and with the potential for linkage to "big data" from the international biological and medical communities (NIH, EBI, etc.), the artificial intelligence and machine learning (AI/ML) era has started for Space Biology.

omics↗

Decoding structure-spectrum relationships with physically organized latent spaces

Here, a semisupervised machine learning method for the discovery of structure-spectrum relationships is developed and then demonstrated using the specific example of interpreting x-ray absorption near-edge structure (XANES) spectra. This method constructs a one-to-one mapping between individual structure descriptors and spectral trends. Specifically, an adversarial autoencoder is augmented with a rank constraint (RankAAE). The RankAAE methodology produces a continuous and interpretable latent space, where each dimension can track an individual structure descriptor. As a part of this process, the model provides a robust and quantitative measure of the structure-spectrum relationship by decoupling intertwined spectral contributions from multiple structural characteristics. This makes it ideal for spectral interpretation and the discovery of descriptors. The capability of this procedure is showcased by considering five local structure descriptors and a database of >50 000 simulated XANES spectra across eight first-row transition metal oxide families. The resulting structure-spectrum relationships not only reproduce known trends in the literature but also reveal unintuitive ones that are visually indiscernible in large datasets. The results suggest that the RankAAE methodology has great potential to assist researchers in interpreting complex scientific data, testing physical hypotheses, and revealing patterns that extend scientific insight.

36 MATERIALS SCIENCE↗

Technical Characterization and Benefit Evaluation of 5G-Enabled Grid Data Transport and Applications

This report summarizes the Year 1 work of Pacific Northwest National Laboratory’s (PNNL’s) 5G Fabricated Resource and Asset Management Encompassment for energy infrastructure (Energy FRAME) project funded by the Department of Energy Office of Science’s Advanced Scientific Computing Research Program. 5G is a breakthrough technology that enables a fully mobile and connected society, and a 5G-enabled digital continuum will be one of the critical foundations for a clean energy economy and grid modernization. In collaboration with PNNL’s Advanced Wireless Communication team and Center for Advanced Technology Evaluation team, the project team has been evaluating the system performance of 5G testbeds in the PNNL 5G Innovation Studio, and has formulated a co-simulation test case of power system transmission, distribution, and communication (T&D&C) networks considering 5G technology and high penetration of distributed energy resources. The methodology developed in the 5G Energy FRAME project can be customized to fit different future grid scenarios to evaluate multiple (dynamic) configurations (computing, sensing, communication, environment) for different stakeholders. In summary, our main technical highlights in project Year 1 are as follows: 1) Technical characterization of 5G standalone architectures, 2) Formulation of co-simulation test case of T&D&C networks embedded with 5G, 3) Initial benefit evaluation of 5G communication platform for grid use cases, and 4) Additional extended discussions on edge computing, artificial intelligence and machine learning, and high-performance computing and cloud computing adoptions. In addition, a collection of system performance data is shared through the publicly available weblink, https://www.pnnl.gov/projects/5g-energy-frame/publications

24 POWER TRANSMISSION AND DISTRIBUTION↗

Physics-guided neural networks with engineering domain knowledge for hybrid process modeling

As neural networks are more frequently used to solve problems in science and engineering, the methods used to incorporate scientific knowledge into these networks are becoming increasingly complex. Here, this work breaks down these complicated techniques into a set of basic strategies which can easily be applied to diverse situations. Several novel neural networks are built using the categories laid out in this work. These networks are tested on simulated data from a continuous stirred tank reactor (CSTR) model to evaluate the advantages provided by each network. The three points demonstrated in this work are: (1) architectural hybrid models can speed up convergence and reduce the amount of data necessary to train a model; (2) adding a physics-guided loss function can improve model generalization and make models more physically consistent; (3) using physics-guided initialization and transfer learning improves accuracy and speeds up convergence, but can harm generalizability if used incorrectly.

42 ENGINEERING↗

End-to-End Differentiable Modeling and Management of the Environment

Focal Area: (2) Data acquisition and assimilation enabled by machine learning, AI, and advanced methods including experimental/network design/optimization. We emphasize the importance of leveraging optimization techniques from AI/machine learning (ML) to solve challenging problems in Earth system modeling. Science Challenge: Automatic differentiation has had a transformative effect on ML by allowing the calculation of gradients of arbitrary functions in an incredibly large class of models. We can potentially realize similar improvements in parameter estimation and control for Earth system models (ESMs) by reimplementing them in computational frameworks from ML. Practitioners working with large (>10 7 parameters) models in ML and AI can obtain good predictive performance in a range of spatiotemporal tasks by making use of optimization via stochastic gradient descent and incorporating prior knowledge at multiple levels. We propose writing ESMs in open-source computational frameworks such as Torch, Tensorflow, and JAX to greatly expand the scope of environmental forecasting and management challenges, which can be addressed by leveraging automatic differentiation and gradient descent-like algorithms. We do not call for a wholesale replacement of physical models with data-driven surrogates, but rather advocate for interleaving physical and empirical equations in a manner that is most faithful to the extent of our scientific knowledge and observational data. Central to this topic is the merging of differentiable physical simulations with differentiable optimization layers, which are now both beginning to come to the forefront.

54 ENVIRONMENTAL SCIENCES↗