Machine Learning for Turbulence
New Deep Learning architectures that embed physics as hard constraints with numerical methods, increasing accuracy and reducing computational cost.
71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
SEARCH · Engineering Papers
Results for “physical learning”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
The Bias-Variance-Correlation Tradeoff and Its Implications for ML Applications in HEP
The bias-variance tradeoff is a well-recognized phenomenon in statistics and machine learning. In this talk, I will discuss an extension, dubbed the bias-variance-correlation tradeoff. Roughly speaking, as the flexibility of a model decreases, the correlations in the outputs of a trained model for different inputs increases. Such correlations have implications for several applications of machine learning in high energy physics, e.g., the use generative models for event generation. In particular, I will argue that claims in the literature of data amplification by generative models stem from ignoring important correlations between the model's outputs for different inputs.
Shyamsundar, Prasanth [Fermilab] (ORCID:0000000227↗
OEDI—Solar Grid Integration Data and Analytics Library
As a part of the Open Energy Data Initiative, this effort aims to develop and demonstrate novel distribution state estimation, control optimization, and transient analysis as well as provide access to data, data integration, and mapping information. More specifically, the focus of the effort will be on physics-based distribution system state estimation, hybrid (physics-based and machine learning) distribution optimal power flow, and event detection/analysis for solar integration and analytics. This work will enable reproducible, robust, replicable, and generalizable R&D in simulation and emulation of solar system integration. These test models and datasets will provide an integrated library for developing and testing power system operation technologies. To make the library user-friendly, this project will provide data curation tools such as data translators, mapping scripts and APIs, database schemas and metadata, interfaces and user dashboard, source code for the reference algorithms, description of the use-cases/scenarios, and comprehensive information on all the assumptions.
14 SOLAR ENERGY↗
Effect of barren plateaus on gradient-free optimization
Barren plateau landscapes correspond to gradients that vanish exponentially in the number of qubits. Such landscapes have been demonstrated for variational quantum algorithms and quantum neural networks with either deep circuits or global cost functions. For obvious reasons, it is expected that gradient-based optimizers will be significantly affected by barren plateaus. However, whether or not gradient-free optimizers are impacted is a topic of debate, with some arguing that gradient-free approaches are unaffected by barren plateaus. Here we show that, indeed, gradient-free optimizers do not solve the barren plateau problem. Our main result proves that cost function differences, which are the basis for making decisions in a gradient-free optimization, are exponentially suppressed in a barren plateau. Hence, without exponential precision, gradient-free optimizers will not make progress in the optimization. We numerically confirm this by training in a barren plateau with several gradient-free optimizers (Nelder-Mead, Powell, and COBYLA algorithms), and show that the numbers of shots required in the optimization grows exponentially with the number of qubits.
71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
Solar Irradiance Ramp Forecasting Based on All-Sky Imagers
Solar forecasting constitutes a critical tool for operating, producing and storing generated power from solar farms. In the framework of the International Energy Agency’s Photovoltaic Power Systems Program Task 16, the solar irradiance nowcast algorithms, based on five all-sky imagers (ASIs), are used to investigate the feasibility of ASIs to foresee ramp events. ASIs 1–2 and ASIs 3–5 can capture the true ramp events by 26.0–51.0% and 49.0–92.0% of the cases, respectively. ASIs 1–2 provided the lowest (<10.0%) falsely documented ramp events while ASIs 3–5 recorded false ramp events up to 85.0%. On the other hand, ASIs 3–5 revealed the lowest falsely documented no ramp events (8.0–51.0%). ASIs 1–2 are developed to provide spatial solar irradiance forecasts and have been delimited only to a small area for the purposes of this benchmark, which penalizes these approaches. These findings show that ASI-based nowcasts could be considered as a valuable tool for predicting solar irradiance ramp events for a variety of solar energy technologies. The combination of physical and deep learning-based methods is identified as a potential approach to further improve the ramp event forecasts.
14 SOLAR ENERGY↗
RODEM Jet Datasets
We present the RODEM Jet Datasets, a comprehensive collection of simulated large-radius jets designed to support the development and evaluation of machine-learning algorithms in particle physics. These datasets encompass a diverse range of jet sources, including quark/gluon jets, jets from the decay of W bosons, top quarks, and heavy new-physics particles. The datasets provide detailed substructure information, including jet kinematics, constituent kinematics, and track displacement details, enabling a wide range of applications in jet tagging, anomaly detection, and generative modelling.
FOS: Physical sciences↗
Efficient Subset Simulation using Hamiltonian Neural Network enhanced Markov Chain Monte Carlo Methods
The Monte Carlo method delivers an unbiased estimate of the probability of failure. However, the variance of the estimate depends on the number of evaluated samples. This number must be very large for estimations of a low probability of failure. If the evaluation of each sample is computationally expensive, the crude Monte Carlo simulation strategy is impracticable. Therefore, subset simulations are used to reduce the required number of evaluations. Subset simulations require a Markov Chain Monte Carlo sampler, such as the random walk Metropolis-Hastings algorithm. The algorithm, however, struggles with sampling in low-probability regions, especially if they are narrow. As a consequence, advanced Markov Chain Monte Carlo simulations have been developed. In particular, the Hamiltonian Monte Carlo method explores the target distribution rapidly. Driven by the idea of Hamiltonian dynamics, this sampler provides a non-random walk through the target distribution. The incorporation of subset simulation and Hamiltonian Monte Carlo methods has shown promising results for reliability analysis. One downside of the Hamiltonian Monte Carlo method is that gradient evaluations are computationally expensive, especially when dealing with high-dimensional problems and evaluating long trajectories. We show that integrating Hamiltonian neural networks in Hamiltonian Monte Carlo simulations significantly speeds up the sampling task. Furthermore, the enhancement of adaptive trajectory length within the Hamiltonian Monte Carlo results in the efficient proposal of the following states. Based on this recent enhancement, we provide a fast sampling strategy for subset simulations using Hamiltonian neural networks to replace the evaluation of the gradient and significantly speed up the Hamiltonian Monte Carlo simulation.
97 MATHEMATICS AND COMPUTING↗
Bayesian Inference with Latent Hamiltonian Neural Networks (L-HNNs)
When sampling for Bayesian inference, one popular approach is to use Hamiltonian Monte Carlo (HMC) and the No-U-Turn Sampler (NUTS). However, HMC and NUTS can require numerous numerical gradients of the target density and can prove slow in practice. We propose Hamiltonian neural networks (HNNs) with HMC and NUTS for solving Bayesian inference problems [1, 2]. Once trained, HNNs do not require gradients of the target density while sampling. Moreover, they satisfy important properties such as perfect time reversibility and Hamiltonian conservation, making them well suited for use within HMC and NUTS because stationarity can be shown. We also propose an HNN extension called latent HNNs (L-HNNs), which predict latent variable outputs. Compared to HNNs, L-HNNs offer improved expressivity and a reduction in integration errors. Finally, we propose employing L-HNNs in NUTS with an online error monitoring scheme to prevent degeneracy of the sampling in regions of low probability density. We demonstrate L-HNNs in NUTS with online error monitoring by using several example cases involving complex, heavy-tailed, and high local curvature probability densities. Overall, L-HNNs in NUTS with online error monitoring satisfactorily inferred these probability densities. Compared to traditional NUTS, L-HNNs in NUTS with online error monitoring improved the effective sample size (ESS) per gradient by an order of magnitude.
97 MATHEMATICS AND COMPUTING↗
AGN-201 Digital Twin Concept of Operations
The AGN-201 is a nuclear reactor at Idaho State University (ISU) and is currently being used in the development of a digital twin (DT) with Idaho National Laboratory (INL). The goal is to create a DT (called the AGN-201 DT) which can monitor an operating nuclear reactor to determine when the reactor is being operated normally; normal operations are any operations that is declared by the ISU staff. To accomplish this, researchers at INL developed a DT ecosystem which can ingest data from the AGN-201 reactor. This data is fed into a series of machine learning (ML) and reactor physics models. The ML and reactor physics models assess the data and determine if the reactor is operating normally. Event(s) that are flagged as anomalies are investigated by the INL staff to determine if any undeclared experiments were conducted. The AGN-201 DT was verified in July/August 2023, when the ISU staff performed undeclared experiments and the INL staff were able to assess the anomalous events and determine what likely caused each event. This project was a steppingstone to promote the use of DTs for international safeguards and marks the first time a DT was able to monitor a nuclear reactor for this purpose. This document provides the concept of operations (CONOPS) for the AGN-201 digital twin (DT). The CONOPS describes how the AGN-201 and AGN-201 DT are expected to operate and details how each are established.
21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
Scientific machine learning for closure models in multiscale problems: A review
Here, closure problems are omnipresent when simulating multiscale systems, where some quantities and processes cannot be fully prescribed despite their effects on the simulation's accuracy. Recently, scientific machine learning approaches have been proposed as a way to tackle the closure problem, combining traditional (physics-based) modeling with data-driven (machine-learned) techniques, typically through enriching differential equations with neural networks. This paper reviews the different reduced model forms, distinguished by the degree to which they include known physics, and the different objectives of a priori and a posteriori learning. The importance of adhering to physical laws (such as symmetries and conservation laws) in choosing the reduced model form and choosing the learning method is discussed. The effect of spatial and temporal discretization and recent trends toward discretization-invariant models are reviewed. In addition, we make the connections between closure problems and several other research disciplines: inverse problems, Mori-Zwanzig theory, and multi-fidelity methods. In conclusion, much progress has been made with scientific machine learning approaches for solving closure problems, but many challenges remain. In particular, the generalizability and interpretability of learned models is a major issue that needs to be addressed further.
97 MATHEMATICS AND COMPUTING↗
Quantum Annealing for Real-World Machine Learning Applications
Optimizing the training of a machine learning pipeline is important for reducing training costs and improving model performance. One such optimizing strategy is quantum annealing, which is an emerging computing paradigm that has shown potential in optimizing the training of a machine learning model. The implementation of a physical quantum annealer has been realized by D-Wave systems and is available to the research community for experiments. Recent experimental results on a variety of machine learning applications have shown interesting results especially under the conditions where the performance of classical machine learning techniques are limited such as limited training data and high dimensional features. This chapter explores the application of D-Wave’s quantum annealer for optimizing machine learning pipelines for real-world classification problems. We review the application domains on which a physical quantum annealer has been used to train machine learning classifiers. We discuss and analyze the experiments performed on the D-Wave quantum annealer for applications such as image recognition, remote sensing imagery, security, computational biology, biomedical sciences, and physics. We discuss the possible advantages and the problems for which quantum annealing is likely to be advantageous over classical computation.
Kumar nath, Rajdeep↗
Modeling Protein–Protein and Protein–Ligand Interactions by the ClusPro Team in CASP16
ABSTRACT In the CASP16 experiment, our team employed hybrid computational strategies to predict both protein–protein and protein–ligand complex structures. For protein–protein docking, we combined physics‐based sampling—using ClusPro FFT docking and molecular dynamics—with AlphaFold (AF)‐based sampling, followed by AF‐based refinement. Our method produced numerous high‐accuracy complex models, including cases where AF alone failed, underscoring the critical role of physics‐based sampling alongside deep learning‐based refinement. For protein–ligand docking, we integrated the ClusPro LigTBM template‐based approach with a machine learning‐based confidence model for rescoring. The method preserves conserved interaction fragments derived from homologous complexes, followed by local resampling using physics‐based sampling and a diffusion model. Our template‐based strategy achieved a mean lDDT‐PLI of 0.69 across 233 targets, which was highly competitive. These results demonstrate that combining physics‐based modeling with AI‐driven refinement can significantly enhance the accuracy of both protein–protein and protein–ligand structure predictions.
Ashizawa, Ryota [Department of Applied Mathematics↗
Federated Learning for Efficient Condition Monitoring and Anomaly Detection in Industrial Cyber-Physical Systems
Detecting and localizing anomalies in cyber-physical systems (CPS) has become increasingly challenging as systems grow in complexity, particularly due to varying sensor reliability and node failures in distributed environments. While federated learning (FL) offers a foundation for distributed model training, existing approaches lack mechanisms to handle these CPS-specific challenges. This paper presents an enhanced FL framework that introduces three key innovations: adaptive model aggregation based on sensor reliability, dynamic node selection for resource optimization, and Weibull-based checkpointing for fault tolerance. Our framework enables reliable condition monitoring while addressing the computational and reliability challenges of industrial CPS deployments. Experiments on NASA Bearing and Hydraulic System Datasets demonstrate superior performance over state-of-the-art FL methods, achieving 99.5% AUC-ROC in anomaly detection and maintaining accuracy under node failures. Statistical validation using Mann-Whitney (U) test confirms significant improvements (p < 0.05) in both detection accuracy and computational efficiency across diverse operational scenarios.1
Marfo, William [University of Texas at El Paso,Dep↗
Hierarchical Reinforcement Learning of a Short-Range Bond-Order Potential for Silica: Analytic Embedding of Coordination with Classical Efficiency
Reinforcement learning (RL) has recently emerged as a data-efficient strategy to parametrize short-range interatomic potentials. Building on our past RL optimization of pairwise silica models, we extend the framework to a bond-order (Tersoff-type) potential that provides an analytic embedding of local coordination through a three-body term. A hierarchical RL workflow combining continuous-action Monte Carlo Tree Search and property-based rewards efficiently explores the 26-dimensional parameter space, sequentially optimizing lattice parameters, densities, angles, and cohesive energies of 21 silica polymorphs. The resulting models, Q-Tersoff and ML-Tersoff, reproduce the energetic ordering of low-energy phases and capture the angular correlations and amorphous structure factors of silica with improved fidelity over pairwise force fields, while remaining orders of magnitude faster than high-dimensional machine-learned potentials. Both models underperform for elastic constants and high-energy frameworks, delineating the limits of the current analytic form. The approach establishes a general and interpretable route to angle-aware, short-range potentials that bridge physics-based and machine-learned descriptions of silicate materials.
36 MATERIALS SCIENCE↗
An analysis of physics limited dispatch of nuclear renewable integrated energy systems using deep reinforcement learning and dynamic modeling
Previous approaches to dispatching nuclear integrated energy systems (NIES) have focused on the profitability and flexibility of these systems to operate on energy grids with highly variable pricing. However, due to the complexity involved in modeling and designing these systems, there has been less emphasis on ensuring that these dispatch strategies are physically achievable. It is imperative to develop methods that allow the system to remain within the desired NIES operating conditions and perform this based on realistic limited forecasted information. This research employs next generation artificial intelligence, namely deep reinforcement learning (DRL), and a dynamic system model written in Modelica to find a safe and profitable dispatch strategy for a solar nuclear hybrid design. The DRL agent is shown to find a novel dispatch strategy that manages both power ramping and power levels while respecting operational limits. This DRL-based dispatch is compared to other dispatching strategies including an optimal design solution from mixed integer linear programming (MILP). It is found that incorporating the physics of such a tightly coupled NIES limits the profitability of the MILP-based dispatch strategy. As a result, the MILP solution overestimates the design’s generated revenue. In contrast, DRL significantly reduces the number of breaches of safe operational conditions during energy arbitrage while maintaining profitability. Furthermore, this work paves the way for a more detailed assessment of NIES profitability and could be used to aid operator decisions on future NIES projects.
14 - SOLAR ENERGY↗
Construction of generalized quasilinear diffusion coefficient using neural networks with physical restrictions
The quasilinear diffusion coefficient (D QL ) derived from our machine learning framework shows comparable trends with the ground truth D QL obtained from GENRAY-CQL3D simulations. Additionally, for the strong absorption cases, the radial current drive profiles generated using the D QL from our model exhibit consistent behavior with those obtained from the original simulation. These findings indicate the potential of our surrogate modeling approach with physical restrictions to replicate key wave–plasma interaction characteristics while reducing computational costs. Traditionally, calculating D QL for wave–particle interactions relies on computationally intensive wave simulations coupled with Fokker–Planck solvers. To address this challenge, we developed a machine learning-based surrogate model with physical restrictions derived from cold plasma theory and bounce-averaged damping effects. First, we establish the propagation domain of Lower Hybrid Waves in the (N∥, ρ) space by identifying the accessibility limit and determining the upper and lower bounds of N∥ using the Potential Power Deposition (PPD) method. Subsequently, leveraging a database constructed using Latin hypercube sampling alongside the underlying physical restrictions (e.g. PPD), machine learning methods including U-Net and Recurrent Neural Networks are employed to design a physics-restricted machine learning framework capable of reconstructing D QL .
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
Sub-millisecond keyhole pore detection in laser powder bed fusion using sound and light sensors and machine learning
Laser powder bed fusion is a mainstream additive manufacturing technology widely used to manufacture complex parts in prominent sectors, including aerospace, biomedical, and automotive industries. However, during the printing process, the presence of an unstable vapor depression can lead to a type of defect called keyhole porosity, which is detrimental to the part quality. In this study, we developed an effective approach to locally detect the generation of keyhole pores during the printing process by leveraging machine learning and a suite of optical and acoustic sensors. Simultaneous synchrotron x-ray imaging allows the direct visualization of pore generation events inside the sample, offering high-fidelity ground truth. A neural network model adopting SqueezeNet architecture using single-sensor data was developed to evaluate the fidelity of each sensor for capturing keyhole pore generation events. Our comparative study shows that the near infrared images gave the highest prediction accuracy, followed by 100 kHz and 20 kHz microphones, and the photodiode sensitive to processing laser wavelength had the lowest accuracy. Using a single sensor, over 90% prediction accuracy can be achieved with a temporal resolution as short as 0.1 ms. A data fusion scheme was also developed with features extracted using SqueezeNet neural network architecture and classification using different machine learning algorithms. Our work demonstrates the correlation between the characteristic optical and acoustic emissions and the keyhole oscillation behavior, and thereby provides strong physics support for the machine learning approach.
36 MATERIALS SCIENCE↗