Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “physical learning, networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22

Designing observables for measurements with deep learning

Many analyses in particle and nuclear physics use simulations to infer fundamental, effective, or phenomenological parameters of the underlying physics models. When the inference is performed with unfolded cross sections, the observables are designed using physics intuition and heuristics. We propose to design targeted observables with machine learning. Unfolded, differential cross sections in a neural network output contain the most information about parameters of interest and can be well-measured by construction. The networks are trained using a custom loss function that rewards outputs that are sensitive to the parameter(s) of interest while simultaneously penalizing outputs that are different between particle-level and detector-level (to minimize detector distortions). We demonstrate this idea in simulation using two physics models for inclusive measurements in deep inelastic scattering. We find that the new approach is more sensitive than classical observables at distinguishing the two models and also has a reduced unfolding uncertainty due to the reduced detector distortions.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

A learning-augmented approach for AC optimal power flow

Because of the high nonlinearity of AC optimal power flow (OPF), numerous efforts have been made in recent decades to find efficient methods. Machine learning (ML) has proven to significantly reduce the computational costs in many real-world problems. Thus, this paper develops a learning-augmented method for solving AC OPF, which integrates both power network equations and ML to yield near-optimal solutions. More specifically, ML models are developed to first predict bus voltage magnitudes and angles. Then, physics-based network equations are employed to calculate the power injection at different buses. Three ML algorithms, i.e., random forest, multi-target decision tree, and extreme learning machine, are explored and compared. To evaluate the efficiency of the proposed learning-augmented AC OPF solver, the MATPOWER Interior Point Solver is adopted as a baseline. Case studies on both 500-bus and 4918-bus test networks show that the proposed learning-augmented method has reduced the computational time by 15–100 times depending on the network size with a minimal loss in optimality.

42 ENGINEERING↗

Communication Network Layer State Estimation Measurement Model for a Cyber-Secure Smart Grid

Network communication has been proven to be a very important tool and a key factor in the recent development and progress of the power grid operation. It is also considered as the foundation for the smart grid because information and communication are integrated into electricity distribution to achieve reliable and accurate knowledge of the power grid. In previous years, absorbing energy from substations and delivering it to customers was the only type of interaction we knew between utility companies and customers. Presently, the growing connections of small distributed generation units caused by the cost reduction of most of the technologies used in generation and storage of electrical energy, along with the potential benefits of renewable energy have pushed many researchers to look into the improvement of information and communication technologies (ICT) in order to ensure a bidirectional flow of power and data. Moreover, the evolution of information and communication technologies and its applications to smart grid have converted the smart grid into a cyber-physical system where vulnerabilities and additional security challenges such as cyber-threats and cyber-attacks have emerged. Previously, we have demonstrated that using machine learning-based processing on data gathered from communication networks and the power grid was a promising solution for detecting cyber threats by implementing a co-simulation of cyber-security for cross-layer strategy. Since the majority of the challenges observed can only be solved in the network communication layer, we present in this work a physics-based state estimation model of the communication network system towards enhanced cyber-physical security of the smart grid. Information integration with the previously developed machine learning model is developed, providing a enhanced cyber-physical security application for the smart grid. Easy-to-implement model, without hard-to-derive parameters, highlight potential aspects of the model for real-life applications.

Mathieu, Reynold↗

Real-Time and Adaptive Reservoir Computing With Application to Profile Prediction in Fusion Plasma

Nuclear fusion is a promising alternative to address the problem of sustainable energy production. The tokamak is an approach to fusion based on magnetic plasma confinement, constituting a complex physical system with many control challenges. Here we study the characteristics and optimization of reservoir computing (RC) for real-time and adaptive prediction of plasma profiles in the DIII-D tokamak. Our experiments demonstrate that RC achieves comparable results to state-of-the-art (deep) convolutional neural networks (CNNs) and long short-term memory (LSTM) models, with a significantly easier and faster training procedure. This efficient approach allows for fast and frequent adaptation of the model to new situations, such as changing plasma conditions or different fusion devices.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Reliable extrapolation of deep neural operators informed by physics or sparse observations

Deep neural operators can learn nonlinear mappings between infinite-dimensional function spaces via deep neural networks. As promising surrogate solvers of partial differential equations (PDEs) for real-time prediction, deep neural operators such as deep operator networks (DeepONets) provide a new simulation paradigm in science and engineering. Pure data-driven neural operators and deep learning models, in general, are usually limited to interpolation scenarios, where new predictions utilize inputs within the support of the training set. However, in the inference stage of real-world applications, the input may lie outside the support, i.e., extrapolation is required, which may result to large errors and unavoidable failure of deep learning models. Here, we address this challenge of extrapolation for deep neural operators. First, we systematically investigate the extrapolation behavior of DeepONets by quantifying the extrapolation complexity, via the 2-Wasserstein distance between two function spaces and propose a new strategy of bias–variance trade-off for extrapolation with respect to model capacity. Subsequently, we develop a complete workflow, including extrapolation determination, and we propose five reliable learning methods that guarantee a safe prediction under extrapolation by requiring additional information—the governing PDEs of the system or sparse new observations. The proposed methods are based on either fine-tuning a pre-trained DeepONet or multifidelity learning. We demonstrate the effectiveness of the proposed framework for various types of parametric PDEs. Furthermore, our systematic comparisons provide practical guidelines for selecting a proper extrapolation method depending on the available information, desired accuracy, and required inference speed.

42 ENGINEERING↗

Improving materials property predictions for graph neural networks with minimal feature engineering *

Graph neural networks (GNNs) have been employed in materials research to predict physical and functional properties, and have achieved superior performance in several application domains over prior machine learning approaches. Recent studies incorporate features of increasing complexity such as Gaussian radial functions, plane wave functions, and angular terms to augment the neural network models, with the expectation that these features are critical for achieving a high performance. Here, we propose a GNN that adopts edge convolution where hidden edge features evolve during training and extensive attention mechanisms, and operates on simple graphs with atoms as nodes and distances between them as edges. As a result, the same model can be used for very different tasks as no other domain-specific features are used. With a model that uses no feature engineering, we achieve performance comparable with state-of-the-art models with elaborate features for formation energy and band gap prediction with standard benchmarks; we achieve even better performance when the dataset size increases. Although some domain-specific datasets still require hand-crafted features to achieve state-of-the-art results, our selected architecture choices greatly reduce the need for elaborate feature engineering and still maintain predictive power in comparison.

42 ENGINEERING↗

Cross-Layered Cyber-Physical Power System State Estimation towards a Secure Grid Operation

In the Smart Grid paradigm, this critical infrastructure operation is increasingly exposed to cyber-threats due to the increased dependency on communication networks. An adversary can launch an attack on a power grid operation through False Data Injection into system measurements and/or through attacks on the communication network, such as flooding the communication channels with unnecessary data or intercepting messages. A cross-layered strategy that combines power grid data, communication grid monitoring and Machine Learning based processing is a promising solution for detecting cyberthreats. In this paper, an implementation of an integrated solution of a cross-layer framework is presented. The advantage of such a framework is the augmentation of valuable data that enhances the detection of anomalies in the operation of power grid. IEEE 118-bus system is built in Simulink to provide a power grid testing environment and communication network data is emulated using SimComponents. The performance of the framework is investigated under various FDI and communication attacks.

cyber security, network security, cyber-physical s↗

Learning viscoelasticity models from indirect data using deep neural networks

In this study, we propose a novel approach to model viscoelasticity materials, where rate-dependent and non-linear constitutive relationships are approximated with deep neural networks. We assume that inputs and outputs of the neural networks are not directly observable, and therefore common training techniques with input–output pairs for the neural networks are inapplicable. To that end, we develop a novel computational approach to both calibrate parametric and learn neural-network-based constitutive relations of viscoelasticity materials from indirect displacement data in the context of multiple-physics systems. We show that limited displacement data holds sufficient information to quantify the viscoelasticity behavior. We formulate the inverse computation – modeling viscoelasticity properties from observed displacement data – as a PDE-constrained optimization problem and minimize the error functional using a gradient-based optimization method. The gradients are computed by a combination of automatic differentiation and implicit function differentiation rules. The effectiveness of our method is demonstrated through numerous benchmark problems in geomechanics and porous media transport.

97 MATHEMATICS AND COMPUTING↗

Machine learning-enhanced hybrid modeling approach for better identification of a building thermal network model and improved prediction

The gray-box modeling approach, which uses a semi-physical thermal network model, has been widely used in building prediction applications, such as model predictive control (MPC). However, unmeasured disturbances, such as occupants, lighting, and in/exfiltration loads, make it challenging to apply this approach to practical buildings. In this word, we propose a hybrid modeling approach that integrates the gray-box model with a model for unmeasured disturbance. After reviewing several system identification approaches, we systematically designed the unmeasured disturbance model with a model selection process based on statistical tests to make it robust. We generated data based on the building model calibrated by real operational data and then trained the hybrid model for two different weather conditions. The hybrid model approach demonstrates an RMSE reduction of approximately 0.2–0.9 °C and 0.3–2 °C on 1-day ahead temperature prediction compared to the Conventional approach for mild (Berkeley, CA) and cold (Chicago, IL) climates, respectively. In addition, this approach was applied to experimental data obtained from the laboratory building to be used for the MPC application, showing superior prediction performances.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

nPINNs: Nonlocal physics-informed neural networks for a parametrized nonlocal universal Laplacian operator. Algorithms and applications

Physics-informed neural networks (PINNs) are effective in solving inverse problems based on differential and integro-differential equations with sparse, noisy, unstructured, and multifidelity data. PINNs incorporate all available information, including governing equations (reflecting physical laws), initial-boundary conditions, and observations of quantities of interest, into a loss function to be minimized, thus recasting the original problem into an optimization problem. In this paper, we extend PINNs to parameter and function inference for integral equations such as nonlocal Poisson and nonlocal turbulence models, and we refer to them as nonlocal PINNs (nPINNs). The contribution of the paper is three-fold. First, we propose a unified nonlocal Laplace operator, which converges to the classical Laplacian as one of the operator parameters, the nonlocal interaction radius $\delta$ goes to zero, and to the fractional Laplacian as $\delta$ goes to infinity. This universal operator forms a super-set of classical Laplacian and fractional Laplacian operators and, thus, has the potential to fit a broad spectrum of data sets. We also provide theoretical convergence rates with respect to $\delta$ and verify them via numerical experiments. Second, we use nPINNs to estimate the two parameters, $\delta$ and $\alpha$, characterizing the kernel of the unified operator. The strong non-convexity of the loss function yielding multiple (good) local minima reveals the occurrence of the operator mimicking phenomenon, that is, different pairs of estimated parameters could produce multiple solutions of comparable accuracy. Third, we propose another nonlocal operator with spatially variable order $\alpha(y)$, which is more suitable for modeling turbulent Couette flow. Our results show that nPINNs can jointly infer this function as well as $\delta$. More importantly, these parameters exhibit a universal behavior with respect to the Reynolds number, a finding that contributes to our understanding of nonlocal interactions in wall-bounded turbulence.

97 MATHEMATICS AND COMPUTING↗

Parameter inference from event ensembles and the top-quark mass

One of the key tasks of any particle collider is measurement. In practice, this is often done by fitting data to a simulation, which depends on many parameters. Sometimes, when the effects of varying different parameters are highly correlated, a large ensemble of data may be needed to resolve parameter-space degeneracies. An important example is measuring the top-quark mass, where other physical and unphysical parameters in the simulation must be profiled when fitting the top-quark mass parameter. We compare four different methodologies for top-quark mass measurement: a classical histogram fit similar to one commonly used in experiment augmented by soft-drop jet grooming; a 2D profile likelihood fit with a nuisance parameter; a machine-learning method called DCTR; and a linear regression approach, either using a least-squares fit or with a dense linearly-activated neural network. Despite the fact that individual events are totally uncorrelated, we find that the linear regression methods work most effectively when we input an ensemble of events sorted by mass, rather than training them on individual events. Although all methods provide robust extraction of the top-quark mass parameter, the linear network does marginally best and is remarkably simple. For the top study, we conclude that the Monte-Carlo-based uncertainty on current extractions of the top-quark mass from LHC data can be reduced significantly (by perhaps a factor of 2) using networks trained on sorted event ensembles. More generally, machine learning from ensembles for parameter estimation has broad potential for collider physics measurements.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

nPINNs: nonlocal Physics-Informed Neural Networks for a parametrized nonlocal universal Laplacian operator. Algorithms and Applications

Physics-informed neural networks (PINNs) are effective in solving inverse problems based on differential and integro-differential equations with sparse, noisy, unstructured, and multifidelity data. PINNs incorporate all available information, including governing equations (reflecting physical laws), initial-boundary conditions, and observations of quantities of interest, into a loss function to be minimized, thus recasting the original problem into an optimization problem. In this paper, we extend PINNs to parameter and function inference for integral equations such as nonlocal Poisson and nonlocal turbulence models, and we refer to them as nonlocal PINNs (nPINNs). The contribution of the paper is three-fold. First, we propose a unified nonlocal Laplace operator, which converges to the classical Laplacian as one of the operator parameters, the nonlocal interaction radius δ goes to zero, and to the fractional Laplacian as δ goes to infinity. This universal operator forms a super-set of classical Laplacian and fractional Laplacian operators and, thus, has the potential to fit a broad spectrum of data sets. We provide theoretical convergence rates with respect to δ and verify them via numerical experiments. Second, we use nPINNs to estimate the two parameters, δ and α, characterizing the kernel of the unified operator. The strong non-convexity of the loss function yielding multiple (good) local minima reveals the occurrence of the operator mimicking phenomenon, that is, different pairs of estimated parameters could produce multiple solutions of comparable accuracy. Third, we propose another nonlocal operator with spatially variable order α(γ), which is more suitable for modeling turbulent Couette flow. Our results show that nPINNs can jointly infer this function as well as δ. More importantly, these parameters exhibit a universal behavior with respect to the Reynolds number, a finding that contributes to our understanding of nonlocal interactions in wall-bounded turbulence.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Data assimilation empowered neural network parametrizations for subgrid processes in geophysical flows

In the past couple of years, there has been a proliferation in the use of machine learning approaches to represent subgrid-scale processes in geophysical flows with an aim to improve the forecasting capability and to accelerate numerical simulations of these flows. Despite its success for different types of flow, the online deployment of a data-driven closure model can cause instabilities and biases in modeling the overall effect of subgrid-scale processes, which in turn leads to inaccurate prediction. To tackle this issue, we exploit the data assimilation technique to correct the physics-based model coupled with the neural network as a surrogate for unresolved flow dynamics in multiscale systems. In particular, we use a set of neural network architectures to learn the correlation between resolved flow variables and the parametrizations of unresolved flow dynamics and formulate a data assimilation approach to correct the hybrid model during their online deployment. We illustrate our framework in a set of applications of the multiscale Lorenz 96 system for which the parametrization model for unresolved scales is exactly known, and the two-dimensional Kraichnan turbulence system for which the parametrization model for unresolved scales is not known a priori. Our analysis, therefore, comprises a predictive dynamical core empowered by (i) a data-driven closure model for subgrid-scale processes, (ii) a data assimilation approach for forecast error correction, and (iii) both data-driven closure and data assimilation procedures. We show significant improvement in the long-term prediction of the underlying chaotic dynamics with our framework compared to using only neural network parametrizations for future prediction. Moreover, we demonstrate that these data-driven parametrization models can handle the non-Gaussian statistics of subgrid-scale processes, and effectively improve the accuracy of outer data assimilation workflow loops in a modular nonintrusive way.

42 ENGINEERING↗

Deep-Learning Electron Diffractive Imaging

Here, we report the development of deep-learning coherent electron diffractive imaging at subangstrom resolution using convolutional neural networks (CNNs) trained with only simulated data. We experimentally demonstrate this method by applying the trained CNNs to recover the phase images from electron diffraction patterns of twisted hexagonal boron nitride, monolayer graphene, and a gold nanoparticle with comparable quality to those reconstructed by a conventional ptychographic algorithm. Fourier ring correlation between the CNN and ptychographic images indicates the achievement of a resolution in the range of 0.70 and 0.55 Å. We further develop CNNs to recover the probe function from the experimental data. The ability to replace iterative algorithms with CNNs and perform real-time atomic imaging from coherent diffraction patterns is expected to find applications in the physical and biological sciences.

47 OTHER INSTRUMENTATION↗

Novel usage of deep learning and high-performance computing in long-baseline neutrino oscillation experiments

Mención Internacional en el título de doctorDeep-learning methods are playing a crucial role in numerous scientific and industrialapplications. Over the past two decades, these techniques have helped in the collection,reconstruction, and analysis of large data samples in particle physics experiments. Themain topic of this PhD research is the study of deep-learning techniques in long-baselineneutrino oscillation experiments. Neutrinos are mysterious light elementary particles,and their investigation is essential to shed light on some of the remaining open questionsin physics. The work presented here describes an algorithm based on a convolutionalneural network developed to provide highly accurate and efficient selections of electronneutrino and muon neutrino interactions in the Deep Underground Neutrino Experiment(DUNE). With this algorithm, the electron neutrino (antineutrino) selection efficiencypeaks at 90% (94%) and exceeds 85% (90%) for reconstructed neutrino energies between2-5 GeV. The selection efficiency for muon neutrino (antineutrino) interactions is foundto have a maximum of 96% (97%) and exceeds 90% (95%) efficiency for reconstructedneutrino energies above 2 GeV. When considering all electron neutrino and antineutrinointeractions as signal (both those appearing from oscillations and those intrinsic tothe beam), a selection purity of 90% is achieved. These event selections are criticalto maximise the sensitivity of the experiment to CP-violating effects, key to furtherunderstand the matter-antimatter asymmetry of the Universe.In high-energy physics experiments, deep learning has also been explored for producingfast simulations and physically-motivated manipulations of simulated images. Some ofthose simulations, such as the light production and detection, are very computationallyexpensive and require novel methods to produce the necessary samples while controllingthe varied underlying physics model parameters. To do so, we invented the model-assistedgenerative adversarial network (MAGAN), first validated on simple generic case studiesand then successfully applied to the DUNE photon-detector simulation.Moreover, we also developed graph neural networks for 3D-voxel classification ofambiguities and optical crosstalk for a different particle physics experiment, most preciselyfor the proposed SuperFGD. This novel 3D-granular plastic-scintillator neutrino detectorwill be used to upgrade the near detector of the T2K neutrino oscillation experiment, and our method reports efficiencies and purities of 94-96% per event in the classificationof particle track voxels.Due to the growth and complexity of deep neural networks, researchers have beeninvestigating techniques to train those networks in a more computationally-efficient way.Many efforts have been made by the community to optimise deep-learning models byparallelising or distributing their training computation across multiple devices. In thisthesis, we study an approach based on data locality for those neural networks that cannotbenefit from scaling their computation due to a significant bottleneck in the data I/O.The research also includes a detailed study on the performance of deep neural networkson hardware accelerator boards.Los métodos de aprendizaje profundo son cada vez más utilizados en numerosas aplicacionescientíficas e industriales hoy en día. Durante las dos últimas décadas, estastécnicas se han empleado en la recolección, reconstrucción y análisis de la gran cantidadde datos generados por experimentos de física de partículas. El tema principal de estatesis doctoral es el uso de estos modelos de aprendizaje profundo en experimentos defísica de neutrinos, en concreto en los experimentos de larga distancia DUNE y T2K. Losneutrinos, partículas fundamentales neutras, de las más ligeras del Universo, pueden serclave para explicar algunas de las cuestiones todavía sin resolver en física fundamental.Entre las diferentes contribuciones que esta tesis ha hecho a su estudio, cabe destacar eldesarrollo de un algoritmo basado en una red de neuronas convolucional para seleccionarcon gran eficiencia y precisión las interacciones de neutrinos electrónicos y muónicos enel Deep Underground Neutrino Experiment (DUNE). La eficiencia de selección obtenidapara neutrinos (antineutrinos) electrónicos alcanza un máximo del 90% (94%) y supera el85% (90%) para neutrinos con energías reconstruidas en el rango 2-5 GeV. La selección deneutrinos (antineutrinos) muónicos tiene una eficiencia máxima del 96% (97%) y excedeel 90% (95%) para neutrinos con energías reconstruidas de más de 2 GeV. Considerandocomo señal todas las interacciones de neutrinos y antineutrinos electrónicos (procedentestanto de oscilaciones como intrínsecos en el haz inicial), se logra una pureza en la seleccióndel 90%. Dichas selecciones de eventos son fundamentales para maximizar la sensibilidaddel experimento a los efectos de violació...

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Enhancing Docking Accuracy with PECAN2, a 3D Atomic Neural Network Trained without Co-Complex Crystal Structures

Decades of drug development research have explored a vast chemical space for highly active compounds. The exponential growth of virtual libraries enables easy access to billions of synthesizable molecules. Computational modeling, particularly molecular docking, utilizes physics-based calculations to prioritize molecules for synthesis and testing. Nevertheless, the molecular docking process often yields docking poses with favorable scores that prove to be inaccurate with experimental testing. To address these issues, several approaches using machine learning (ML) have been proposed to filter incorrect poses based on the crystal structures. However, most of the methods are limited by the availability of structure data. Here, we propose a new pose classification approach, PECAN2 (Pose Classification with 3D Atomic Network 2), without the need for crystal structures, based on a 3D atomic neural network with Point Cloud Network (PCN). The new approach uses the correlation between docking scores and experimental data to assign labels, instead of relying on the crystal structures. We validate the proposed classifier on multiple datasets including human mu, delta, and kappa opioid receptors and SARS-CoV-2 Mpro. Our results demonstrate that leveraging the correlation between docking scores and experimental data alone enhances molecular docking performance by filtering out false positives and false negatives.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

KamNet: An integrated spatiotemporal deep neural network for rare event searches in KamLAND-Zen

Rare event searches allow us to search for new physics at energy scales inaccessible with other means by leveraging specialized large-mass detectors. Machine learning provides a new tool to maximize the information provided by these detectors. The information is sparse, which forces these algorithms to start from the lowest level data and exploit all symmetries in the detector to produce results. In this work we present KamNet, which harnesses breakthroughs in geometric deep learning and spatiotemporal data analysis to maximize the physics reach of KamLAND-Zen, a kiloton scale spherical liquid scintillator detector searching for 0⁢νβ⁢β. Using a simplified background model for KamLAND, we show that KamNet outperforms a conventional convolutional neural network (CNN) on benchmarking Monte Carlo simulations with an increasing level of robustness. Using simulated data, we then demonstrate KamNet's ability to increase KamLAND-Zen's sensitivity to 0ν⁢ββ and 2⁢ν⁢β⁢β decay to excited states. A key component of this work is the addition of an attention mechanism to elucidate the underlying physics KamNet is using for the background rejection.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Physically Informed Machine Learning Prediction of Electronic Density of States

The electronic structure of a material, such as its density of states (DOS), provides key insights into its physical and functional properties and serves as a valuable source of high-quality features for many materials screening and discovery workflows. Still, the computational cost of calculating the DOS, most commonly with density functional theory (DFT), becomes prohibitive for meeting high-fidelity or high-throughput requirements, necessitating a cheaper but sufficiently accurate surrogate. To fulfill this demand, we develop a general machine learning method based on graph neural networks for predicting the DOS purely from atomic positions, six orders of magnitude faster than DFT. This approach can effectively use large materials databases and be applied generally across the entire periodic table to materials classes of arbitrary compositional and structural diversity. We furthermore devise a highly adaptable scheme for physically informed learning which encourages the DOS prediction to favor physically reasonable solutions defined by any set of desired constraints. This functionality provides a means for ensuring that the predicted DOS is reliable enough to be used as an input to downstream materials screening workflows to predict more complex functional properties, which rely on accurate physical features.

36 MATERIALS SCIENCE↗