Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “physical learning, networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 631 records · Page 35
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Denoising Convolutional Networks to Accelerate Detector Simulation [Poster]

The high accuracy of detector simulation is crucial for modern particle physics experiments. However, this accuracy comes with a high computational cost, which will be exacerbated by the large datasets and complex detector upgrades associated with next-generation facilities such as the High Luminosity LHC. We explore the viability of regression-based machine learning (ML) approaches using convolutional neural networks (CNN) to ``denoise'' faster, lower-quality detector simulations, augmenting them to produce a higher-quality final result with a reduced computational burden. The denoising CNN works in concert with classical detector simulation software rather than replacing it entirely, increasing its reliability compared to other ML approaches to simulation. We obtain promising results from a prototype based on photon showers in the CMS electromagnetic calorimeter. Future directions are also discussed.

43 PARTICLE ACCELERATORS↗
DOE OSTI2021

Development of a Filtered CFD-DEM Drag Model with Multiscale Markers Using an Artificial Neural Network and Nonlinear Regression

Here, the accuracy of coarse-grained Euler-Lagrangian simulations of fluidized beds heavily depends on the mesoscale drag models to account for the influences of the unresolved sub-grid structures. Traditional filtered drag models are regressed with mesoscale markers such as voidage and slip velocities. In this research, a filtered drag was regressed with both mesoscale and macro-scale markers using fine grid Computational Fluid Dynamics - Discrete Element Method (CFD-DEM) simulations. The traditional non-linear regression method was compared with machine learning regression using an Artificial Neural Network (ANN) implemented in PyTorch and coupled with MFiX. The new drag showed higher accuracy than the Wen-Yu drag and another filtered drag derived from the two-fluid model. The nonlinear regression shows slightly better results than ANN regression in cases with similar R 2 values. The utilization of the gas inlet velocity as an additional macro-scale marker reduced the errors by up to 55.3% in the tested cases.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

A Physics-Constrained Deep Learning Model for Simulating Multiphase Flow in 3D Heterogeneous Porous Media

Physics-based simulators for multiphase flow in porous media emulate nonlinear processes with coupled physics, and usually require extensive computational resources for software development, maintenance and simulation execution. As a result, a huge demand exists for fast modeling of coupled processes in a wide range of subsurface applications including geological sequestration, hydrocarbon recovery and geothermal energy extraction. In this work, an efficient physics-constrained deep learning model is developed for solving multiphase flow in 3-Dimensional (3D) heterogeneous porous media. The model fully leverages the spatial topology predictive capability of convolutional neural networks, specifically U-Net with successive contracting and expansive steps, and is coupled with an efficient continuity-based smoother to predict flow responses that need spatial continuity. Furthermore, the transient regions are penalized to steer the training process such that the model can accurately capture flow in these regions. The model takes inputs including properties of porous media, fluid properties and well controls, and predicts the temporal-spatial evolution of the state variables (pressure and saturation). While maintaining the continuity of fluid flow, the 3D spatial domain is decomposed into 2D images for reducing training cost, and the decomposition results in an increased number of training data samples and better training efficiency. Additionally, a surrogate model is separately constructed as a postprocessor to calculate well flow rate based on the predictions of state variables from the deep learning model. We use the example of CO 2 injection into saline aquifers, and apply the physics-constrained deep learning model that is trained from physics-based simulation data and emulates the physics process. The model performs prediction with a speedup of ~ 1400 times compared to physics-based simulations, and the average temporal errors of predicted pressure and saturation plumes are 0.27% and 0.099% respectively. Furthermore, water production rate is efficiently predicted by a surrogate model for well flow rate, with a mean error less than 5%. Therefore, with its unique scheme to cope with the fidelity in fluid flow in porous media, the physics-constrained deep learning model can become an efficient predictive model for computationally demanding inverse problems or other coupled processes.

58 GEOSCIENCES↗
DOE OSTI2026

Variable rate neural compression for sparse detector data

Particle colliders produce data at extraordinary rates, posing major challenges for transmission and storage. High-throughput compression algorithms are therefore essential. In the sPHENIX experiment taking data at the Relativistic Heavy Ion Collider, a time projection chamber records three-dimensional (3D) particle trajectories that are highly sparse, making conventional learning-free lossy compression ineffective. Convolutional neural networks have surpassed traditional methods in compression ratio and accuracy. However, they fail to exploit sparsity for efficiency. To address these gaps, we present BCAE-VS, a bicephalous convolutional autoencoder with variable compression ratio for sparse data, which adapts compression to input complexity through key-point identification and sparse convolution. BCAE-VS achieves higher accuracy and compression ratios than prior neural approaches while being orders of magnitude smaller. Moreover, its throughput increases with sparsity—a property not observed in other methods. Although it was developed for collider experiments, BCAE-VS readily extends to other sparse data domains, such as light detection and ranging (LiDAR) sensing and 3D microscopy.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Machine learning for continuous quantum error correction on superconducting qubits

Abstract Continuous quantum error correction has been found to have certain advantages over discrete quantum error correction, such as a reduction in hardware resources and the elimination of error mechanisms introduced by having entangling gates and ancilla qubits. We propose a machine learning algorithm for continuous quantum error correction that is based on the use of a recurrent neural network to identify bit-flip errors from continuous noisy syndrome measurements. The algorithm is designed to operate on measurement signals deviating from the ideal behavior in which the mean value corresponds to a code syndrome value and the measurement has white noise. We analyze continuous measurements taken from a superconducting architecture using three transmon qubits to identify three significant practical examples of non-ideal behavior, namely auto-correlation at temporal short lags, transient syndrome dynamics after each bit-flip, and drift in the steady-state syndrome values over the course of many experiments. Based on these real-world imperfections, we generate synthetic measurement signals from which to train the recurrent neural network, and then test its proficiency when implementing active error correction, comparing this with a traditional double threshold scheme and a discrete Bayesian classifier. The results show that our machine learning protocol is able to outperform the double threshold protocol across all tests, achieving a final state fidelity comparable to the discrete Bayesian classifier.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2021

Identification of Non-Fermi Liquid Physics in a Quantum Critical Metal via Quantum Loop Topography

Non-Fermi liquid physics is ubiquitous in strongly correlated metals, manifesting itself in anomalous transport properties, such as a $\textit{T}$-linear resistivity in experiments. However, its theoretical understanding in terms of microscopic models is lacking, despite decades of conceptual work and attempted numerical simulations. In this work, we demonstrate that a combination of sign-problem-free quantum Monte Carlo sampling and quantum loop topography, a physics-inspired machine-learning approach, can map out the emergence of non-Fermi liquid physics in the vicinity of a quantum critical point (QCP) with little prior knowledge. Using only three parameter points for training the underlying neural network, we are able to robustly identify a stable non-Fermi liquid regime tracing the fans of metallic QCPs at the onset of both spin-density wave and nematic order. In particular, we establish for the first time that a spin-density wave QCP commands a wide fan of non-Fermi liquid region that funnels into the quantum critical point. Our study thereby provides an important proof-of-principle example that new physics can be detected via unbiased machine-learning approaches.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2025

Learning Physically Interpretable Atmospheric Models From Data With WSINDy

The multiscale and turbulent nature of Earth's atmosphere has historically rendered accurate weather modeling a hard problem. Recently, there has been an explosion of interest surrounding data-driven approaches to weather modeling, which in many cases show improved forecasting accuracy and computational efficiency when compared to traditional methods. However, many of the current data-driven approaches employ highly parameterized neural networks, often resulting in uninterpretable models and limited gains in scientific understanding. In this work, we address the interpretability problem by explicitly discovering partial differential equations governing atmospheric phenomena, identifying symbolic mathematical models with direct physical interpretations. The purpose of this paper is to demonstrate that, in particular, the weak-form sparse identification of nonlinear dynamics (WSINDy) algorithm can learn effective atmospheric models from both simulated and assimilated data. Our approach adapts the standard WSINDy algorithm to work with high-dimensional fluid data of arbitrary spatial dimension.

58 GEOSCIENCES↗
DOE OSTI2021

Deep ML for inference from indirect measurements

The magic of deep machine learning (DML): Neural networks that can beat Go grandmasters. Networks that have better than human accuracy at classifying images in over 100 categories. Making still pictures move realistically. Super-resolution versions of very old film clips. Realistic, high-resolution, images of people that have never existed. Realistic insertion of people into video clips. Network generated music and writing that passes for human created. How can this technology be leveraged for physics and engineering?

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Machine Learning of Key Variables Impacting Extreme Precipitation in Various Regions of the Contiguous United States

Abstract Amplification in extreme precipitation intensity and frequency can cause severe flooding and impose significant social and economic consequences. Variations in extreme precipitation intensity, frequencies, and return periods can be attributed to many physical variables across spatial and temporal scales. Here we employ ensemble machine learning (ML) methods, namely random forest (RF), eXtreme Gradient Boosting (XGB), and artificial neural networks (ANN), to explore key contributing variables to monthly extreme precipitation intensity and frequency in six regions over the United States. We further establish emulators for return periods. Results show that the ML models for intensity perform better in regions with obvious seasonality (i.e., Northern Great Plains, Southern Great Plains, and West Coast) than the other three regions (Northeast, Southwest, and Rocky Mountains), while for frequency the models perform well for most regions. The Shapley additive explanation is used to help explain the relationships between extreme precipitation characteristics and identify top variables for RF and XGB. We find that latent heat flux, relative humidity, soil moisture, and large‐scale subsidence are key common variables across the regions for both monthly intensity and frequency, and their compound effects are non‐negligible. The developed ML models capture the probability and return period of extreme precipitation well for all regions and may be used for decision making (e.g., infrastructure planning and design).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

A deep learning approach to fast analysis of collective Thomson scattering spectra

Fast analysis of collective Thomson scattering ion acoustic wave features using a deep convolutional neural network model is presented. The network was trained from spectra to predict the plasma parameters, including ion velocities, population fractions, and ion and electron temperatures. A fully kinetic particle-in-cell simulation was used to model a laboratory astrophysics experiment and simulate a diagnostic image of the ion acoustic wave feature. Network predictions were compared with Bayesian inference of the plasma model parameters for both the simulated and experimentally measured images. Both approaches were fairly accurate predicting the simulated image and the network predictions matched a good portion of the Bayesian results for the experimentally measured image. The Bayesian approach is more robust to noise and motivates future work to train deep learning models with realistic noise. The advantage of the deep learning model is making thousands of predictions in a few hundred milliseconds, compared to a few seconds to minutes per prediction for the optimization and Bayesian approaches presented here. The results demonstrate promising capabilities of deep learning models to analyze Thomson data orders of magnitude faster than conventional methods when using the neural network for standalone analysis. If more rigorous analysis is needed, neural network predictions can be used to quickly initialize other optimization methods and increase chances of success. This is especially useful when the dataset becomes very large or highly dimensional and manually refining initial conditions for the entire dataset are no longer tractable.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

Cyber-Physical Power Systems Protection: The Byzantine Cybersecurity Framework

Cybersecurity of smart grids have been topic of much interest in recent years. As this critical infrastructure operation increases dependency on automated processes and controls, exposure to cyber-physical threats become inevitable. Considering cyber-physical security of the grid, much focus of attention has been made towards smart grids real-time monitoring solutions, including the state estimation process. Analyzing the relevant literature, one can note though that seldom research has been done on cyber-physical security of smart grids protection systems. Protection systems have intangible value towards grid reliability. This paper presents a cybersecurity framework for smart grids protection systems. A physics-based inspired machine learning solution is at the core process of the framework. Processed relay inputs and outputs are used by a deep predictive coding network. Formal models, a quasi-static state estimator, provides an oracle when low confidence decision is reached. Evolving knowledge is derived through reinforcement learning. Implementation aspects considering the Pacific Northwest National Laboratory Electricity Infrastructure Operations Center are presented. Built as an extra control layer to protection systems, without hard-to-derive parameters, highlights potential aspects towards real-life applications.

Bretas, Arturo Suman↗
DOE OSTI2021

Machine-Learned Manifold-Based Models for Large Eddy Simulation of Turbulent Combustion

Reduced-order manifold combustion models are commonly used to lower the cost of reacting Large Eddy Simulations (LES) and may be derived either from physical principles as in flamelet models or from data-driven methods like Principal Component Analysis (PCA). In either case, neural networks are increasingly used as part of these models to provide a nonlinear mapping between a small set of pre-defined variables that parameterize the manifold and outputs of interest, such as reaction rates. In this work, we propose a new manifold-based modeling approach that combines the definition of the manifold-parameterizing variables (linear combinations of species), the nonlinear mapping to the outputs, and closure of filtered quantities for LES into the structure of a single neural network. This allows the process used to train the neural network to simultaneously optimize both the functional form of the model and the identities of the inputs to the model. The new approach can flexibly incorporate thermochemical data from any combustion system; if trained on data from 1D flames it can be interpreted as an optimized flamelet model, but it can also be used to learn models from data from more complex configurations. This work presents a priori evaluations of the new approach in both contexts. Evaluation using data from 1D premixed flames demonstrates the physical interpretability of the manifold variables generated by the new approach. Evaluation using data from direct numerical simulations of turbulent flames shows improved predictions relative to either flamelet or PCA-based models in a more complex configuration.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

Probabilistic Physics-Informed Graph Convolutional Network for Active Distribution System Voltage Prediction

Here this letter proposes a novel data-driven probabilistic physics-informed graph convolutional network (GCN) for active distribution system voltage prediction with PVs and EVs. It leverages both measurements and network topology to accurately and efficiently predict node voltages without the need for an accurate distribution system power flow model. The dropout-enabled Bayesian inference is developed to achieve uncertainty quantification of the voltage prediction. Thanks to the network model embedding, it also has robustness against topology changes, a key difference with existing machine learning-based approaches. Comparison results with other state-of-the-art machine learning methods on a realistic 759-node distribution system demonstrate that the proposed method can achieve better accuracy and robustness under different scenarios.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

AF2Complex predicts direct physical interactions in multimeric proteins with deep learning

Abstract Accurate descriptions of protein-protein interactions are essential for understanding biological systems. Remarkably accurate atomic structures have been recently computed for individual proteins by AlphaFold2 (AF2). Here, we demonstrate that the same neural network models from AF2 developed for single protein sequences can be adapted to predict the structures of multimeric protein complexes without retraining. In contrast to common approaches, our method, AF2Complex, does not require paired multiple sequence alignments. It achieves higher accuracy than some complex protein-protein docking strategies and provides a significant improvement over AF-Multimer, a development of AlphaFold for multimeric proteins. Moreover, we introduce metrics for predicting direct protein-protein interactions between arbitrary protein pairs and validate AF2Complex on some challenging benchmark sets and the E. coli proteome. Lastly, using the cytochrome c biogenesis system I as an example, we present high-confidence models of three sought-after assemblies formed by eight members of this system.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Prediction of carbon nanostructure mechanical properties and the role of defects using machine learning

Graphene-based nanostructures hold immense potential as strong and lightweight materials, however, their mechanical properties such as modulus and strength are difficult to fully exploit due to challenges in atomic-scale engineering. This study presents a database of over 2,000 pristine and defective nanoscale CNT bundles and other graphitic assemblies, inspired by microscopy, with associated stress–strain curves from reactive molecular dynamics (MD) simulations using the reactive INTERFACE force field (IFF-R). These 3D structures, containing up to 80,000 atoms, enable detailed analyses of structure-stiffness-failure relationships. By leveraging the database and physics- and chemistry-informed machine learning (ML), accurate predictions of elastic moduli and tensile strength are demonstrated at speeds 1,000 to 10,000 times faster than efficient MD simulations. Hierarchical Graph Neural Networks with Spatial Information (HS-GNNs) are introduced, which integrate chemistry knowledge. HS-GNNs as well as extreme gradient boosted trees (XGBoost) achieve forecasts of mechanical properties of arbitrary carbon nanostructures with only 3 to 6% mean relative error. The reliability equals experimental accuracy and is up to 20 times higher than other ML methods. Predictions maintain 8 to 18% accuracy for large CNT bundles, CNT junctions, and carbon fiber cross-sections outside the training distribution. The physics- and chemistry-informed HS-GNN works remarkably well for data outside the training range while XGBoost works well with limited training data inside the training range. The carbon nanostructure database is designed for integration with multimodal experimental and simulation data, scalable beyond 100 nm size, and extendable to chemically similar compounds and broader property ranges. The ML approaches have potential for applications in structural materials, nanoelectronics, and carbon-based catalysts.

Winetrout, Jordan J.↗
DOE OSTI2022

Data reduction through optimized scalar quantization for more compact neural networks

Raw data generation for several existing and planned large physics experiments now exceeds TB/s rates, generating untenable data sets in very little time. Those data often demonstrate high dimensionality while containing limited information. Meanwhile, Machine Learning algorithms are now becoming an essential part of data processing and data analysis. Those algorithms can be used offline for post processing and post data analysis, or they can be used online for real time processing providing ultra low latency experiment monitoring. Both use cases would benefit from data throughput reduction while preserving relevant information: one by reducing the offline storage requirements by several orders of magnitude and the other by allowing ultra fast online inferencing with low complexity Machine Learning models. Moreover, reducing the data source throughput also reduces material cost, power and data management requirements. In this work we demonstrate optimized nonuniform scalar quantization for data source reduction. This data reduction allows lower dimensional representations while preserving the relevant information of the data, thus enabling high accuracy Tiny Machine Learning classifier models for online fast inferences. We demonstrate this approach with an initial proof of concept targeting the CookieBox, an array of electron spectrometers used for angular streaking, that was developed for LCLS-II as an online beam diagnostic tool. We used the Lloyd-Max algorithm with the CookieBox dataset to design an optimized nonuniform scalar quantizer. Optimized quantization lets us reduce input data volume by 69% with no significant impact on inference accuracy. When we tolerate a 2% loss on inference accuracy, we achieved 81% of input data reduction. Finally, the change from a 7-bit to a 3-bit input data quantization reduces our neural network size by 38%.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

PECAN2

PECAN2, Pose Classification with 3D Atomic Network 2, presents an innovative approach to pose classification in computational modeling, particularly in the context of molecular docking. Traditional methods like docking, which rely on physics-based calculations, are prone to inaccuracies in predicting binding poses during experimental testing. While machine learning (ML) approaches, such as ML-driven pose classification, have been introduced to address these issues, they often depend on the availability of crystal structure data. In contrast, PECAN2 introduces a novel pose classification method that operates independently of crystal structures. Instead, it utilizes a 3D atomic neural network with Point Cloud Network (PCN) to establish correlations between docking scores and experimental data. This approach marks a departure from previous studies that heavily relied on crystal structures for labeling. The key innovation lies in the ability of PECAN2 to enhance the performance of molecular docking by filtering out false positives and false negatives through the correlation between docking scores and experimental data.

Shim, Heesung↗