Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “neural algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 829 records · Page 46

Physics-informed machine learning

Despite great progress in simulating multiphysics problems using the numerical discretization of partial differential equations (PDEs), one still cannot seamlessly incorporate noisy data into existing algorithms, mesh generation remains complex, and high-dimensional problems governed by parameterized PDEs cannot be tackled. Moreover, solving inverse problems with hidden physics is often prohibitively expensive and requires different formulations and elaborate computer codes. Machine learning has emerged as a promising alternative, but training deep neural networks requires big data, not always available for scientific problems. Instead, such networks can be trained from additional information obtained by enforcing the physical laws (for example, at random points in the continuous space-time domain). Such physics-informed learning integrates (noisy) data and mathematical models, and implements them through neural networks or other kernel-based regression networks. Moreover, it may be possible to design specialized network architectures that automatically satisfy some of the physical invariants for better accuracy, faster training and improved generalization. Furthermore, we review some of the prevailing trends in embedding physics into machine learning, present some of the current capabilities and limitations and discuss diverse applications of physics-informed learning both for forward and inverse problems, including discovering hidden physics and tackling high-dimensional problems.

97 MATHEMATICS AND COMPUTING↗

Deep Learning Approach for High-accuracy Electron Counting of Monolithic Active Pixel Sensor-type Direct Electron Detectors at Increased Electron Dose

Abstract Electron counting can be performed algorithmically for monolithic active pixel sensor direct electron detectors to eliminate readout noise and Landau noise arising from the variability in the amount of deposited energy for each electron. Errors in existing counting algorithms include mistakenly counting a multielectron strike as a single electron event, and inaccurately locating the incident position of the electron due to lateral spread of deposited energy and dark noise. Here, we report a supervised deep learning (DL) approach based on Faster region-based convolutional neural network (R-CNN) to recognize single electron events at varying electron doses and voltages. The DL approach shows high accuracy according to the near-ideal modulation transfer function (MTF) and detector quantum efficiency for sparse images. It predicts, on average, 0.47 pixel deviation from the incident positions for 200 kV electrons versus 0.59 pixel using the conventional counting method. The DL approach also shows better robustness against coincidence loss as the electron dose increases, maintaining the MTF at half Nyquist frequency above 0.83 as the electron density increases to 0.06 e−/pixel. Thus, the DL model extends the advantages of counting analysis to higher dose rates than conventional methods.

Materials Science↗

What drives the scatter of local star-forming galaxies in the BPT diagrams? A Machine Learning based analysis

ABSTRACT We investigate which physical properties are most predictive of the position of local star forming galaxies on the BPT diagrams, by means of different Machine Learning (ML) algorithms. Exploiting the large statistics from the Sloan Digital Sky Survey (SDSS), we define a framework in which the deviation of star-forming galaxies from their median sequence can be described in terms of the relative variations in a variety of observational parameters. We train artificial neural networks (ANN) and random forest (RF) trees to predict whether galaxies are offset above or below the sequence (via classification), and to estimate the exact magnitude of the offset itself (via regression). We find, with high significance, that parameters primarily associated to variations in the nitrogen-over-oxygen abundance ratio (N/O) are the most predictive for the [N ii]-BPT diagram, whereas properties related to star formation (like variations in SFR or EW(H α)) perform better in the [S ii]-BPT diagram. We interpret the former as a reflection of the N/O–O/H relationship for local galaxies, while the latter as primarily tracing the variation in the effective size of the S+ emitting region, which directly impacts the [S ii] emission lines. This analysis paves the way to assess to what extent the physics shaping local BPT diagrams is also responsible for the offsets seen in high redshift galaxies or, instead, whether a different framework or even different mechanisms need to be invoked.

79 ASTRONOMY AND ASTROPHYSICS↗

Randomized probe imaging through deep k-learning

Randomized probe imaging (RPI) is a single-frame diffractive imaging method that uses highly randomized light to reconstruct the spatial features of a scattering object. The reconstruction process, known as phase retrieval, aims to recover a unique solution for the object without measuring the far-field phase information. Typically, reconstruction is done via time-consuming iterative algorithms. In this work, we propose a fast and efficient deep learning based method to reconstruct phase objects from RPI data. The method, which we call deep k-learning, applies the physical propagation operator to generate an approximation of the object as an input to the neural network. This way, the network no longer needs to parametrize the far-field diffraction physics, dramatically improving the results. Deep k-learning is shown to be computationally efficient and robust to Poisson noise. The advantages provided by our method may enable the analysis of far larger datasets in photon starved conditions, with important applications to the study of dynamic phenomena in physical science and biological engineering.

Guo, Zhen (ORCID:0000000213473451)↗

A General Framework to Learn Tertiary Structure for Protein Sequence Characterization

During the past five years, deep-learning algorithms have enabled ground-breaking progress towards the prediction of tertiary structure from a protein sequence. Very recently, we developed SAdLSA, a new computational algorithm for protein sequence comparison via deep-learning of protein structural alignments. SAdLSA shows significant improvement over established sequence alignment methods. In this contribution, we show that SAdLSA provides a general machine-learning framework for structurally characterizing protein sequences. By aligning a protein sequence against itself, SAdLSA generates a fold distogram for the input sequence, including challenging cases whose structural folds were not present in the training set. About 70% of the predicted distograms are statistically significant. Although at present the accuracy of the intra-sequence distogram predicted by SAdLSA self-alignment is not as good as deep-learning algorithms specifically trained for distogram prediction, it is remarkable that the prediction of single protein structures is encoded by an algorithm that learns ensembles of pairwise structural comparisons, without being explicitly trained to recognize individual structural folds. As such, SAdLSA can not only predict protein folds for individual sequences, but also detects subtle, yet significant, structural relationships between multiple protein sequences using the same deep-learning neural network. The former reduces to a special case in this general framework for protein sequence annotation.

59 BASIC BIOLOGICAL SCIENCES↗

Implicit learning of convective organization explains precipitation stochasticity

Accurate prediction of precipitation intensity is crucial for both human and natural systems, especially in a warming climate more prone to extreme precipitation. Yet, climate models fail to accurately predict precipitation intensity, particularly extremes. One missing piece of information in traditional climate model parameterizations is subgrid-scale cloud structure and organization, which affects precipitation intensity and stochasticity at coarse resolution. Here, using global storm-resolving simulations and machine learning, we show that, by implicitly learning subgrid organization, we can accurately predict precipitation variability and stochasticity with a low-dimensional set of latent variables. Using a neural network to parameterize coarse-grained precipitation, we find that the overall behavior of precipitation is reasonably predictable using large-scale quantities only; however, the neural network cannot predict the variability of precipitation (R 2 ~ 0.45) and underestimates precipitation extremes. The performance is significantly improved when the network is informed by our organization metric, correctly predicting precipitation extremes and spatial variability (R 2 ~ 0.9). The organization metric is implicitly learned by training the algorithm on a high-resolution precipitable water field, encoding the degree of subgrid organization. The organization metric shows large hysteresis, emphasizing the role of memory created by subgrid-scale structures. We demonstrate that this organization metric can be predicted as a simple memory process from information available at the previous time steps. These findings stress the role of organization and memory in accurate prediction of precipitation intensity and extremes and the necessity of parameterizing subgrid-scale convective organization in climate models to better project future changes of water cycle and extremes.

54 ENVIRONMENTAL SCIENCES↗

End-to-End Optimization for Battery Materials and Molecules by Combining Graph Neural Networks and Reinforcement Learning

The National Renewable Energy Laboratory (NREL), together with the Colorado School of Mines (CSM) and Colorado State University (CSU), has developed a machine learning-enhanced approach to the design of new battery materials. Currently, such materials are designed in part via numerous expensive high-fidelity computational simulations that predict the performance of a given composition. Even with computational screening tools, the vast landscape of possible molecular or crystal structures exceeds current and future computational capacity. Improving the efficiency by which new materials can be optimized will therefore disrupt the cost, risk, and time required to bring new energy solutions to the marketplace. Predicting the properties of an organic molecule or periodic crystalline material given its structure has grown increasingly common. These approaches leverage large-scale computational and experimental databases and ML approaches such as graph neural networks. The inverse design problem of finding a material that possesses desired properties is substantially more challenging, since enumerating all valid material structures is not feasible. In this project, we leveraged recent success in reinforcement learning to efficiently navigate this high-dimensional search space. Just as algorithms can find the optimal chess moves from nearly limitless options, we train an approach to evolve a simple starting structure into a complex structure that possess the desired properties. Our solution has been demonstrated by applying it to two related design application tasks for short- and long-term energy storage, respectively: (1) the design of solid-state ion conductors and (2) the design of organic redox-active materials. The project has resulted an open-source software library for material design, documented examples of applying the library to both organic and inorganic material optimization, and peer-reviewed publications detailing the data, computational models, and resulting candidate materials.

25 ENERGY STORAGE↗

Improved Architectures and Training Algorithms for Deep Operator Networks

Operator learning techniques have recently emerged as a powerful tool for learning maps between infinite-dimensional Banach spaces. Trained under appropriate constraints, they can also be effective in learning the solution operator of partial differential equations (PDEs) in an entirely self-supervised manner. In this work we analyze the training dynamics of deep operator networks (DeepONets) through the lens of Neural Tangent Kernel theory, and reveal a bias that favors the approximation of functions with larger magnitudes. To correct this bias we propose to adaptively re-weight the importance of each training example, and demonstrate how this procedure can effectively balance the magnitude of back-propagated gradients during training via gradient descent. We also propose a novel network architecture that is more resilient to vanishing gradient pathologies. Taken together, our developments provide new insights into the training of DeepONets and consistently improve their predictive accuracy by a factor of 10-50x, demonstrated in the challenging setting of learning PDE solution operators in the absence of paired input-output observations.

97 MATHEMATICS AND COMPUTING↗

Detecting rare neutral atomic-carbon absorbers with a deep neural network

ABSTRACT C i absorbers play an important role as indicators for exploring the presence of cold gas in the interstellar medium of galaxies. However, the current data base of C i absorbers is very limited due to their weak absorption feature and rarity. Here, we report results from a search of C i λλ1560, 1656 absorption lines using Mg ii absorbers as signposts with modified deep learning algorithms, which provides a very quick way to search for weak C i absorber candidates. A total of 107 C i absorbers were detected, which nearly doubles the size of previously known samples. In addition, we found 17 C i absorbers to be associated with 2175 Å dust absorbers (2DAs), i.e. about 16 per cent C i absorbers are associated with 2DAs. Comparing the average dust depletion patterns of C i absorbers with those of damped Lyman α absorbers (DLAs), Mg ii absorbers, Ca ii absorbers, and 2175 Å dust absorbers (2DAs) shows that C i absorbers generally have environments with more dust than DLAs, Mg ii, and Ca ii absorbers, but similar to dust in 2DAs. Similarity between the dust depletion pattern of C i absorbers to that of the warm disc in the Milky Way indicates that C i absorption clouds are possibly associated with disc components in distant galaxies. Therefore, C i absorbers are confirmed to be excellent probes to trace cold gas and dust in the Universe.

Ge, Jian↗

Bias-Variance Trade-Off in Physics-Informed Neural Networks with Randomized Smoothing for High-Dimensional PDEs

Physics-Informed Neural Networks (PINNs) have triggered a paradigm shift in scientific computing, leveraging mesh-free properties and robust approximation capabilities. While proving effective for low-dimensional partial differential equations (PDEs), the computational cost of PINNs remains a hurdle in high-dimensional scenarios. This is particularly pronounced when computing high-order and high-dimensional derivatives in the physics-informed loss. Randomized Smoothing PINN (RS-PINN) introduces Gaussian noise for stochastic smoothing of the original neural net model, enabling the use of Monte Carlo methods for derivative approximation, which eliminates the need for costly automatic differentiation. Despite its computational efficiency, especially in the approximation of high-dimensional derivatives, RS-PINN introduces biases in both loss and gradients, negatively impacting convergence, especially when coupled with stochastic gradient descent (SGD) algorithms. We present a comprehensive analysis of biases in RS-PINN, attributing them to the nonlinearity of the Mean Squared Error (MSE) loss as well as the intrinsic nonlinearity of the PDE itself. We propose tailored bias correction techniques, delineating their application based on the order of PDE nonlinearity. The derivation of an unbiased RS-PINN allows for a detailed examination of its advantages and disadvantages compared to the biased version. Specifically, the biased version has a lower variance and runs faster than the unbiased version, but it is less accurate due to the bias. To optimize the bias-variance trade-off, we combine the two approaches in a hybrid method that balances the rapid convergence of the biased version with the high accuracy of the unbiased version. In addition to methodological contributions, we present an enhanced implementation of RS-PINN. Extensive experiments on diverse high-dimensional PDEs, including Fokker-Planck, Hamilton-Jacobi-Bellman (HJB), viscous Burgers’, Allen-Cahn, and Sine-Gordon equations, illustrate the bias-variance trade-off and highlight the effectiveness of the hybrid RS-PINN. Empirical guidelines are provided for selecting biased, unbiased, or hybrid versions, depending on the dimensionality and nonlinearity of the specific PDE problem.

97 MATHEMATICS AND COMPUTING↗

Large-scale deep learning for metastasis detection in pathology reports

Objectives No existing algorithm can reliably identify metastasis from pathology reports across multiple cancer types and the entire US population. In this study, we develop a deep learning model that automatically detects patients with metastatic cancer by using pathology reports from many laboratories and of multiple cancer types. Materials and Methods We use 60 471 unstructured pathology reports from 4 Surveillance, Epidemiology, and End Results (SEER) registries. The reports were coded into 1 of 3 labels: metastasis negative, metastases positive, or metastasis undetermined. We utilize a task-specific deep neural network trained from scratch and compare its performance with a widely used large language model (LLM). Results Our deep learning architecture trained on task-specific data outperforms a general-purpose LLM, with a recall of 0.894 compared to 0.824. We quantified model uncertainty and used it to defer reports for human review. We found that retaining 72.9% of reports increased recall from 0.894 to 0.969. Discussion A smaller deep learning architecture trained on task-specific data outperforms a general LLM. Equally critical to model performance is the incorporation of uncertainty quantification, achieved here through an abstention mechanism. Conclusions This study’s finding demonstrate the feasibility of developing algorithms to automatically identify metastatic cancer cases from unstructured pathology reports.

machine learning↗

Machine learning without a processor: Emergent learning in a nonlinear analog network

Standard deep learning algorithms require differentiating large nonlinear networks, a process that is slow and power-hungry. Electronic contrastive local learning networks (CLLNs) offer potentially fast, efficient, and fault-tolerant hardware for analog machine learning, but existing implementations are linear, severely limiting their capabilities. These systems differ significantly from artificial neural networks as well as the brain, so the feasibility and utility of incorporating nonlinear elements have not been explored. Here, we introduce a nonlinear CLLN—an analog electronic network made of self-adjusting nonlinear resistive elements based on transistors. We demonstrate that the system learns tasks unachievable in linear systems, including XOR (exclusive or) and nonlinear regression, without a computer. We find our decentralized system reduces modes of training error in order (mean, slope, curvature), similar to spectral bias in artificial neural networks. The circuitry is robust to damage, retrainable in seconds, and performs learned tasks in microseconds while dissipating only picojoules of energy across each transistor. This suggests enormous potential for fast, low-power computing in edge systems like sensors, robotic controllers, and medical devices, as well as manufacturability at scale for performing and studying emergent learning.

Science & Technology - Other Topics↗

Machine Learning for DUNE Supernova Trigger

One of the major scientific goals of the Deep Underground Neutrino Experiment (DUNE) is to detect and measure the neutrino flux originating from galactic core-collapse supernovae. These neutrinos provide an opportunity to study the end of life evolution of massive stars, and reveal information about the structure of core-collapse that is not visible in observations of the electromagnetic spectrum. Because of the rarity of these events, it is crucial that DUNE is able to detect supernova neutrino interactions when they occur. However, this will require sifting through a large quantity of data, motivating the development of a trigger algorithm to identify significant events and discard irrelevant data. Machine learning provides a potential approach to building this trigger. This project generates ADC and ground truth images of simulated neutrino interactions in a LArTPC detector to be used for machine learning, and uses them to train a sparse Convolutional Neural Network (C NN). The performance of this model when applied to the task of pixel classification based on interaction type is examined. This project found that the sparse CNN approach has the potential to have high accuracy in pixel classification, meaning it may be highly relevant to the development of a supernova neutrino trigger for the DUNE far detector.

Damish, S.↗

Accelerating Scientific Computing in the Post-Moore’s Era

Novel uses of graphical processing units for accelerated computation revolutionized the field of high-performance scientific computing by providing specialized workflows tailored to algorithmic requirements. As the era of Moore’s law draws to a close, many new non–von Neumann processors are emerging as potential computational accelerators, including those based on the principles of neuromorphic computing, tensor algebra, and quantum information. While development of these new processors is continuing to mature, the potential impact on accelerated computing is anticipated to be profound. We discuss how different processing models can advance computing in key scientific paradigms: machine learning and constraint satisfaction. Significantly, each of these new processor types utilizes a fundamentally different model of computation, and this raises questions about how to best use such processors in the design and implementation of applications. While many processors are being developed with a specific domain target, the ubiquity of spin-glass models and neural networks provides an avenue for multi-functional applications. Furthermore, this also hints at the infrastructure needed to integrate next-generation processing units into future high-performance computing systems.

97 MATHEMATICS AND COMPUTING↗

Dark Energy Survey Year 3 results: $w$CDM cosmology from simulation-based inference with persistent homology on the sphere

We present cosmological constraints from Dark Energy Survey Year 3 (DES Y3) weak lensing data using persistent homology, a topological data analysis technique that tracks how features like clusters and voids evolve across density thresholds. For the first time, we apply spherical persistent homology to galaxy survey data through the algorithm TopoS2, which is optimized for curved-sky analyses and HEALPix compatibility. Employing a simulation-based inference framework with the Gower Street simulation suite, specifically designed to mimic DES Y3 data properties, we extract topological summary statistics from convergence maps across multiple smoothing scales and redshift bins. After neural network compression of these statistics, we estimate the likelihood function and validate our analysis against baryonic feedback effects, finding minimal biases (under $0.3σ$) in the $Ω_\mathrm{m}-S_8$ plane. Assuming the $w$CDM model, our combined Betti numbers and second moments analysis yields $S_8 = 0.821 \pm 0.018$ and $Ω_\mathrm{m} = 0.304\pm0.037$-constraints 70% tighter than those from cosmic shear two-point statistics in the same parameter plane. Our results demonstrate that topological methods provide a powerful and robust framework for extracting cosmological information, with our spherical methodology readily applicable to upcoming Stage IV wide-field galaxy surveys.

Prat, J. [Nordita; Royal Inst. Tech., Sodertalje; ↗

Comparison of machine learning and electrical resistivity arrays to inverse modeling for locating and characterizing subsurface targets

Here, this study evaluates the performance of multiple machine learning (ML) algorithms and electrical resistivity (ER) arrays for inversion with comparison to a conventional Gauss-Newton numerical inversion method. Four different ML models and four arrays were used for the estimation of only six variables for locating and characterizing hypothetical subsurface targets. The combination of dipole-dipole with Multilayer Perceptron Neural Network (MLP-NN) had the highest accuracy. Evaluation showed that both MLP-NN and Gauss-Newton methods performed well for estimating the matrix resistivity while target resistivity accuracy was lower, and MLP-NN produced sharper contrast at target boundaries for the field and hypothetical data. Both methods exhibited comparable target characterization performance, whereas MLP-NN had increased accuracy compared to Gauss-Newton in prediction of target width and height, which was attributed to numerical smoothing present in the Gauss-Newton approach. MLP-NN was also applied to a field dataset acquired at U.S. DOE Hanford site.

54 ENVIRONMENTAL SCIENCES↗

Intelligent experiments through real-time AI: Fast Data Processing and Autonomous Detector Control for sPHENIX and future EIC detectors

This R&D project, initiated by the DOE Nuclear Physics AI-Machine Learning initiative in 2022, leverages AI to address data processing challenges in high-energy nuclear experiments (RHIC, LHC, and future EIC). Our focus is on developing a demonstrator for real-time processing of high-rate data streams from sPHENIX experiment tracking detectors. The limitations of a 15 kHz maximum trigger rate imposed by the calorimeters can be negated by intelligent use of streaming technology in the tracking system. The approach efficiently identifies low momentum rare heavy flavor events in high-rate p+p collisions (3MHz), using Graph Neural Network (GNN) and High Level Synthesis for Machine Learning (hls4ml). Success at sPHENIX promises immediate benefits, minimizing resources and accelerating the heavy-flavor measurements. The approach is transferable to other fields. For the EIC, we develop a DIS-electron tagger using Artificial Intelligence - Machine Learning (AI-ML) algorithms for real-time identification, showcasing the transformative potential of AI and FPGA technologies in high-energy nuclear and particle experiments real-time data processing pipelines.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Domain Adaptation for Measurements of Strong Gravitational Lenses

Upcoming surveys are predicted to discover galaxy-scale strong lenses on the magnitude of 105, making deep learning methods necessary in lensing data analysis. Currently, there is insufficient real lensing data to train deep learning algorithms, but training only on simulated data results in poor performance on real data. Domain adaptation can bridge the gap between simulated and real datasets. We adopt domain adaptation on the estimation of Einstein radius in simulated galaxy-scale gravitational lensing images. We evaluate two domain adaptation techniques - domain adversarial neural networks (DANN) and maximum mean discrepancy (MMD). We train on a source domain of simulated lenses and apply it to a target domain with emulation of DES survey conditions. We show that both domain adaptation techniques can significantly improve the model performance on the more complex target domain datasets. Our results show the potential of using domain adaptation to perform analysis on future survey data with a deep neural network trained on simulated data.

79 ASTRONOMY AND ASTROPHYSICS↗