Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Neural Networks (Computer)”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,585 records · Page 88

Stimulus-dependent suppression of chaos in recurrent neural networks

Neuronal activity arises from an interaction between ongoing firing generated spontaneously by neural circuits and responses driven by external stimuli. Using mean-field analysis, we ask how a neural network that intrinsically generates chaotic patterns of activity can remain sensitive to extrinsic input. We find that inputs not only drive network responses, but they also actively suppress ongoing activity, ultimately leading to a phase transition in which chaos is completely eliminated. The critical input intensity at the phase transition is a nonmonotonic function of stimulus frequency, revealing a 'resonant' frequency at which the input is most effective at suppressing chaos even though the power spectrum of the spontaneous activity peaks at zero and falls exponentially. A prediction of our analysis is that the variance of neural responses should be most strongly suppressed at frequencies matching the range over which many sensory systems operate.

97 MATHEMATICS AND COMPUTING↗

A novel technique for minimizing energy functional using neural networks

An energy functional describes the equilibrium state of a system. In this work, we present a novel technique, Functional Optimization using Neural Networks (FONN), for minimizing the system’s energy. FONN utilizes neural networks to process information at discrete grid points, considering their interactions with neighboring grid points, to update the state of the system. The training process involves formulating a loss function based on the system’s energy, and with the help of multiple fine-tuning steps, the method employs a progressive energy reduction technique that decreases the energy in multiple steps. FONN’s effectiveness is demonstrated across various problems, including the minimization of the heat and Lyapunov energy. Furthermore, the paper explores the minimization of the elastic bending energy with an area constraint.

97 MATHEMATICS AND COMPUTING↗

Measurement of the single top production cross section in proton-antiproton collisions at 1.96 TeV

This thesis describes a search for singly produced top quarks via an electroweak vertex in head-on proton-antiproton collisions at a center of mass energy of √s = 1.96 TeV. The analysis uses a total of 2.3 fb -1 of data collected with the D0 detector at Fermilab, corresponding to two different run periods of the Tevatron collider. Two channels contribute to single top quark production at the Tevatron, the s-channel and the t-channel. In the s-channel, a virtual W boson is produced from the aniquilation of a quark and an antiquark and a top and a bottom quarks are produced from the W decay. The top quark decays almost exclusively into a W boson and a bottom quark. Final states are considered in which the W boson decays leptonically into an electron or a muon plus a neutrino. Thus, at the detector level, the final state characterizing the s-channel contains one lepton, missing energy accounting for the neutrino, and two jets from the two bottom quarks. In the t-channel, the final state has an additional jet coming from a light quark. Clearly, a precise reconstruction of the events requires a precise measurement of the energy of the jets. A multivariate technique, Bayesian neural networks, is used to extract the single top signal from the overwhelming background still left after event selection. A Bayesian likelihood probability is then computed to measure the single top cross section. Assuming the observed excess is due to single top events, the measured single top quark production cross section is σ(p$\bar{p}$ → tb + X, tqb + X) = 4.70 +1.18 -0.93 pb. The observed excess is associated with a p-value of (3.2 ± 2.3) x 10 -8 , assuming the background-only hypothesis. This p-value corresponds to an excess over background of 5.4 standard deviations for a Gaussian density. The p-value computed using the standard model signal cross section of 3.46 pb is (22.7 ± 0.6) x 10 -6 , corresponding to an expected significance of 4.08 standard deviations.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Prediction and Computation of Corrosion Rates of A36 Mild Steel in Oilfield Seawater

The parameters which primarily control the corrosion rate and life of steel structures are several and they vary across the different ocean and seawater as well as along the depth. While the effect of single parameter on corrosion behavior is known, the conjoint effects of multiple parameters and the interrelationship among the variables are complex. Millions sets of experiments are required to understand the mechanism of corrosion failure. Statistical modeling such as ANN is one solution that can reduce the number of experimentation. ANN model was developed using 170 sets of experimental data of A35 mild steel in simulated seawater, varying the corrosion influencing parameters SO{sub 4}{sup 2−}, Cl{sup −}, HCO{sub 3}{sup −},CO{sub 3}{sup 2−}, CO{sub 2}, O{sub 2}, pH and temperature as input and the corrosion current as output. About 60% of experimental data were used to train the model, 20% for testing and 20% for validation. The model was developed by programming in Matlab. 80% of the validated data could predict the corrosion rate correctly. Corrosion rates predicted by the ANN model are displayed in 3D graphics which show many interesting phenomenon of the conjoint effects of multiple variables that might throw new ideas of mitigation of corrosion by simply modifying the chemistry of the constituents. The model could predict the corrosion rates of some real systems.

36 MATERIALS SCIENCE↗

Deep learning for time series forecasting: a survey of recent advances

Time series forecasting plays a critical role in numerous real-world applications, such as finance, healthcare, transportation, and scientific computing. In recent years, deep learning has become a powerful tool for modeling complex temporal patterns and improving forecasting accuracy. This survey provides an overview of recent deep learning approaches for time series forecasting, involving various architectures including RNNs, CNNs, GNNs, transformers, large language models, MLP-based models, and diffusion models. We first identify key challenges in the field, such as temporal dependency, efficiency, and cross-variable dependency, which drive the development of forecasting techniques. Then, the general advantages and limitations of each architecture are discussed to contextualize their adaptation in time series forecasting. Furthermore, we highlight promising design trends like multi-scale modeling, decomposition, and frequency-domain techniques, which are shaping the future of the field. This paper serves as a compact reference for researchers and practitioners seeking to understand the current landscape and future trajectory of deep learning in time series forecasting.

97 MATHEMATICS AND COMPUTING↗

Physics-guided machine learning for improved accuracy of the National Solar Radiation Database

The National Solar Radiation Database (NSRDB) provides high-resolution spatiotemporal solar irradiance data for the entire globe. The NSRDB uses a two-step Physical Solar Model (PSM) to compute the effects of clouds and other atmospheric variables on the solar radiation reaching the surface of the Earth. Physical and optical cloud properties are fundamental inputs to the PSM and are derived from the National Oceanic and Atmospheric Administration's Geostationary Operational Environmental Satellites. This paper describes recent improvements to the NSRDB driven by physics-guided machine learning methods for cloud property retrieval. The impacts of these new methods on the NSRDB irradiance data are validated using an extensive set of ground measurement sites, showing significant improvement for all sites. We report on average, the mean absolute percentage error for global horizontal irradiance and direct normal irradiance show reductions of 2.16 and 3.95 percentage points respectively for all daylight conditions, 5.92 and 17.39 percentage points respectively for cloudy conditions, and 9.00 and 22.59 percentage points respectively for gap-filled cloudy conditions. These new methods will help improve the quality and accuracy of the irradiance and cloud data in the NSRDB.

14 SOLAR ENERGY↗

Perovskite neural trees

Trees are used by animals, humans and machines to classify information and make decisions. Natural tree structures displayed by synapses of the brain involves potentiation and depression capable of branching and is essential for survival and learning. Demonstration of such features in synthetic matter is challenging due to the need to host a complex energy landscape capable of learning, memory and electrical interrogation. We report experimental realization of tree-like conductance states at room temperature in strongly correlated perovskite nickelates by modulating proton distribution under high speed electric pulses. This demonstration represents physical realization of ultrametric trees, a concept from number theory applied to the study of spin glasses in physics that inspired early neural network theory dating almost forty years ago. We apply the tree-like memory features in spiking neural networks to demonstrate high fidelity object recognition, and in future can open new directions for neuromorphic computing and artificial intelligence.

36 MATERIALS SCIENCE↗

A Lie algebraic theory of barren plateaus for deep parameterized quantum circuits

Variational quantum computing schemes train a loss function by sending an initial state through a parametrized quantum circuit, and measuring the expectation value of some operator. Despite their promise, the trainability of these algorithms is hindered by barren plateaus (BPs) induced by the expressiveness of the circuit, the entanglement of the input data, the locality of the observable, or the presence of noise. Up to this point, these sources of BPs have been regarded as independent. In this work, we present a general Lie algebraic theory that provides an exact expression for the variance of the loss function of sufficiently deep parametrized quantum circuits, even in the presence of certain noise models. Our results allow us to understand under one framework all aforementioned sources of BPs. This theoretical leap resolves a standing conjecture about a connection between loss concentration and the dimension of the Lie algebra of the circuit’s generators.

97 MATHEMATICS AND COMPUTING↗

Analytical ab initio hessian from a deep learning potential for transition state optimization

Identifying transition states—saddle points on the potential energy surface connecting reactant and product minima—is central to predicting kinetic barriers and understanding chemical reaction mechanisms. In this work, we train a fully differentiable equivariant neural network potential, NewtonNet, on thousands of organic reactions and derive the analytical Hessians. By reducing the computational cost by several orders of magnitude relative to the density functional theory (DFT) ab initio source, we can afford to use the learned Hessians at every step for the saddle point optimizations. We show that the full machine learned (ML) Hessian robustly finds the transition states of 240 unseen organic reactions, even when the quality of the initial guess structures are degraded, while reducing the number of optimization steps to convergence by 2–3× compared to the quasi-Newton DFT and ML methods. All data generation, NewtonNet model, and ML transition state finding methods are available in an automated workflow.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Off-the-shelf deep learning is not enough, and requires parsimony, Bayesianity, and causality

Abstract Deep neural networks (‘deep learning’) have emerged as a technology of choice to tackle problems in speech recognition, computer vision, finance, etc. However, adoption of deep learning in physical domains brings substantial challenges stemming from the correlative nature of deep learning methods compared to the causal, hypothesis driven nature of modern science. We argue that the broad adoption of Bayesian methods incorporating prior knowledge, development of solutions with incorporated physical constraints and parsimonious structural descriptors and generative models, and ultimately adoption of causal models, offers a path forward for fundamental and applied research.

97 MATHEMATICS AND COMPUTING↗

Machine learning surrogates for surface complexation model of uranium sorption to oxides

Abstract The safety assessments of the geological storage of spent nuclear fuel require understanding the underground radionuclide mobility in case of a leakage from multi-barrier canisters. Uranium, the most common radionuclide in non-reprocessed spent nuclear fuels, is immobile in reduced form (U(IV) and highly mobile in an oxidized state (U(VI)). The latter form is considered one of the most dangerous environmental threats in the safety assessments of spent nuclear fuel repositories. The sorption of uranium to mineral surfaces surrounding the repository limits their mobility. We quantify uranium sorption using surface complexation models (SCMs). Unfortunately, numerical SCM solvers often encounter convergence problems due to the complex nature of convoluted equations and correlations between model parameters. This study explored two machine learning surrogates for the 2-pK Triple Layer Model of uranium retention by oxide surfaces if released as U(IV) in the oxidizing conditions: random forest regressor and deep neural networks. Our surrogate models, particularly DNN, accurately reproduce SCM model predictions at a fraction of the computational cost without any convergence issues. The safety assessment of spent fuel repositories, specifically the migration of leaked radioactive waste, will benefit from having ultrafast AI/ML surrogates for the computationally expensive sorption models that can be easily incorporated into larger-scale contaminant migration models. One such model is presented here.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Energy efficient photonic memory based on electrically programmable embedded III-V/Si memristors: switches and filters

Abstract Over the past few years, extensive work on optical neural networks has been investigated in hopes of achieving orders of magnitude improvement in energy efficiency and compute density via all-optical matrix-vector multiplication. However, these solutions are limited by a lack of high-speed power power-efficient phase tuners, on-chip non-volatile memory, and a proper material platform that can heterogeneously integrate all the necessary components needed onto a single chip. We address these issues by demonstrating embedded multi-layer HfO 2 /Al 2 O 3 memristors with III-V/Si photonics which facilitate non-volatile optical functionality for a variety of devices such as Mach-Zehnder Interferometers, and (de-)interleaver filters. The Mach-Zehnder optical memristor exhibits non-volatile optical phase shifts > π with ~33 dB signal extinction while consuming 0 electrical power consumption. We demonstrate 6 non-volatile states each capable of 4 Gbps modulation. (De-) interleaver filters were demonstrated to exhibit memristive non-volatile passband transformation with full set/reset states. Time duration tests were performed on all devices and indicated non-volatility up to 24 hours and beyond. We demonstrate non-volatile III-V/Si optical memristors with large electric-field driven phase shifts and reconfigurable filters with true 0 static power consumption. As a result, co-integrated photonic memristors offer a pathway for in-memory optical computing and large-scale non-volatile photonic circuits.

Cheung, Stanley (ORCID:0000000248860013)↗

Electrical coupled Morris-Lecar neurons: From design to pattern analysis

In this study, an experimental electronic neuron based on Morris-Lecar model is presented, able to become an experimental unit tool to study collective association of robust coupled neurons. The circuit design is given according to the ionic currents of this model. A weak coupling of such neurons under Multisim Software can generate clusters based on the boundary conditions of the neurons and their initial conditions. For this study, we work in the region close to the fold bifurcation of limit cycles. In this region two limit cycles exist, one of the cycles is stable and another one is unstable.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Accelerating data acquisition with FPGA-based edge machine learning: a case study with LCLS-II

New scientific experiments and instruments generate vast amounts of data that need to be transferred for storage or further processing, often overwhelming traditional systems. Edge machine learning (EdgeML) addresses this challenge by integrating machine learning (ML) algorithms with edge computing, enabling real-time data processing directly at the point of data generation. EdgeML is particularly beneficial for environments where immediate decisions are required, or where bandwidth and storage are limited. In this paper, we demonstrate a high-speed configurable ML model in a fully customizable EdgeML system using a field programmable gate array (FPGA). Our demonstration focuses on an angular array of electron spectrometers, referred to as the ‘CookieBox,’ developed for the Linac Coherent Light Source II project. The EdgeML system captures 51.2 Gbps from a 6.4 GS s −1 analog to digital converter and is designed to integrate data pre-processing and ML inside an FPGA. Our implementation achieves an inference latency of 0.2 µs for the ML model, and a total latency of 0.4 µs for the complete EdgeML system, which includes pre-processing, data transmission, digitization, and ML inference. The modular design of the system allows it to be adapted for other instrumentation applications requiring low-latency data processing.

97 MATHEMATICS AND COMPUTING↗

Scalable Topological Data Analysis and Visualization for Evaluating Data-Driven Models in Scientific Applications

With the rapid adoption of machine learning techniques for large-scale applications in science and engineering comes the convergence of two grand challenges in visualization. First, the utilization of black box models (e.g., deep neural networks) calls for advanced techniques in exploring and interpreting model behaviors. Second, the rapid growth in computing has produced enormous datasets that require techniques that can handle millions or more samples. Although some solutions to these interpretability challenges have been proposed, they typically do not scale beyond thousands of samples, nor do they provide the high-level intuition scientists are looking for. Here, we present the first scalable solution to explore and analyze high-dimensional functions often encountered in the scientific data analysis pipeline. By combining a new streaming neighborhood graph construction, the corresponding topology computation, and a novel data aggregation scheme, namely topology aware datacubes, we enable interactive exploration of both the topological and the geometric aspect of high-dimensional data. Following two use cases from high-energy-density (HED) physics and computational biology, we demonstrate how these capabilities have led to crucial new insights in both applications.

97 MATHEMATICS AND COMPUTING↗