Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Generative Neural Networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Smart Pixels: In-pixel AI for on-sensor data filtering

We present a smart pixel prototype readout integrated circuit (ROIC) designed in CMOS 28 nm bulk process, with in-pixel implementation of an artificial intelligence (AI) / machine learning (ML) based data filtering algorithm designed as proof-of-principle for a Phase III upgrade at the Large Hadron Collider (LHC) pixel detector. The first version of the ROIC consists of two matrices of 256 smart pixels, each 25$\times$25 $\mu$m$^2$ in size. Each pixel consists of a charge-sensitive preamplifier with leakage current compensation and three auto-zero comparators for a 2-bit flash-type ADC. The frontend is capable of synchronously digitizing the sensor charge within 25 ns. Measurement results show an equivalent noise charge (ENC) of $\sim$30e$^-$ and a total dispersion of $\sim$100e$^-$ The second version of the ROIC uses a fully connected two-layer neural network (NN) to process information from a cluster of 256 pixels to determine if the pattern corresponds to highly desirable high-momentum particle tracks for selection and readout. The digital NN is embedded in-between analog signal processing regions of the 256 pixels without increasing the pixel size and is implemented as fully combinatorial digital logic to minimize power consumption and eliminate clock distribution, and is active only in the presence of an input signal. The total power consumption of the neural network is $\sim$ 300 $\mu$W. The NN performs momentum classification based on the generated cluster patterns and even with a modest momentum threshold, it is capable of 54.4% - 75.4% total data rejection, opening the possibility of using the pixel information at 40MHz for the trigger. The total power consumption of analog and digital functions per pixel is $\sim$ 6 $\mu$W per pixel, which corresponds to $\sim$ 1 W/cm$^2$ staying within the experimental constraints.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

3D_MolGNN_RL

3D-MolGNNRL, couples reinforcement learning (RL) to a deep generative model based on 3D-Scaffold to generate target candidates specific to a protein pocket building up atom by atom from the core scaffold. 3D-MolGNNRL provides an efficient way to optimize key features within a protein pocket using a parallel graph neural network model. The agent learns to build molecules in 3D space while optimizing the binding affinity, potency, and synthetic accessibility of the candidates generated for the SARS-CoV-2 Main protease

Kumar, Neeraj↗
DOE OSTI2020

Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations

We developed a new class of physics-informed generative adversarial networks (PI-GANs) to solve forward, inverse, and mixed stochastic problems in a unified manner based on a limited number of scattered measurements. Unlike standard GANs relying solely on data for training, here we encoded into the architecture of GANs the governing physical laws in the form of stochastic differential equations (SDEs) using automatic differentiation. In particular, we applied Wasserstein GANs with gradient penalty (WGAN-GP) for its enhanced stability compared to vanilla GANs. We first tested WGAN-GP in approximating Gaussian processes of different correlation lengths based on data realizations collected from simultaneous reads at sparsely placed sensors. We obtained good approximation of the generated stochastic processes to the target ones even if there is a mismatch between the input noise dimensionality and the effective dimensionality of the target stochastic processes. We also studied the overfitting issue for both the discriminator and the generator, and we found that overfitting occurs also in the generator in addition to the discriminator as previously reported. Subsequently, we considered the solution of elliptic SDEs requiring approximations of three stochastic processes, namely the solution, the forcing, and the diffusion coefficient. Here again, we assumed data collected from simultaneous reads at a limited number of sensors for the multiple stochastic processes. Furthermore, three generators were used for the PI-GANs: two of them were feed forward deep neural networks (DNNs), while the other one was the neural network induced by the SDE. For the case where we have one group of data, we employed one feed forward DNN as the discriminator, while for the case of multiple groups of data we employed multiple discriminators in PI-GANs. We solved forward, inverse, and mixed problems without changing the framework of PI-GANs, obtaining both the means and the standard deviations of the stochastic solution and the diffusion coefficient in good agreement with benchmarks. In this work, we have demonstrated the effectiveness of PI-GANs in solving SDEs for about 120 dimensions. In principle, PI-GANs could tackle very high dimensional problems given more sensor data with low-polynomial growth in computational cost.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Leveraging dendritic complexity for neuromorphic computing

Abstract Beyond-von Neumann computing approaches are necessary to sustain the growth of microelectronics and the increasing appetite for artificial intelligence/machine learning algorithms. Neuromorphic computing is an emerging paradigm that takes inspiration from the brain to provide a path forward to improve the computational efficiency and computational density of next-generation computing architectures. In nature, we observe brains performing complex computations with a much smaller energy footprint than conventional computing approaches. Current neuromorphic systems are focused primarily on scalability, namely, increasing the number of computational units (neurons) and connections between units (synapses). However, for brain-like cognition and efficiency in next-generation computing hardware, we need increased complexity in function, as well as improved connection density for scalability. Here, we present our work that aims to incorporate dendrites for ‘compute-on-wire’ in neuromorphic architectures to increase the computational complexity (e.g. number of programmable parameters, nonlinear dynamics) as well as computational efficiency (energy/compute) of artificial neural networks (ANNs). We do this by showcasing neuromorphic dendrite elements that can be leveraged for various applications. We will present examples of neuroscience-inspired direction-selective circuits and an ANN with active dendrites leveraging shunting inhibition. We also demonstrate the benefits of using dendrites in deep neural networks. To conclude, we discuss how we can utilize emerging hardware devices in these systems and design next-generation neuromorphic architectures with dendrites.

Cardwell, Suma G. (ORCID:0000000226575545)↗
DOE OSTI2022

Discovering faint and high apparent motion rate near-Earth asteroids using a deep learning program

ABSTRACT Although many near-Earth objects have been found by ground-based telescopes, some fast-moving ones, especially those near detection limits, have been missed by observatories. We developed a convolutional neural network for detecting faint fast-moving near-Earth objects. It was trained with artificial streaks generated from simulations and was able to find these asteroid streaks with an accuracy of 98.7 per cent and a false positive rate of 0.02 per cent on simulated data. This program was used to search image data from the Zwicky Transient Facility (ZTF) in four nights in 2019, and it identified six previously undiscovered asteroids. The visual magnitudes of our detections range from ∼19.0 to 20.3 and motion rates range from ∼6.8 to 24 deg d−1, which is very faint compared to other ZTF detections moving at similar motion rates. Our asteroids are also ∼1–51 m diameter in size and ∼5–60 lunar distances away at close approach, assuming their albedo values follow the albedo distribution function of known asteroids. The use of a purely simulated data set to train our model enables the program to gain sensitivity in detecting faint and fast-moving objects while still being able to recover nearly all discoveries made by previously designed neural networks which used real detections to train neural networks. Our approach can be adopted by any observatory for detecting fast-moving asteroid streaks.

Wang, Franklin (ORCID:0000000344887355)↗
DOE OSTI2023

Development of a hybrid neural network and transfer learning model for optimized ICP-MS/MS operation

Correct function and calibration of instrumentation is a crucial assumption for any scientific experiment. One such instrument, tandem inductively coupled plasma mass spectrometer (ICP-MS/MS), has in-depth calibration settings that range across 30+ different parameters, making it difficult to determine optimal conditions without expertise and some degree of trial and error. Often, these settings are hand-tuned, a time-intensive process prone to local maxima and human error. While some automation is available, the automation also may favor local optimizations over a global optimum. In addition to these difficulties, day to day instrument variability can further complicate the calibration process. We propose a solution to this problem as a machine learning (ML) algorithm that learns how each parameter helps determine the calibration sensitivity across several elements, and re-weights parameters over time as instrument variability changes (e.g., a global neural network (NN) with a time-dependent transfer learning (TL) component). This model would be able to generate a surface of predicted calibration sensitivities and their respective parameters, and a simple multivariate algorithm would be able to pull out the optimum results with the settings associated with them. Here-in, we describe our initial findings in working towards this goal, including data extraction from historical files, exploratory data analysis, and some initial model building to better describe the data and the feasibility of our goal.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Physics-Informed Recurrent Neural Networks to Predict Reactor Operations of the AGN-201 Nuclear Reactor

4 page paper submitted to ANS Student conference. Summary of paper similar to the following abstract: The ability to predict how a reactor will operate, understand when anomalous conditions arise, and ensure a reactor is being operated as expected is crucial for deploying new nuclear facilities. Digital twins serve as a unique solution to recognizing reactor behavior; however, they require data to be useful. For next-generation reactors, this data may not currently be available. To explore how synthetic physics-informed reactor data can be used to predict reactor operations, a recurrent neural network was implemented for the Idaho State University AGN-201 digital twin. The goal of this work is to determine how synthetic data can be used to train a recurrent neural network model for predicting the reactor power of the AGN-201. The recurrent neural network was validated using both synthetic and real operational data. We envision this approach will help bridge the gap between the virtual and physical sides of a digital twin, where reactor physics models based on as-built data can be corrected for actual operating parameters to ensure the virtual model mirrors reality.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2022

Invertible Neural Networks for Aerodynamic Design of Wind Turbine Blades

The state-of-the-practice methods for aerodynamic design of wind turbine blads use linearized blade element momentum theory (BEM) to optimize the twist and chord profiles from a pre-selected set of 2D airfoil shapes. In this work, we apply invertible neural network (INN) tools to enable the rapid inverse aerodynamic design of wind turbine blades including component airfoils. The INN is trained on data obtained through the use of robust automated mesh generation and the HAMSTR computational fluid dynamics solver with advanced turbulence and transition models validated for turbine applications. Our design technique is a significant improvement over the state-of-the-practice linearized blade element momentum (BEM) techniques in capturing 3D nonlinear aerodynamic effects that are critical for optimal design of the rotors. This is made possible by developing sparse, invertible neural networks (INNs) for inverse design and optimization that realize a 100x cost reduction compared to adjoint-based computational fluid dynamics (CFD) approaches, while enabling increased robustness of the final design. We demonstrate the INN tool for design of a section of the NREL 5-MW blade. All generated shapes satisfy the desired aerodynamic characteristics, demonstrating the success of the INN approach for inverse design of wind turbine blades.

aerodynamic design↗
DOE OSTI2020

Data-Driven Template Discovery Using Graph Convolutional Neural Networks

Modeling adversarial activities is a critical component of developing high-con?dence indicators of efforts to acquire, fabricate, proliferate, and/or deploy weapons of mass terror (WMTs). Current approaches to generating representative patterns of interest (a.k.a templates) from the real-world domains involve a Subject Matter Expert (SME)-guided manual process. The goal of Data-Driven Template Discovery (DDTD) is to use a (potentially small) set of SME generated templates to discover other previously unknown and interesting templates in an attributed graph. A template is an activity pattern describing a set of interactions among a group of nodes in the graph. The motivation behind DDTD is to expand the original set of templates, without having SMEs craft all the templates by hand. DDTD also provides seed templates to SMEs, to help them construct larger, high-?delity, and scenario-oriented templates. In these cases, obtaining a larger set of templates that are related (contain similar signals) to the original set is of great value. In this work, we propose to use Graph Convolutional Neural Networks (GCNs) to discover new templates that are heavily related to the original set. GCNs are a family of Neural Network (NN) architectures especially designed to work directly on graphs. In contrast to the traditional NNs, that require considerable amounts of labeled data, GCNs do not require a big labeled training set because they can directly leverage the graph structure instead. This property makes GCNs the perfect tool for creating activity templates.

Joaristi, Mikel↗
DOE OSTI2023

Fabrication-conscious neural network based inverse design of single-material variable-index multilayer films

Multilayer films with continuously varying indices for each layer have attracted great deal of attention due to their superior optical, mechanical, and thermal properties. However, difficulties in fabrication have limited their application and study in scientific literature compared to multilayer films with fixed index layers. In this work we propose a neural network based inverse design technique enabled by a differentiable analytical solver for realistic design and fabrication of single material variable-index multilayer films. This approach generates multilayer films with excellent performance under ideal conditions. We furthermore address the issue of how to translate these ideal designs into practical useful devices which will naturally suffer from growth imperfections. By integrating simulated systematic and random errors just as a deposition tool would into the optimization process, we demonstrated that the same neural network that produced the ideal device can be retrained to produce designs compensating for systematic deposition errors. Furthermore, the proposed approach corrects for systematic errors even in the presence of random fabrication imperfections. The results outlined in this paper provide a practical and experimentally viable approach for the design of single material multilayer film stacks for an extremely wide variety of practical applications with high performance.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Inverse Design of Photonic Surfaces via High throughput Femtosecond Laser Processing and Tandem Neural Networks

Abstract This work demonstrates a method to design photonic surfaces by combining femtosecond laser processing with the inverse design capabilities of tandem neural networks that directly link laser fabrication parameters to their resulting textured substrate optical properties. High throughput fabrication and characterization platforms are developed that generate a dataset comprising 35280 unique microtextured surfaces on stainless steel with corresponding measured spectral emissivities. The trained model utilizes the nonlinear one‐to‐many mapping between spectral emissivity and laser parameters. Consequently, it generates predominantly novel designs, which reproduce the full range of spectral emissivities (average root‐mean‐squared‐error < 2.5%) using only a compact region of laser parameter space 25 times smaller than what is represented in the training data. Finally, the inverse design model is experimentally validated on a thermophotovoltaic emitter design application. By synergizing laser‐matter interactions with neural network capabilities, the approach offers insights into accelerating the discovery of photonic surfaces, advancing energy harvesting technologies.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Demonstration of sub-micron UCN position resolution using room-temperature CMOS sensor

High spatial resolution of ultracold neutron (UCN) measurement is of growing interest to UCN experiments such as UCN spectrometers, UCN polarimeters, quantum physics of UCNs, and quantum gravity. Here we utilize physics informed deep learning to enhance the experimental position resolution and to demonstrate sub-micron spatial resolutions for UCN position measurements obtained using a room-temperature CMOS sensor, extending our previous work that demonstrated a position uncertainty of 1.5 microns. We explore the use of the open-source software Allpix Squared to generate experiment-like synthetic hit images with ground-truth position labels. We use physics-informed deep learning by training a fully connected neural network (FCNN) to learn a mapping from input hit images to output hit position. The automated analysis for sub-micron position resolution in UCN detection combined with the fast data rates of current and next generation UCN sources will enable improved precision for future UCN research and applications.

10B nanometer thin film↗
DOE OSTI2020

PANNA: Properties from Artificial Neural Network Architectures

We report prediction of material properties from first principles is often a computationally expensive task. Recently, artificial neural networks and other machine learning approaches have been successfully employed to obtain accurate models at a low computational cost by leveraging existing example data. Here, we present a software package “Properties from Artificial Neural Network Architectures” (PANNA) that provides a comprehensive toolkit for creating neural network models for atomistic systems following the Behler–Parrinello topology. Besides the core routines for neural network training, it includes data parser, descriptor builder for Behler–Parrinello class of symmetry functions and force-field generator suitable for integration within molecular dynamics packages. PANNA offers a variety of activation and cost functions, regularization methods, as well as the possibility of using fully-connected networks with custom size for each atomic species. PANNA benefits from the optimization and hardware-flexibility of the underlying TensorFlow engine which allows it to be used on multiple CPU/GPU/TPU systems, making it possible to develop and optimize neural network models based on large datasets.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Generative learning of densities on manifolds

A generative modeling framework is proposed that combines diffusion models and manifold learning to efficiently sample data densities on manifolds. The approach utilizes Diffusion Maps to uncover possible low-dimensional underlying (latent) spaces in the high-dimensional data (ambient) space. Two approaches for sampling from the latent data density are described. The first is a score-based diffusion model, which is trained to map a standard normal distribution to the latent data distribution using a neural network. The second one involves solving an Itô stochastic differential equation in the latent space. Additional realizations of the data are generated by lifting the samples back to the ambient space using Double Diffusion Maps , a recently introduced technique typically employed in studying dynamical system reduction; here the focus lies in sampling densities rather than system dynamics. The proposed approaches enable sampling high dimensional data densities restricted to low-dimensional, a priori unknown manifolds. The efficacy of the proposed framework is demonstrated through a benchmark problem and a material with multiscale structure.

Double diffusion maps↗
DOE OSTI2022

Long-time integration of parametric evolution equations with physics-informed DeepONets

Ordinary and partial differential equations (ODEs/PDEs) play a paramount role in analyzing and simulating complex dynamic processes across all corners of science and engineering. In recent years machine learning tools are aspiring to introduce new effective ways of simulating such equations, however existing approaches are not able to reliably return stable and accurate predictions across long temporal horizons. We aim to address this challenge by introducing an effective framework for learning evolution operators that map random initial conditions to associated ODE/PDE solutions within a short time interval. Such operators can be parametrized by deep neural networks that are trained in an entirely self-supervised manner without requiring one to generate any paired input-output observations. Global long-time predictions across a range of initial conditions can be then obtained by iteratively evaluating the trained model using each prediction as the initial condition for the next evaluation step. Here, this introduces a new approach to temporal domain decomposition that is shown to be effective in performing accurate long-time simulations for a wide range of parametric ODE and PDE systems, from wave propagation, to reaction-diffusion dynamics and stiff chemical kinetics, introducing a new way of rapidly emulating non-equilibrium processes in science and engineering.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Adsorbate Partition Functions via Phase Space Integration: Quantifying the Effect of Translational Anharmonicity on Thermodynamic Properties

A new method for computing anharmonic thermophysical properties for adsorbates on metal surfaces is presented. Classical Monte Carlo phase space integration is performed to calculate the partition function for the motion of a hydrogen atom on Cu(111). Here, a minima-preserving neural network potential energy surface is used within the integration routine. Two different sampling schema for generating the training data are presented, and two different density functionals are used. The results are benchmarked against direct state counting results by using discrete variable representation. The phase space integration results are in excellent quantitative agreement with the benchmark results. Additionally, both the discrete variable representation and the phase space integration results confirm that the motion of H on Cu(111) is highly anharmonic. The results were applied to calculate the free energy of dissociative adsorption of H 2 and the resulting Langmuir isotherms at 400, 800, and 1200 K in a partial pressure range of 0–1 bar. It shows that the anharmonic effects lead to significantly higher predicted surface site fractions of hydrogen.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

TrioSim: A Lightweight Simulator for Large-Scale DNN Workloads on Multi-GPU Systems

Deep Neural Networks (DNNs) have become increasingly capable of performing tasks ranging from image recognition to content generation. The training and inference of DNNs heavily rely on GPUs, as GPUs' massively parallel architecture delivers extremely high computing capability. With the growing complexity of DNNs and the size of training datasets, training DNNs with a large number of GPUs is becoming a prevalent strategy. Researchers have been exploring how to design software and hardware systems for GPU farms to achieve the best utilization, efficiency, and DNN accuracy during training or inference. However, when designing and deploying such systems, designers usually rely on testing on physical hardware platforms equipped with many GPUs, incurring high costs that are almost prohibitive for system designers to test different configurations and designs, even for highly resourceful companies. While an alternative solution is to test on GPU simulators, they are often too slow for these l

Li, Ying [William & Mary, Williamsburg, VA, USA] (↗
DOE OSTI2023

Machine Learning Benchmarks for the Classification of Equivalent Circuit Models from Electrochemical Impedance Spectra

Analysis of Electrochemical Impedance Spectroscopy (EIS) data for electrochemical systems often consists of defining an Equivalent Circuit Model (ECM) using expert knowledge and then optimizing the model parameters to deconvolute various resistance, capacitive, inductive, or diffusion responses. For small data sets, this procedure can be conducted manually; however, it is not feasible to manually define a proper ECM for extensive data sets with a wide range of EIS responses. Automatic identification of an ECM would substantially accelerate the analysis of large sets of EIS data. We showcase machine learning methods to classify the ECMs of 9,300 impedance spectra provided by QuantumScape for the BatteryDEV hackathon. The best-performing approach is a gradient-boosted tree model utilizing a library to automatically generate features, followed by a random forest model using the raw spectral data. A convolutional neural network using boolean images of Nyquist representations is presented as an alternative, although it achieves a lower accuracy. We publish the data and open source the associated code. The approaches described in this article can serve as benchmarks for further studies. A key remaining challenge is the identifiability of the labels, underlined by the model performances and the comparison of misclassified spectra.

25 ENERGY STORAGE↗