Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Neural Networks (Computer)”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,207 records · Page 67

Analyzing rocket plume spectral data with neural networks

The Optical Plume Anomaly Detection (OPAD) system is under development to provide early-warning failure detection in support of ground-level testing of the Space Shuttle Main Engine (SSME). Failure detection is to be achieved through the acquisition of spectrally resolved plume emissions and subsequent identification of abnormal levels indicative of engine corrosion or component failure. Two computer codes (one linear and the other non-linear) are used by the OPAD system to iteratively determine specific element concentrations in the SSME plume, given emission intensity and wavelength information. Since this analysis is extremely labor intensive, a study was initiated to develop neural networks that would model the 'inverse' of these computer codes. Optimally connected feed-forward networks with imperceptible prediction error have been developed for each element modeled by the linear code, SPECTRA4. Radial basis function networks were developed for the non-linear code, SPECTRA5, and predict combustion temperature in addition to element concentrations.

Whitaker, Kevin W.↗

Deep Learning-Assisted Real-Time Forward Modeling of Electromagnetic Logging in Complex Formations

Higher dimensional (i.e., 2-D and 3-D) modeling is indispensable to correctly evaluate the responses of electromagnetic logging tools in complex formation environments. However, limited by the high computational cost of rigorous modeling such as finite difference method and the finite element method, the real-time applications in the well logging industry primarily rely on the 1-D forward solver, which would result in erroneous formation evaluation for complex scenarios. As a result, aiming at realizing fast modeling for electromagnetic logging tools in complex formations, this paper proposes a general framework assisted by deep neural networks (DNNs). The framework consists of three modules: earth model classification, parameter extraction, and surrogate construction. Separate DNNs are trained and tested for different modules. The accuracy and efficiency of the DNN assisted fast modeling are validated by several experiments. Here, this study finds that the fast modeling assisted by DNNs is able to calculate the tool responses and reconstruct the subsurface formations in real-time.

97 MATHEMATICS AND COMPUTING↗

TNet: A Model-Constrained Tikhonov Network Approach for Inverse Problems

Deep learning (DL), in particular deep neural networks, by default is purely data-driven and in general does not require physics. This is the strength of DL but also one of its key limitations when applied to science and engineering problems in which underlying physical properties—such as stability, conservation, and positivity—and accuracy are required. DL methods in their original forms are often not capable of respecting the underlying mathematical models or achieving desired accuracy even in big-data regimes. On the other hand, many data-driven science and engineering problems, such as inverse problems, typically have limited experimental or observational data, and DL would overfit the data in this case. Leveraging information encoded in the underlying mathematical models, we argue, not only compensates for missing information in low data regimes but also provides opportunities to equip DL methods with the underlying physics, hence promoting better generalization. This paper develops a model-constrained DL approach and its variant TNet—a Tikhonov neural network—which are capable of learning not only information hidden in the training data but also in the underlying mathematical models to solve inverse problems governed by partial differential equations in low data regimes. We provide the constructions and some theoretical results for the proposed approaches for both linear and nonlinear inverse problems. Since TNet is designed to learn inverse solutions with Tikhonov regularization, it is interpretable: in fact it recovers Tikhonov solutions for linear cases while potentially approximating Tikhonov solutions for nonlinear inverse problems. We also prove that data randomization can enhance not only the smoothness of the networks but also their generalizations. Comprehensive numerical results confirm the theoretical findings and show that with even as little as 1 training data sample for one-dimensional (1D) deconvolution, 5 for an inverse 2D heat conductivity problem, 100 for inverse initial conditions for a time-dependent 2D Burgers’s equation, and 50 for inverse initial conditions for 2D Navier–Stokes equations, TNet solutions can be as accurate as Tikhonov solutions while being several orders of magnitude faster. Furthermore, this is possible owing to the model-constrained term, replications, and randomization.

97 MATHEMATICS AND COMPUTING↗

Differentiable Neural Architecture, Mixed Precision and Accelerator Co-Search

Quantization, effective Neural Network architecture, and efficient accelerator hardware are three important design paradigms to maximize accuracy and efficiency. Mixed Precision Quantization is a process of assigning different precision to different Neural Network layers for optimized inference. Neural Architecture Search (NAS) is a process of automatically designing the neural network for a task and can also be extended to search for the precision of each weight and activation matrix. In this paper, we develop the following three methods: (i) Fast Differentiable Hardware-aware Mixed Precision Quantization Search method to find optimal precision, (ii) Joint Differentiable hardware-aware Architecture and Mixed Precision Quantization Co-search, (iii) Joint Accelerator, Architecture, and Precision triple co-search to find best possibilities in all the three worlds. We demonstrate the effectiveness of our proposed methods targeting Bitfusion accelerator by searching mixed precision models on MobilenetV2. We achieve better accuracy-latency trade-off models than the manually designed and previously proposed search methods.

97 MATHEMATICS AND COMPUTING↗

Knowledge-guided learning with curated prior genetic biomarkers for robust model interpretation

Abstract Motivation Knowledge-guided learning offers effective and robust model training strategies in data-scarce settings by incorporating established domain knowledge, thereby enhancing generalization, robustness, and interpretability. By contrast, conventional deep learning approaches rely purely on data-driven learning, which can limit robust model interpretability, particularly in high-dimensional settings with limited size samples. In computational biology, knowledge-guided learning has primarily leveraged network- and structural-based knowledge, leading to biologically interpretable representations and enhanced predictive performance compared to conventional approaches. However, curated biomarkers, one of the most accessible forms of biological knowledge, remain largely unexplored within knowledge-guided paradigms. Results In this study, we propose a model-agnostic training paradigm, Biomarker-driven Explainable Prior-guided Learning (BioExPL), that can be applied to any neural networks that incorporates curated prior knowledge. BioExPL enforces neural networks to reflect curated biomarker priors in their latent representations through a novel knowledge-alignment loss. BioExPL consistently demonstrated significantly improved predictive performance and enhanced model interpretability with minimized computational overhead in simulation studies and intensive experiments on multiple cancer datasets. BioExPL not only integrates prior curated knowledge into the model but also accurately identifies unknown associated signals additionally. BioExPL is model-agnostic and domain-independent, enabling its integration into diverse neural network architectures. Availability and implementation The open-source is publicly available at: https://github.com/datax-lab/BioExPL.

Baek, Beomsu [Department of Computer Science, Univ↗

The expressivity of classical and quantum neural networks on entanglement entropy

Abstract Analytically continuing the von Neumann entropy from Rényi entropies is a challenging task in quantum field theory. While then-th Rényi entropy can be computed using the replica method in the path integral representation of quantum field theory, the analytic continuation can only be achieved for some simple systems on a case-by-case basis. In this work, we propose a general framework to tackle this problem using classical and quantum neural networks with supervised learning. We begin by studying several examples with known von Neumann entropy, where the input data is generated by representing$${\text {Tr}}\rho _A^n$$ Tr ρ A n with a generating function. We adopt KerasTuner to determine the optimal network architecture and hyperparameters with limited data. In addition, we frame a similar problem in terms of quantum machine learning models, where the expressivity of the quantum models for the entanglement entropy as a partial Fourier series is established. Our proposed methods can accurately predict the von Neumann and Rényi entropies numerically, highlighting the potential of deep learning techniques for solving problems in quantum information theory.

Physics↗

Integrated data-driven and experimental approaches to accelerate lead optimization targeting SARS-CoV- 2 main protease

Identification of potential therapeutic candidates can be expedited by integrating computational modeling with domain aware machine learning (ML) approaches followed by experimental validation. Generative deep learning models have been recently developed that can generate thousands of new candidates, but their physiochemical properties are typically not optimized. Using our deep learning models and a scaffold as a starting point, we generated tens of thousands of compounds for SARS-CoV-2 M pro that preserve the core scaffold. Here we utilized and implemented several computational tools such as structural alert and toxicity analysis, high throughput virtual screening, ML-based 3D quantitative structure–activity relationships, multi-parameter optimization, and graph neural networks on libraries of generated candidates to predict biological activity and binding affinity a priori. From these collective computational results, eight promising candidates were identified and tested experimentally using Native Mass Spectrometry (MS) and FRET-based functional assays. Two compounds, with quinazoline-2-thiol and acetylpiperidine core moiety showed IC 50 values in the low micromolar range: 2.95±0.0017 µM and 3.41±0.0015 µM, respectively. The molecular dynamics simulations further highlight that binding of these compounds results in allosteric modulations in the chain B and the interface domains of the M pro . The key fragments from these top hits can be used as input for closed loop lead optimization in the integrated pipeline.

60 APPLIED LIFE SCIENCES↗

Real-time X-ray phase-contrast imaging using SPINNet—a speckle-based phase-contrast imaging neural network

X-ray phase-contrast imaging has become indispensable for visualizing samples with low absorption contrast. In this regard, speckle-based techniques have shown significant advantages in spatial resolution, phase sensitivity, and implementation flexibility compared with traditional methods. However, the computational cost associated with data inversion has hindered their wider adoption. By exploiting the power of deep learning, we developed a speckle-based phase-contrast imaging neural network (SPINNet) that significantly improves the imaging quality and boosts the phase retrieval speed by at least 2 orders of magnitude compared to existing methods. To achieve this performance, we combined SPINNet with a coded-mask-based technique, an enhanced version of the speckle-based method. Using this scheme, we demonstrate the simultaneous reconstruction of absorption and phase images on the order of 100 ms, where a traditional correlation-based analysis would take several minutes even with a cluster. In addition to significant improvement in speed, our experimental results show that the imaging and phase retrieval quality of SPINNet outperform existing single-shot speckle-based methods. Furthermore, we successfully demonstrate SPINNet application in x-ray optics metrology and 3D x-ray phase-contrast tomography. Our result shows that SPINNet could enable many applications requiring high-resolution and fast data acquisition and processing, such as in situ and in operando 2D and 3D phase-contrast imaging and real-time at-wavelength metrology and wavefront sensing.

36 MATERIALS SCIENCE↗

Learning Constitutive Relations From Soil Moisture Data via Physically Constrained Neural Networks

Abstract The constitutive relations of the Richardson‐Richards equation encode the macroscopic properties of soil water retention and conductivity. These soil hydraulic functions are commonly represented by models with a handful of parameters. The limited degrees of freedom of such soil hydraulic models constrain our ability to extract soil hydraulic properties from soil moisture data via inverse modeling. We present a new free‐form approach to learning the constitutive relations using physically constrained neural networks. We implemented the inverse modeling framework in a differentiable modeling framework, JAX, to ensure scalability and extensibility. For efficient gradient computations, we implemented implicit differentiation through a nonlinear solver for the Richardson‐Richards equation. We tested the framework against synthetic noisy data and demonstrated its robustness against varying magnitudes of noise and degrees of freedom of the neural networks. We applied the framework to soil moisture data from an upward infiltration experiment and demonstrated that the neural network‐based approach was better fitted to the experimental data than a parametric model and that the framework can learn the constitutive relations.

54 ENVIRONMENTAL SCIENCES↗

Understanding Proton Movement in [Fe-Fe] Hydrogenases

Nature uses specialized metalloenzymes to carry out small molecule activation reactions, including CO 2 fixation, O 2 activation, and proton reduction, with unparalleled efficiency, rates, and selectivity. The latter reactions are performed by hydrogenases, protein metallo-complexes that interconverts H 2 to protons and electrons (H 2 oxidation) and the reverse reaction (H 2 production) with incredibly low energy input and amazingly fast kinetics. Reproducing both the activity and efficiency of metalloenzymes in sustainable anthropogenic systems remains one of the “holy grails” of inorganic chemistry. However, identifying the precise molecular components responsible for these desirable properties has been challenging in the natural metalloenzymes, hindering efforts to develop analogous processes in synthetic compounds. Considering the inherent complexity of a metalloenzyme and the many interactions, both strong and weak, that contribute to the function of an enzyme, we have elected to model natural metalloenzymes on a biochemical platform. Towards this end, we have a developed structural, functional, and mechanistic mimic of the [Ni-Fe] hydrogenases within a robust protein scaffold, rubredoxin, to understand the influence of the secondary coordination environment on the metal center. This involved preparing a series of three rubredoxin constructs containing a single point mutation at Val positions and making the NMR chemical shift assignments for the paramagnetic (nickel-substituted) and non-paramagnetic (zinc-substituted) form. These physical studies were complemented with in silico molecular dynamic studies on proteins with [Fe-S] clusters to engineer new proton channels to test in vitro . To assist our search for new natural metalloprotein scaffolds within the vast number of sequenced genomes, we created a neural network-based program to identify proteins with specific metal-binding sites. These computational and physical studies with metalloenzymes provide direct insight into the fundamental chemical principles driving the natural systems and offer design principles for developing catalysts that utilize analogous principles.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Autoencoder node saliency: Selecting relevant latent representations

The autoencoder is an artificial neural network that performs nonlinear dimension reduction and learns hidden representations of unlabeled data. With a linear transfer function it is similar to the principal component analysis (PCA). While both methods use weight vectors for linear transformations, the autoencoder does not come with any indication similar to the eigenvalues in PCA that are paired with eigenvectors. In this work, we propose a novel autoencoder node saliency method that examines whether the features constructed by autoencoders exhibit properties related to known class labels. The supervised node saliency ranks the nodes based on their capability of performing a learning task. It is coupled with the normalized entropy difference (NED). We establish a property for NED values to verify classifying behaviors among the top ranked nodes. Lastly, by applying our methods to real datasets, we demonstrate their ability to provide indications on the performing nodes and explain the learned tasks in autoencoders.

97 MATHEMATICS AND COMPUTING↗

Real-Time and Adaptive Reservoir Computing With Application to Profile Prediction in Fusion Plasma

Nuclear fusion is a promising alternative to address the problem of sustainable energy production. The tokamak is an approach to fusion based on magnetic plasma confinement, constituting a complex physical system with many control challenges. Here we study the characteristics and optimization of reservoir computing (RC) for real-time and adaptive prediction of plasma profiles in the DIII-D tokamak. Our experiments demonstrate that RC achieves comparable results to state-of-the-art (deep) convolutional neural networks (CNNs) and long short-term memory (LSTM) models, with a significantly easier and faster training procedure. This efficient approach allows for fast and frequent adaptation of the model to new situations, such as changing plasma conditions or different fusion devices.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Deep learning to estimate permeability using geophysical data

Time-lapse electrical resistivity tomography (ERT) is a popular geophysical method to estimate three-dimensional (3D) permeability fields from electrical potential difference measurements. Traditional inversion and data assimilation methods are used to ingest this ERT data into hydrogeophysical models to estimate permeability. Due to ill-posedness and the curse of dimensionality, existing inversion strategies provide poor estimates and low resolution of the 3D permeability field. Recent advances in deep learning provide us with powerful algorithms to overcome this challenge. This paper presents a deep learning (DL) framework to estimate the 3D subsurface permeability from time-lapse ERT data. To test the feasibility of the proposed framework, we train DL-enabled inverse models on simulation data. Each measurement in both synthetic and field data is standardized by removing the mean and scaling the time-series to unit variance. This pre-processing step is necessary to bring simulation data closer to field observations. Subsurface process models based on hydrogeophysics are used to generate this synthetic data. Training performed on limited simulation data resulted in the DL model over-fitting. An advanced data augmentation based on mixup is implemented to generate additional training samples to overcome this issue. This mixup technique creates weakly labeled (low-fidelity) samples from strongly labeled (high-fidelity) data. The weakly labeled training data is then used to develop DL-enabled inverse models and reduce over-fitting. As both time-lapse ERT (1133048 features/realization) and 3D permeability (585453 features/realization) data samples are from a high-dimensional space, principal component analysis (PCA) is employed to reduce dimensionality. Encoded ERT and encoded permeability are generated using the trained PCA estimators. A deep neural network is then trained to map the encoded ERT to encoded permeability. This mixup training and unsupervised learning allowed us to build a fast and reasonably accurate DL-based inverse model under limited simulation data. Results show that proposed weak supervised learning can capture salient spatial features in the 3D permeability field. Quantitatively, the average mean squared error (in terms of the natural log) on the strongly labeled training, validation, and test datasets is less than 0.5. The R 2 -score (global metric) is greater than 0.75, and the percent error in each cell (local metric) is less than 10%. Finally, an added benefit in terms of computational cost is that the proposed DL-based inverse model is at least O(10 4 ) times faster than running a forward model once it is trained. Data generation, DL model training, and hyperparameter tuning to identify optimal neural network architectures utilized high-performance computing resources while the DL inference is performed on a standard laptop. Approximately, O(10 5 ) processor hours are used for generating data and DL tuning and training. We acknowledge that the data generation and DL model development are expensive. But once a DL model is trained, it can be re-used for inversion rapidly for the given system, with set physics and domain. Note that traditional inversion may require multiple forward model simulations (e.g., in the order of 10 to 1000), which are very expensive. This computational savings ≈ O(10 5 ) – O(10 7 )) makes the proposed DL-based inverse model attractive for subsurface imaging and real-time ERT monitoring applications due to fast and yet reasonably accurate estimations of permeability field.

58 GEOSCIENCES↗

Modeling MTS pyrolysis and SiC deposition kinetics using principal component analysis and neural networks

Accurate chemical kinetics modeling is crucial for improving the efficiency of chemical processing and synthesis of ceramic matrix composites. Detailed kinetic models are computationally expensive due to the large number of transported chemical species, while the simplified physics-based models, such as single-step global mechanisms, are efficient but often overlook key chemical intermediates and pathways. Recent deep learning approaches promise accurate and cost-effective models. Yet, they require additional closures for the transported nonlinear latent variables, complicating integration with existing solvers. In this work, we develop a hybrid linear—nonlinear reduced model for silicon carbide deposition from methyltrichlorosilane precursor by combining principal component analysis (PCA) and autoencoder (AE) neural network (NN) approaches. PCA is used to identify a smaller set of linear transport variables, enabling direct reuse of conventional transport solvers. NNs then reconstruct the full chemical state from these reduced variables. We demonstrate the method on a chemical vapor deposition reactor—comprising a gas-phase pyrolysis plug flow reactor and a heterogeneous surface reactor—over a wide range of temperatures, pressures, and residence times. Our PCA–AE model achieves high accuracy with only five transported scalars, achieving an eightfold cost reduction compared to detailed mechanisms, in both a priori (using data from the test set only) and a posteriori (coupled with a differential equation solver). In conclusion, notable errors arise primarily near training domain boundaries and for long residence times, indicating the need for domain shift indicators and better long-horizon predictions in future reduced chemistry model development.

autoencoder neural networks↗

Neural network methods for radiation detectors and imaging

Recent advances in image data proccesing through deep learning allow for new optimization and performance-enhancement schemes for radiation detectors and imaging hardware. This enables radiation experiments, which includes photon sciences in synchrotron and X-ray free electron lasers as a subclass, through data-endowed artificial intelligence. We give an overview of data generation at photon sources, deep learning-based methods for image processing tasks, and hardware solutions for deep learning acceleration. Most existing deep learning approaches are trained offline, typically using large amounts of computational resources. However, once trained, DNNs can achieve fast inference speeds and can be deployed to edge devices. A new trend is edge computing with less energy consumption (hundreds of watts or less) and real-time analysis potential. While popularly used for edge computing, electronic-based hardware accelerators ranging from general purpose processors such as central processing units (CPUs) to application-specific integrated circuits (ASICs) are constantly reaching performance limits in latency, energy consumption, and other physical constraints. These limits give rise to next-generation analog neuromorhpic hardware platforms, such as optical neural networks (ONNs), for high parallel, low latency, and low energy computing to boost deep learning acceleration (LA-UR-23-32395).

edge computing↗

Computational scanning tunneling microscope image database

We introduce the systematic database of scanning tunneling microscope (STM) images obtained using density functional theory (DFT) for two-dimensional (2D) materials, calculated using the Tersoff-Hamann method. It currently contains data for 716 exfoliable 2D materials. Examples of the five possible Bravais lattice types for 2D materials and their Fourier-transforms are discussed. All the computational STM images generated in this work are made available on the JARVIS-STM website ( https://jarvis.nist.gov/jarvisstm ). We find excellent qualitative agreement between the computational and experimental STM images for selected materials. As a first example application of this database, we train a convolution neural network model to identify the Bravais lattice from the STM images. We believe the model can aid high-throughput experimental data analysis. These computational STM images can directly aid the identification of phases, analyzing defects and lattice-distortions in experimental STM images, as well as be incorporated in the autonomous experiment workflows.

47 OTHER INSTRUMENTATION↗

Multiphysics Meshfree Degradation Modeling of Energy Storage Materials with Kernel Enrichment

Energy storage materials exhibit strong electro-chemo-mechanical coupling and highly anisotropic material properties, contributing to the formation and propagation of micro-cracking during charge/discharge cycling and ultimately diminishing performance and service life. With microstructural images supplied by the National Renewable Energy Laboratory (NREL), pixel-based meshfree model construction by the reproducing kernel particle method (RKPM) is used to represent the complex material microstructures that dictate the coupled physics of these systems. Traditional electro-chemo-mechanical models rely on mesh-based finite element methods, which can lead to difficulties in meshing such complex geometries and capturing crack propagation due to mesh dependency. The first kernel enrichment discussed will be the interface modified reproducing kernel (IM-RK) [1, 2], constructed by scaling a smooth kernel function with an interface-distance function to achieve strategic discontinuity types (i.e. weak discontinuities for strain discontinuities and strong discontinuities for cracks) and alleviate Gibbs oscillations near these transition zones. The IM-RK is especially useful for areas in which a known discontinuity-type is expected a priori. The second kernel enrichment to be discussed is a neural network-enhanced reproducing kernel (NN-RK) [3, 4], which is introduced to effectively model non-obvious damage and crack propagation in the material microstructures; the location, orientation, and solution transition near a localization are automatically captured by superimposed block-level NN optimizations. This NN enrichment approach allows for effective modeling of localizations via a fixed background discretization, relieving tedious efforts for adaptive refinement in traditional mesh-based methods. Applications to the heterogeneous microstructures of Li-ion battery cathodes will be presented to demonstrate the effectiveness of the proposed methods. NN-RK is additionally used to inform how crack opening and closure in turn affect the electro-chemo-mechanical responses in the material microstructure. Reference: [1] Wang, Y., Baek, J., Tang, Y. et al. "Support vector machine guided reproducing kernel particle method for image-based modeling of microstructures," Comput Mech 73, 907-942 (2024). https://doi.org/10.1007/s00466-023-02394-9. [2] Susuki, K., Allen, J. & Chen, J. S.. "Image-based modeling of coupled electro-chemo-mechanical behavior of Li-ion battery cathode using an interface-modified reproducing kernel particle method," Engineering with Computers (2024). https://doi.org/10.1007/s00366-024-02016-9. [3] Baek, J., Chen, J. S., Susuki, K., "Neural Network enhanced Reproducing Kernel Particle Method for Modeling Localizations," International Journal for Numerical Methods in Engineering, Vol. 123, 4422-4454 (2022). https://doi.org/10.1002/nme.7040.

25 ENERGY STORAGE↗