Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “neural encoding”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Super Resolving Unrolled Neural Networks for Remote Sensing

In remote sensing systems, the capabilities of the system are constrained by the complex interactions between size, weight, and power (SWAP) of potential designs. In electro-optical (EO) systems, examples of these critical parameters include the system’s sensitivity and resolution. Those parameters can be increased by ever larger optical apertures and focal planes but at the cost of more SWAP. Multi-image super resolution (MISR) techniques allow resolution to be enhanced via computation rather than more sophisticated optical hardware. These algorithms combine multiple images together into a single, higher resolution image, trading temporal resolution and computation for spatial resolution. Fielded MISR techniques, such as Drizzle, can require several hundred images to create a single super resolved image, implying reduced temporal resolution, increased data acquisition load, and limiting mission applications. Iterative techniques, such as model-based image reconstruction and compressive sensing, have been shown to create super resolved images using fewer images than Drizzle. They do this by posing an optimization problem that balances accuracy between a highly accurate physical model and an image model. In the case of super resolution, the physical model is defined by the relation between low resolution input images and the desired high resolution output image. The image model encodes some assumptions about the super resolved image. These assumptions are meant to suppress reconstruction artifacts that arise due to deterministic physical model error, stochastic measurement noise, and potential undersampling. In practice, the performance of iterative methods are limited by imaging models compatible with optimization. Deep learning-based methods can effectively learn image models of arbitrary complexity, but lack the theoretical explainability and robustness of iterative techniques. Consensus equilibrium (CE) generalizes the iterative techniques beyond optimization, enabling blackbox algorithms such as traditional and neural image denoisers to be used as the image model. CE-based approaches retain much of the explainability and robustness of iterative techniques while allowing the expressiveness of machine learning image models to be used. Additionally, by unrolling iterations of CE with an embedded image denoiser, the image denoiser can be further trained and specialized to the specific application with potentially higher quality reconstructions. Under this project, we demonstrated the feasibility of training an unrolled neural network based upon CE. While we didn’t train one, we showed that the CE process is differentiable and its gradient can be tractably computed. We also explored the usage of a variants of CE akin to generative neural works. Most importantly, we applied the CE framework to a number of problems including non-blind deconvolution, upsampling, single-image super resolution, MISR, event-based sensing, and saturated deconvolution. Our MISR prototype creates high quality reconstructions with an order of magnitude fewer images than previous approaches and, critically, produces these reconstructions fast enough for practical usage.

47 OTHER INSTRUMENTATION↗
DOE OSTI2020

Deep Ensemble Confidence Levels for Multi-hot Categorization

NOICE (Neural Optical Image Categorizer for the Elog) is a small collaboration tasked with categorizing the images in the Fermilab Accelerator Division electronic logbook by using Artificial Intelligence. To do so, we manually categorized a subset of the images in the E-log into nine independent labels. Each image was then multi-hot-encoded into a nine-dimensional binary vector because each image could have more then one label.

43 PARTICLE ACCELERATORS↗
DOE OSTI2024

Forecasting high-dimensional spatio-temporal systems from sparse measurements

This paper introduces a new neural network architecture designed to forecast high-dimensional spatio-temporal data using only sparse measurements. The architecture uses a two-stage end-to-end framework that combines neural ordinary differential equations (NODEs) with vision transformers. Initially, our approach models the underlying dynamics of complex systems within a low-dimensional space; and then it reconstructs the corresponding high-dimensional spatial fields. Many traditional methods involve decoding high-dimensional spatial fields before modeling the dynamics, while some other methods use an encoder to transition from high-dimensional observations to a latent space for dynamic modeling. In contrast, our approach directly uses sparse measurements to model the dynamics, bypassing the need for an encoder. This direct approach simplifies the modeling process, reduces computational complexity, and enhances the efficiency and scalability of the method for large datasets. We demonstrate the effectiveness of our framework through applications to various spatio-temporal systems, including fluid flows and global weather patterns. Although sparse measurements have limitations, our experiments reveal that they are sufficient to forecast system dynamics accurately over long time horizons. Our results also indicate that the performance of our proposed method remains robust across different sensor placement strategies, with further improvements as the number of sensors increases. This robustness underscores the flexibility of our architecture, particularly in real-world scenarios where sensor data is often sparse and unevenly distributed.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Disease modeling for Mucopolysaccharidosis type IIIB using patient derived induced pluripotent stem cells

Highlights: • Establishment of a patient-specific neuronal model for Mucopolysaccharidosis type IIIB using iPSCs. • Lysosomes enlarged and secondary lipid accumulated in Mucopolysaccharidosis type IIIB iPSC-derived neuronal cells. • Recombinant NAGLU, δ-tocopherol, and HPBCD reduced the disease phenotypes in MPS IIIB iPSC-derived neuronal cells. Mucopolysaccharidosis type IIIB (MPS IIIB) is a lysosomal disease caused by mutations in the NAGLU gene encoding α-N-acetylglucosaminidase (NAGLU) which degrades heparan sulfate in lysosomes. Deficiency in NAGLU results in lysosomal accumulation of glycosaminoglycans (GAGs) and neurological symptoms. Currently, there is no effective treatment or cure for this disease. In this study, induced pluripotent stem cell lines were established from two MPS IIIB patient fibroblast lines and differentiated into neural stem cells and neurons. MPS IIIB neural stem cells exhibited NAGLU deficiency accompanied with GAG accumulation, as well as lysosomal enlargement and secondary lipid accumulation. Treatments with recombinant NAGLU, δ-tocopherol, and 2-hydroxypropyl-b-cyclodextrin significantly reduced the disease phenotypes in these cells. These results indicate the MPS IIIB neural stem cells and neurons have the disease relevant phenotype and can be used as a cell-based disease model system for evaluation of drug efficacy and compound screening for drug development.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2021

Physics-informed neural networks for solving forward and inverse flow problems via the Boltzmann-BGK formulation

We report the Boltzmann equation with the Bhatnagar-Gross-Krook collision model (Boltzmann-BGK equation) has been employed to describe multiscale flows, i.e., from the hydrodynamic limit to free molecular flow. In this study we employ physics-informed neural networks (PINNs) to solve forward and inverse problems via the Boltzmann-BGK formulation (PINN-BGK), enabling PINNs to model flows in both the continuum and rarefied regimes. In particular, the PINN-BGK is composed of three sub-networks, i.e., the first for approximating the equilibrium distribution function, the second for approximating the non-equilibrium distribution function, and the third one for encoding the Boltzmann-BGK equation as well as the corresponding boundary/initial conditions. By minimizing the residuals of the governing equations and the mismatch between the predicted and provided boundary/initial conditions, we can approximate the Boltzmann-BGK equation for both continuous and rarefied flows. For forward problems, the PINN-BGK is utilized to solve various benchmark flows given boundary/initial conditions, e.g. Kovasznay flow, Taylor-Green flow, cavity flow, and micro Couette flow for Knudsen number up to 5. For inverse problems, we focus on rarefied flows in which accurate boundary conditions are difficult to obtain. We employ the PINN-BGK to infer the flow field in the entire computational domain given a limited number of interior scattered measurements on the velocity without using the (unknown) boundary conditions. Results for the two-dimensional micro Couette and micro cavity flows with Knudsen numbers ranging from 0.1 to 10 indicate that the PINN-BGK can infer the velocity field in the entire domain with good accuracy. Finally, we also present some results on using transfer learning to accelerate the training process. Specifically, we can obtain a three-fold speedup comparted to the standard training process (e.g., Adam plus L-BFG-B) for the two-dimensional flow problems considered in our work.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Implicit learning of convective organization explains precipitation stochasticity

Accurate prediction of precipitation intensity is crucial for both human and natural systems, especially in a warming climate more prone to extreme precipitation. Yet, climate models fail to accurately predict precipitation intensity, particularly extremes. One missing piece of information in traditional climate model parameterizations is subgrid-scale cloud structure and organization, which affects precipitation intensity and stochasticity at coarse resolution. Here, using global storm-resolving simulations and machine learning, we show that, by implicitly learning subgrid organization, we can accurately predict precipitation variability and stochasticity with a low-dimensional set of latent variables. Using a neural network to parameterize coarse-grained precipitation, we find that the overall behavior of precipitation is reasonably predictable using large-scale quantities only; however, the neural network cannot predict the variability of precipitation (R 2 ~ 0.45) and underestimates precipitation extremes. The performance is significantly improved when the network is informed by our organization metric, correctly predicting precipitation extremes and spatial variability (R 2 ~ 0.9). The organization metric is implicitly learned by training the algorithm on a high-resolution precipitable water field, encoding the degree of subgrid organization. The organization metric shows large hysteresis, emphasizing the role of memory created by subgrid-scale structures. We demonstrate that this organization metric can be predicted as a simple memory process from information available at the previous time steps. These findings stress the role of organization and memory in accurate prediction of precipitation intensity and extremes and the necessity of parameterizing subgrid-scale convective organization in climate models to better project future changes of water cycle and extremes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Neuroevolution of Spiking Neural Networks Using Compositional Pattern Producing Networks

Spiking neural networks (SNNs) offer tremendous potential for the future of AI, including the ability to be implemented efficiently on neuromorphic systems. One of the challenges in building functioning SNNs is the training process, as standard error back-propagation cannot be easily applied. In this work, we extend an evolutionary approach for training SNNs by implementing an indirect encoding of individuals. Specifically, we evolve SNNs using Compositional Pattern Producing Networks, which are able to learn the connectivity patterns between neurons defined in a coordinate space. We validate the approach on multiple control and classification tasks.

Elbrecht, Daniel↗
DOE OSTI2022

Supervised learning and the finite-temperature string method for computing committor functions and reaction rates

A central object in the computational studies of rare events is the committor function. Though costly to compute, the committor function encodes complete mechanistic information of the processes involving rare events, including reaction rates and transition-state ensembles. Under the framework of transition path theory, Rotskoff et al. [Proceedings of the 2nd Mathematical and Scientific Machine Learning Conference, Proceedings of Machine Learning Research (PLMR, 2022), Vol. 145, pp. 757–780] proposes an algorithm where a feedback loop couples a neural network that models the committor function with importance sampling, mainly umbrella sampling, which collects data needed for adaptive training. Here, in this work, we show additional modifications are needed to improve the accuracy of the algorithm. The first modification adds elements of supervised learning, which allows the neural network to improve its prediction by fitting to sample-mean estimates of committor values obtained from short molecular dynamics trajectories. The second modification replaces the committor-based umbrella sampling with the finite-temperature string (FTS) method, which enables homogeneous sampling in regions where transition pathways are located. We test our modifications on low-dimensional systems with non-convex potential energy where reference solutions can be found via analytical or finite element methods, and show how combining supervised learning and the FTS method yields accurate computation of committor functions and reaction rates. We also provide an error analysis for algorithms that use the FTS method, using which reaction rates can be accurately estimated during training with a small number of samples. The methods are then applied to a molecular system in which no reference solution is known, where accurate computations of committor functions and reaction rates can still be obtained.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Decode the Workload: Training Deep Learning Models for Efficient Compute Cluster Representation

Monitoring the status of a high throughput computing cluster running computationally intensive production jobs is a crucial yet challenging system administration task due to the complexity of such systems. To this end, we train autoencoders using the Linux kernel CPU metrics of the cluster. Additionally, we explore assisting these models with graph neural networks to share information across threads within a compute node. The models are compared in terms of their ability to: 1) Produce a compressed latent representation that captures the salient features of the input, 2) Detect anomalous activity, and 3) Make distinction between different kinds of jobs run at Jefferson Lab. The goal is to have a robust encoder whose compressed embeddings are used for several downstream tasks. We extend this study further by deploying these models in a human-in-the-loop production-based setting for the anomaly detection task and discuss the associated implementation aspects such as continual learning and the criterion to generate alarms. This study represents a first step in the endeavor towards building self-supervised large-scale foundation models for computing centers.

Mohammed, Ahmed↗
DOE OSTI2022

Explainable machine learning of the underlying physics of high-energy particle collisions

We present an implementation of an explainable and physics-aware machine learning model capable of inferring the underlying physics of high-energy particle collisions using the information encoded in the energy-momentum four-vectors of the final state particles. We demonstrate the proof-of-concept of our White Box AI approach using a Generative Adversarial Network (GAN) which learns from a DGLAP-based parton shower Monte Carlo event generator. The constrained generator network architecture mimics the structure of a parton shower exhibiting similarities with Recurrent Neural Networks (RNNs). We show, for the first time, that our approach leads to a network that is able to learn not only the final distribution of particles, but also the underlying parton branching mechanism, i.e. the Altarelli-Parisi splitting function, the ordering variable of the shower, and the scaling behavior. While the current work is focused on perturbative physics of the parton shower, we foresee a broad range of applications of our framework to areas that are currently difficult to address from first principles in QCD. Examples include nonperturbative and collective effects, factorization breaking and the modification of the parton shower in heavy-ion, and electron-nucleus collisions.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2025

MSU IETC LSTM Ethernet Decode (AN EDGE)

This research explores the ability of machine learning to perform signal separation of an Ethernet style encoded, full-duplex communication. Typical signal separation currently requires an active tap of the communication line, followed by a recombination and retransmission of the data. The purpose of this research is to study a passive approach to data acquisition from a full-duplex signal. The machine learning model used in this research is a long-short-term memory recurrent neural network (LSTM-RNN). The results show that the LSTM was largely successful in recreating the transmission signal from the measured data points, though the separated signals have not yet been tested using a decoding method.

Full Duplex Signals↗
DOE OSTI2023

Deep learning Hamiltonians from disordered image data in quantum materials

The capabilities of image probe experiments are rapidly expanding, providing new information about quantum materials on unprecedented length- and timescales. Many such materials feature inhomogeneous electronic properties with intricate pattern formation on the observable surface. This rich spatial structure contains information about interactions, dimensionality, and disorder—a spatial encoding of the Hamiltonian driving the pattern formation. Image recognition techniques from machine learning are an excellent tool for interpreting information encoded in the spatial relationships in such images. Here, we develop a deep learning framework for using the rich information available in these spatial correlations in order to discover the underlying Hamiltonian driving the patterns. We first vet the method on a known case, scanning near-field optical microscopy on a thin film of V⁢O 2 . We then apply our trained convolutional neural network architecture to new optical microscope images of a different V⁢O 2 film as it goes through the metal-insulator transition. We find that a two-dimensional Hamiltonian with both interactions and random field disorder is required to explain the intricate, fractal intertwining of metal and insulator domains during the transition. This detailed knowledge about the underlying Hamiltonian paves the way for using the model to control the pattern formation via, e.g., tailored hysteresis protocols. Finally, we also introduce a distribution-based confidence measure on the results of a multilabel classifier, which does not rely on adversarial training. In addition, we propose a machine-learning-based criterion for diagnosing a physical system's proximity to criticality.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2026

SENTRA: A Modular Computational Graph Framework for Critical Mineral and Materials Supply Chains: Part I: Network Construction Latent-Quantity Estimation, and Temporal Graph Forecasting

Global supply chains for critical minerals and materials are complex, evolving networks of countries, products, production stages, and trade relationships. Existing analytical approaches are limited by fragmented data and static network representations that do not capture the dynamic production dependencies linking raw materials, intermediate products, and final goods across multiple countries. Trade and production statistics provide only a partial view of domestic production, inventories, and material flows, making it difficult to identify indirect sourcing pathways, hidden dependencies, and embedded foreign exposures. This paper introduces the Supply Chain Exposure Network Tracking and Risk Assessment (SENTRA) framework, a modular graph-based computational framework for constructing, analyzing, and forecasting dynamic supply chain networks. As the first paper in a three-part methodological series, it establishes the computational foundation of SENTRA by constructing a temporal attributed multi-relational graph whose nodes represent product–country pairs and whose edges encode observed trade and within-country value-chain relationships. Statistical estimation and constrained optimization recover latent production, final demand, and product input dependency coefficients while enforcing economic accounting constraints. Graph-derived exposure measures quantify direct, transshipment, value-chain, and multi-hop supply chain dependencies independently of the forecasting model. A temporal graph forecasting architecture based on a relational graph neural network then forecasts the evolution of the graph under mass-balance constraints with distribution-free conformal uncertainty quantification. Validation on the global aluminum supply chain shows that the learned graph representations recover economically meaningful supply chain structure, accurately forecast out-of-sample trade relationships, and produce well-calibrated prediction intervals. Subsequent papers apply this computational foundation to exposure assessment, disruption analysis, and scenario-based policy analysis, and extend the framework to multimaterial supply chain modeling and decision support.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Benchmarking machine learning interatomic potentials via phonon anharmonicity

Abstract Machine learning approaches have recently emerged as powerful tools to probe structure-property relationships in crystals and molecules. Specifically, machine learning interatomic potentials (MLIPs) can accurately reproduce first-principles data at a cost similar to that of conventional interatomic potential approaches. While MLIPs have been extensively tested across various classes of materials and molecules, a clear characterization of the anharmonic terms encoded in the MLIPs is lacking. Here, we benchmark popular MLIPs using the anharmonic vibrational Hamiltonian of ThO 2 in the fluorite crystal structure, which was constructed from density functional theory (DFT) using our highly accurate and efficient irreducible derivative methods. The anharmonic Hamiltonian was used to generate molecular dynamics (MD) trajectories, which were used to train three classes of MLIPs: Gaussian approximation potentials, artificial neural networks (ANN), and graph neural networks (GNN). The results were assessed by directly comparing phonons and their interactions, as well as phonon linewidths, phonon lineshifts, and thermal conductivity. The models were also trained on a DFT MD dataset, demonstrating good agreement up to fifth-order for the ANN and GNN. Our analysis demonstrates that MLIPs have great potential for accurately characterizing anharmonicity in materials systems at a fraction of the cost of conventional first principles-based approaches.

interatomic potentials↗
DOE OSTI2025

Data Quality Monitoring for the Hadron Calorimeters Using Transfer Learning for Anomaly Detection

The proliferation of sensors brings an immense volume of spatio-temporal (ST) data in many domains, including monitoring, diagnostics, and prognostics applications. Data curation is a time-consuming process for a large volume of data, making it challenging and expensive to deploy data analytics platforms in new environments. Transfer learning (TL) mechanisms promise to mitigate data sparsity and model complexity by utilizing pre-trained models for a new task. Despite the triumph of TL in fields like computer vision and natural language processing, efforts on complex ST models for anomaly detection (AD) applications are limited. In this study, we present the potential of TL within the context of high-dimensional ST AD with a hybrid autoencoder architecture, incorporating convolutional, graph, and recurrent neural networks. Motivated by the need for improved model accuracy and robustness, particularly in scenarios with limited training data on systems with thousands of sensors, this research investigates the transferability of models trained on different sections of the Hadron Calorimeter of the Compact Muon Solenoid experiment at CERN. The key contributions of the study include exploring TL’s potential and limitations within the context of encoder and decoder networks, revealing insights into model initialization and training configurations that enhance performance while substantially reducing trainable parameters and mitigating data contamination effects.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

Solving high-dimensional inverse problems using amortized likelihood-free inference with noisy and incomplete data

Here, we present a likelihood-free probabilistic inversion method based on normalizing flows for high-dimensional inverse problems. The proposed method is composed of two complementary networks: a summary network for data compression and an inference network for parameter estimation. The summary network encodes raw observations into a fixed-size vector of summary features, while the inference network generates samples of the approximate posterior distribution of the model parameters based on these summary features. The posterior samples are produced in a deep generative fashion by sampling from a latent Gaussian distribution and passing these samples through an invertible transformation. We construct this invertible transformation by sequentially alternating conditional invertible neural network and conditional neural spline flow layers. The summary and inference networks are trained simultaneously. We apply the proposed method to an inversion problem in groundwater hydrology to estimate the posterior distribution of the log-conductivity field conditioned on spatially sparse time-series observations of the system’s hydraulic head responses. The conductivity field is represented with 706 degrees of freedom in the considered problem. Comparison with the likelihood-based iterative ensemble smoother PEST-IES method demonstrates that the proposed method accurately estimates the parameter posterior distribution and the observations’ predictive posterior distribution at a fraction of the inference time of PEST-IES.

conditional invertible neural network↗
DOE OSTI2024

A Quantum-Classical Collaborative Training Architecture Based on Quantum State Fidelity

Recent advancements have highlighted the limitations of current quantum systems, particularly the restricted number of qubits available on near-term quantum devices. This constraint greatly inhibits the range of applications that can leverage quantum computers. Moreover, as the available qubits increase, the computational complexity grows exponentially, posing additional challenges. Consequently, there is an urgent need to use qubits efficiently and mitigate both present limitations and future complexities. To address this, existing quantum applications attempt to integrate classical and quantum systems in a hybrid framework. In this study, we concentrate on quantum deep learning and introduce a collaborative classical-quantum architecture called co-TenQu. The classical component employs a tensor network for compression and feature extraction, enabling higher-dimensional data to be encoded onto logical quantum circuits with limited qubits. On the quantum side, we propose a quantum-state-fidelity-based evaluation function to iteratively train the network through a feedback loop between the two sides. co-TenQu has been implemented and evaluated with both simulators and the IBM-Q platform. Compared to state-of-the-art approaches, co-TenQu enhances a classical deep neural network by up to 41.72% in a fair setting. Additionally, it outperforms other quantum-based methods by up to 1.9 times and achieves similar accuracy while utilizing 70.59% fewer qubits.

42 ENGINEERING↗
DOE OSTI2021

Global transcriptome profile of the developmental principles of in vitro iPSC-to-motor neuron differentiation

Abstract Background Human induced pluripotent stem cells (iPSC) have opened new avenues for regenerative medicine. Consequently, iPSC-derived motor neurons have emerged as potentially viable therapies for spinal cord injuries and neurodegenerative disorders including Amyotrophic Lateral Sclerosis. However, direct clinical application of iPSC bears in itself the risk of tumorigenesis and other unforeseeable genetic or epigenetic abnormalities. Results Employing RNA-seq technology, we identified and characterized gene regulatory networks triggered by in vitro chemical reprogramming of iPSC into cells with the molecular features of motor neurons (MNs) whose function in vivo is to innervate effector organs. We present meta-transcriptome signatures of 5 cell types: iPSCs, neural stem cells, motor neuron progenitors, early motor neurons, and mature motor neurons. In strict response to the chemical stimuli, along the MN differentiation axis we observed temporal downregulation of tumor growth factor-β signaling pathway and consistent activation of sonic hedgehog, Wnt/β-catenin, and Notch signaling. Together with gene networks defining neuronal differentiation (neurogenin 2, microtubule-associated protein 2, Pax6, and neuropilin-1), we observed steady accumulation of motor neuron-specific regulatory genes, including Islet-1 and homeobox protein HB9. Interestingly, transcriptome profiling of the differentiation process showed that Ca 2+ signaling through cAMP and LPC was downregulated during the conversion of the iPSC to neural stem cells and key regulatory gene activity of the pathway remained inhibited until later stages of motor neuron formation. Pathways shaping the neuronal development and function were well-represented in the early motor neuron cells including, neuroactive ligand-receptor interactions, axon guidance, and the cholinergic synapse formation. A notable hallmark of our in vitro motor neuron maturation in monoculture was the activation of genes encoding G-coupled muscarinic acetylcholine receptors and downregulation of the ionotropic nicotinic acetylcholine receptors expression. We observed the formation of functional neuronal networks as spontaneous oscillations in the extracellular action potentials recorded on multi-electrode array chip after 20 days of differentiation. Conclusions Detailed transcriptome profile of each developmental step from iPSC to motor neuron driven by chemical induction provides the guidelines to novel therapeutic approaches in the re-construction efforts of muscle innervation.

59 BASIC BIOLOGICAL SCIENCES↗