Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “neural”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Quantum-inspired weight-constrained neural network: Reducing variable numbers by 100× compared to standard neural networks

Although quantum machine learning has shown great promise, the practical application of quantum computers remains constrained in the noisy intermediate-scale quantum era. To take advantage of quantum machine learning, we investigate the underlying mathematical principles of these quantum models and find that the quantum neural network with amplitude encoding is equivalent to a weight-constrained neural network. Motivated by this discovery, we develop a classical weight-constrained neural network. We find that this approach can reduce the number of variables in a classical neural network by a factor of 135 while preserving its accuracy. In addition, we develop a dropout method to enhance the robustness of quantum machine learning models, which are highly susceptible to adversarial attacks. This technique can also be applied to improve the adversarial robustness of the classical weight-constrained neural network, which is essential for industry applications, such as self-driving vehicles. Our work offers an approach to reduce the complexity of large classical neural networks, addressing a critical challenge in machine learning.

quantum algorithms & computation↗
DOE OSTI2025

Accelerating Hamiltonian Monte Carlo for Bayesian inference in neural networks and neural operators

Hamiltonian Monte Carlo (HMC) is a powerful and accurate method to sample from the posterior distribution in Bayesian inference. However, HMC techniques are computationally demanding for Bayesian neural networks due to the high dimensionality of the network’s parameter space and the non-convexity of their posterior distributions. Therefore, various approximation techniques, such as variational inference (VI) or stochastic gradient MCMC, are often employed to infer the posterior distribution of the network parameters. Such approximations introduce inaccuracies in the inferred distributions, resulting in unreliable uncertainty estimates. In this work, we propose a hybrid approach that combines inexpensive VI and accurate HMC methods to efficiently and accurately quantify uncertainties in neural networks and neural operators. The proposed approach leverages an initial VI training on the full network. We examine the influence of individual parameters on the prediction uncertainty, which shows that a large proportion of the parameters do not contribute substantially to uncertainty in the network predictions. This information is then used to significantly reduce the dimension of the parameter space, and HMC is performed only for the subset of network parameters that strongly influence prediction uncertainties. This yields a framework for accelerating the full batch HMC for posterior inference in neural networks. We demonstrate the efficiency and accuracy of the proposed framework on deep neural networks and operator networks, showing that inference can be performed for large networks with tens to hundreds of thousands of parameters. Finally, we show that this method can effectively learn surrogates for complex physical systems by modeling the operator that maps from upstream conditions to wall-pressure data on a cone in hypersonic flow.

Bayesian inference↗
DOE OSTI2026

Neural entropy-stable conservative flux form neural networks for learning hyperbolic conservation laws

We propose a neural entropy-stable conservative flux form neural network (NESCFN) for learning hyperbolic conservation laws and their associated entropy functions directly from solution trajectories, without requiring any predefined numerical discretization. While recent neural network architectures have successfully integrated classical numerical principles into learned models, most rely on prior knowledge of the governing equations or assume a fixed discretization. Our approach removes this dependency by embedding entropy-stable design principles into the learning process itself, enabling the discovery of physically consistent dynamics in a fully data-driven setting. By jointly learning both the flux function and a corresponding entropy, NESCFN promotes conservation and entropy dissipation, which is critical for long-term stability and fidelity in the system of hyperbolic conservation laws. Furthermore, numerical results demonstrate that the method achieves stability and conservation over extended time horizons and accurately captures shock propagation speeds, even without oracle access to future-time solution profiles in the training data.

Conservative flux form↗
NASA NTRS1990

3-D components of a biological neural network visualized in computer generated imagery. II - Macular neural network organization

Computer-assisted reconstructions of small parts of the macular neural network show how the nerve terminals and receptive fields are organized in 3-dimensional space. This biological neural network is anatomically organized for parallel distributed processing of information. Processing appears to be more complex than in computer-based neural network, because spatiotemporal factors figure into synaptic weighting. Serial reconstruction data show anatomical arrangements which suggest that (1) assemblies of cells analyze and distribute information with inbuilt redundancy, to improve reliability; (2) feedforward/feedback loops provide the capacity for presynaptic modulation of output during processing; (3) constrained randomness in connectivities contributes to adaptability; and (4) local variations in network complexity permit differing analyses of incoming signals to take place simultaneously. The last inference suggests that there may be segregation of information flow to central stations subserving particular functions.

Ross, Muriel D.↗
DOE OSTI2025

Co-training of multiple neural networks for simultaneous optimization and training of physics-informed neural networks for composite curing

This paper introduces a Physics-Informed Neural Network (PINN) technique that co-trains neural networks (NNs) that represent each function in a system of equations to simultaneously solve equations representing an out-of-autoclave (OOA) cure process while conducting optimization in adherence to process requirements. Specifically, this co-training approach benefits from using NNs to represent OOA inputs (air temperature profile) and outputs (part and tool temperature profiles and degree of cure). Production requirements can then be levied on the inputs, such as maximum air temperature and minimum cure cycle, and simultaneously on the outputs, such as degree of cure, maximum part temperature, and part temperature rate limits. The technique is validated with finite element (FE) simulations and physical experiments for curing a Toray T830H-6 K/3900-2D composite panel. Furthermore, this novel approach efficiently models and optimizes the OOA cure process.

Composite curing↗
DOE OSTI2025

Neural chaos: A spectral stochastic neural operator

Building surrogate models for operators with uncertainty quantification capabilities is essential for many engineering applications where randomness–such as variability in material properties, boundary conditions, and initial conditions–is unavoidable. Polynomial Chaos Expansion (PCE) is widely recognized as a go-to method for constructing stochastic surrogates in both intrusive and non-intrusive ways, and it has recently been used in the context of operator learning. However, its application becomes challenging for complex or high-dimensional processes, as achieving accuracy requires higher-order polynomials, which can increase computational demand and/or the risk of overfitting. Furthermore, PCE requires specialized treatments to manage random variables that are not independent, and these treatments may be problem-dependent or may fail with increasing complexity. Here, in this work, we adopt the same formalism as the spectral expansion used in PCE; however, we replace the classical polynomial basis functions with neural network (NN) basis functions to leverage their expressivity. To achieve this, we propose an algorithm that identifies NN-parameterized basis functions in a purely data-driven manner, without any prior assumptions about the joint distribution of the random variables involved, whether independent or dependent, or about their marginal distributions. The proposed algorithm identifies each NN-parameterized basis function sequentially, ensuring they are orthogonal with respect to the data distribution. The basis functions are constructed directly on the joint stochastic variables without requiring a tensor product structure or assuming independence of the random variables. This approach may offer greater flexibility for complex stochastic models, while simplifying implementation compared to the tensor product structures typically used in PCE to handle random vectors. This is particularly advantageous given the current state of open-source packages, where building and training neural networks can be done with just a few lines of code and extensive community support. We demonstrate the effectiveness of the proposed scheme through several numerical examples of varying complexity and provide comparisons with classical PCE.

Polynomial chaos expansion↗
DOE OSTI2025

Large-scale experimental validation of thermochemical water-splitting oxides discovered by defect graph neural networks

Thermochemical water-splitting (TCH) based on 2-step thermal redox cycles in metal oxides is a promising approach to generating H 2 , but state-of-the-art (SOTA) CeO 2 has several practical limitations, which has motivated continued materials discovery efforts in this field. Here, in this study, we improve upon a SOTA defect graph neural network (dGNN) surrogate model's oxygen vacancy predictions and combine them with materials project phase diagrams to down-select and discover structurally diverse, experimentally known metal oxides whose TCH performance was previously unknown. Amongst twelve candidates selected based on our high-throughput screening and down-selection criteria, we achieved ∼80% accuracy in identifying materials with stable redox cycling and hydrogen production in stagnation flow reactor water-splitting experiments. Closer to 100% accuracy can be achieved if higher-accuracy, hybrid DFT-predicted vacancy formation energies were computed and used in lieu of the most uncertain dGNN-based screening predictions, as they correct false positives to true negatives. Notably, two discovered candidates, Sr 3 PrMn 2 O 8 and Ba 2 Fe 2 O 5 , display hydrogen yields greater than CeO 2 under specific redox conditions. In conclusion, these results demonstrate our ability to computationally predict and experimentally validate promising candidate TCH materials that have the potential to compete with CeO 2 .

08 HYDROGEN↗
DOE OSTI2026

ProtNHF: Neural Hamiltonian Flows for Controllable Protein Sequence Generation

This dataset accompanies the publication "ProtNHF: Neural Hamiltonian Flows for Controllable Protein Sequence Generation". This paper introduces a new AI model for protein sequence generation. This dataset contains data related to experiments discussed in the publication. This includes generated sequences and evaluation metrics supporting all unconditional and bias-controlled experiments in the ProtNHF paper.

60 APPLIED LIFE SCIENCES↗
NASA NTRS1992

Resource constrained design of artificial neural networks using comparator neural network

We present a systematic design method executed under resource constraints for automating the design of artificial neural networks using the back error propagation algorithm. Our system aims at finding the best possible configuration for solving the given application with proper tradeoff between the training time and the network complexity. The design of such a system is hampered by three related problems. First, there are infinitely many possible network configurations, each may take an exceedingly long time to train; hence, it is impossible to enumerate and train all of them to completion within fixed time, space, and resource constraints. Second, expert knowledge on predicting good network configurations is heuristic in nature and is application dependent, rendering it difficult to characterize fully in the design process. A learning procedure that refines this knowledge based on examples on training neural networks for various applications is, therefore, essential. Third, the objective of the network to be designed is ill-defined, as it is based on a subjective tradeoff between the training time and the network cost. A design process that proposes alternate configurations under different cost-performance tradeoff is important. We have developed a Design System which schedules the available time, divided into quanta, for testing alternative network configurations. Its goal is to select/generate and test alternative network configurations in each quantum, and find the best network when time is expended. Since time is limited, a dynamic schedule that determines the network configuration to be tested in each quantum is developed. The schedule is based on relative comparison of predicted training times of alternative network configurations using comparator network paradigm. The comparator network has been trained to compare training times for a large variety of traces of TSSE-versus-time collected during back-propagation learning of various applications.

Wah, Benjamin W.↗
NASA NTRS2009

Evolvable Neural Software System

The Evolvable Neural Software System (ENSS) is composed of sets of Neural Basis Functions (NBFs), which can be totally autonomously created and removed according to the changing needs and requirements of the software system. The resulting structure is both hierarchical and self-similar in that a given set of NBFs may have a ruler NBF, which in turn communicates with other sets of NBFs. These sets of NBFs may function as nodes to a ruler node, which are also NBF constructs. In this manner, the synthetic neural system can exhibit the complexity, three-dimensional connectivity, and adaptability of biological neural systems. An added advantage of ENSS over a natural neural system is its ability to modify its core genetic code in response to environmental changes as reflected in needs and requirements. The neural system is fully adaptive and evolvable and is trainable before release. It continues to rewire itself while on the job. The NBF is a unique, bilevel intelligence neural system composed of a higher-level heuristic neural system (HNS) and a lower-level, autonomic neural system (ANS). Taken together, the HNS and the ANS give each NBF the complete capabilities of a biological neural system to match sensory inputs to actions. Another feature of the NBF is the Evolvable Neural Interface (ENI), which links the HNS and ANS. The ENI solves the interface problem between these two systems by actively adapting and evolving from a primitive initial state (a Neural Thread) to a complicated, operational ENI and successfully adapting to a training sequence of sensory input. This simulates the adaptation of a biological neural system in a developmental phase. Within the greater multi-NBF and multi-node ENSS, self-similar ENI s provide the basis for inter-NBF and inter-node connectivity.

Curtis, Steven A.↗
NASA NTRS2003

Neural crest contributions to the lamprey head

The neural crest is a vertebrate-specific cell population that contributes to the facial skeleton and other derivatives. We have performed focal DiI injection into the cranial neural tube of the developing lamprey in order to follow the migratory pathways of discrete groups of cells from origin to destination and to compare neural crest migratory pathways in a basal vertebrate to those of gnathostomes. The results show that the general pathways of cranial neural crest migration are conserved throughout the vertebrates, with cells migrating in streams analogous to the mandibular and hyoid streams. Caudal branchial neural crest cells migrate ventrally as a sheet of cells from the hindbrain and super-pharyngeal region of the neural tube and form a cylinder surrounding a core of mesoderm in each pharyngeal arch, similar to that seen in zebrafish and axolotl. In addition to these similarities, we also uncovered important differences. Migration into the presumptive caudal branchial arches of the lamprey involves both rostral and caudal movements of neural crest cells that have not been described in gnathostomes, suggesting that barriers that constrain rostrocaudal movement of cranial neural crest cells may have arisen after the agnathan/gnathostome split. Accordingly, neural crest cells from a single axial level contributed to multiple arches and there was extensive mixing between populations. There was no apparent filling of neural crest derivatives in a ventral-to-dorsal order, as has been observed in higher vertebrates, nor did we find evidence of a neural crest contribution to cranial sensory ganglia. These results suggest that migratory constraints and additional neural crest derivatives arose later in gnathostome evolution.

Non-NASA Center↗
NASA NTRS2004

Real-Time Adaptive Color Segmentation by Neural Networks

Artificial neural networks that would utilize the cascade error projection (CEP) algorithm have been proposed as means of autonomous, real-time, adaptive color segmentation of images that change with time. In the original intended application, such a neural network would be used to analyze digitized color video images of terrain on a remote planet as viewed from an uninhabited spacecraft approaching the planet. During descent toward the surface of the planet, information on the segmentation of the images into differently colored areas would be updated adaptively in real time to capture changes in contrast, brightness, and resolution, all in an effort to identify a safe and scientifically productive landing site and provide control feedback to steer the spacecraft toward that site. Potential terrestrial applications include monitoring images of crops to detect insect invasions and monitoring of buildings and other facilities to detect intruders. The CEP algorithm is reliable and is well suited to implementation in very-large-scale integrated (VLSI) circuitry. It was chosen over other neural-network learning algorithms because it is better suited to realtime learning: It provides a self-evolving neural-network structure, requires fewer iterations to converge and is more tolerant to low resolution (that is, fewer bits) in the quantization of neural-network synaptic weights. Consequently, a CEP neural network learns relatively quickly, and the circuitry needed to implement it is relatively simple. Like other neural networks, a CEP neural network includes an input layer, hidden units, and output units (see figure). As in other neural networks, a CEP network is presented with a succession of input training patterns, giving rise to a set of outputs that are compared with the desired outputs. Also as in other neural networks, the synaptic weights are updated iteratively in an effort to bring the outputs closer to target values. A distinctive feature of the CEP neural network and algorithm is that each update of synaptic weights takes place in conjunction with the addition of another hidden unit, which then remains in place as still other hidden units are added on subsequent iterations. For a given training pattern, the synaptic weight between (1) the inputs and the previously added hidden units and (2) the newly added hidden unit is updated by an amount proportional to the partial derivative of a quadratic error function with respect to the synaptic weight. The synaptic weight between the newly added hidden unit and each output unit is given by a more complex function that involves the errors between the outputs and their target values, the transfer functions (hyperbolic tangents) of the neural units, and the derivatives of the transfer functions.

Duong, Tuan A.↗
NASA NTRS2002

Amphioxus and lamprey AP-2 genes: implications for neural crest evolution and migration patterns

The neural crest is a uniquely vertebrate cell type present in the most basal vertebrates, but not in cephalochordates. We have studied differences in regulation of the neural crest marker AP-2 across two evolutionary transitions: invertebrate to vertebrate, and agnathan to gnathostome. Isolation and comparison of amphioxus, lamprey and axolotl AP-2 reveals its extensive expansion in the vertebrate dorsal neural tube and pharyngeal arches, implying co-option of AP-2 genes by neural crest cells early in vertebrate evolution. Expression in non-neural ectoderm is a conserved feature in amphioxus and vertebrates, suggesting an ancient role for AP-2 genes in this tissue. There is also common expression in subsets of ventrolateral neurons in the anterior neural tube, consistent with a primitive role in brain development. Comparison of AP-2 expression in axolotl and lamprey suggests an elaboration of cranial neural crest patterning in gnathostomes. However, migration of AP-2-expressing neural crest cells medial to the pharyngeal arch mesoderm appears to be a primitive feature retained in all vertebrates. Because AP-2 has essential roles in cranial neural crest differentiation and proliferation, the co-option of AP-2 by neural crest cells in the vertebrate lineage was a potentially crucial event in vertebrate evolution.

NASA Discipline Evolutionary Biology↗
NASA NTRS1993

An overview on development of neural network technology

The study has been to obtain a bird's-eye view of the current neural network technology and the neural network research activities in NASA. The purpose was two fold. One was to provide a reference document for NASA researchers who want to apply neural network techniques to solve their problems. Another one was to report out survey results regarding NASA research activities and provide a view on what NASA is doing, what potential difficulty exists and what NASA can/should do. In a ten week study period, we interviewed ten neural network researchers in the Langley Research Center and sent out 36 survey forms to researchers at the Johnson Space Center, Lewis Research Center, Ames Research Center and Jet Propulsion Laboratory. We also sent out 60 similar forms to educators and corporation researchers to collect general opinions regarding this field. Twenty-eight survey forms, 11 from NASA researchers and 17 from outside, were returned. Survey results were reported in our final report. In the final report, we first provided an overview on the neural network technology. We reviewed ten neural network structures, discussed the applications in five major areas, and compared the analog, digital and hybrid electronic implementation of neural networks. In the second part, we summarized known NASA neural network research studies and reported the results of the questionnaire survey. Survey results show that most studies are still in the development and feasibility study stage. We compared the techniques, application areas, researchers' opinions on this technology, and many aspects between NASA and non-NASA groups. We also summarized their opinions on difficulties encountered. Applications are considered the top research priority by most researchers. Hardware development and learning algorithm improvement are the next. The lack of financial and management support is among the difficulties in research study. All researchers agree that the use of neural networks could result in cost saving. Fault tolerance has been claimed as one important feature of neural computing. However, the survey indicates that very few studies address this issue. Fault tolerance is important in space mission and aircraft control. We believe that it is worthy for NASA to devote more efforts into the utilization of this feature.

Lin, Chun-Shin↗
DOE OSTI2025

Understanding latent timescales in neural ordinary differential equation models of advection-dominated dynamical systems

The neural ordinary differential equation (ODE) framework has shown considerable promise in recent years in developing highly accelerated surrogate models for complex physical systems characterized by partial differential equations (PDEs). For PDE-based systems, state-of-the-art neural ODE strategies leverage a two-step procedure to achieve this acceleration: a nonlinear dimensionality reduction step provided by an autoencoder, and a time integration step provided by a neural-network based model for the resultant latent space dynamics (the neural ODE). This work explores the applicability of such autoencoder-based neural ODE strategies for PDEs in which advection terms play a critical role. More specifically, alongside predictive demonstrations, physical insight into the sources of model acceleration (i.e., how the neural ODE achieves its acceleration) is the scope of the current study. Such investigations are performed by quantifying the effects of both autoencoder and neural ODE components on latent system time-scales using eigenvalue analysis of dynamical system Jacobians. To this end, the sensitivity of various critical training parameters – de-coupled versus end-to-end training, latent space dimensionality, and the role of training trajectory length, for example – to both model accuracy and the discovered latent system timescales is quantified. Furthermore, this work specifically uncovers the key role played by the training trajectory length (the number of rollout steps in the loss function during training) on the latent system timescales: larger trajectory lengths correlate with an increase in limiting neural ODE time-scales, and optimal neural ODEs are found to recover the largest time-scales of the full-order (ground-truth) system. Demonstrations are performed across fundamentally different unsteady fluid dynamics configurations influenced by advection: (1) the Kuramoto–Sivashinsky equations (2) Hydrogen-Air channel detonations (the compressible reacting Navier–Stokes equations with detailed chemistry), and (3) 2D Atmospheric flow.

Advection-dominated dynamical systems↗
NASA NTRS2002

An amphioxus winged helix/forkhead gene, AmphiFoxD: insights into vertebrate neural crest evolution

During amphioxus development, the neural plate is bordered by cells expressing many genes with homologs involved in vertebrate neural crest induction. However, these amphioxus cells evidently lack additional genetic programs for the cell delaminations, migrations, and differentiations characterizing definitive vertebrate neural crest. We characterize an amphioxus winged helix/forkhead gene (AmphiFoxD) closely related to vertebrate FoxD genes. Phylogenetic analysis indicates that the AmphiFoxD is basal to vertebrate FoxD1, FoxD2, FoxD3, FoxD4, and FoxD5. One of these vertebrate genes (FoxD3) consistently marks neural crest during development. Early in amphioxus development, AmphiFoxD is expressed medially in the anterior neural plate as well as in axial (notochordal) and paraxial mesoderm; later, the gene is expressed in the somites, notochord, cerebral vesicle (diencephalon), and hindgut endoderm. However, there is never any expression in cells bordering the neural plate. We speculate that an AmphiFoxD homolog in the common ancestor of amphioxus and vertebrates was involved in histogenic processes in the mesoderm (evagination and delamination of the somites and notochord); then, in the early vertebrates, descendant paralogs of this gene began functioning in the presumptive neural crest bordering the neural plate to help make possible the delaminations and cell migrations that characterize definitive vertebrate neural crest. Copyright 2002 Wiley-Liss, Inc.

NASA Discipline Evolutionary Biology↗