Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Equation learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Adjoint-Operator Learning For A Neural Network

Electronic neural networks made to synthesize initially unknown mathematical models of time-dependent phenomena or to learn temporally evolving patterns by use of algorithms based on adjoint operators. Algorithms less complicated, involve less computation and solve learning equations forward in time possibly simultaneously with equations of evolution of neural network, thereby both increasing computational efficiency and making real-time applications possible.

Barhen, Jacob↗

Application of adjoint operators to neural learning

A technique for the efficient analytical computation of such parameters of the neural architecture as synaptic weights and neural gain is presented as a single solution of a set of adjoint equations. The learning model discussed concentrates on the adiabatic approximation only. A problem of interest is represented by a system of N coupled equations, and then adjoint operators are introduced. A neural network is formalized as an adaptive dynamical system whose temporal evolution is governed by a set of coupled nonlinear differential equations. An approach based on the minimization of a constrained neuromorphic energylike function is applied, and the complete learning dynamics are obtained as a result of the calculations.

Barhen, J.↗

Non-Lipschitzian neural dynamics

A novel approach is presented which is motivated by an attempt to remove one of the most fundamental limitations of artificial neural networks: their rigid behavior as compared with even the simplest biological systems. It is demonstrated that non-Lipschitzian dynamics, based on the faliure of the Lipschitz conditions at repellers, displays a new qualitative effect, i.e., a multichoice response to periodic external excitations. This makes it possible to construct unpredictable systems, represented in the form of coupled activation and learning dynamical equations. It is shown that unpredictable systems can be controlled by sign strings which uniquely define the system behavior by specifying the direction of the motions at the critical points. Unpredictable systems driven by sign strings are extremely flexible and can serve as a powerful tool for complex pattern recognition.

Barhen, Jacob↗

Neurocomputing

The present work is intended to give technologists, research scientists, and mathematicians a graduate-level overview of the field of neurocomputing. After exploring the relationship of this field to general neuroscience, attention is given to neural network building blocks, the self-adaptation equations of learning laws, the data-transformation structures of associative networks, and the multilayer data-transformation structures of mapping networks. Also treated are the neurocomputing frontiers of spatiotemporal, stochastic, and hierarchical networks, 'neurosoftware', the creation of neural network-based computers, and neurocomputing applications in sensor processing, control, and data analysis.

Hecht-Nielsen, Robert↗

IAC control system analysis development

The MultiOptimal Differential Equation Language (MODEL) is described. It provides a means for generating numerical solutions to systems of differential equations using a digital computer. The notation of this language is similar to that usually used in describing physical systems by differential equations. Thus, the learning process is simplified, programming becomes easier, and debugging is more readily accomplished. Programs written in the MultiOptimal Differential Equation Language are machine translated into FORTRAN 4 code which is optimal in several respects. The interactive version of MODEL makes use of interactive system routines so that the user may observe the solution as it is being generated and interact with the program in a manner similar to that associated with analog simulation. The DISCOS-Control version of the MODEL simulation language is used to describe the control system for a plant which is modelled by the DISCOS program. DISCOS plant variables may be referenced in the control system description and all the sensor signals, coupling torques, momentum devices and external loads are automatically linked to the DISCOS plant model.

Frisch, H. P.↗

Neural network error correction for solving coupled ordinary differential equations

A neural network is presented to learn errors generated by a numerical algorithm for solving coupled nonlinear differential equations. The method is based on using a neural network to correctly learn the error generated by, for example, Runge-Kutta on a model molecular dynamics (MD) problem. The neural network programs used in this study were developed by NASA. Comparisons are made for training the neural network using backpropagation and a new method which was found to converge with fewer iterations. The neural net programs, the MD model and the calculations are discussed.

Shelton, R. O.↗

Adjoint-operators and non-adiabatic learning algorithms in neural networks

Adjoint sensitivity equations are presented, which can be solved simultaneously (i.e., forward in time) with the dynamics of a nonlinear neural network. These equations provide the foundations for a new methodology which enables the implementation of temporal learning algorithms in a highly efficient manner.

Toomarian, N.↗

Unstructured-grid methods development: Lessons le arned

The development is summarized of unstructured grid methods for the solution of the equations of fluid flow and some of the lessons learned are shared. The 3-D Euler equations are solved, including spatial discretizations, temporal discretizations, and boundary conditions. An example calculation with an upwind implicit method using a CFL (Courant Friedricks Lewy) number of infinity is presented for the Boeing 747 aircraft. The results obtained in less than one hour of CPU time on a Cray-2 computer, thus demonstrating the speed and robustness of the present capability.

Batina, John T.↗

The Emergence of Agent-Based Technology as an Architectural Component of Serious Games

The evolution of games as an alternative to traditional simulations in the military context has been gathering momentum over the past five years, even though the exploration of their use in the serious sense has been ongoing since the mid-nineties. Much of the focus has been on the aesthetics of the visuals provided by the core game engine as well as the artistry provided by talented development teams to produce not only breathtaking artwork, but highly immersive game play. Consideration of game technology is now so much a part of the modeling and simulation landscape that it is becoming difficult to distinguish traditional simulation solutions from game-based approaches. But games have yet to provide the much needed interactive free play that has been the domain of semi-autonomous forces (SAF). The component-based middleware architecture that game engines provide promises a great deal in terms of options for the integration of agent solutions to support the development of non-player characters that engage the human player without the deterministic nature of scripted behaviors. However, there are a number of hard-learned lessons on the modeling and simulation side of the equation that game developers have yet to learn, such as: correlation of heterogeneous systems, scalability of both terrain and numbers of non-player entities, and the bi-directional nature of simulation to game interaction provided by Distributed Interactive Simulation (DIS) and High Level Architecture (HLA).

Phillips, Mark↗

Turbo-Design: Open-Source Radial Equilibrium Turbomachinery Solver: Part I - Turbines

Advances in 3D Geometrical Designs and Cooling have played a significant role in improving the efficiency of turbomachinery. However, these advancements must be effectively translated back to the modeler. Machine learning can facilitate this transition. Specifically, machine learning–based loss models can bridge the gap between 3D and 1D designs, enabling modelers not only to predict velocity triangles but also to extract additional geometric features. Currently, the design tools used at NASA have not been updated to support such integration—until now. TurboDesign is an open-source, Python-based framework that replaces TD2 (LEW-11029-1) and AXOD2 (LEW-16323-1), both of which are radial equilibrium solvers for axial turbines. The goal of this update is to enable the integration of machine learning loss models into radial equilibrium equations. Additionally, TurboDesign is designed to support radial machines. This paper presents the governing equations, the assumptions underlying the code, the integration of legacy loss models, an example of machine learning model integration, and a validation comparison with CFD. All code, tutorials, and documentation are available at: https://www.github.com/nasa/turbo-design

Radial Equilibrium↗

Atmospheric Chemistry Modeling Using a Regression Forest Model

Atmospheric chemistry is central to many environmental issues such as air pollution, climate change, and stratospheric ozone loss. Chemistry Transport Models (CTM) are a central tool for understanding these issues, whether for research or for forecasting. These models split the atmosphere in a large number of grid-boxes and consider the emission of compounds into these boxes and their subsequent transport, deposition, and chemical processing. The chemistry is represented through a series of simultaneous ordinary differential equations, one for each compound. Given the difference in life-times between the chemical compounds (milli-seconds for O1D to years for CH4) these equations are numerically stiff and solving them consists of a significant fraction of the computational burden of a CTM. We have investigated a machine learning approach to solving the differential equations instead of solving them numerically. From an annual simulation of the GEOS-Chem model we have produced a training dataset consisting of the concentration of compounds before and after the differential equations are solved, together with some key physical parameters for every grid-box and time-step. From this dataset we have trained a machine learning algorithm (regression forest) to be able to predict the concentration of the compounds after the integration step based on the concentrations and physical state at the beginning of the time step. We have then included this algorithm back into the GEOS-Chem model, bypassing the need to integrate the chemistry. This machine learning approach shows many of the characteristics of the full simulation and has the potential to be substantially faster. There are a wide range of application for such an approach - generating boundary conditions, for use in air quality forecasts, chemical data assimilation systems, centennial scale climate simulations etc. We discuss our approches' speed and accuracy, and highlight some potential future directions for improving this approach.

Keller, Christoph A.↗

Learning-Based State-Dependent Coefficient Form Task Space Tracking Control of Soft Robot

n this paper, a data-driven modeling and control framework is developed for task space control of a soft robot gripper which consists of four individual soft fingers. Each of the four fingers is modeled as a manipulator with high degrees of freedom. The corresponding task space dynamics of the manipulator are derived using a rigid-link approximation of the continuum manipulator. A neural network approach is used to learn the derived dynamics in State Dependent Coefficient (SDC) form. Using the learned SDC matrices, an asymptotically stable optimal closed-loop tracking controller which is based on solving the State Dependent Riccati Equation (SDRE) is derived. The model learning and trajectory tracking controller is implemented on an open source Soft Motion (SoMo) platform simulating the soft gripper motion and corresponding tracking results are presented.

Rounak Bhattacharya↗

Progress in unstructured-grid methods development for unsteady aerodynamic applications

The development of unstructured-grid methods for the solution of the equations of fluid flow and what was learned over the course of the research are summarized. The focus of the discussion is on the solution of the time-dependent Euler equations including spatial discretizations, temporal discretizations, and boundary conditions. An example calculation with an implicit upwind method using a CFL number of infinity is presented for the Boeing 747 aircraft. The results were obtained in less than one hour CPU time on a Cray-2 computer, thus, demonstrating the speed and robustness of the capability. Additional calculations for the ONERA M6 wing demonstrate the accuracy of the method through the good agreement between calculated results and experimental data for a standard transonic flow case.

Batina, John T.↗

Application of Sparse Identification of Nonlinear Dynamics for Physics-Informed Learning

Advances in machine learning and deep neural networks has enabled complex engineering tasks like image recognition, anomaly detection, regression, and multi-objective optimization, to name but a few. The complexity of the algorithm architecture, e.g., the number of hidden layers in a deep neural network, typically grows with the complexity of the problems they are required to solve, leaving little room for interpreting (or explaining) the path that results in a specific solution. This drawback is particularly relevant for autonomous aerospace and aviation systems, where certifications require a complete understanding of the algorithm behavior in all possible scenarios. Including physics knowledge in such data-driven tools may improve the interpretability of the algorithms, thus enhancing model validation against events with low probability but relevant for system certification. Such events include, for example, spacecraft or aircraft sub-system failures, for which data may not be available in the training phase. This paper investigates a recent physics-informed learning algorithm for identification of system dynamics, and shows how the governing equations of a system can be extracted from data using sparse regression. The learned relationships can be utilized as a surrogate model which, unlike typical data-driven surrogate models, relies on the learned underlying dynamics of the system rather than large number of fitting parameters. The work shows that the algorithm can reconstruct the differential equations underlying the observed dynamics using a single trajectory when no uncertainty is involved. However, the training set size must increase when dealing with stochastic systems, e.g., nonlinear dynamics with random initial conditions.

Corbetta, Matteo↗

Motor System Development Depends on Experience: A Microgravity Study of Rats

Animals move about their environment by sensing their surroundings and making adjustments according to need. All animals take the force of gravity into account when the brain and spinal cord undertake the planning and execution of movements. To what extent must animals learn to factor in the force of gravity when making neural calculations about movement? Are animals born knowing how to respond to gravity, or must the young nervous system learn to enter gravity into the equation? To study this issue, young rats were reared in two different gravitational environments (the one-G of Earth and the microgravity of low Earth orbit) that necessitated two different types of motor operations (movements) for optimal behavior. We inquired whether those portions of the young nervous system involved in movement, the motor system, can adapt to different gravitational levels and, if so, the cellular basis for this phenomenon. We studied two groups of rats that had been raised for 16 days in microgravity (eight or 14 days old at launch) and compared their walking and righting (ability to go from upside down to upright) and brain structure to those of control rats that developed on Earth. Flight rats were easily distinguished from the age-matched ground control rats in terms of both motor function and central nervous system structure. Mature surface righting predominated in control rats on the day of landing (R+O), while immature righting predominated in the flight rats on landing day and 30 days after landing. Some of these changes appear to be permanent. Several conclusions can be drawn from these studies: (1) Many aspects of motor behavior are preprogrammed into the young nervous system. In addition, several aspects of motor behavior are acquired as a function of the interaction of the developing organism and the rearing environment; (2) Widespread neuroanatomical differences between one-G- and microgravity-reared rats indicate that there is a structural basis for the adaptation to the rearing environment. These observations provide support for the idea that an animal's motor system adapts for optimal function within the environment experienced during a critical period in early postnatal life.

Walton, Kerry D.↗

Neural network training by integration of adjoint systems of equations forward in time

A method and apparatus for supervised neural learning of time dependent trajectories exploits the concepts of adjoint operators to enable computation of the gradient of an objective functional with respect to the various parameters of the network architecture in a highly efficient manner. Specifically, it combines the advantage of dramatic reductions in computational complexity inherent in adjoint methods with the ability to solve two adjoint systems of equations together forward in time. Not only is a large amount of computation and storage saved, but the handling of real-time applications becomes also possible. The invention has been applied it to two examples of representative complexity which have recently been analyzed in the open literature and demonstrated that a circular trajectory can be learned in approximately 200 iterations compared to the 12000 reported in the literature. A figure eight trajectory was achieved in under 500 iterations compared to 20000 previously required. The trajectories computed using our new method are much closer to the target trajectories than was reported in previous studies.

Toomarian, Nikzad↗

Neural Network Training by Integration of Adjoint Systems of Equations Forward in Time

A method and apparatus for supervised neural learning of time dependent trajectories exploits the concepts of adjoint operators to enable computation of the gradient of an objective functional with respect to the various parameters of the network architecture in a highly efficient manner. Specifically. it combines the advantage of dramatic reductions in computational complexity inherent in adjoint methods with the ability to solve two adjoint systems of equations together forward in time. Not only is a large amount of computation and storage saved. but the handling of real-time applications becomes also possible. The invention has been applied it to two examples of representative complexity which have recently been analyzed in the open literature and demonstrated that a circular trajectory can be learned in approximately 200 iterations compared to the 12000 reported in the literature. A figure eight trajectory was achieved in under 500 iterations compared to 20000 previously required. Tbc trajectories computed using our new method are much closer to the target trajectories than was reported in previous studies.

Toomarian, Nikzad↗

Differential Equation Approximation Using Gradient-Boosted Quantile Regression

The operation of cyber-physical-human (CPH) systems is subject to various epistemic and aleatory uncertainties. Overall trustworthiness of CPH systems relies on the trustworthiness of its components and their interactions. It is important that computational models comprising the cyber component of CPH provide predictions accompanied by a measure of confidence in model outcomes. Uncertainty quantification (UQ) and propagation are especially important in safety critical CPH systems. Gradient-boosted trees is a modeling approach capable both of learning the dynamics of a system and performing UQ. In this paper, we devise a method for using gradient boosting to learn the dynamics of a second order differential equation and estimate uncertainty at the same time. We do this by creating a custom loss function that trains the model to approximate the second derivative of a noisy time series, and to penalize based on a parameter that corresponds to the desired quantile. The resulting gradient boosting model can simulate stochastic trajectories of the system given a single starting point, that is, it can estimate both the expected trajectory and its uncertainty. We show that the uncertainty estimation is well calibrated and that the model can learn the dynamics even in the presence of noise. We demonstrate the approach on a simple cartpole system.

Autonomous systems↗