Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “reinforcement Learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 955 records · Page 53

Rapid motor learning in the translational vestibulo-ocular reflex

Motor learning was induced in the translational vestibulo-ocular reflex (TVOR) when monkeys were repeatedly subjected to a brief (0.5 sec) head translation while they tried to maintain binocular fixation on a visual target for juice rewards. If the target was world-fixed, the initial eye speed of the TVOR gradually increased; if the target was head-fixed, the initial eye speed of the TVOR gradually decreased. The rate of learning acquisition was very rapid, with a time constant of approximately 100 trials, which was equivalent to <1 min of accumulated stimulation. These learned changes were consolidated over >or=1 d without any reinforcement, indicating induction of long-term synaptic plasticity. Although the learning generalized to targets with different viewing distances and to head translations with different accelerations, it was highly specific for the particular combination of head motion and evoked eye movement associated with the training. For example, it was specific to the modality of the stimulus (translation vs rotation) and the direction of the evoked eye movement in the training. Furthermore, when one eye was aligned with the heading direction so that it remained motionless during training, learning was not expressed in this eye, but only in the other nonaligned eye. These specificities show that the learning sites are neither in the sensory nor the motor limb of the reflex but in the sensory-motor transformation stage of the reflex. The dependence of the learning on both head motion and evoked eye movement suggests that Hebbian learning may be one of the underlying cellular mechanisms.

Non-NASA Center↗

Program Helps Simulate Neural Networks

Neural Network Environment on Transputer System (NNETS) computer program provides users high degree of flexibility in creating and manipulating wide variety of neural-network topologies at processing speeds not found in conventional computing environments. Supports back-propagation and back-propagation-related algorithms. Back-propagation algorithm used is implementation of Rumelhart's generalized delta rule. NNETS developed on INMOS Transputer(R). Predefines back-propagation network, Jordan network, and reinforcement network to assist users in learning and defining own networks. Also enables users to configure other neural-network paradigms from NNETS basic architecture. Small portion of software written in OCCAM(R) language.

Villarreal, James↗

Assessing the Effects of Momentary Priming on Memory Retention During an Interference Task

A memory aid, that used brief (33ms) presentations of previously learned information (target words), was assessed on its ability to reinforce memory for target words while the subject was performing an interference task. The interference task required subjects to learn new words and thus interfered with their memory of the target words. The brief presentation (momentary memory priming) was hypothesized to refresh the subjects memory of the target words. 143 subjects, in a within subject design, were given a 33ms presentation of the target memory words during the interference task in a treatment condition and a blank 33ms presentation in the control condition. The primary dependent measure, memory loss over the interference trial, was not significantly different between the two conditions. The memory prime did not appear to hinder the subjects performance on the interference task. This paper describes the experiment and the results along with suggestions for future research.

Schutte, Paul C.↗

Modeling the behavioral substrates of associate learning and memory - Adaptive neural models

Three adaptive single-neuron models based on neural analogies of behavior modification episodes are proposed, which attempt to bridge the gap between psychology and neurophysiology. The proposed models capture the predictive nature of Pavlovian conditioning, which is essential to the theory of adaptive/learning systems. The models learn to anticipate the occurrence of a conditioned response before the presence of a reinforcing stimulus when training is complete. Furthermore, each model can find the most nonredundant and earliest predictor of reinforcement. The behavior of the models accounts for several aspects of basic animal learning phenomena in Pavlovian conditioning beyond previous related models. Computer simulations show how well the models fit empirical data from various animal learning paradigms.

Lee, Chuen-Chien↗

Physics-Informed Neural Network (PINN) Prediction of Mixed Mass-Heat-Crystallization Limited Methane Hydrate Formation and Dissociation in Micro-Confinement

The creation and use of Physics-Informed Neural Networks (PINNs) for simulating the dynamics of methane hydrate formation and dissociation will be presented. The PINN framework's main benefit is its capacity to impose physical consistency with only a partial comprehension of the governing equations. This makes the algorithm especially useful for systems with little experimental evidence or a lack of theoretical knowledge. A strong basis for forecasting methane hydrate behavior over the verified operating ranges of 30.0-80.9 bar pressure and 1.0-4.0 K sub-cooling conditions is provided by the combination of conductive heat transfer equations and mixed mass-transfer–crystallization kinetics. PINNs were more accurate at predicting the mixed mass-heat-crystallization limited kinetics than conventional Artificial Neural Networks (ANNs), demonstrating remarkable predictive accuracy for methane hydrate production over the ANN model. The efficiency of incorporating physical limitations from first principles into machine learning frameworks for methane hydrate crystallizations is reinforced by these findings. For hydrate-related applications in energy generation, carbon sequestration, and climate modelling, our study establishes PINNs as a computational tool that is both scalable and efficient. The proven capacity to close the gap between conventional physics-based simulations and solely data-driven models creates new opportunities for expedited hydrate research and practical applications.

Hartman, Ryan L [NYU Tandon School of Engineering]↗

MC-1 Engine Valves, Lessons Learned

Many lessons were learned during the development of the valves for the MC-1 engine. The purpose of this report is to focus on a variety of issues related to the engine valves and convey the lessons learned. This paper will not delve into detailed technical analysis of the components. None of the lessons learned are new or surprising, but simply reinforce the importance of addressing the details of the design early, at the component level. The Marshall Space Flight Center (MSFC), Huntsville, Alabama developed the MC-1 engine, a LOX / FW-1, 60,000 pound thrust engine. This engine was developed under the Low Cost Boost Technology office at MSFC and proved to be a very successful project for the MSFC Propulsion team and the various subcontractors working the development of the engine and its components.

Laszar, John↗

Engineering integration in elementary science classrooms: Effects of disciplinary language scaffolds on English learners' content learning and engineering identity

Abstract Background Limited research examines the effects of integrated science and engineering (SE) instruction emphasizing disciplinary literacy and language activities on engineering identity and content understanding. Far fewer studies target English learners (ELs). Purpose The impact of an SE intervention on the development of science, engineering, and technology knowledge as well as engineering identity was examined. To address ELs' learning needs, the curricular design was built on a validated SE model by integrating (1) developmental, (2) language scaffolds, and (3) culturally based accommodations. Design/Method Separate analysis of variance examined the effects of the intervention on science, engineering, and technology knowledge as well as engineering identity. The relationship among engineering identity and content outcomes was also examined. ELs from kindergarten to second grade classrooms were randomly assigned to the integrated SE group or control group. Results Integrated SE instruction significantly increased ELs' science, engineering, and technology knowledge as well as a substantially developed engineering identity. Overall, ELs' engineering identity is associated with an increase in science, engineering, and technology content knowledge. However, second grade girls' identity development was not associated with learning measures. These correlations suggest the context of the engineering activity may have reinforced gendered stereotypes and reduced the effects for girls' engineering attitudes. Conclusions Integrated SE instruction emphasizing disciplinary literacy and cultural accommodations increases early elementary ELs' learning and engineering identity. Future studies should examine the unique effects of language scaffolds and cultural modifications on student learning and the impact of gender stereotypes on girls' engineering attitudes.

Aguirre‐Muñoz, Zenaida↗

Predicting Mechanical Properties from Microstructure Images in Fiber-Reinforced Polymers Using Convolutional Neural Networks

Evaluating the mechanical response of fiber-reinforced composites can be extremely time-consuming and expensive. Machine learning (ML) techniques offer a means for faster predictions via models trained on existing input–output pairs and have exhibited success in composite research. This paper explores a fully convolutional neural network modified from StressNet, which was originally used for linear elastic materials, and extended here for a non-linear finite element (FE) simulation to predict the stress field in 2D slices of segmented tomography images of a fiber-reinforced polymer specimen. The network was trained and evaluated on data generated from the FE simulations of the exact microstructure. The testing results show that the trained network accurately captures the characteristics of the stress distribution, especially on fibers, solely from the segmented microstructure images. The trained model can make predictions within seconds in a single forward pass on an ordinary laptop, given the input microstructure, compared to 92.5 h to run the full FE simulation on a high-performance computing cluster. These results show promise in using ML techniques to conduct fast structural analysis for fiber-reinforced composites and suggest a corollary that the trained model can be used to identify the location of potential damage sites in fiber-reinforced polymers.

Sun, Yixuan (ORCID:0000000311093380)↗

Hydropower Cyber-Physical Configurations

The U.S. Department of Energy’s Water Power Technologies Office funded Pacific Northwest National Laboratory, Argonne National Laboratory, and the National Renewable Energy Laboratory to develop a typology to characterize the variety and pervasiveness of cyber-physical configurations across the nation’s hydropower fleet. Outreach to owners and operators returned configurations for 275 hydropower plants or approximately 12% of the fleet. Components (OT and IT), systems, and connections among systems differed among plants according to function, age, position in the river cascade, and many other factors. Seven cyber-physical configuration types labeled A through I, included from 2 to dozens of plants. They were differentiated by how pervasive data and control connections were among cyber-physical components and how frequently control signals paired with data signals in a feedback loop. The flow of data and control within each type implies what cybersecurity vulnerabilities may exist, and what mitigation actions may be most effective. A self-assessment approach allows plant operators to identify the configuration type similar to their plant and link to the lessons learned and best practices information. The cyber-physical typology reinforces the idea that hydropower facilities vary widely, but it also identifies groups that highlight similarities in how their cyber-physical components interact. This helps address fleetwide cybersecurity needs by identifying a reasonable number of configuration types that share risks, vulnerabilities, and potential mitigations.

13 HYDRO ENERGY↗

CRCNS US-France Research Proposal: Collaborative Research: Encoding reward expectation in Drosophilia

The fruit fly Drosophila melanogaster has been a valuable model for investigating the genetic and neural bases that underlie learning and memory. Early and most current studies use basic behavior conditioning protocols to study learning in controlled laboratory settings. More recently, the ability to transgenically manipulate many of the brain neurons in the fruit fly with exquisite specificity, and the recent knowledge of the synaptic ‘connectome’ of the fruit fly brain, makes these animals almost unique as a comprehensive model for studies of learning, memory and motivated behavior. In fact, the connectome has revealed many types of new connections that had until now been overlooked. Within this context, the thesis of this proposal is that studies of learning and memory will be greatly enhanced by using more sophisticated means for evaluating memory representations, such as have been developed in vertebrates, and combining those studies with information from the connectome guided by computational modelling. We propose to push beyond the boundaries of existing conditioning protocols for fruit flies to investigate more complex memory representations. In particular, we will investigate the function of reinforcement pathways in relation to the absence of expected reinforcement. More specifically, we propose a series of experiments designed to investigate the memory representations in fruit flies when an expected consequence of a Conditioned Stimulus (CS) fails to occur. Although studies have evaluated how this failure can establish extinction memory for the CS, our studies will go beyond studying extinction. Specifically, we predict that in Drosophila when a CS is associated with a failed expectation of an appetitive food reinforcement it will acquire aversive value, and vice versa for a failed expectation of an aversive reinforcer. We combine these studies with manipulations of reinforcement pathways in the CNS inspired from the connectome, iteratively knitted in with established computational models. Intellectual Merit: The concept of reinforcement expectation and incentive contrast have been influential in the development of studies of associative learning in mammals. These questions are particularly challenging to answer in vertebrates because they require exquisite cellular, temporal, and genetic specificity of experimental manipulations. The recent development of work with identified neurons and their connectomes makes the larval and adult fly brains ripe as models for pushing our understanding of neural bases for these higher- order conditioning phenomena. Broader Impacts: Public health: These analyses and the conceptual framework of prediction error processing underlying them have a profound impact on our understanding of reinforcement-related behavior in humans, including monetary rewards and the mnemonic consequences of traumatic experiences, and for pathologies of the dopamine reinforcement system. Educational: This project will provide interdisciplinary training for postdoctoral researchers, Ph.D. and undergraduate students. The PIs will act as co-supervisors or mentors of students working in the different labs via face-to-face and internet-based technologies. We will also work with ASU’s award-winning Ask- A-Biologist program. This is an online science program designed to enrich the learning experiences of students of all ages and to provide classroom material for use by K-12 teachers. We will develop an extension of a game developed under a prior NSF award, and the new game will include modules to teach K-12 students about how insects learn. We will also integrate into the AAB site a program developed by a collaborator (B Gerber) at the Leibniz Institut für Neurobiologie, Magdeburg, and now in use in schools in Germany, to teach K-12 students how to train animals using the fruit fly larval learning paradigm. Underrepresented groups: All PIs will work with their university offices of Academic Diversity and Equal Opportunity for reaching underrepresented students.

59 BASIC BIOLOGICAL SCIENCES↗

Operation, Manufacturing, and Supply Chain of Lithium-ion Batteries for Electric Vehicles

The US Department of Energy (DOE) has a rich history of funding key innovations in batteries through multiple programs, including the Energy Efficiency and Renewable Energy programs sponsored by the Vehicle Technologies Office and Advanced Manufacturing Office. These DOE-sponsored innovations have been critical in lowering the cost of lithium-ion batteries to less than $160/kWh and increasing their energy density to more than 250 Wh/kg, enhancing the nation’s electrification system and achieving higher degrees of decarbonization. These investments continue to be crucial for positioning the United States as the hub for the most cumulative battery R&D programs worldwide. However, major breakthroughs in battery manufacturing are still needed to enable an electrified future evolving to predominantly incorporate rechargeable lithium-ion batteries. This will require a strong battery manufacturing base that will not only enable cutting-edge advancements in materials and manufacturing sciences but will also allow the United States to become a world leader in this fast-growing area. This review aims at supporting DOE’s Advanced Technology Vehicle Manufacturing Loan Program to give technology background and information that can be used to develop the basis for loan topics and evaluate battery applications. Such a program can represent a significant stride toward reinforcing the US position in advanced manufacturing by introducing artificial intelligence, machine learning, and sensing and smart controls to facilitate, expedite, and lead the battery industry by 2030.

25 ENERGY STORAGE↗

Transfer learning for probabilistic localization of hidden cracks in concrete structures

Abstract The utility of discriminative supervised learning models built using multiple training-data sources is investigated for hidden crack localization in concrete. Feed-forward neural network (FFNN) is chosen as the model architecture, and transfer learning is used to assimilate the information obtained from different sources (computational physics simulations and laboratory experiments). The labeled training data consists of values of a damage index and the known locations of hidden cracks. The classification models need to learn how the presence of damage (hidden cracks) affects the damage index at different sensors for different test conditions. To this end, diagnostic FFNN models are built by sequentially adding and training new hidden layers to assimilate labeled information from computer models (different model geometries, test conditions, crack lengths, crack locations) and laboratory experiments on a plain cement slab. These transfer learning-based models are then used to localize damage in concrete specimens that reflect real-world conditions (i.e., specimens with steel reinforcement and randomly distributed aggregate). The actual damage state in these specimens is determined by extracting cores and performing petrographic studies on the extracted cores. The damage probability estimated by transfer learning-based models is compared with the petrographic damage rating index (DRI) to identify the most suitable approach to train the diagnostic models. The transfer learning-based diagnostic methodology shows promise and could be used in various structural health monitoring applications, where sufficient labeled data are typically not available from a single data source.

Miele, S.↗

Design and Control of Large Collections of Learning Agents

The intelligent control of multiple autonomous agents is an important yet difficult task. Previous methods used to address this problem have proved to be either too brittle, too hard to use, or not scalable to large systems. The 'Collective Intelligence' project at NASA/Ames provides an elegant, machine-learning approach to address these problems. This approach mathematically defines some essential properties that a reward system should have to promote coordinated behavior among reinforcement learners. This work has focused on creating additional key properties and algorithms within the mathematics of the Collective Intelligence framework. One of the additions will allow agents to learn more quickly, in a more coordinated manner. The other will let agents learn with less knowledge of their environment. These additions will allow the framework to be applied more easily, to a much larger domain of multi-agent problems.

Agogino, Adrian↗

Research rocket test RR-1 (Black Brant VC) and RR-2 (Aerobee 170A): Investigations of the stability of bubbles in plain and fiber-reinforced metal and solidified in a near-zero-g environment

The results of the first two of a series of research rocket flights are presented. The objectives of these flights were (1) to learn about the capabilities of these rockets, (2) to learn how to interface the payloads and rockets, and (3) to process some of the composite casting demonstration capsules intended originally for Apollo 15. The capsules contained experiments for investigating the stability of gas bubbles in plain and fiber-reinforced metal melted and solidified in a near-zero-g (0.0119g) environment. The characteristics of the two research rockets, an Aerobee 170A and a Black Brant VC, used to obtain the periods of near-zero-g and the temperature control unit used for processing the contents of the two experiment capsules are discussed.

Yates, I. C.↗

Machine learning-enabled multiscale modeling of mechanical deformation of aluminum and Al-SiC nanocomposites

A machine learning-enabled multiscale framework is developed for modeling the mechanical response of both pure metal and nanoparticle-reinforced metal matrix nanocomposites (MMNCs). Using aluminum–silicon carbide (Al-SiC) as an example MMNC, atomistic simulations reveal three distinct deformation mechanisms (i.e., defect-free, dislocation-based, and interface separation) governed by the interfaces between the Al matrix and SiC nanoparticles. As compared with single crystal Al, the lattice undergoes a more abrupt failure once the dislocation network becomes extensive and void nucleation initiates, whereas in Al-SiC, nanoparticle interfaces enable a more gradual progression of damage. These mechanisms are captured through a combined classification-regression neural network surrogate model that bridges atomic-scale insights with continuum-scale finite element analysis. Machine learning-enabled multiscale modeling of pure Al accurately predicted strain localization and confirmed by in-situ scanning electron microscopic tensile testing on perforated Al specimens. This study underscores the promise of integrating physics-informed machine learning with hierarchical modeling to capture the interface dominated phenomena and guide the design of advanced MMNCs.

Al-SiC↗

Using Gamification to Enhance Mastery of Network Security Concepts

Gamification has proven to be effective in engaging and encouraging people to work towards and achieve goals. Many students struggle to focus on schoolwork, due to a lack of interest, lack of understanding, or other factors unique to the student. Applying gamification elements to education can help engage these students in learning their course material and help them excel academically. This study examines the effectiveness of using gamification techniques to enhance the learning experience in college Computer Science courses. A video game application is utilized to review and reinforce cybersecurity concepts that students have already been taught in class. Previous work has been made on a prototype game build that teaches about ARP (Address Resolution Protocol) components. The focus of this study is to refine and develop the structure of the prototype into a more interactive and enjoyable format with non-competitive and captivating activities that allow students to study at their own pace. An updated version of the game was created that focused on reaching a balance between education and entertainment. The game was used by students enrolled in a cybersecurity class, where pre-survey, post-survey and a focus group interview were conducted to determine how effective the updated version is compared to the current build, in addition to how effective the gamification method is regarding student retention of taught material. The pre-survey and post-survey results revealed an increase in interest and mastery of cybersecurity concepts as a result of playing the game. Students found value in the game as both a method of reviewing material taught in class and an entertaining and engaging game. These results show potential in using gamification in cybersecurity and education.

Hilliard, Kevin↗