Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “learned”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Use of machine learning to analyze chemistry card sort tasks

Education researchers are deeply interested in understanding the way students organize their knowledge. Card sort tasks, which require students to group concepts, are one mechanism to infer a student’s organizational strategy. However, the limited resolution of card sort tasks means they necessarily miss some of the nuance in a student’s strategy. Here in this work, we propose new machine learning strategies that leverage a potentially richer source of student thinking: free-form written language justifications associated with student sorts. Using data from a university chemistry card sort task, we use vectorized representations of language and unsupervised learning techniques to generate qualitatively interpretable clusters, which can provide unique insight in how students organize their knowledge. We compared these to machine learning analysis of the students’ sorts themselves. Machine learning-generated clusters revealed different organizational strategies than those built into the task; for example, sorts by difficulty or even discipline. There were also many more categories generated by machine learning for what we would identify as more novice-like sorts and justifications than originally built into the task, suggesting students’ organizational strategies converge when they become more expert-like. Finally, we learned that categories generated by machine learning for students’ justifications did not always match the categories for their sorts, and these cases highlight the need for future research on students’ organizational strategies, both manually and aided by machine learning. In sum, the use of machine learning to analyze results from a card sort task has helped us gain a more nuanced understanding of students’ expertise, and demonstrates a promising tool to add to existing analytic methods for card sorts.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Versatile feature learning with graph convolutions and graph structures

Graphs represent real world relationships, and graph embedding projects nodes in a graph to a latent space that can help simplify downstream tasks. Recent development of graph convolutions in deep learning significantly improves the performance of many learning tasks on graphs. Unfortunately, prior embedding methods either do not embed graphs with node features, or fail to produce high-quality embeddings for downstream learning tasks that result in large performance gap in comparison to direct learning on graphs.We present a versatile and effective embedding method, Conv2Vec, for embedding graphs with or without node features. It is based on graph convolutions with objective functions motivated by concepts and structures from classical graph algorithms. Conv2Vec produce high-quality embedding for both plain graphs and graphs with node features for downstream tasks.We evaluate the embeddings generated by Conv2Vec with a transductive node classification task. With the generated embeddings and very simple machine learning approaches, we are able to achieve accuracies similar to those achieved by direct learning with graph convolutions. Interestingly, if we strip the node features from the graph and thus learning an embedding has to rely entirely on the graph topology, node classification with our embedding significantly outperforms direct learning with various graph convolutions. This suggests that structures from classical graph algorithms may play an important role in learning on graphs.

Cong, Guojing↗
DOE OSTI2021

Evolutionary NAS in Light of Model Stability for Accurate Continual Learning

Continual learning, the capability to learn new knowledge from streaming data without forgetting the previous knowledge, is a critical requirement for dynamic learning systems, especially for emerging edge devices such as self-driving cars and drones. However, continual learning is still facing the catastrophic forgetting problem. Previous work illustrate that model performance on continual learning is not only related to the learning algorithms but also strongly dependent on the inherited model, i.e., the model where continual learning starts. The better stability of the inherited model, the less catastrophic forgetting and thus, the inherited model should be elaborately selected. Inspired by this finding, we develop an evolutionary neural architecture search (ENAS) algorithm that emphasizes the Stability of the inherited model, namely ENAS-S. ENAS-S aims to find optimal architectures for accurate continual learning on edge devices. On CIFAR-10 and CIFAR-100, we present that ENAS-S achieves competitive architectures with lower catastrophic forgetting and smaller model size when learning from a data stream, as compared with handcrafted DNNs.

Du, Xiaocong↗
DOE OSTI2019

Computationally Efficient Training of Deep Neural Networks via Transfer Learning

Transfer learning is a highly successful approach to train deep neural networks for customized image recognition tasks. Transfer learning leverages deep learning models that are previously trained on massive datasets and re-trains the network for a novel image recognition dataset. Typically, the advantage of transfer learning has been measured in sample efficiency, but instead, we investigate the computational efficiency of transfer learning. A good pre-trained model provides features that can be used as input to a new classifier (usually the top layers of a neural network). We show that if a good pre-trained model is selected, that training a new classifier can be much more computationally efficient than training a deep neural network without transfer learning. The first step in transfer learning is to select a pre-trained model to use as a feed-forward network for generating features. Here this selection process either uses human intuition/convenience or a methodical but computationally intensive validation loop. Instead, we would prefer a method to select the pre-trained model that will produce the best transfer results for the least amount of computation. To this end, we provide a computationally efficient metric for the fit between a pre-trained model and a novel image recognition task. The better the fit, the less computation will be needed to re-train the pre-trained model for the novel task. As machine learning becomes ubiquitous, highly-accurate trained models will proliferate and computationally efficient transfer learning methods will enable rapid development of new image recognition models.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

A Data-Agnostic, Continuous Machine Learning Framework for Application in High Energy Physics and Beyond: Phase 1 Final Scientific/Technical Report

This Phase 1 effort has focused on the development of continual learning frameworks for use in machine learning, specifically in the applied context of High Energy Physics (HEP). Machine learning (ML) is a transformative technology by which computers, typically through the use of neural networks, are able to perform tasks with proficiency that rivals or surpasses that of human users. Model Degradation & Catastrophic Forgetting are two undesired phenomena which can occur in ML where the performance of a model degrades when either deployed on novel data streams, or trained on novel data which are sufficiently different than the data the models were initially trained on. A natural example where these sorts of effects can be observed is in the performance of detectors in harsh environments, where the detector signature may change over the lifetime of the detector as it ages and deteriorates — precisely what occurs in the experiments conducted in HEP. Real world HEP data is therefore an excellent test-ground and use-case for Continual Learning paradigms, which are techniques used in ML to counteract these problems. Ensemble learning is one such technique, where multiple smaller models are trained on subsets of the overall data and are ensembled together during inference. The intuition behind this technique is that, although there are shifts in the distributions which govern the incoming data streams, these shifts are not expected to be homogeneous or global. If a sufficient diversity in solutions within the various sub-models has been achieved, then at least one sub-model is expected to retain its performance within the overall ensemble. One further strength of this approach is that the architectures of the various models do not need to be identical, and in fact even different modalities of data can naturally be combined in this way. This work focused on applying ensemble learning techniques to derive results using two main datasets, anomaly detection in HEP data & time-series forecasting in semiconductor manufacturing data. Semiconductor manufacturing involves data with surprising similarity to that of HEP (e.g. wafer maps look very similar to digi-occupancy maps) and Cerium Lab’s prominence within the semiconductor industry makes semiconductor manufacturing a natural opportunity for commercialization of this work. Our efforts have led to two strong results. The first is that we evaluated the proposed ensembling techniques using previously proposed machine learning architectures for use in anomaly detection, namely AutoEncoder based models and their derivatives. We also developed new architectures which have not been evaluated in this context before. In fact, this work marks the first use of Vision Transformers for anomaly detection in HEP. Second, we demonstrated that ensemble learning significantly improves model performance in scenarios prone to degradation, validating its effectiveness across both HEP and semiconductor datasets. These results further support ensemble learning as a powerful strategy for mitigating catastrophic forgetting and maintaining robust performance in evolving data environments.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS1992

Indirect decentralized learning control

The new field of learning control develops controllers that learn to improve their performance at executing a given task, based on experience performing this specific task. In a previous work, the authors presented a theory of indirect learning control based on use of indirect adaptive control concepts employing simultaneous identification and control. This paper develops improved indirect learning control algorithms, and studies the use of such controllers in decentralized systems. The original motivation of the learning control field was learning in robots doing repetitive tasks such as on an assembly line. This paper starts with decentralized discrete time systems, and progresses to the robot application, modeling the robot as a time varying linear system in the neighborhood of the nominal trajectory, and using the usual robot controllers that are decentralized, treating each link as if it is independent of any coupling with other links. The basic result of the paper is to show that stability of the indirect learning controllers for all subsystems when the coupling between subsystems is turned off, assures convergence to zero tracking error of the decentralized indirect learning control of the coupled system, provided that the sample time in the digital learning controller is sufficiently short.

Longman, Richard W.↗
NASA NTRS1998

Preliminary Work for Examining the Scalability of Reinforcement Learning

Researchers began studying automated agents that learn to perform multiple-step tasks early in the history of artificial intelligence (Samuel, 1963; Samuel, 1967; Waterman, 1970; Fikes, Hart & Nilsonn, 1972). Multiple-step tasks are tasks that can only be solved via a sequence of decisions, such as control problems, robotics problems, classic problem-solving, and game-playing. The objective of agents attempting to learn such tasks is to use the resources they have available in order to become more proficient at the tasks. In particular, each agent attempts to develop a good policy, a mapping from states to actions, that allows it to select actions that optimize a measure of its performance on the task; for example, reducing the number of steps necessary to complete the task successfully. Our study focuses on reinforcement learning, a set of learning techniques where the learner performs trial-and-error experiments in the task and adapts its policy based on the outcome of those experiments. Much of the work in reinforcement learning has focused on a particular, simple representation, where every problem state is represented explicitly in a table, and associated with each state are the actions that can be chosen in that state. A major advantage of this table lookup representation is that one can prove that certain reinforcement learning techniques will develop an optimal policy for the current task. The drawback is that the representation limits the application of reinforcement learning to multiple-step tasks with relatively small state-spaces. There has been a little theoretical work that proves that convergence to optimal solutions can be obtained when using generalization structures, but the structures are quite simple. The theory says little about complex structures, such as multi-layer, feedforward artificial neural networks (Rumelhart & McClelland, 1986), but empirical results indicate that the use of reinforcement learning with such structures is promising. These empirical results make no theoretical claims, nor compare the policies produced to optimal policies. A goal of our work is to be able to make the comparison between an optimal policy and one stored in an artificial neural network. A difficulty of performing such a study is finding a multiple-step task that is small enough that one can find an optimal policy using table lookup, yet large enough that, for practical purposes, an artificial neural network is really required. We have identified a limited form of the game OTHELLO as satisfying these requirements. The work we report here is in the very preliminary stages of research, but this paper provides background for the problem being studied and a description of our initial approach to examining the problem. In the remainder of this paper, we first describe reinforcement learning in more detail. Next, we present the game OTHELLO. Finally we argue that a restricted form of the game meets the requirements of our study, and describe our preliminary approach to finding an optimal solution to the problem.

Clouse, Jeff↗
NASA NTRS1996

Cascade Error Projection: A Learning Algorithm for Hardware Implementation

In this paper, we workout a detailed mathematical analysis for a new learning algorithm termed Cascade Error Projection (CEP) and a general learning frame work. This frame work can be used to obtain the cascade correlation learning algorithm by choosing a particular set of parameters. Furthermore, CEP learning algorithm is operated only on one layer, whereas the other set of weights can be calculated deterministically. In association with the dynamical stepsize change concept to convert the weight update from infinite space into a finite space, the relation between the current stepsize and the previous energy level is also given and the estimation procedure for optimal stepsize is used for validation of our proposed technique. The weight values of zero are used for starting the learning for every layer, and a single hidden unit is applied instead of using a pool of candidate hidden units similar to cascade correlation scheme. Therefore, simplicity in hardware implementation is also obtained. Furthermore, this analysis allows us to select from other methods (such as the conjugate gradient descent or the Newton's second order) one of which will be a good candidate for the learning technique. The choice of learning technique depends on the constraints of the problem (e.g., speed, performance, and hardware implementation); one technique may be more suitable than others. Moreover, for a discrete weight space, the theoretical analysis presents the capability of learning with limited weight quantization. Finally, 5- to 8-bit parity and chaotic time series prediction problems are investigated; the simulation results demonstrate that 4-bit or more weight quantization is sufficient for learning neural network using CEP. In addition, it is demonstrated that this technique is able to compensate for less bit weight resolution by incorporating additional hidden units. However, generation result may suffer somewhat with lower bit weight quantization.

Duong, Tuan A.↗
NASA NTRS1994

Goal Directed Model Inversion: Learning Within Domain Constraints

Goal Directed Model Inversion (GDMI) is an algorithm designed to generalize supervised learning to the case where target outputs are not available to the learning system. The output of the learning system becomes the input to some external device or transformation, and only the output of this device or transformation can be compared to a desired target. The fundamental driving mechanism of GDMI is to learn from success. Given that a wrong outcome is achieved, one notes that the action that produced that outcome "would have been right if the outcome had been the desired one." The algorithm makes use of these intermediate "successes" to achieve the final goal. A unique and potentially very important feature of this algorithm is the ability to modify the output of the learning module to force upon it a desired syntactic structure. This differs from ordinary supervised learning in the following way: in supervised learning the exact desired output pattern must be provided. In GDMI instead, it is possible to require simply that the output obey certain rules, i.e., that it "make sense" in some way determined by the knowledge domain. The exact pattern that will achieve the desired outcome is then found by the system. The ability to impose rules while allowing the system to search for its own answers in the context of neural networks is potentially a major breakthrough in two ways: 1) it may allow the construction of networks that can incorporate immediately some important knowledge, i.e. would not need to learn everything from scratch as normally required at present, and 2) learning and searching would be limited to the areas where it is necessary, thus facilitating and speeding up the process. These points are illustrated with examples from robotic path planning and parametric design.

Colombano, Silvano P.↗
NASA NTRS1994

Goal Directed Model Inversion: Learning Within Domain Constraints

Goal Directed Model Inversion (GDMI) is an algorithm designed to generalize supervised learning to the case where target outputs are not available to the learning system. The output of the learning system becomes the input to some external device or transformation, and only the output of this device or transformation can be compared to a desired target. The fundamental driving mechanism of GDMI is to learn from success. Given that a wrong outcome is achieved, one notes that the action that produced that outcome "would have been right if the outcome had been the desired one." The algorithm makes use of these intermediate "successes" to achieve the final goal. A unique and potentially very important feature of this algorithm is the ability to modify the output of the learning module to force upon it a desired syntactic structure. This differs from ordinary supervised learning in the following way: in supervised learning the exact desired output pattern must be provided. In GDMI instead, it is possible to require simply that the output obey certain rules, i.e., that it "make sense" in some way determined by the knowledge domain. The exact pattern that will achieve the desired outcome is then found by the system. The ability to impose rules while allowing the system to search for its own answers in the context of neural networks is potentially a major breakthrough in two ways: (1) it may allow the construction of networks that can incorporate immediately some important knowledge, i.e., would not need to learn everything from scratch as normally required at present; and (2) learning and searching would be limited to the areas where it is necessary, thus facilitating and speeding up the process. These points are illustrated with examples from robotic path planning and parametric design.

Colombano, Silvano P.↗
NASA NTRS2001

Lessons Learned and Technical Standards: A Logical Marriage

A comprehensive database of lessons learned that corresponds with relevant technical standards would be a boon to technical personnel and standards developers. The authors discuss the emergence of one such database within NASA, and show how and why the incorporation of lessons learned into technical standards databases can be an indispensable tool for government and industry. Passed down from parent to child, teacher to pupil, and from senior to junior employees, lessons learned have been the basis for our accomplishments throughout the ages. Government and industry, too, have long recognized the need to systematically document And utilize the knowledge gained from past experiences in order to avoid the repetition of failures and mishaps. The use of lessons learned is a principle component of any organizational culture committed to continuous improvement. They have formed the foundation for discoveries, inventions, improvements, textbooks, and technical standards. Technical standards are a very logical way to communicate these lessons. Using the time-honored tradition of passing on lessons learned while utilizing the newest in information technology, the National Aeronautics and Space Administration (NASA) has launched an intensive effort to link lessons learned with specific technical standards through various Internet databases. This article will discuss the importance of lessons learned to engineers, the difficulty in finding relevant lessons learned while engaged in an engineering project, and the new NASA project that can help alleviate this difficulty. The article will conclude with recommendations for more expanded cross-sectoral uses of lessons learned with reference to technical standards.

Gill, Paul↗
NASA NTRS2006

QUICR-learning for Multi-Agent Coordination

Coordinating multiple agents that need to perform a sequence of actions to maximize a system level reward requires solving two distinct credit assignment problems. First, credit must be assigned for an action taken at time step t that results in a reward at time step t > t. Second, credit must be assigned for the contribution of agent i to the overall system performance. The first credit assignment problem is typically addressed with temporal difference methods such as Q-learning. The second credit assignment problem is typically addressed by creating custom reward functions. To address both credit assignment problems simultaneously, we propose the "Q Updates with Immediate Counterfactual Rewards-learning" (QUICR-learning) designed to improve both the convergence properties and performance of Q-learning in large multi-agent problems. QUICR-learning is based on previous work on single-time-step counterfactual rewards described by the collectives framework. Results on a traffic congestion problem shows that QUICR-learning is significantly better than a Q-learner using collectives-based (single-time-step counterfactual) rewards. In addition QUICR-learning provides significant gains over conventional and local Q-learning. Additional results on a multi-agent grid-world problem show that the improvements due to QUICR-learning are not domain specific and can provide up to a ten fold increase in performance over existing methods.

Agogino, Adrian K.↗
NASA NTRS2011

Collaborative Supervised Learning for Sensor Networks

Collaboration methods for distributed machine-learning algorithms involve the specification of communication protocols for the learners, which can query other learners and/or broadcast their findings preemptively. Each learner incorporates information from its neighbors into its own training set, and they are thereby able to bootstrap each other to higher performance. Each learner resides at a different node in the sensor network and makes observations (collects data) independently of the other learners. After being seeded with an initial labeled training set, each learner proceeds to learn in an iterative fashion. New data is collected and classified. The learner can then either broadcast its most confident classifications for use by other learners, or can query neighbors for their classifications of its least confident items. As such, collaborative learning combines elements of both passive (broadcast) and active (query) learning. It also uses ideas from ensemble learning to combine the multiple responses to a given query into a single useful label. This approach has been evaluated against current non-collaborative alternatives, including training a single classifier and deploying it at all nodes with no further learning possible, and permitting learners to learn from their own most confident judgments, absent interaction with their neighbors. On several data sets, it has been consistently found that active collaboration is the best strategy for a distributed learner network. The main advantages include the ability for learning to take place autonomously by collaboration rather than by requiring intervention from an oracle (usually human), and also the ability to learn in a distributed environment, permitting decisions to be made in situ and to yield faster response time.

Wagstaff, Kiri L.↗
NASA NTRS2011

Category Learning Research in the Interactive Online Environment Second Life

The interactive online environment Second Life allows users to create novel three-dimensional stimuli that can be manipulated in a meaningful yet controlled environment. These features suggest Second Life's utility as a powerful tool for investigating how people learn concepts for unfamiliar objects. The first of two studies was designed to establish that cognitive processes elicited in this virtual world are comparable to those tapped in conventional settings by attempting to replicate the established finding that category learning systematically influences perceived similarity . From the perspective of an avatar, participants navigated a course of unfamiliar three-dimensional stimuli and were trained to classify them into two labeled categories based on two visual features. Participants then gave similarity ratings for pairs of stimuli and their responses were compared to those of control participants who did not learn the categories. Results indicated significant compression, whereby objects classified together were judged to be more similar by learning than control participants, thus supporting the validity of using Second Life as a laboratory for studying human cognition. A second study used Second Life to test the novel hypothesis that effects of learning on perceived similarity do not depend on the presence of verbal labels for categories. We presented the same stimuli but participants classified them by selecting between two complex visual patterns designed to be extremely difficult to label. While learning was more challenging in this condition , those who did learn without labels showed a compression effect identical to that found in the first study using verbal labels. Together these studies establish that at least some forms of human learning in Second Life parallel learning in the actual world and thus open the door to future studies that will make greater use of the enriched variety of objects and interactions possible in simulated environments compared to traditional experimental situations.

Andrews, Jan↗
NASA NTRSDate not supplied

Long-term Engagement in Authentic Research with NASA (LEARN): Innovative Practices Suggested by a New Model for Teacher Research Experiences

NASA's LEARN Project is an innovative program that provides long-term immersion in the practice of atmospheric science for middle and high school in-service teachers. Working alongside NASA scientists and using authentic NASA Science Mission Directorate research data, teachers develop individual research topics of interest during two weeks in the summer while on-site at NASA Langley. With continued, intensive mentoring by NASA scientists, the teachers further develop their research throughout the academic year through virtual group meetings and data team meetings mirroring scientific collaborations. At the end of the first year, LEARN teachers present scientific posters. The LEARN experience has had such an impact that multiple teachers from the first two cohorts have elected to continue their research. The LEARN project evaluation has provided insights into particularly effective elements of this new approach. Findings indicate that teachers? perceptions of the scientific enterprise have changed, and that LEARN provided substantial resources to help them take real-world research to their students. This presentation will focus on key factors from LEARN?s implementation that inform best practices for the incorporation of authentic scientific research into teacher professional development experiences. We suggest that these factors should be considered in the development of other such experiences, including: (1) The involvement of a single scientist as both the project leader/manager and the project scientist, to ensure that the project can meet teachers? needs. (2) An emphasis on framing and approaching scientific research questions, so that teachers can learn to evaluate the feasibility of studies based on scope, scale, and availability of data. (3) Long term, ongoing relationships where teachers and scientists work as collaborators, beyond the workshop ?mold.? (4) A focus on meeting the needs of individual teachers, whether their needs relate to elements of research and analysis, or to their tight professional schedules. (5) Above all, flexibility and patience. LEARN builds relationships with teachers slowly, over a long period of time. In the middle, life often intervenes. LEARN has emphasized that teachers? success is more important than deadlines or following a rigid protocol.

M Pippin↗
NASA NTRS2020

Efficient Active Learning for New Domains

The promise of active learning is to reduce the number of labeled examples required by supervised machine learning algorithms. The largest potential benefits lie in entirely new domains, for which no labeled examples yet exist. Yet to date, most active learning studies are retroactive and demonstrate the benefits that could have been gained if active learning had been used. What are the barriers to true adoption and utilization of active learning? We focus on two: (1) the cold start or class discovery problem, in which active learning methods may struggle to make progress with zero labeled examples, and (2) the cost of having the classifier in the loop to select the next example to be labeled. We assess different active learning approaches in the context of these two barriers and conclude with recommendations for how to employ active learning in new domains. As an example, we report on the use of active learning on a large, novel data set of Mars surface images.

Lu, Steven↗
DOE OSTI2020

Reinforcement learning based automated history matching for improved hydrocarbon production forecast

History matching aims to find a numerical reservoir model that can be used to predict the reservoir performance. An engineer and model calibration (data inversion) method are required to adjust various parameters/properties of the numerical model in order to match the reservoir production history. In this study, we develop deep neural networks within the reinforcement learning framework to achieve automated history matching that will reduce engineers’ efforts, human bias, automatically and intelligently explore the parameter space, and remove the need of large set of labeled training data. To that end, a fast-marching-based reservoir simulator is encapsulated as an environment for the proposed reinforcement learning. The deep neural-network-based learning agent interacts with the reservoir simulator within reinforcement learning framework to achieve the automated history matching. Reinforcement learning techniques, such as discrete Deep Q Network and continuous Deep Deterministic Policy Gradients, are used toth, used to train the learning agents. The continuous actions enable the Deep Deterministic Policy Gradients to explore more states at each iteration in a a learning episode; consequently, a better history matching is achieved using this algorithm as compared to Deep Q Network. For simplified dual-target composite reservoir models, the best history-matching performances of the discrete and continuous learning methods in terms of normalized root mean square errors are 0.0447 and 0.0038, respectively. Furthermore, our study shows that continuous action space achieved by the deep deterministic policy gradient drastically outperforms deep Q network.

42 ENGINEERING↗
DOE OSTI2023

Inverse reinforcement learning control for building energy management

Reinforcement learning (RL) based control is widely considered a promising approach in building automation and control as it has demonstrated the potential to deal with complex objectives in adjacent domains like robotics, autonomous vehicles, gaming applications, and advertisement recommendations. When applied to any environment, model-free RL learns to improve its control performance over time without requiring a control model, by receiving and then analyzing feedback from the building environment after each control action. Operational objectives are becoming increasingly complex through the simultaneous consideration of thermal comfort, carbon emissions, grid services, and indoor air quality. In this context, conventional rule-based control approaches are proving sub-optimal, mostly heuristic, and inadequate. The model-free and self learning nature of RL appears promising and attractive as it may address the scalability issues associated with advanced control approaches. However, it suffers from long training times and unstable control behavior during the early stages of its learning process, which makes it unsuitable to be applied directly to buildings. This paper addresses these issues using an inverse reinforcement learning approach (IRL), a technique utilized to learn the objective of a controller agent which is considered an expert in its respective domain. Here, we consider a rule-based control as the expert demonstrator. IRL is different from a direct imitation (i.e., direct mapping of states to actions) of control actions as it tries to find the underlying intent of an expert's policy, providing the controller with a better-generalized policy for unseen states or environments with slightly different dynamics. This approach propels the RL controller's policy to levels similar to or better than that of a rule-based policy before it starts learning by interacting with the building. This makes RL for building energy management applications more practical as it prevents the erratic and exploratory behavior in the initial training period, simultaneously speeding up the learning process when compared to applying an untrained RL agent directly to a building environment.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗