Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Online Learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17

An Online Approach to Solve the Dynamic Vehicle Routing Problem with Stochastic Trip Requests for Paratransit Services

Many transit agencies operating paratransit and microtransit services have to respond to trip requests that arrive in real-time, which entails solving hard combinatorial and sequential decision-making problems under uncertainty. To avoid decisions that lead to significant inefficiency in the long term, vehicles should be allocated to requests by optimizing a non-myopic utility function or by batching requests together and optimizing a myopic utility function. While the former approach is typically offline, the latter can be performed online. We point out two major issues with such approaches when applied to paratransit services in practice. First, it is difficult to batch paratransit requests together as they are temporally sparse. Second, the environment in which transit agencies operate changes dynamically (e.g., traffic conditions can change over time), causing the estimates that are learned offline to become stale. To address these challenges, we propose a fully online approach to solve the dynamic vehicle routing problem (DVRP) with time windows and stochastic trip requests that is robust to changing environmental dynamics by construction. We focus on scenarios where requests are relatively sparse—our problem is motivated by applications to paratransit services. We formulate DVRP as a Markov decision process and use Monte Carlo tree search to evaluate actions for any given state. Accounting for stochastic requests while optimizing a non-myopic utility function is computationally challenging; indeed, the action space for such a problem is intractably large in practice. To tackle the large action space, we leverage the structure of the problem to design heuristics that can sample promising actions for the tree search. Our experiments using real-world data from our partner agency show that the proposed approach outperforms existing state-of-the-art approaches both in terms of performance and robustness.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Chapter 18: Learning and Tracking Ad Hoc Fiducial Markers in Spatial Augmented Reality

We describe a spatial augmented reality system with a tangible user interface used to control computer simulations of complex systems. In spatial augmented reality, the user’s physical space is augmented with projected imagery, blending real objects with projected information, and a tangible user interface enables users to manipulate physical objects as controllers for interactive visualizations. Our system learns ad hoc objects in the user’s environment as fiducial markers (i.e., objects that are visually recognized and tracked). When combined with simulation and visualization tools, these interfaces allow the user to control simulations or ensembles of simulations via physical objects using apt metaphors. While other research has leveraged the use of depth cameras, our system enables the use of standard cameras in readily available smartphones and webcams and has an implementation that runs completely in JavaScript in the web browser. We discuss the prerequisite object-recognition requirements for such tangible user interfaces and describe computer-vision and machine-learning algorithms meeting those requirements. We conclude by presenting example applications, which are also available online.

energy distribution↗

Deep Reinforcement Learning From Demonstrations to Assist Service Restoration in Islanded Microgrids

Microgrids can be operated in island mode during utility grid outages to support service restoration and improve system resilience. To schedule and dispatch distributed energy resources (DERs) in an islanded microgrid, conventional model-based methods rely on accurate distribution network models and lack generalization and adaptability. Data-driven methods are promising for DER coordination but face practical challenges such as potential hazards to microgrids during online training and insufficient online training opportunities due to low outage rates. This paper presents a novel two-stage learning framework that builds on the deep deterministic policy gradient from demonstrations to identify an optimal restoration strategy. At the pre-training stage, imitation learning is applied to equip the control agent with expert experiences to guarantee acceptable initial performance. At the online training stage, action clipping, reward shaping, and expert demonstrations are leveraged to ensure safe exploration while accelerating the training process. In conclusion, the proposed method is illustrated using the IEEE 123-node system and compared with a representative model-based method and the standard deep deterministic policy gradient method to prove solution accuracy and demonstrate increased computational efficiency.

Du, Yan↗

Machine Learning-Based PV Reserve Determination Strategy for Frequency Control on the WECC System: Preprint

Frequency control from Photovoltaic (PV) plants has great potential to address the frequency response challenge of the power system with high renewable penetration. However, using model-based approaches to determine the optimal PV headroom reserve requires significant online computation and is intractable for an interconnection level system. This paper proposes a machine learning based strategy, that is suitable for real-time operation, to determine the optimal PV reserve for frequency control. The proposed machine learning algorithm is trained and tested on 1,987 offline simulations of a 60% renewable penetration Western Electricity Coordinating Council (WECC) system. Furthermore, the proposed reserve determination strategy is applied on a realistic one-day operation profile of the WECC system and demonstrates over 40% PV headroom saving compared to a conservative approach. It is evident that the proposed strategy can efficiently and effectively determine the optimal PV frequency control reserve for realistic interconnection systems.

41 EE - Solar Energy Technologies Office (EE-4S)↗

A Hybrid Optimization and Deep Learning Algorithm for Cyber-Resilient DER Control

With the proliferation of distributed energy resources (DERs) in the distribution grid, it is a challenge to effectively control a large number of DERs resilient to the communication and security disruptions, as well as to provide the online grid services, such as voltage regulation and virtual power plant (VPP) dispatch. To this end, a hybrid feedback-based optimization algorithm along with deep learning forecasting technique is proposed to specifically address the cyber-related issues. The online decentralized feedback-based DER optimization control requires timely, accurate voltage measurement from the grid. However, in practice such information may not be received by the control center or even be corrupted. Therefore, the long short-term memory (LSTM) deep learning algorithm is employed to forecast delayed/missed/attacked messages with high accuracy. The IEEE 37-node feeder with high penetration of PV systems is used to validate the efficiency of the proposed hybrid algorithm. The results show that 1) the LSTM-forecasted lost voltage can effectively improve the performance of the DER control algorithm in the practical cyber-physical architecture; and 2) the LSTM forecasting strategy outperforms other strategies of using previous message and skipping dual parameter update.

cyber-resilient algorithm↗

Locomotion training of legged robots using hybrid machine learning techniques

In this study artificial neural networks and fuzzy logic are used to control the jumping behavior of a three-link uniped robot. The biped locomotion control problem is an increment of the uniped locomotion control. Study of legged locomotion dynamics indicates that a hierarchical controller is required to control the behavior of a legged robot. A structured control strategy is suggested which includes navigator, motion planner, biped coordinator and uniped controllers. A three-link uniped robot simulation is developed to be used as the plant. Neurocontrollers were trained both online and offline. In the case of on-line training, a reinforcement learning technique was used to train the neurocontroller to make the robot jump to a specified height. After several hundred iterations of training, the plant output achieved an accuracy of 7.4%. However, when jump distance and body angular momentum were also included in the control objectives, training time became impractically long. In the case of off-line training, a three-layered backpropagation (BP) network was first used with three inputs, three outputs and 15 to 40 hidden nodes. Pre-generated data were presented to the network with a learning rate as low as 0.003 in order to reach convergence. The low learning rate required for convergence resulted in a very slow training process which took weeks to learn 460 examples. After training, performance of the neurocontroller was rather poor. Consequently, the BP network was replaced by a Cerebeller Model Articulation Controller (CMAC) network. Subsequent experiments described in this document show that the CMAC network is more suitable to the solution of uniped locomotion control problems in terms of both learning efficiency and performance. A new approach is introduced in this report, viz., a self-organizing multiagent cerebeller model for fuzzy-neural control of uniped locomotion is suggested to improve training efficiency. This is currently being evaluated for a possible patent by NASA, Johnson Space Center. An alternative modular approach is also developed which uses separate controllers for each stage of the running stride. A self-organizing fuzzy-neural controller controls the height, distance and angular momentum of the stride. A CMAC-based controller controls the movement of the leg from the time the foot leaves the ground to the time of landing. Because the leg joints are controlled at each time step during flight, movement is smooth and obstacles can be avoided. Initial results indicate that this approach can yield fast, accurate results.

Simon, William E.↗

Load altering attack-tolerant defense strategy for load frequency control system

Cyber attacks are emerging threats to every information-oriented energy management system. By violating the cyber systems, the hacker can disrupt the security and stability due to the strong coupling between the cyber and physical facilities. In this paper, one type of cyber attacks designated as the load altering attack is studied for the power system frequency control, and corresponding defense strategies are proposed to improve the frequency control performance. Considering the difficulty of the application of model-based controller into large-scale power systems, a novel model-free defense framework is for the first time presented. Under this framework, both active defense and passive defense strategies are designed. The former assumes that the defender has the initiative to learn different attack scenarios. Adaptive defense strategies are implemented using the online attack identification information and off-line trained strategy pool. The latter assumes that the defender passively tolerates various attack scenarios via the pre-trained off-line strategy. Both approaches prove to be effective through validation based on the IEEE benchmark systems. The proposed defense framework and defense strategies can be extended to other energy control systems to enhance their attack tolerance capability.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Fingerprinting Anomalous Computation with RNN for GPU-accelerated HPC Machines

This paper presents a workload classification framework that discriminates illicit computation from authorized workloads on GPU-accelerated HPC systems. As such systems become more and more powerful, they are exploited by attackers to run malicious and for-profit programs that typically require extremely high computing ability to be successful. Our classification framework leverages the distinctive signatures between illicit and authorized workloads, and explore machine learning methods to learn the workloads and classify them. The framework uses lightweight, non-intrusive workload profiling to collect model input data, and explores multiple machine learning methods, particularly recurrent neural network (RNN) that is suitable for online anomalous workload detection. Evaluation results on four generations of GPU machines demonstrate that the workload classification framework can tell apart the illicit authorized workloads with a high accuracy of over 95%.

Zou, Pengfei↗

RAP: Resource-aware Automated GPU Sharing for Multi-GPU Recommendation Model Training and Input Preprocessing

Ensuring high-quality recommendations for newly onboarded users requires the continuous retraining of Deep Learning Recommendation Models (DLRMs) with freshly generated data. To serve the online DLRM retraining, existing solutions use hundreds of CPU computing nodes designated for input preprocessing, causing significant power consumption that surpasses even the power usage of GPU trainers. To this end, we propose RAP, an end-to-end DLRM training framework that supports Resource-aware Automated GPU sharing for DLRM input Preprocessing and Training. The core idea of RAP is to accurately capture the remaining GPU computing resources during DLRM training for input preprocessing, achieving superior training efficiency without requiring additional resources. Specifically, RAP utilizes a co-running cost model to efficiently assess the costs of various input preprocessing operations, and it implements a resource-aware horizontal fusion technique that adaptively merges smaller kernels according to GPU availability, circumventing any interference with DLRM training. In addition, RAP leverages a heuristic searching algorithm that jointly optimizes both the input preprocessing graph mapping and the co-running schedule to maximize the end-to-end DLRM training throughput. The comprehensive evaluation shows that RAP achieves 78.3× speedup on average over CPU-based DLRM input preprocessing frameworks. In addition, the end-to-end training throughput of RAP is only 2.04% lower than the ideal case, which has no input preprocessing overhead.

Wang, Zheng↗

Small Satellite Reliability Initiative (SSRI) Knowledge Base Tool: Update and Future Direction

NASA’s Small Satellite Reliability Initiative (SSRI), in conjunction with NASA’s Small Spacecraft Systems Virtual Institute (S3VI), has developed the SSRI Knowledge Base to improve mission confidence for small spacecraft. The SSRI Knowledge Base is a comprehensive and searchable online tool that consolidates and organizes resources, best practices, and lessons learned from previous small satellite missions sponsored by NASA, other government agencies, and academia. This free, publicly available tool is available to the entire SmallSat community at: NASA SSRI Knowledge Base | Explore. The SSRI Knowledge Base provides vetted, high-quality sources of information on elements that are key to successful small satellite missions. These resources include SSRI working group generated documents and presentations in addition to existing guides, publications, standards, software tools, websites, and books. The Knowledge Base is fully searchable, offers downloadable content when possible, and otherwise links to or references content directly from within the tool. All 58 of the planned topic pages that comprise the SSRI Knowledge Base have been recently completed and include over 450 unique resources that are now available for review. Over the past several months significant enhancements to the tool’s capabilities have been developed and implemented. These enhancements consist of the completed baseline content; development of an Application Programming Interface (API); improved user interfaces; scalable and searchable Best Practices and Lessons Learned (BPLL) lists with ratings; and custom website analytics. This presentation and paper will discuss the motivation for and development of the SSRI Knowledge Base, demonstrate the existing tool, and outline plans for further development. The SSRI is a collaborative activity with broad participation from civil, Department of Defense, and both national and international commercial space systems providers and stakeholders. The S3VI is jointly sponsored by NASA’s Space Technology Mission Directorate and Science Mission Directorate.

Small Spacecraft↗

Small Satellite Reliability Initiative (SSRI) Knowledge Base Tool: Use Case Review and Future Functionality and Content Direction

NASA’s Small Satellite Reliability Initiative (SSRI), in conjunction with NASA’s Small Spacecraft Systems Virtual Institute (S3VI), has developed the SSRI Knowledge Base to improve mission confidence for small spacecraft. The SSRI is a collaborative activity with broad participation from civil, Department of Defense, and both national and international commercial space systems providers and stakeholders. The S3VI is jointly sponsored by NASA’s Space Technology Mission Directorate and Science Mission Directorate. The SSRI Knowledge Base is a comprehensive and searchable online tool that consolidates and organizes resources, best practices, and lessons learned from previous small satellite missions sponsored by NASA, other government agencies, and academia. This free, publicly available tool is available to the entire SmallSat community at https://s3vi.ndc.nasa.gov/ssri-kb/. The SSRI Knowledge Base provides vetted, high-quality sources of information on elements that are key to successful small satellite missions. These resources include SSRI working group generated documents and presentations in addition to existing guides, publications, standards, software tools, websites, and books. The Knowledge Base is fully searchable, offers downloadable content when possible, and otherwise links to or references content directly from within the tool. All 58 of the planned topic pages that comprise the SSRI Knowledge Base have been recently completed and include over 450 unique resources that are now available for review. Over the past several months, significant enhancements to the tool’s capabilities have been developed and implemented. These enhancements consist of the completed baseline content, an Application Programming Interface (API), improved user interfaces, new interfaces for crowdsourcing of content and user ratings, and custom website analytics to inform future development. This presentation and paper will discuss the motivation for the SSRI Knowledge Base, review educational use case(s), and outline plans for further development. The 2022 session topic that best fits the abstract (select only one): Coordinating Successful Educational Programs

Small Spacecraft↗

Small Satellite Reliability Initiative (SSRI) Knowledge Base Tool: Update and Future Direction

The Small Satellite Reliability Initiative (SSRI)Knowledge Base is a comprehensive and searchable online tool that consolidates and organizes resources, best practices, and lessons learned from previous small satellite missions sponsored by NASA, other government agencies, and academia. This free, publicly available tool is available to the entire SmallSat community. The SSRI Knowledge Base provides vetted, high-quality sources of information on elements that are key to successful small satellite missions. These resources include SSRI working group generated documents and presentations in addition to existing guides, publications, standards, software tools, websites, and books. The Knowledge Base is fully searchable, offers downloadable content when possible, and otherwise links to or references content directly from within the tool. This presentation and paper will discuss the motivation for and development of the SSRI Knowledge Base, demonstrate the existing tool, and outline plans for further development. The SSRI is a collaborative activity with broad participation from civil, Department of Defense, and both national and international commercial space systems providers and stakeholders. NASA’s Small Spacecraft Systems Virtual Institute (S3VI) funds the SSRI Knowledge Base. The S3VIis jointly sponsored by NASA’s Space Technology Mission Directorate and Science Mission Directorate.

Small Spacecraft↗

Small Satellite Reliability Initiative (SSRI) Knowledge Base Tool: Update and Future Direction

NASA’s Small Satellite Reliability Initiative (SSRI), in conjunction with NASA’s Small Spacecraft Systems Virtual Institute (S3VI), has developed the SSRI Knowledge Base to improve mission confidence for small spacecraft. The SSRI Knowledge Base is a comprehensive and searchable online tool that consolidates and organizes resources, best practices, and lessons learned from previous small satellite missions sponsored by NASA, other government agencies, and academia. This free, publicly available tool is available to the entire SmallSat community at: NASA SSRI Knowledge Base | Explore. The SSRI Knowledge Base provides vetted, high-quality sources of information on elements that are key to successful small satellite missions. These resources include SSRI working group generated documents and presentations in addition to existing guides, publications, standards, software tools, websites, and books. The Knowledge Base is fully searchable, offers downloadable content when possible, and otherwise links to or references content directly from within the tool. All 58 of the planned topic pages that comprise the SSRI Knowledge Base have been recently completed and include over 450 unique resources that are now available for review. Over the past several months significant enhancements to the tool’s capabilities have been developed and implemented. These enhancements consist of the completed baseline content; development of an Application Programming Interface (API); improved user interfaces; scalable and searchable Best Practices and Lessons Learned (BPLL) lists with ratings; and custom website analytics. This presentation and paper will discuss the motivation for and development of the SSRI Knowledge Base, demonstrate the existing tool, and outline plans for further development. The SSRI is a collaborative activity with broad participation from civil, Department of Defense, and both national and international commercial space systems providers and stakeholders. The S3VI is jointly sponsored by NASA’s Space Technology Mission Directorate and Science Mission Directorate.

Small Spacecraft↗

Small Satellite Reliability Initiative (SSRI) Knowledge Base Tool: Use Case Review and Future Functionality and Content Direction

The Small Satellite Reliability Initiative (SSRI) Knowledge Base is a comprehensive and searchable online tool that consolidates and organizes resources, best practices, and lessons learned from previous small satellite missions sponsored by NASA, other government agencies, and academia. This free, publicly available tool is available to the entire SmallSat Community. The SSRI Knowledge Base provides vetted, high-quality sources of information on elements that are key to successful small satellite missions. These resources include SSRI working group generated documents and presentations in addition to existing guides, publications, standards, software tools, websites, and books. The Knowledge Base is fully searchable, offers downloadable content when possible, and otherwise links to or references content directly from within the tool. This presentation and paper will discuss the motivation for the SSRI Knowledge Base, review educational use cases, and outline plans for further development. The SSRI is a collaborative activity with broad participation from civil, U.S. Department of Defense, and both national and international commercial space systems providers and stakeholders. NASA’s Small Spacecraft Systems Virtual Institute (S3VI) funds the SSRI Knowledge Base. The S3VI is jointly sponsored by NASA’s Space Technology Mission Directorate and Science Mission Directorate. This presentation and paper will discuss the motivation for the SSRI Knowledge Base, review educational use case(s), and outline plans for further development.

Small Spacecraft↗

Small Satellite Reliability Initiative (SSRI) Knowledge Base Tool: Use Case Review and Future Functionality and Content Direction

NASA’s Small Satellite Reliability Initiative (SSRI), in conjunction with NASA’s Small Spacecraft Systems Virtual Institute (S3VI), has developed the SSRI Knowledge Base to improve mission confidence for small spacecraft. The SSRI Knowledge Base is a comprehensive and searchable online tool that consolidates and organizes resources, best practices, and lessons learned from previous small satellite missions sponsored by NASA, other government agencies, and academia. This free, publicly available tool is available to the entire SmallSat community at: NASA SSRI Knowledge Base | Explore. The SSRI Knowledge Base provides vetted, high-quality sources of information on elements that are key to successful small satellite missions. These resources include SSRI working group generated documents and presentations in addition to existing guides, publications, standards, software tools, websites, and books. The Knowledge Base is fully searchable, offers downloadable content when possible, and otherwise links to or references content directly from within the tool. All 58 of the planned topic pages that comprise the SSRI Knowledge Base have been recently completed and include over 450 unique resources that are now available for review. Over the past several months significant enhancements to the tool’s capabilities have been developed and implemented. These enhancements consist of the completed baseline content; development of an Application Programming Interface (API); improved user interfaces; scalable and searchable Best Practices and Lessons Learned (BPLL) lists with ratings; and custom website analytics. This presentation and paper will discuss the motivation for and development of the SSRI Knowledge Base, review of potential use case(s), and outline plans for further development and content generation. The SSRI is a collaborative activity with broad participation from civil, Department of Defense, and both national and international commercial space systems providers and stakeholders. The S3VI is jointly sponsored by NASA’s Space Technology Mission Directorate and Science Mission Directorate.

Small Spacecraft↗

Geostellar: Remote Solar Energy Assessments Personalized

Geostellar has produced an online tool that generates a unique solar profile for homeowners to learn about the financial benefits to installing rooftop solar panels on their home. The website incorporates the physical building characteristics of the home, including shading, slope, and orientation of the roof, and applies electricity costs and incentives to determine the best solar energy estimated energy production values against actual installed rooftop photovoltaic systems. The validation conducted by NREL concluded that over three-quarters of Geostellar's potential size estimates are at least as large as the actual installed systems, indicating a correct assessment of roof availability. In addition, 87% of Geostellar's 25-year production estimates are within 90% of the actual PV Watts results.

14 SOLAR ENERGY↗

Detecting Anomalous Computation with RNNs on GPU-Accelerated HPC Machines

This paper presents a workload classification framework that discriminates illicit computation from authorized workloads on GPU-accelerated HPC systems. As such systems become more and more powerful, they are exploited by attackers to run malicious and for-profit programs that typically require extremely high computing ability to be successful. Our classification framework leverages the distinctive signatures between illicit and authorized workloads, and explore machine learning methods to learn the workloads and classify them. The framework uses lightweight, non-intrusive workload profiling to collect model input data, and explores multiple machine learning methods, particularly recurrent neural network (RNN) that is suitable for online anomalous workload detection. Evaluation results on three generations of GPU machines demonstrate that the workload classification framework can tell apart the illicit authorized workloads with a high accuracy of over 95%.

Pengfei, Zou↗

Space Operations Learning Center Facebook Application

The proposed Space Operations Learning Center (SOLC) Facebook module, initially code-named Spaceville, is intended to be an educational online game utilizing the latest social networking technology to reach a broad audience base and inspire young audiences to be interested in math, science, and engineering. Spaceville will be a Facebook application/ game with the goal of combining learning with a fun game and social environment. The mission of the game is to build a scientific outpost on the Moon or Mars and expand the colony. Game activities include collecting resources, trading resources, completing simple science experiments, and building architectures such as laboratories, habitats, greenhouses, machine shops, etc. The player is awarded with points and achievement levels. The player s ability increases as his/her points and levels increase. A player can interact with other players using multiplayer Facebook functionality. As a result, a player can discover unexpected treasures through scientific missions, engineering, and working with others. The player creates his/her own avatar with his/her selection of its unique appearance, and names the character. The player controls the avatar to perform activities such as collecting oxygen molecules or building a habitat. From observations of other successful social online games such as Farmville and Restaurant City, a common element of these games is having eye-catching and cartoonish characters, and interesting animations for all activities. This will create a fun, educational, and rewarding environment. The player needs to accumulate points in order to be awarded special items needed for advancing to higher levels. Trophies will be awarded to the player when certain goals are reached or tasks are completed. In order to acquire some special items needed for advancement in the game, the player will need to visit his/her neighboring towns to discover the items. This is the social aspect of the game that requires the player to go out of his/her own establishment to explore what is in the neighborhood. Spaceville will take advantage of Facebook s successful architecture to inspire a new audience of scientists and engineers for the future.

Lui, Ben↗