Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “learning algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,405 records · Page 78
All sourcesNASA NTRSDOE OSTI
NASA NTRS1999

A Survey of Collective Intelligence

This chapter presents the science of "COllective INtelligence" (COIN). A COIN is a large multi-agent systems where: i) the agents each run reinforcement learning (RL) algorithms; ii) there is little to no centralized communication or control; iii) there is a provided world utility function that, rates the possible histories of tile full system. Tile conventional approach to designing large distributed systems to optimize a world utility does not use agents running RL algorithms. Rather that approach begins with explicit modeling of the overall system's dynamics, followed by detailed hand-tuning of the interactions between the components to ensure that they "cooperate" as far as the world utility is concerned. This approach is labor-intensive, often results in highly non-robust systems, and usually results in design techniques that, have limited applicability. In contrast, with COINs we wish to solve the system design problems implicitly, via the 'adaptive' character of the RL algorithms of each of the agents. This COIN approach introduces an entirely new, profound design problem: Assuming the RL algorithms are able to achieve high rewards, what reward functions for the individual agents will, when pursued by those agents, result in high world utility? In other words, what reward functions will best ensure that we do not have phenomena like the tragedy of the commons, or Braess's paradox? Although still very young, the science of COINs has already resulted in successes in artificial domains, in particular in packet-routing, the leader-follower problem, and in variants of Arthur's "El Farol bar problem". It is expected that as it matures not only will COIN science expand greatly the range of tasks addressable by human engineers, but it will also provide much insight into already established scientific fields, such as economics, game theory, or population biology.

Wolpert, David H.↗
NASA NTRS2000

Optimal Wonderful Life Utility Functions in Multi-Agent Systems

The mathematics of Collective Intelligence (COINs) is concerned with the design of multi-agent systems so as to optimize an overall global utility function when those systems lack centralized communication and control. Typically in COINs each agent runs a distinct Reinforcement Learning (RL) algorithm, so that much of the design problem reduces to how best to initialize/update each agent's private utility function, as far as the ensuing value of the global utility is concerned. Traditional team game solutions to this problem assign to each agent the global utility as its private utility function. In previous work we used the COIN framework to derive the alternative Wonderful Life Utility (WLU), and experimentally established that having the agents use it induces global utility performance up to orders of magnitude superior to that induced by use of the team game utility. The WLU has a free parameter (the clamping parameter) which we simply set to zero in that previous work. Here we derive the optimal value of the clamping parameter, and demonstrate experimentally that using that optimal value can result in significantly improved performance over that of clamping to zero, over and above the improvement beyond traditional approaches.

Wolpert, David H.↗
NASA NTRS1998

Distributed Control with Collective Intelligence

We consider systems of interacting reinforcement learning (RL) algorithms that do not work at cross purposes , in that their collective behavior maximizes a global utility function. We call such systems COllective INtelligences (COINs). We present the theory of designing COINs. Then we present experiments validating that theory in the context of two distributed control problems: We show that COINs perform near-optimally in a difficult variant of Arthur's bar problem [Arthur] (and in particular avoid the tragedy of the commons for that problem), and we also illustrate optimal performance in the master-slave problem.

Wolpert, David H.↗
NASA NTRS1998

Using Collective Intelligence to Route Internet Traffic

A Collective Intelligence (COIN) is a community of interacting reinforcement learning (RL) algorithms designed so that their collective behavior maximizes a global utility function. We introduce the theory of COINs, then present experiments using that theory to design COINs to control internet traffic routing. These experiments indicate that COINs outperform previous RL-based systems for such routing that have previously been investigated.

Wolpert, David H.↗
NASA NTRS2004

Adaptive, Distributed Control of Constrained Multi-Agent Systems

Product Distribution (PO) theory was recently developed as a broad framework for analyzing and optimizing distributed systems. Here we demonstrate its use for adaptive distributed control of Multi-Agent Systems (MASS), i.e., for distributed stochastic optimization using MAS s. First we review one motivation of PD theory, as the information-theoretic extension of conventional full-rationality game theory to the case of bounded rational agents. In this extension the equilibrium of the game is the optimizer of a Lagrangian of the (Probability dist&&on on the joint state of the agents. When the game in question is a team game with constraints, that equilibrium optimizes the expected value of the team game utility, subject to those constraints. One common way to find that equilibrium is to have each agent run a Reinforcement Learning (E) algorithm. PD theory reveals this to be a particular type of search algorithm for minimizing the Lagrangian. Typically that algorithm i s quite inefficient. A more principled alternative is to use a variant of Newton's method to minimize the Lagrangian. Here we compare this alternative to RL-based search in three sets of computer experiments. These are the N Queen s problem and bin-packing problem from the optimization literature, and the Bar problem from the distributed RL literature. Our results confirm that the PD-theory-based approach outperforms the RL-based scheme in all three domains.

Bieniawski, Stefan↗
NASA NTRS2003

Theory of Collective Intelligence

In this chapter an analysis of the behavior of an arbitrary (perhaps massive) collective of computational processes in terms of an associated "world" utility function is presented We concentrate on the situation where each process in the collective can be viewed as though it were striving to maximize its own private utility function. For such situations the central design issue is how to initialize/update the collective's structure, and in particular the private utility functions, so as to induce the overall collective to behave in a way that has large values of the world utility. Traditional "team game" approaches to this problem simply set each private utility function equal to the world utility function. The "Collective Intelligence" (COIN) framework is a semi-formal set of heuristics that recently have been used to construct private utility. functions that in many experiments have resulted in world utility values up to orders of magnitude superior to that ensuing from use of the team game utility. In this paper we introduce a formal mathematics for analyzing and designing collectives. We also use this mathematics to suggest new private utilities that should outperform the COIN heuristics in certain kinds of domains. In accompanying work we use that mathematics to explain previous experimental results concerning the superiority of COIN heuristics. In that accompanying work we also use the mathematics to make numerical predictions, some of which we then test. In this way these two papers establish the study of collectives as a proper science, involving theory, explanation of old experiments, prediction concerning new experiments, and engineering insights.

Nash equilibrium↗
NASA NTRS1997

Efficiently Ranking Hyphotheses in Machine Learning

This paper considers the problem of learning the ranking of a set of alternatives based upon incomplete information (e.g. a limited number of observations). At each decision cycle, the system can output a complete ordering on the hypotheses or decide to gather additional information (e.g. observation) at some cost.

Machine Learning algorithm parametric ranking↗
NASA NTRS2019

Cognitive Communications for NASA Space Systems

The growing complexity of spacecraft constellations, communication relay offerings, and mission architectures drives the need for the development of autonomous communication systems. NASA has traditionally launched single spacecraft missions that are served by the Space Communication and Navigation (SCaN) program. Operations on SCaN networks are typically scheduled weeks in advance, and often each asset serves a single user spacecraft at a time. Recent movement towards swarm missions could make the current approach unsustainable. Additionally, the integration of commercial communication service providers will substantially increase the data transfer options available to new missions. NASA science missions have found benefit in launching swarms of spacecraft, allowing coordinated simultaneous observations from different perspectives. Inter-spacecraft communication (mesh networking) is an enabler for this architecture, as are CubeSats that allow cost-effective provisioning of distributed mission assets. As more complex swarm missions launch, one challenge is coordinating communication within the swarm and choosing the appropriate mechanism for telemetry, tracking, control, and data services to and from Earth. Cognitive communications research conducted by SCaN aims to mitigate the increasing communication complexity for mission users by increasing the autonomy of links, networks, and service scheduling. By considering automation techniques including recent advances in artificial intelligence and machine learning, cognitive algorithms and related approaches enable increased mission science return, improved resource utilization for service provider networks, and resiliency in unpredictable or unplanned environments. The Cognitive Communications Project at the NASA Glenn Research Center develops applications of data-driven, non-deterministic methods to improve the autonomy of space communication. The project emphasizes development of decentralized space networks with artificial intelligence agents optimizing communication link throughput, data routing, and system-wide asset management. This paper discusses the objectives, approaches, and opportunities of the research to address growing needs of the space communications community.

Chelmins, David↗
NASA NTRS2019

Machine Learning Based AFP Inspection: A Tool for Characterization and Integration

Automated Fiber Placement (AFP) has become a standard manufacturing technique in the creation of large scale composite structures due to its high production rates. However, the associated rapid layup that accompanies AFP manufacturing has a tendency to induce defects. We forward an inspection system that utilizes machine learning (ML) algorithms to locate and characterize defects from profilometry scans coupled with a data storage system and a user interface (UI) that allows for informed manufacturing. A Keyence LJ-7080 blue light profilometer is used for fast 2D height profiling. After scans are collected, they are process by ML algorithms, displayed to an operator through the UI, and stored in a database. The overall goal of the inspection system is to add an additional tool for AFP manufacturing. Traditional AFP inspection is done manually adding to manufacturing time and being subject to inspector errors or fatigue. For large parts, the inspection process can be cumbersome. The proposed inspection system has the capability of accelerating this process while still keeping a human inspector integrated and in control. This allows for the rapid capability of the automated inspection software and the robustness of a human checking for defects that the system either missed or misclassified.

Sacco, Christopher↗
NASA NTRS2018

Virtual Sensors Determined Through Machine Learning

We propose a method that increases the capability of a conventional sensor, transforming it into an enhanced virtual sensor. This paper focuses on a virtual thermal Infrared Radiation (IR) sensor based on a conventional visual (RGB) sensor. The estimation of thermal IR images can enhance the ability of terrain classification, which is crucial for autonomous navigation of rovers. The estimate in IR from visual band has inherent limitations, as these are different bands, yet correlations between visual RGB and thermal IR images exist, as different terrains, which visually may appear different, also have different thermal inertia. This paper describes the developed deep learning-based algorithm that estimates thermal IR images from RGB images of terrains, providing the feasibility of the idea with average 1.21 error [degree Celsius].

Torresen, Jim↗
NASA NTRS2018

Virtual Sensors Determined Through Machine Learning

We propose a method that increases the capability of a conventional sensor/instrument, transforming it into an enhanced virtual sensor/instrument. This paper focuses on a virtual thermal IR sensor based on a conventional visual (RGB) sensor. The estimation of thermal IR images can enhance the ability of terrain classification, which is crucial for autonomous navigation of rovers. The estimate in IR from visual band has inherent limitations, as these are different bands, yet correlations between visual/RGB and thermal IR images exist, as different terrains, which visually may appear different, also have different thermal inertia. We developed a deep learning-based algorithm that estimates thermal IR images from RGB images of terrains, providing the feasibility of the idea with average 1.21 error [degree].

Kurazume, Ryo↗
NASA NTRSDate not supplied

Joint Spectrum Access and Power Control in Air-Air Communications - A Deep Reinforcement Learning Based Approach

This paper considers the dynamic spectrum access and power control problem in a single-hop point-to-point Air-Air Communication Network (AACN). Due to spectrum scarcity, we assume the number of Aircraft-to-Aircraft (A2A) communication links is greater than that of the available channels, such that some communication links need to share the same channel, causing co-channel interference. We formulate the joint channel selection and power control optimization problem to maximize the Weighted Sum Spectral Efficiency (WSSE). A distributed and dynamic deep Q learning-based algorithm is proposed to find the optimal solution. Specifically, we design two different policies that are trained by conducting a trial-and-error scheme. Each communication link can achieve the optimal policy by exploiting the local information from its neighbors, and this distributive approach make it scalable to large networks. Finally, our experimental results demonstrate the effectiveness of the proposed solution in various AACN scenarios.

Zhe Wang↗
NASA NTRSDate not supplied

Joint Spectrum Access and Power Control in Air-Air Communications - A Deep Reinforcement Learning Based Approach

This paper considers the dynamic spectrum access and power control problem in a single-hop point-to-point Air-Air Communication Network (AACN). Due to spectrum scarcity, we assume the number of Aircraft-to-Aircraft (A2A) communication links is greater than that of the available channels, such that some communication links need to share the same channel, causing co-channel interference. We formulate the joint channel selection and power control optimization problem to maximize the Weighted Sum Spectral Efficiency (WSSE). A distributed and dynamic deep Q learning-based algorithm is proposed to find the optimal solution. Specifically, we design two different policies that are trained by conducting a trial-and-error scheme. Each communication link can achieve the optimal policy by exploiting the local information from its neighbors, and this distributive approach make it scalable to large networks. Finally, our experimental results demonstrate the effectiveness of the proposed solution in various AACN scenarios.

Zhe Wang↗
NASA NTRSDate not supplied

Trend Analysis of AI/ML Tools and Services in NASA

Usage of Machine Learning (ML) algorithms within NASA’s Science Mission Directorates have been increasing over the years. This can be quantitatively observed in the upward trends of ML usage found by analyzing the publications and presentations (in affiliation with NASA) available through NASA Technical Reports Server (NTRS) and PubMed Central(PMC). Identifying the problem types and class of ML algorithms used to tackle them across the divisions can present opportunities for collaborations, interdisciplinary projects and knowledge transfer for sustainable partnerships. In this presentation, we will present the trend analysis of ML algorithms used in different SMD divisions based on the publications and presentations publicly available. We identify these trends by leveraging ML algorithms which are able to search through the publication texts semantically; which are also highly scalable. We will also present an analysis on the available opensource tools and services in NASA leveraging AI/ML algorithms. This work will provide ample avenues for collaborative efforts across different disciplines based on the surfaced trends.

Slesa Adhikari↗
NASA NTRSDate not supplied

Trend Analysis of AI/ML Tools and Services in NASA

Usage of Machine Learning (ML) algorithms within NASA’s Science Mission Directorates have been increasing over theyears. This can be quantitatively observed in the upward trends of ML usage found by analyzing the publications andpresentations (in affiliation with NASA) available through NASA Technical Reports Server (NTRS) and PubMed Central(PMC). Identifying the problem types and class of ML algorithms used to tackle them across the divisions can presentopportunities for collaborations, interdisciplinary projects and knowledge transfer for sustainable partnerships. In thispresentation, we will present the trend analysis of ML algorithms used in different SMD divisions based on the publicationsand presentations publicly available. We identify these trends by leveraging ML algorithms which are able to search throughthe publication texts semantically; which are also highly scalable. We will also present an analysis on the available opensource tools and services in NASA leveraging AI/ML algorithms. This work will provide ample avenues for collaborativeefforts across different disciplines based on the surfaced trends.

Slesa Adhikari↗
NASA NTRS2021

PIXLISE-C: Exploring The Data Analysis Needs of NASA Scientists for Mineral Identification

NASA JPL scientists working on the micro x-ray fluorescence (microXRF) spectroscopy data collected from Mars surface perform data analysis to look for signs of past microbial life on Mars. Their data analysis workflow mainly involves identifying mineral com- pounds through the element abundance in spatially distributed data points. Working with the NASA JPL team, we identified pain points and needs to further develop their existing data visualization and analysis tool. Specifically, the team desired improvements for the process of creating and interpreting mineral composition groups. To address this problem, we developed an interactive tool that enables scientists to (1) cluster the data using either manual lasso-tool selection or through various machine learning clustering algorithms, and (2) compare the clusters and individual data points to make informed decisions about mineral compositions. Our preliminary tool supports a hybrid data analysis workflow where the user can manually refine the machine-generated clusters.

Davidoff, Scott↗
NASA NTRSDate not supplied

Data Science and the Knowledge Discovery Adventure

This talk will cover the important steps involved in the data science and knowledge discovery process: • Initial fact gathering (interview domain experts, review reports, articles, state-of-the-art) • Identify the problem (prediction, classification, statistical analysis, etc.) • Survey supporting data sources • Understand the data (numerical, categorical, text, sampling rate, data quality issues, etc.) • Selecting relevant features and sources • Acquire the data (set up agreements with the data stewards, APIs to download, etc.) • Merge data sources (temporal, spatial, common key, other ontologies...) • Feature Engineering (non linear domain knowledge or physics-based relationships) • Build data processing pipeline (may need to tap into data stream, develop parallel processing algorithm, federated learning etc.) • Build model and test (tune hyper-parameters, cross validation.) • Analyze/Validate results (do the results make sense. Does it answer the original question). • Deploy/Publish (Monitor and assess benefits)

Data science↗
NASA NTRS2022

Developing Data-Driven Artificial Neural Network for A High Throughput Retrieval of Aerosol Optical Depth and Surface Temperature of Mars

In this work, we aim to developartificial neural network(ANN)techniquesto reproduce the retrieval results ofphysical quantities from spacecraft observations of solar system bodiesusing radiative transfer methods. The particular application here is the retrieval of dust optical depth, water-ice optical depth, and surface temperatureonMarsusingdaytime observationsobtained by the Thermal Emission Spectrometer (TES) onboard the Mars Global Surveyor.Compared against the results obtained from traditional radiative transfer retrieval techniques, our ANN successfully recoveredthe three quantitiesusingdaytime observations. The principal advantage of thesemachine learning(ML) algorithms istheir complete automation and highthroughput. Therefore, the algorithms presented here would be useful for very large datasets andwould make practical the sampling of many different approximations or boundary conditions related to a given observation dataset and retrieval problem.

Rafael Moreno↗