Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “bandit”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

29 records · Page 2

Combining active learning suggestions

We study the problem of combining active learning suggestions to identify informative training examples by empirically comparing methods on benchmark datasets. Many active learning heuristics for classification problems have been proposed to help us pick which instance to annotate next. But what is the optimal heuristic for a particular source of data? Motivated by the success of methods that combine predictors, we combine active learners with bandit algorithms and rank aggregation methods. We demonstrate that a combination of active learners outperforms passive learning in large benchmark datasets and removes the need to pick a particular active learner a priori. We discuss challenges to finding good rewards for bandit approaches and show that rank aggregation performs well.

97 MATHEMATICS AND COMPUTING↗

Artificial intelligence for materials research at extremes

Abstract Materials development is slow and expensive, taking decades from inception to fielding. For materials research at extremes, the situation is even more demanding, as the desired property combinations such as strength and oxidation resistance can have complex interactions. Here, we explore the role of AI and autonomous experimentation (AE) in the process of understanding and developing materials for extreme and coupled environments. AI is important in understanding materials under extremes due to the highly demanding and unique cases these environments represent. Materials are pushed to their limits in ways that, for example, equilibrium phase diagrams cannot describe. Often, multiple physical phenomena compete to determine the material response. Further, validation is often difficult or impossible. AI can help bridge these gaps, providing heuristic but valuable links between materials properties and performance under extreme conditions. We explore the potential advantages of AE along with decision strategies. In particular, we consider the problem of deciding between low-fidelity, inexpensive experiments and high-fidelity, expensive experiments. The cost of experiments is described in terms of the speed and throughput of automated experiments, contrasted with the human resources needed to execute manual experiments. We also consider the cost and benefits of modeling and simulation to further materials understanding, along with characterization of materials under extreme environments in the AE loop. Graphical abstract AI sequential decision-making methods for materials research: Active learning, which focuses on exploration by sampling uncertain regions, Bayesian and bandit optimization as well as reinforcement learning (RL), which trades off exploration of uncertain regions with exploitation of optimum function value. Bayesian and bandit optimization focus on finding the optimal value of the function at each step or cumulatively over the entire steps, respectively, whereas RL considers cumulative value of the labeling function, where the latter can change depending on the state of the system (blue, orange, or green).

36 MATERIALS SCIENCE↗

A learning-based measurement framework for traffic matrix inference in software defined networks

Here, we propose an intelligent framework for Traffic Matrix (TM) inference in Software Defined Networks (SDN) where the Ternary Content Addressable Memory (TCAM) entries of switches are partitioned into two parts to: (1) effectively aggregate part of incoming flows for aggregate measurements, and (2) de-aggregate and directly measure the most informative flows for per-flow measurements. These measurements are then processed to effectively estimate the size of network flows. Under hard resource constraints of limited TCAM sizes, we show how to design the optimal and efficient-compressed flow aggregation matrices. We propose an optimal Multi-Armed Bandit (MAB) based algorithm to adaptively measure the most rewarding flows. We evaluate the performance of our framework using real traffic traces from different network environments and by considering two main applications: TM estimation and Heavy Hitter (HH) detection. Moreover, we have implemented a prototype of our framework in Mininet to demonstrate its effectiveness.

42 ENGINEERING↗

Online Learning and Pricing for Service Systems with Reusable Resources

We consider a price-based revenue management problem with finite reusable resources over a finite time horizon T. Customers arrive following a price-dependent Poisson process, and each customer requests one unit of c homogeneous reusable resources. If there is an available unit, the customer gets served within a price-dependent exponentially distributed service time; otherwise, the customer waits in a queue until the next available unit. In this paper, we assume that the firm does not know how the arrival and service rates depend on posted prices, and thus it makes adaptive pricing decisions in each period based only on past observations to maximize the cumulative revenue. Given a discrete price set with cardinality P, we propose two online learning algorithms, termed batch upper confidence bound (BUCB) and batch Thompson sampling (BTS), and prove that the cumulative regret upper bound is O˜(√PT) , which matches the regret lower bound. In establishing the regret, we bound the transient system performance upon price changes via a novel coupling argument, and also generalize bandits to accommodate subexponential rewards. Here, we also extend our approach to models with balking and reneging customers and discuss a continuous price setting. Our numerical experiments demonstrate the efficacy of the proposed BUCB and BTS algorithms.

97 MATHEMATICS AND COMPUTING↗

Online Learning and Pricing for Network Revenue Management with Reusable Resources

We consider a price-based network revenue management problem with multiple products and multiple reusable resources. Each randomly arriving customer requests a product (service) that needs to occupy a sequence of reusable resources (servers). We adopt an incomplete information setting where the firm does not know the price-demand function for each product and the goal is to dynamically set prices of all products to maximize the total expected revenue of serving customers. We propose novel batched bandit learning algorithms for finding near-optimal pricing policies, and show that they admit a near-optimal cumulative regret bound of $O(J\sqrt{XT})$, where J, X, and T, are the numbers of products, candidate prices, and service periods, respectively. As part of our regret analysis, we develop the first finite-time mixing time analysis of an open network queueing system (i.e., the celebrated Jackson Network), which could be of independent interest. Our numerical studies show that the proposed approaches perform consistently well.

42 ENGINEERING↗

Current Drive and Plasma Heating by Electron Bernstein Waves in MAST

Electron Bernstein waves (EBW) have the potential to provide highly localized heating and current drive (CD). EBWs are predominantly electrostatic and they damp on electrons near electron cyclotron harmonics without momentum injection into the plasma. These features represent a powerful tool for understanding transport and stability phenomena by locally perturbing the plasma and providing complementary CD methods in addition to neutral beams. The Mega-Ampere Spherical Tokamak (MAST) has a large cylindrical vacuum vessel and we have taken advantage of this to consider a number of launcher positions for RF power injection. The feasibility of EBW in the extended parameter space of MAST has been explored. Modelling was conducted with the EBW and BANDIT code package using a 'steady state' reference scenario with near zero loop voltage. Clear heating and CD effects have been identified for different launch configurations and frequencies.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Decision making based on optical excitation transfer via near-field interactions between quantum dots

Optical near-field interactions between nanostructured matters, such as quantum dots, result in unidirectional optical excitation transfer when energy dissipation is induced. This results in versatile spatiotemporal dynamics of the optical excitation, which can be controlled by engineering the dissipation processes and exploited to realize intelligent capabilities such as solution searching and decision making. Here, we experimentally demonstrate the ability to solve a decision making problem on the basis of optical excitation transfer via near-field interactions by using colloidal quantum dots of different sizes, formed on a geometry-controlled substrate. We characterize the energy transfer behavior due to multiple control light patterns and experimentally demonstrate the ability to solve the multi-armed bandit problem. Our work makes a decisive step towards the practical design of nanophotonic systems capable of efficient decision making, one of the most important intellectual attributes of the human brain.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Reduction of matrix wavefront for NASTRAN

The three grid point resequencing algorithms most often run by NASTRAN users are compared for their ability to reduce matrix root-mean-square (rms) wavefront, which is the most critical parameter in determining matrix decomposition time in NASTRAN. The three algorithms are Cuthill-McKee (CM), Gibbs-Poole-Stockmeyer (GPS), and Levy. The first two (CM and GPS) are in the BANDIT program, and the Levy algorithm is in WAVEFRONT. Results are presented for a diversified collection of 30 test problems ranging in size from 59 to 2680 nodes. It is concluded that GPS is exceptionally fast and, for the conditions under which the test was made, the algorithm best able to reduce rms wavefront consistently well.

Everstine, G. C.↗

Coevolutionary Free Lunches

Recent work on the foundations of optimization has begun to uncover its underlying rich structure. In particular, the "No Free Lunch" (NFL) theorems [WM97] state that any two algorithms are equivalent when their performance is averaged across all possible problems. This highlights the need for exploiting problem-specific knowledge to achieve better than random performance. In this paper we present a general framework covering most search scenarios. In addition to the optimization scenarios addressed in the NFL results, this framework covers multi-armed bandit problems and evolution of multiple co-evolving agents. As a particular instance of the latter, it covers "self-play" problems. In these problems the agents work together to produce a champion, who then engages one or more antagonists in a subsequent multi-player game In contrast to the traditional optimization case where the NFL results hold, we show that in self-play there are free lunches: in coevolution some algorithms have better performance than other algorithms, averaged across all possible problems. However in the typical coevolutionary scenarios encountered in biology, where there is no champion, NFL still holds.

Wolpert, David H.↗

Coevolutionary Free Lunches

Recent work on the mathematical foundations of optimization has begun to uncover its rich structure. In particular, the "No Free Lunch" (NFL) theorems state that any two algorithms are equivalent when their performance is averaged across all possible problems. This highlights the need for exploiting problem-specific knowledge to achieve better than random performance. In this paper we present a general framework covering more search scenarios. In addition to the optimization scenarios addressed in the NFL results, this framework covers multi-armed bandit problems and evolution of multiple co-evolving players. As a particular instance of the latter, it covers "self-play" problems. In these problems the set of players work together to produce a champion, who then engages one or more antagonists in a subsequent multi-player game. In contrast to the traditional optimization case where the NFL results hold, we show that in self-play there are free lunches: in coevolution some algorithms have better performance than other algorithms, averaged across all possible problems. We consider the implications of these results to biology where there is no champion.

Wolpert, David H.↗

Comsat Antenna

The antenna shown is the new, multiple-beam, Unattended Earth Terminal, located at COMSAT Laboratories in Clarksburg, Maryland. Seemingly simple, it is actually a complex structure capable of maintaining contact with several satellites simultaneously (conventional Earth station antennas communicate with only one satellite at a time). In developing the antenna, COMSAT Laboratories used NASTRAN, NASA's structural analysis computer program, together with BANDIT, a companion program. The computer programs were used to model several structural configurations and determine the most suitable, The speed and accuracy of the computerized design analysis afforded appreciable savings in time and money.

Source record↗