Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Agents”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,369 records · Page 76

Preliminary Work for Examining the Scalability of Reinforcement Learning

Researchers began studying automated agents that learn to perform multiple-step tasks early in the history of artificial intelligence (Samuel, 1963; Samuel, 1967; Waterman, 1970; Fikes, Hart & Nilsonn, 1972). Multiple-step tasks are tasks that can only be solved via a sequence of decisions, such as control problems, robotics problems, classic problem-solving, and game-playing. The objective of agents attempting to learn such tasks is to use the resources they have available in order to become more proficient at the tasks. In particular, each agent attempts to develop a good policy, a mapping from states to actions, that allows it to select actions that optimize a measure of its performance on the task; for example, reducing the number of steps necessary to complete the task successfully. Our study focuses on reinforcement learning, a set of learning techniques where the learner performs trial-and-error experiments in the task and adapts its policy based on the outcome of those experiments. Much of the work in reinforcement learning has focused on a particular, simple representation, where every problem state is represented explicitly in a table, and associated with each state are the actions that can be chosen in that state. A major advantage of this table lookup representation is that one can prove that certain reinforcement learning techniques will develop an optimal policy for the current task. The drawback is that the representation limits the application of reinforcement learning to multiple-step tasks with relatively small state-spaces. There has been a little theoretical work that proves that convergence to optimal solutions can be obtained when using generalization structures, but the structures are quite simple. The theory says little about complex structures, such as multi-layer, feedforward artificial neural networks (Rumelhart & McClelland, 1986), but empirical results indicate that the use of reinforcement learning with such structures is promising. These empirical results make no theoretical claims, nor compare the policies produced to optimal policies. A goal of our work is to be able to make the comparison between an optimal policy and one stored in an artificial neural network. A difficulty of performing such a study is finding a multiple-step task that is small enough that one can find an optimal policy using table lookup, yet large enough that, for practical purposes, an artificial neural network is really required. We have identified a limited form of the game OTHELLO as satisfying these requirements. The work we report here is in the very preliminary stages of research, but this paper provides background for the problem being studied and a description of our initial approach to examining the problem. In the remainder of this paper, we first describe reinforcement learning in more detail. Next, we present the game OTHELLO. Finally we argue that a restricted form of the game meets the requirements of our study, and describe our preliminary approach to finding an optimal solution to the problem.

Clouse, Jeff↗

Process and Equipment for Nitrogen Oxide Waste Conversion to Fertilizer

The present invention describes a process for converting vapor streams from sources containing at least one nitrogen-containing oxidizing agent therein to a liquid fertilizer composition comprising the steps of: (1) directing a vapor stream containing at least nitrogen-containing oxidizing agent to a first contact zone; (2) contacting said vapor stream with water to form nitrogen oxide(s) from said at least one nitrogen- containing oxidizing agent; (3) directing said acid(s) as a second stream to a second contact zone; (4) exposing said second stream to hydrogen peroxide which is present within said second contact zone in a relative amount of at least 0.1% by weight of said second stream within said second contact zone to convert at least some of any nitrogen oxide species or ions other than in the nitrite form present within said second stream to nitrate ion; (5) sampling said stream within said second contact zone to determine the relative amount of hydrogen peroxide within said second contact zone; (6) adding hydrogen peroxide to said second contact zone when a level on hydrogen peroxide less than 0.1% by weight in said second stream is determined by said sampling; (7) adding a solution comprising potassium hydroxide to said second stream to maintain a pH between 6.0 and 11.0 within said second stream within said second contact zone to form a solution of potassium nitrate; and (8) removing sais solution of potassium nitrate from said second contact zone.

Lueck, Dale E.↗

Autonomy Architectures for a Constellation of Spacecraft

Until the past few years, missions typically involved fairly large expensive spacecraft. Such missions have primarily favored using older proven technologies over more recently developed ones, and humans controlled spacecraft by manually generating detailed command sequences with low-level tools and then transmitting the sequences for subsequent execution on a spacecraft controller. This approach toward controlling a spacecraft has worked spectacularly on previous missions, but it has limitations deriving from communications restrictions - scheduling time to communicate with a particular spacecraft involves competing with other projects due to the limited number of deep space network antennae. This implies that a spacecraft can spend a long time just waiting whenever a command sequence fails. This is one reason why the New Millennium program has an objective to migrate parts of mission control tasks onboard a spacecraft to reduce wait time by making spacecraft more robust. The migrated software is called a "remote agent" and has 4 components: a mission manager to generate the high level goals, a planner/scheduler to turn goals into activities while reasoning about future expected situations, an executive/diagnostics engine to initiate and maintain activities while interpreting sensed events by reasoning about past and present situations, and a conventional real-time subsystem to interface with the spacecraft to implement an activity's primitive actions. In addition to needing remote planning and execution for isolated spacecraft, a trend toward multiple-spacecraft missions points to the need for remote distributed planning and execution. The past few years have seen missions with growing numbers of probes. Pathfinder has its rover (Sojourner), Cassini has its lander (Huygens), and the New Millenium Deep Space 3 (DS3) proposal involves a constellation of 3 spacecraft for interferometric mapping. This trend is expected to continue to progressively larger fleets. For example, one mission proposed to succeed DS3 would have 18 spacecraft flying in formation in order to detect earth-sized planets orbiting other stars. A proposed magnetospheric constellation would involve 5 to 500 spacecraft in Earth orbit to measure global phenomena within the magnetosphere. This work describes and compares three autonomy architectures for a system that continuously plans to control a fleet of spacecraft using collective mission goals instead of goals or command sequences for each spacecraft. A fleet of self-commanding spacecraft would autonomously coordinate itself to satisfy high level science and engineering goals in a changing partially-understood environment making feasible the operation of tens or even a hundred spacecraft (such as for interferometry or plasma physics missions). The easiest way to adapt autonomous spacecraft research to controlling constellations involves treating the constellation as a single spacecraft. Here one spacecraft directly controls the others as if they were connected. The controlling "master" spacecraft performs all autonomy reasoning, and the slaves only have real-time subsystems to execute the master's commands and transmit local telemetry/observations. The executive/diagnostics module starts actions and the master's real-time subsystem controls the action either locally or remotely through a slave. While the master/slave approach benefits from conceptual simplicity, it relies on an assumption that the master spacecraft's executive can continuously monitor the slaves' real-time subsystems, and this relies on high-bandwidth highly-reliable communications. Since unintended results occur fairly rarely, one way to relax the bandwidth requirements involves only monitoring unexpected events in spacecraft. Unfortunately, this disables the ability to monitor for unexpected events between spacecraft and leads to a host of coordination problems among the slaves. Also, failures in the communications system can result in losing slaves. The other two architectures improve robustness while reducing communications by progressively distributing more of the other three remote agent components across the constellation. In a teamwork architecture, all spacecraft have executives and real-time subsystems - only the leader has the planner/scheduler and mission manager. Finally, distributing all remote agent components leads to a peer-to-peer approach toward constellation control.

Barrett, Anthony↗

Rheology of Foam Near the Order-Disorder Phase Transition

Foams are extremely important in a variety of industrial applications. Foams are widely used in fire-fighting applications, and are especially effective in fighting flammable liquid fires. In fact the Fire Suppression System aboard the Space Shuttle utilizes cylinders of Halon foam, which, when fired, force a rapidly expanding foam into the convoluted spaces behind instrument panels. Foams are critical in the process of enhanced oil recovery, due to their surface-active and highly viscous nature. They are also used as drilling fluids in underpressurized geologic formations. They are used as transport agents, and as trapping agents. They are also used as separation agents, where ore refinement is accomplished by froth flotation of the typically lighter and hydrophobic contaminants. The goal of the proposed investigation is the determination of the mechanical and rheological properties of foams, utilizing the microgravity environment to explore foam rheology for foams which cannot exist, or only exist for a short time, in 1g.

Holt, R. Glynn↗

Multiagent Work Practice Simulation: Progress and Challenges

Modeling and simulating complex human-system interactions requires going beyond formal procedures and information flows to analyze how people interact with each other. Such work practices include conversations, modes of communication, informal assistance, impromptu meetings, workarounds, and so on. To make these social processes visible, we have developed a multiagent simulation tool, called Brahms, for modeling the activities of people belonging to multiple groups, situated in a physical environment (geographic regions, buildings, transport vehicles, etc.) consisting of tools, documents, and a computer system. We are finding many useful applications of Brahms for system requirements analysis, instruction, implementing software agents, and as a workbench for relating cognitive and social theories of human behavior. Many challenges remain for representing work practices, including modeling: memory over multiple days, scheduled activities combining physical objects, groups, and locations on a timeline (such as a Space Shuttle mission), habitat vehicles with trajectories (such as the Shuttle), agent movement in 3D space (e.g., inside the International Space Station), agent posture and line of sight, coupled movements (such as carrying objects), and learning (mimicry, forming habits, detecting repetition, etc.).

Clancey, William J.↗

Multiagent Work Practice Simulation: Progress and Challenges

Modeling and simulating complex human-system interactions requires going beyond formal procedures and information flows to analyze how people interact with each other. Such work practices include conversations, modes of communication, informal assistance, impromptu meetings, workarounds, and so on. To make these social processes visible, we have developed a multiagent simulation tool, called Brahms, for modeling the activities of people belonging to multiple groups, situated in a physical environment (geographic regions, buildings, transport vehicles, etc.) consisting of tools, documents, and computer systems. We are finding many useful applications of Brahms for system requirements analysis, instruction, implementing software agents, and as a workbench for relating cognitive and social theories of human behavior. Many challenges remain for representing work practices, including modeling: memory over multiple days, scheduled activities combining physical objects, groups, and locations on a timeline (such as a Space Shuttle mission), habitat vehicles with trajectories (such as the Shuttle), agent movement in 3d space (e.g., inside the International Space Station), agent posture and line of sight, coupled movements (such as carrying objects), and learning (mimicry, forming habits, detecting repetition, etc.).

Clancey, William J.↗

Transition-Independent Decentralized Markov Decision Processes

There has been substantial progress with formal models for sequential decision making by individual agents using the Markov decision process (MDP). However, similar treatment of multi-agent systems is lacking. A recent complexity result, showing that solving decentralized MDPs is NEXP-hard, provides a partial explanation. To overcome this complexity barrier, we identify a general class of transition-independent decentralized MDPs that is widely applicable. The class consists of independent collaborating agents that are tied up by a global reward function that depends on both of their histories. We present a novel algorithm for solving this class of problems and examine its properties. The result is the first effective technique to solve optimally a class of decentralized MDPs. This lays the foundation for further work in this area on both exact and approximate solutions.

Becker, Raphen↗

Morphologies, Processing and Properties of Ceramic Foams and Their Potential as TPS Materials

The current research is focused on processing ceramic foams with compositions that have potential as a thermal protection material. The use of pre-ceramic polymers with the addition of sacrificial blowing agents or sacrificial fillers offers a viable approach to form either open or closed cell insulation. Our work demonstrates that this is a feasible method to form refractory ceramic foams at relatively low processing temperatures. It is possible to foam complex shapes then pyrolize the system to form a ceramic while retaining the shape of the unfired foam. Initial work focused on identifying suitable pre-ceramic polymers with desired properties such as ceramic yield and chemical make up of the pyrolysis product after firing. We focused on making foams in the Si system (Sic, Si02, Si-0-C), which is in use in current acreage TPS systems. Ceramic foams with different architectures were formed from the pyrolysis of pre-ceramic polymers at 1200 C in different atmospheres. In some systems a sacrificial polyurethane was used as the blowing agent. We have also processed foams using sacrificial fillers to introduce controlled cell sizes. Each sacrificial filler or blowing agent leads to a unique morphology. The effect of different fillers on foam morphologies and the characterization of these foams in terms of mechanical and thermal properties are presented. We have conducted preliminary arc jet testing on selected foams with the materials being exposed to typical re-entry conditions for acreage TPS and these results will be discussed. Foams processed using these approaches have bulk densities ranging from 0.15 to 0.9 g/cm3 and cell sizes ranging from 5 to 500 pm. Compression strengths ranged from 2 to 7 MPa for these systems. Finally, preliminary oxidation studies have been conducted on selected systems and will be discussed.

Stackpoole, Mairead↗

INFUSION: Modeling Robot and Crew Interaction

Brahms is a multi-agent modeling and simulation language and distributed runtime system developed at NASA. It can be used to model and run a simualtion of the distributed work activities of multiple agents, such as humans,robots, and software agents, to coordinate a mission on one or more locations.Brahms is being used to model activities at the Flashline Man Arctic Research Station for possible use in planning Mars missions. The station is located at Haughton Crater, Devon Island, Nutiavut, Arctic Canada.

Laufenberg, Lawrence↗

Process for Nitrogen Oxide Waste Conversion to Fertilizer

The present invention describes a process for converting vapor streams from sources containing at least one nitrogen-containing oxidizing agent therein to a liquid fertilizer composition comprising the steps of: a) directing a vapor stream containing at least one nitrogen-containing oxidizing agent to a first contact zone; b) contacting said vapor stream with water to form nitrogen oxide(s) from said at least one nitrogen-containing oxidizing agent; c) directing said acid(s) as a second stream to a second contact zone; d) exposing said second stream to hydrogen peroxide which is present within said second contact zone in a relative amount of at least 0.1% by weight of said second stream within said second contact zone to convert at least some of any nitrogen oxide species or ions other than in the nitrate form present within said second stream to nitrate ion; e) sampling said stream within said second contact zone to determine the relative amount of hydrogen peroxide within said second contact zone; f) adding hydrogen peroxide to said second contact zone when a level of hydrogen peroxide less than 0.1 % by weight in said second stream is determined by said sampling; g) adding a solution comprising potassium hydroxide to said second stream to maintain a pH between 6.0 and 11.0 within said second stream within said second contact zone to form a solution of potassium nitrate; and h) removing said solution of potassium nitrate from said second contact zone.

Lueck, Dale E.↗

A Survey of Collectives

Due to the increasing sophistication and miniaturization of computational components, complex, distributed systems of interacting agents are becoming ubiquitous. Such systems, where each agent aims to optimize its own performance, but where there is a well-defined set of system-level performance criteria, are called collectives. The fundamental problem in analyzing/designing such systems is in determining how the combined actions of self-interested agents leads to 'coordinated' behavior on a iarge scale. Examples of artificial systems which exhibit such behavior include packet routing across a data network, control of an array of communication satellites, coordination of multiple deployables, and dynamic job scheduling across a distributed computer grid. Examples of natural systems include ecosystems, economies, and the organelles within a living cell. No current scientific discipline provides a thorough understanding of the relation between the structure of collectives and how well they meet their overall performance criteria. Although still very young, research on collectives has resulted in successes both in understanding and designing such systems. It is eqected that as it matures and draws upon other disciplines related to collectives, this field will greatly expand the range of computationally addressable tasks. Moreover, in addition to drawing on them, such a fully developed field of collective intelligence may provide insight into already established scientific fields, such as mechanism design, economics, game theory, and population biology. This chapter provides a survey to the emerging science of collectives.

Tumer, Kagan↗

Time course of vasodilatory responses in skeletal muscle arterioles: role in hyperemia at onset of exercise

At the onset of dynamic exercise, muscle blood flow increases within 1-2 s. It has been postulated that local vasodilatory agents produced by the vascular endothelium or the muscle itself contribute to this response. We hypothesized that only vasodilators that act directly on the vascular smooth muscle could produce vasodilation of skeletal muscle arterioles in <2 s. To test this hypothesis, we determined the time course of the vasodilatory response of isolated skeletal muscle arterioles to direct application of potassium chloride, adenosine, acetylcholine, and sodium nitroprusside. Soleus and gastrocnemius muscles were dissected from the hindlimbs of male Sprague-Dawley rats. First-order arterioles (100-200 microm) were isolated, cannulated on micropipettes, and pressurized to 60 cmH(2)O in an organ bath. Vasodilatory agents were added directly to the bath, and diameter responses of the arterioles were recorded in real time on a videotape recorder. Frame-by-frame analysis of the diameter responses indicated that none of the vasodilator agents tested produced significant diameter increases in <4 s in either soleus or gastrocnemius muscle arterioles. These results indicate that, although these local vasodilators produce significant vasodilation of skeletal muscle resistance arterioles, these responses are not rapid enough (within 1-2 s) to contribute to the initiation of the exercise hyperemic response at the onset of dynamic exercise.

NASA Discipline Musculoskeletal↗

Ten-fold augmentation of endothelial uptake of vascular endothelial growth factor with ultrasound after systemic administration

OBJECTIVES: In this study, the feasibility of delivering and enhancing the uptake of vascular endothelial growth factor (VEGF) into the intact endothelium by using ultrasound (US) facilitation was determined. BACKGROUND: A limitation of tissue-targeted drug delivery is the need for direct arterial cannulation. We postulate a mechanism by which agents injected intravenously may be targeted to a tissue using US and ultrasonic contrast agents. METHODS: We used a rat model to test the ability of US and an ultrasonic contrast agent perflurocarbon exposed sonicated dextrose albumin (PESDA) to increase uptake of VEGF in the myocardium. Continuous wave Doppler US (0.6 W/cm2 at 1 MHz for 15 min) was applied to the chest wall overlying the myocardium during intravenous injection with either VEGF (100 microg/kg) alone or a combination of VEGF and PESDA (0.1%). Control rats had VEGF infused without US or PESDA. The VEGF uptake was measured quantitatively in the heart, lung, liver and kidneys by enzyme-linked immunosorbent assay (ng/g of tissue) and morphologically by fluorescence microscopy. RESULTS: There was an eight-fold increase in VEGF uptake in the heart by US alone (16.86 +/- 1.56 vs. 2.11 +/- 0.953 ng/g of tissue, p < 0.0001) and a 13-fold increase with US + PESDA (26.78 +/- 2.88 vs. 2.11 +/- 0.953 ng/g of tissue, p < 0.0001) compared with control rats. Fluorescence microscopy revealed deposition of VEGF in the endothelium of small intramyocardial arterioles. CONCLUSIONS: These results show a marked increase in endothelial VEGF uptake with US and US + PESDA. Thus, US may be used to augment endothelial VEGF uptake 10-fold to 13-fold.

Non-NASA Center↗

Restoring and maintaining bone in osteopenic female rat skeleton: I. Changes in bone mass and structure

This experiment contains the crucial data for the lose, restore, and maintain (LRM) concept, a practical approach for reversing existing osteoporosis. The LRM concept uses anabolic agents to restore bone mass and architecture (+ phase) and then switches to an agent with the established ability to maintain bone mass, to keep the new bone (+/- phase). The purpose of this study was to learn whether switching to an agent known chiefly for its ability to maintain existing bone mass preserves new bone induced by PGE2 in osteopenic, estrogen-depleted rats. The current study had three phases, the bone loss (-), restore (+), and maintain (+/-) phases. We ovariectomized (OX) or sham ovariectomized (sham-OX) 5.5-month-old female rats (- phase). The OX rats were treated 5 months postovariectomy with 1-6 mg PGE2 per kg/day for 75 days to restore lost cancellous bone mass (+ phase), and then PGE2 treatment was stopped and treatment began with 1 or 5 micrograms/kg of risedronate, a bisphosphonate, twice a week for 60 days (+/- phase). During the loss (-) phase, the cancellous bone volume of the proximal tibial metaphysis in the OX rat fell to 19% of initial and 30% of age-matched control levels. During the restore (+) phase, the cancellous bone volume in OX rats doubled. When PGE2 treatment was stopped, however, and no special maintenance efforts were made during the maintain (+/-) phase, the PGE2-induced cancellous bone disappeared. In contrast, the PGE2-induced cancellous bone persisted when the PGE2 treatment was followed by either a 1 or 5 micrograms treatment of risedronate per kg given twice a week for 60 days during the maintain (+/-) phase. The tibial shaft demonstrated very little cortical bone loss during the loss (-) phase in OX rats. The tibial shaft cortical bone fell some 8%. During the restore (+) phase, new cortical bone in OX rats increased by 22%. When PGE2 treatment was stopped and nothing was given during the maintain (+/-) phase, however, all but the PGE2-induced subperiosteal bone disappeared. In contrast, when PGE2 treatment was stopped and 1 micron risedronate per kg twice a week for 60 days was administered during the maintenance (+/-) phase, the PGE2-induced subperiosteal bone and some of the subendocortical bone and marrow trabeculae persisted.(ABSTRACT TRUNCATED AT 400 WORDS).

NASA Discipline Musculoskeletal↗

Serotonin, atherosclerosis, and collateral vessel spasm

Studies on animal models demonstrate that platelet products contribute to vascular spasm in ischemic syndromes and that this is reversible with administration of ketanserin and thromboxane synthesis inhibitors. Laboratory animals (dogs, rabbits, and rats) that had femoral artery ligations exhibited supersensitivity to serotonin within days in their collateral blood vessels. This supersensitivity lasted at least 6 months. The response to serotonin was reversed by ketanserin, but not by 5HT-1 antagonists. Supersensitivity does not extend to norepinephrine, and alpha blockers do not influence the response to serotonin. It appears that platelet activation by endothelial injury contributes to ischemia through blood vessel occlusion and vascular spasm. When platelet activation occurs in vivo, blood vessel occlusion and vascular spasm are reversible in part by using ketanserin or agents that block thromboxane synthesis or its action. Combining both classes of agents reverses spasm completely. These findings support existing evidence that platelet products contribute to vascular disease, and provide an approach to improved management with currently available pharmacologic agents.

Review, Tutorial↗

Adaptive Bio-Inspired Wireless Network Routing for Planetary Surface Exploration

Wireless mobile networks suffer connectivity loss when used in a terrain that has hills, and valleys when line of sight is interrupted or range is exceeded. To resolve this problem and achieve acceptable network performance, we have designed an adaptive, configurable, hybrid system to automatically route network packets along the best path between multiple geographically dispersed modules. This is very useful in planetary surface exploration, especially for ad-hoc mobile networks, where computational devices take an active part in creating a network infrastructure, and can actually be used to route data dynamically and even store data for later transmission between networks. Using inspiration from biological systems, this research proposes to use ant trail algorithms with multi-layered information maps (topographic maps, RF coverage maps) to determine the best route through ad-hoc network at real time. The determination of best route is a complex one, and requires research into the appropriate metrics, best method to identify the best path, optimizing traffic capacity, network performance, reliability, processing capabilities and cost. Real ants are capable of finding the shortest path from their nest to a food source without visual sensing through the use of pheromones. They are also able to adapt to changes in the environment using subtle clues. To use ant trail algorithms, we need to define the probability function. The artificial ant is, in this case, a software agent that moves from node to node on a network graph. The function to calculate the fitness (evaluate the better path) includes: length of the network edge, the coverage index, topology graph index, and pheromone trail left behind by other ant agents. Each agent modifies the environment in two different ways: 1) Local trail updating: As the ant moves between nodes it updates the amount of pheromone on the edge; and 2) Global trail updating: When all ants have completed a tour the ant that found the shortest route updates the edges in its path.

Alena, Richard I.↗

Acquisition of Autonomous Behaviors by Robotic Assistants

Our research achievements under the NASA-JSC grant contributed significantly in the following areas. Multi-agent based robot control architecture called the Intelligent Machine Architecture (IMA) : The Vanderbilt team received a Space Act Award for this research from NASA JSC in October 2004. Cognitive Control and the Self Agent : Cognitive control in human is the ability to consciously manipulate thoughts and behaviors using attention to deal with conflicting goals and demands. We have been updating the IMA Self Agent towards this goal. If opportunity arises, we would like to work with NASA to empower Robonaut to do cognitive control. Applications 1. SES for Robonaut, 2. Robonaut Fault Diagnostic System, 3. ISAC Behavior Generation and Learning, 4. Segway Research.

Peters, R. A., II↗

Formalism Challenges of the Cougaar Model Driven Architecture

The Cognitive Agent Architecture (Cougaar) is one of the most sophisticated distributed agent architectures developed today. As part of its research and evolution, Cougaar is being studied for application to large, logistics-based applications for the Department of Defense (DoD). Anticipiting future complex applications of Cougaar, we are investigating the Model Driven Architecture (MDA) approach to understand how effective it would be for increasing productivity in Cougar-based development efforts. Recognizing the sophistication of the Cougaar development environment and the limitations of transformation technologies for agents, we have systematically developed an approach that combines component assembly in the large and transformation in the small. This paper describes some of the key elements that went into the Cougaar Model Driven Architecture approach and the characteristics that drove the approach.

Bohner, Shawn A.↗