Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “agent based modeling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 667 records · Page 37
All sourcesNASA NTRSDOE OSTI
NASA NTRS2003

Principles for Integrating Mars Analog Science, Operations, and Technology Research

During the Apollo program, the scientific community and NASA used terrestrial analog sites for understanding planetary features and for training astronauts to be scientists. Human factors studies (Harrison, Clearwater, & McKay 1991; Stuster 1996) have focused on the effects of isolation in extreme environments. More recently, with the advent of wireless computing, we have prototyped advanced EVA technologies for navigation, scheduling, and science data logging (Clancey 2002b; Clancey et al., in press). Combining these interests in a single expedition enables tremendous synergy and authenticity, as pioneered by Pascal Lee's Haughton-Mars Project (Lee 2001; Clancey 2000a) and the Mars Society s research stations on a crater rim on Devon Island in the High Canadian Arctic (Clancey 2000b; 2001b) and the Morrison Formation of southeast Utah (Clancey 2002a). Based on this experience, the following principles are proposed for conducting an integrated science, operations, and technology research program at analog sites: 1) Authentic work; 2) PI-based projects; 3) Unencumbered baseline studies; 4) Closed simulations; and 5) Observation and documentation. Following these principles, we have been integrating field science, operations research, and technology development at analog sites on Devon Island and in Utah over the past five years. Analytic methods include work practice simulation (Clancey 2002c; Sierhuis et a]., 2000a;b), by which the interaction of human behavior, facilities, geography, tools, and procedures are formalized in computer models. These models are then converted into the runtime EVA system we call mobile agents (Clancey 2002b; Clancey et al., in press). Furthermore, we have found that the Apollo Lunar Surface Journal (Jones, 1999) provides a vast repository or understanding astronaut and CapCom interactions, serving as a baseline for Mars operations and quickly highlighting opportunities for computer automation (Clancey, in press).

Clancey, William J.↗
NASA NTRS2021

Developing a Set of Indicators to Identify, Monitor, and Track Impacts and Change in Forests of the United States

United States forestland is an important ecosystem type, land cover, land use, and economic resource that is facing several drivers of change including climatic. Because of its significance, forestland was identified through the National Climate Assessment(NCA) as a key sector and system of concern to be included in a system of climate indicators as part of a sustained assessment effort. Here, we describe 11 informative core indicators of forests and climate change impacts with metrics available or nearly available for use in the NCA efforts. The recommended indicators are based on a comprehensive conceptual model which recognizes forests as a land use, an ecosystem, and an economic sector. The indicators cover major forest attributes such as extent, structural components such as biomass, functions such as growth and productivity, and ecosystem services such as biodiversity and outdoor recreation. Interactions between humans and forests are represented through indicators focused on the wildland-urban interface, cost to mitigate wildfire risk, and energy produced from forest-based biomass. Selected indicators also include drought and disturbance from both wildfires and biotic agents. The forest indicators presented are an initial set that will need further refinement in coordination with other NCA indicator teams. Our effort ideally will initiate the collection of critical measurements and observations and lead to additional research on forest-climate indicators.

U.S. forests↗
DOE OSTI2022

In Situ EXAFS-Based Nanothermometry of Heterodimer Nanocrystals under Induction Heating

The ability to heat nanocrystalline materials through magnetic induction has been used in the fields of catalysis, biomedical sciences, and polymer degradation. However, the working temperature to which the inductively coupled material rises is still poorly understood. Herein, we use extended X-ray absorption fine structure spectroscopy in conjunction with thermal imaging to improve the understanding of heating in inductively coupled systems. After extraction of the Debye–Waller factor from the spectroscopy, we obtain the temperature of inductively heated nanocrystals from the correlated Debye model. We combine carbon-supported iron oxide nanocrystals as induction heating agents with platinum nanocrystals as thermal probes. By testing these nanocrystal species as both unattached nanocrystals and heterodimers, we report that nanostructured systems show a significant temperature difference of up to 73.60 °C when compared to their local support environment. Furthermore, this result has implications for inductively heated catalysis, magnetic hyperthermia for targeted cell death, and polymer synthesis.

09 BIOMASS FUELS↗
DOE OSTI2024

Co-orchestration of multiple instruments to uncover structure–property relationships in combinatorial libraries

The rapid growth of automated and autonomous instrumentation brings forth opportunities for the co-orchestration of multimodal tools that are equipped with multiple sequential detection methods or several characterization techniques to explore identical samples. This is exemplified by combinatorial libraries that can be explored in multiple locations via multiple tools simultaneously or downstream characterization in automated synthesis systems. In co-orchestration approaches, information gained in one modality should accelerate the discovery of other modalities. Correspondingly, an orchestrating agent should select the measurement modality based on the anticipated knowledge gain and measurement cost. Herein, we propose and implement a co-orchestration approach for conducting measurements with complex observables, such as spectra or images. The method relies on combining dimensionality reduction by variational autoencoders with representation learning for control over the latent space structure and integration into an iterative workflow via multi-task Gaussian Processes (GPs). This approach further allows for the native incorporation of the system's physics via a probabilistic model as a mean function of the GPs. We illustrate this method for different modes of piezoresponse force microscopy and micro-Raman spectroscopy on a combinatorial Sm-BiFeO3 library. However, the proposed framework is general and can be extended to multiple measurement modalities and arbitrary dimensionality of the measured signals.

47 OTHER INSTRUMENTATION↗
NASA NTRS2016

HST-STIS Spectra of Saturn's Rings and Implications for Their Reddening Agent

We obtained HST-STIS spectra of Saturn's main rings in May 2011, using the G230L (and G430L) gratings, with final averaged radial resolution of 160 (and 330) km/pixel. The dataset filled a previous 200-330nm "spectral gap" between Cassini and ground-based spectra. The data provide radial profiles as a function of wavelength, but our most basic product at this point is a set of very low-noise spectra, radially averaged over broad regions of the rings (A, B, C, and Cassini Division). The raw spectra required special processing to remove artifacts due to extended-source grating scatter. We have modeled the spectra using a new particle surface model, which corrects for on-surface shadowing due to the likely very rough ring particle surfaces, and avoids overestimation of intra-mixed "neutral absorber". We correct for non-classical layer effects and finite ring optical depth, and relate our observed reflectivities to the spherical albedos of individual smooth particles. We model these smooth particle albedos using standard Hapke theory for regolith grain mixtures that are either homogeneous and "intramixed" (nonicy absorbers dispersed in water ice regolith grains) or heterogeneous "intimate" mixtures. As candidates for the nonicy contaminants we have considered amorphous carbon, aromatic-rich and aliphatic-rich organic tholins, silicates, hematite and iron metal. For the A and B rings, we find that iron metal (including a new theoretical estimate of the refractive indices of nanometer-sized grains of iron) is not spectrally steep enough in the 200-300nm range, and that aliphatic-rich tholins are either too steep at short wavelengths or too flat at long wavelengths. However, less than 1% by mass of aromatic-rich tholins provides a very good fit across the entire spectral range with no gratuitous "neutral absorber" needed, and a minimum of additional free parameters. The best fits require forward-scattering regolith grains. For the C Ring and Cassini Division, additional absorbers are needed (updated results will be given).

Saturn's Rings↗
DOE OSTI2020

Indium and thallium extraction into betainium bis(trifluoromethylsulfonyl)imide ionic liquid from aqueous hydrochloric acid media

The partitioning behavior of indium and thallium ions between aqueous hydrochloric acid solutions and hydrophobic betainium bis(trifluoromethylsulfonyl)imide ionic liquid was studied using a standard liquid–liquid extraction technique. The experiments were mainly carried out by means of a radiochemical method, using 111 In and 201 Tl medical radionuclides at an ultra-trace concentration. Both mono- and trivalent thallium were utilized and the highest oxidation state (3+) was produced by adding either chlorine or bromine water to the aqueous phase prior to extraction. The metal ion transfer was optimized by varying the aqueous concentration of hydrochloric acid and by introducing zwitterionic betaine as an extracting agent into the aqueous phase. It was found that the highest D values of In(III), Tl(III) and Tl(I) in the investigated chemical systems were up to 200, 70, and 1, respectively. The influence of the addition of lithium salt, having common with the ionic liquid counterion, and the variation of mixing time on the distribution ratio of the metal ions were also studied. Finally, to understand the underlying extraction mechanism of the metal ions and to propose the predominant extracted species, mathematical models based on ion pair formation and ion exchange were developed.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS1989

A layered abduction model of perception: Integrating bottom-up and top-down processing in a multi-sense agent

A layered-abduction model of perception is presented which unifies bottom-up and top-down processing in a single logical and information-processing framework. The process of interpreting the input from each sense is broken down into discrete layers of interpretation, where at each layer a best explanation hypothesis is formed of the data presented by the layer or layers below, with the help of information available laterally and from above. The formation of this hypothesis is treated as a problem of abductive inference, similar to diagnosis and theory formation. Thus this model brings a knowledge-based problem-solving approach to the analysis of perception, treating perception as a kind of compiled cognition. The bottom-up passing of information from layer to layer defines channels of information flow, which separate and converge in a specific way for any specific sense modality. Multi-modal perception occurs where channels converge from more than one sense. This model has not yet been implemented, though it is based on systems which have been successful in medical and mechanical diagnosis and medical test interpretation.

Josephson, John R.↗
NASA NTRSDate not supplied

How Autonomous Intelligent Systems Can Facilitate Earth-independent Medical Care: Going Beyond Telepresence

During the last decade, teleoperated robotic systems have extended humans’ sensorimotor competence to digitally fly beyond the physical barrier of distance and scale and thus transmit sensorimotor skills of the human through direct communication. Telepresence capabilities have enabled tele-physical remote access at small scales thanks to telerobotic mediums. Although the concept was initially motivated by space applications, such technologies quickly have expanded into the medical domain and resulted in teleoperated medical robots, including telerobotic surgical systems (such as the da Vinci surgical system). Effective telepresence fundamentally depends on an agile, reliable, and secure communication medium that can transmit real-time information between the operator and a remote device. However, direct telepresence may not be achievable for long-duration exploration spaceflight missions. Thus, autonomous systems and local intelligence represent potential solutions to the aforementioned issues. One example solution employs demonstration systems which enable learning from the pre-captured inputs of a skilled human operator. These will be computationally modeled and later probabilistically replicated toward the completion of remote physical tasks when direct telepresence is not viable - such as under communication blackout conditions. In other words, trained autonomous systems (e.g., robots) can perform remote operations that mimic the physical performance of experts during remote operations/training. Beyond learning the physics of the task, autonomous agents can also be used to conduct algorithmic decision-making that mimics the higher-level cognition of the expert. Thus, using an autonomous system, pre-trained cognitive and manipulation-based skills can be leveraged (acquired during pre-mission events) to produce digital twins of an intelligent operator. Such systems can be used for the real-time conduction of intricate tasks in complex and unstructured environments. Such systems will operationalize “cognitive digital twins” and can expand the reach of human cognition and manipulation through the power of data-driven learning from demonstration algorithms. This system category will be discussed as a fully autonomous operation in this talk. In addition to the above, we will also propose and discuss the possibility of partial-automation using remote intelligence and remote sensing. In contrast to full automation, partial automation can close the loop through a local operator equipped with augmented sensory awareness through wearable systems. Such technologies will allow the local operator to conduct delicate tasks while being guided using sensory augmentation and being monitored to gauge her/his level of cognitive focus and performance. The difference with the previous category is that a remote human will conduct the task. Further, rather than making a digital twin of human cognition, we will augment the control inputs of the local human to match those of the skilled expert operator who is not accessible in real-time. Going beyond classic telepresence and thus approaching intelligent telepresence, our vision is that autonomous agents will eventually enable the safe, consistent and efficient delivery of complex, remote and smart medical care during space exploration across operators in an Earth-independent fashion. We will discuss our collective vision from NASA and MERIIT@NYU lab in this talk.

Telepresence↗
DOE OSTI2024

Distributed optimization for multi-commodity urban traffic control

A distributed method for concurrent traffic signal and routing control of traffic networks is proposed. The method is based on the multi-commodity store-and-forward model, in which the destinations are the commodities. The system benefits from the communication between vehicles and infrastructure, providing optimal signal timings to intersections and routes to vehicles on a link-by-link basis. Using the augmented Lagrangian to model the constraints into the objective, the baseline centralized problem is decomposed into a set of objective-coupled subproblems, one for each intersection, enabling the solution to be computed by a distributed- gradient projection algorithm. Further, the intersection agents only need to communicate and coordinate with neighboring intersections to ensure convergence to the optimal solution while tolerating suboptimal iterations that offer more flexibility, unlike other distributed approaches. Through microsimulation, we demonstrate the effectiveness of the proposed algorithm in traffic networks with time-varying demand. Computational analysis shows that the distributed problem is suitable for real-time applications. A robustness analysis show that the distributed formulation enables a graceful degradation of the system in case of failure.

Augmented Lagrangian↗
NASA NTRS2019

Mitigation Strategies for Space Radiation Health Risks

Astronauts embarking on missions beyond low Earth orbit (LEO) will be exposed to a radiation field that may increase the risks of developing cancer, cardiovascular diseases, central nervous system disorders, and immune decrements. Operational parameters will be the primary determinants of crew radiation exposure. NASA uses integrated design tools and risk models to optimize these parameters to minimize radiation exposure. NASA is also considering medical countermeasures (MCMs) to reduce radiation-associated health risks. MCMs for potential use in space-based applications can be developed from a variety of sources, including: a) population-based chemoprevention trials against targeted diseases b) drug development efforts focused on treating acute effects from accidental radiation exposures c) drug development to mitigate side effects of radiotherapy d) mechanistic studies of distinct damage caused by high charge (Z) and energy (HZE) radiation. Use of agents developed for other applications, or repurposed, is advantageous because long-term safety in humans is already established.

Huff, Janice L.↗
DOE OSTI2021

Distributional Deep Reinforcement Learning-Based Emergency Frequency Control

Emergency frequency control is one of the most critical approaches to maintain power system stability after major disturbances. With the increasing number of grid-connected renewable energy sources, existing model-based methods of frequency control are facing up with challenges of computational speed and scalability for large-scale systems. In this paper, the emergency frequency control problem is formulated as a Markov Decision Process (MDP) and solved through a novel Distributional Deep Reinforcement Learning (DDRL) method, namely the distributional soft actor critic (DSAC) method. Compared with other RL methods that only estimate the mean value, the proposed DSAC model estimates the distribution of value function over returns. This advancement can lead to more insights and knowledge for the agent, with the benefit of a much faster and more stable learning process, and the improved frequency control performance. Here, the simulation results on IEEE 39-bus and IEEE 118-bus systems demonstrate the effectiveness and robustness of proposed models, as well as the advantage compared to other state-of-the-art DRL algorithms.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

An Edge-Cloud Integrated Solution for Buildings Demand Response Using Reinforcement Learning

Buildings, as major energy consumers, can provide great untapped demand response (DR) resources for grid services. However, their participation remains low in real-life. One major impediment for popularizing DR in buildings is the lack of cost-effective automation systems that can be widely adopted. Existing optimization-based smart building control algorithms suffer from high costs on both building-specific modeling and on-demand computing resources. To tackle these issues, this paper proposes a cost-effective edge-cloud integrated solution using reinforcement learning (RL). Beside RL’s ability to solve sequential optimal decision-making problems, its adaptability to easy-to-obtain building models and the off-line learning feature are likely to reduce the controller’s implementation cost. Using a surrogate building model learned automatically from building operation data, an RL agent learns an optimal control policy on cloud infrastructure, and the policy is then distributed to edge devices for execution. Simulation results demonstrate the control efficacy and the learning efficiency in buildings of different sizes. A preliminary cost analysis on a 4-zone commercial building shows the annual cost for optimal policy training is only 2.25% of the DR incentive received. Results of this study show a possible approach with higher return on investment for buildings to participate in DR programs.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2019

Nanoparticle Injection Technology for Remediating Leaks of CO 2 Storage Formation (Final Report)

Leakage of a well can occur through the wellbore, the annulus between well tubing and steel casing, or on the outside of the casing. The overall goal of the project is to develop a new technology that can repair or reduce the leakage of wellbore. The new technology is based on simultaneously injecting nanoparticles electrochemically and removing some of the harmful ions in the system so the distressed cement materials can be densified and the corrosion risk of steel casing can be reduced. There were two objectives. One is the development of the injection technology for leakage repair and the other is the development of new numerical simulation models that can simulate and predict the performance of the new repair technology. An electro-migration testing unit and a in-house wellbore testing system were developed; healing agents with nanoparticles were selected, numerical models were developed to simulate the electro-migration technology, and the effectiveness of the new technology were evaluated.

42 ENGINEERING↗
DOE OSTI2019

Nanoparticle Injection Technology for Remediating Leaks of CO 2 Storage Formation (Final Technical Report)

Leakage of a well can occur through the wellbore, the annulus between well tubing and steel casing, or on the outside of the casing. The overall goal of the project is to develop a new technology that can repair or reduce the leakage of wellbore. The new technology is based on simultaneously injecting nanoparticles electrochemically and removing some of the harmful ions in the system so the distressed cement materials can be densified and the corrosion risk of steel casing can be reduced. There were two objectives. One is the development of the injection technology for leakage repair and the other is the development of new numerical simulation models that can simulate and predict the performance of the new repair technology. An electro-migration testing unit and an in-house wellbore testing system were developed; healing agents with nanoparticles were selected, numerical models were developed to simulate the electro-migration technology, and the effectiveness of the new technology was evaluated.

42 ENGINEERING↗
NASA NTRS2004

Applications of Intelligent Tutoring Systems to Human-Robotic Exploration of Mars

Space missions with small crews extending over several years with time-delay preventing normal conversations with people on earth will raise many challenges for training. Of special interest are possible three-year missions to Mars, requiring refresher instruction and learning new skills based on unexpected problems with machines and environmental conditions. For example, the crew will be required to monitor and repair more complex life support systems for air and water recycling than we even know how to build today. Highly educated astronauts, often with several doctorate degrees, require a very different mode of interaction than we have developed for school children or even typical college students. Explanation methods may need to differ-using analogies and techniques from different domains-depending on whether the astronaut is an astrophysicist, a pilot, or a geologist.Virtual reality (e.g., for Hubble repair missions) and "integrated" simulations (involving role-playing and emphasizing failure scenarios) are the most common advanced forms of instruction used in space flight today. The emphasis is on collaborative, embodied interaction with the same workstations and tools used in practice (e.g., a cockpit simulator). Otherwise, computerized instructional technology used by NASA is not model-based or tutorial in nature. This discussion will review some of the key instructional methods used at NASA over the past two decades and consider why ITS methods have not been exploited. Some of the problems and opportunities for training for Mars missions are examined, including how using robots in exploration activities will help but raise new training problems. These ideas will be illustrated with examples from the BrahmsVE system in which a browser- based virtual reality display with avatars allows interacting with a distributed multiagent system, in which agents can be people, robots, or software programs. Using BrahmsVE may provide a way for astronauts to interact with proxies of people who serve as instructional coaches on Mars.

Clancey, William J.↗
DOE OSTI2022

The role of information structures in game-theoretic multi-agent learning

Multi-agent learning (MAL) studies how agents learn to behave optimally and adaptively from their experience when interacting with other agents in dynamic environments. The outcome of a MAL process is jointly determined by all agents’ decision-making. Hence, each agent needs to think strategically about others’ sequential moves, when planning future actions. The strategic interactions among agents makes MAL go beyond the direct extension of single-agent learning to multiple agents. With the strategic thinking, each agent aims to build a subjective model of others decision-making using its observations. Such modeling is directly influenced by agents’ perception during the learning process, which is called the information structure of the agent’s learning. As it determines the input to MAL processes, information structures play a significant role in the learning mechanisms of the agents. This review creates a taxonomy of MAL and establishes a unified and systematic way to understand MAL from the perspective of information structures. Here we define three fundamental components of MAL: the information structure (i.e., what the agent can observe), the belief generation (i.e., how the agent forms a belief about others based on the observations), as well as the policy generation (i.e., how the agent generates its policy based on its belief). In addition, this taxonomy enables the classification of a wide range of state-of-the-art algorithms into four categories based on the belief-generation mechanisms of the opponents, including stationary, conjectured, calibrated, and sophisticated opponents. We introduce Value of Information (VoI) as a metric to quantify the impact of different information structures on MAL. Finally, we discuss the strengths and limitations of algorithms from different categories and point to promising avenues of future research.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Reduced Order Model of Transactive Bidding Loads

Transactive energy (TE) has been identified to provide better grid efficiency and reliability by market-based transactive exchanges between energy producers and energy consumers. Simulations of TE systems are crucial to evaluate the benefits and impacts of different transactive mechanisms. However, such simulations can be time consuming due to the information exchange between various participants and complex co-simulation environments. In this paper, we develop a reduced order model to speed up the simulation of transactive systems in TE simulation platform (TESP) while achieving very low error between the reduced order and full model results. Specifically, the developed reduced order model consists of an aggregate responsive load agent which utilizes two Recurrent Neural Networks (RNNs) with Long Short-Term Memory units (LSTMs) to enable transactive elements to collectively participate in the TE system. The proposed aggregate responsive load (ARL) agent is able to produce similar transactive behaviors to the full simulation model while achieving significant simulation time reduction. Finally, we also show that the developed model enables generalization of simulation results across different dates and across different number of loads included in the simulations.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Multiscale modeling of HBV infection integrating intra- and intercellular viral propagation to analyze extracellular viral markers

Chronic infection with hepatitis B virus (HBV) is caused by the persistence of closed circular DNA (cccDNA) in the nucleus of infected hepatocytes. Despite available therapeutic anti-HBV agents, eliminating the cccDNA remains challenging. Thus, quantifying and understanding the dynamics of cccDNA are essential for developing effective treatment strategies and new drugs. However, such study requires repeated liver biopsy to measure the intrahepatic cccDNA, which is basically not accepted because liver biopsy is potentially morbid and not common during hepatitis B treatment. We here aimed to develop a noninvasive method for quantifying cccDNA in the liver using surrogate markers in peripheral blood. We constructed a multiscale mathematical model that explicitly incorporates both intracellular and intercellular HBV infection processes. The model, based on age-structured partial differential equations, integrates experimental data from in vitro and in vivo investigations. By applying this model, we roughly predicted the amount and dynamics of intrahepatic cccDNA within a certain range using specific viral markers in serum samples, including HBV DNA, HBsAg, HBeAg, and HBcrAg. Our study represents a significant step towards advancing the understanding of chronic HBV infection. The noninvasive quantification of cccDNA using our proposed method holds promise for improving clinical analyses and treatment strategies. By comprehensively describing the interactions of all components involved in HBV infection, our multiscale mathematical model provides a valuable framework for further research and the development of targeted interventions.

59 BASIC BIOLOGICAL SCIENCES↗