Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “agent based modeling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 739 records · Page 41
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Efficient Reinforcement Learning for Real-Time Hardware-Based Energy System Experiments: Preprint

In the context of urgent climate challenges and the pressing need for rapid technology development, Reinforcement Learning (RL) stands as a compelling data-driven method for controlling real-world physical systems. However, RL implementation often entails time-consuming and computationally intensive data collection and training processes, rendering them inefficient for real-time applications that lack non-real-time models. To address these limitations, real-time emulation techniques have emerged as valuable tools for the lab-scale rapid prototyping of intricate energy systems. While emulated systems offer a bridge between simulation and reality, they too face constraints, hindering comprehensive characterization, testing, and development. In this research, we construct a surrogate model using limited data from simulated systems, enabling an efficient and effective training process for a Double Deep Q-Network (DDQN) agent for future deployment. Our approach is illustrated through a hydropower application, demonstrating the practical impact of our approach on climate-related technology development.

deep Q-learning↗
DOE OSTI2024

SEAS Communication Engine: An Extensible, Flexible Wrapper for Co-Simulation Agents

When modeling and analyzing the power grid and other large scale systems, researchers often express scenarios as optimization problems and feed them into advanced software solvers. In order to allow multiple solvers to communicate with each other and share data from different domains, the National Renewable Energy Laboratory (NREL) and associated Department of Energy (DOE) labs have developed a software framework called the Hierarchical Engine for Large-scale Infrastructure Co-Simulation (HELICS). HELICS allows cosimulation via a collection of client libraries for different languages that can be called from the appropriate optimization software. However, these client libraries do not provide a higher level of abstraction beyond reading and writing data off of the shared HELICS bus. In this paper, we describe a new software library called the SEAS Communication Engine that exposes a higher-level API for running cosimulation problems. The SEAS Engine provides a class-based abstraction on top of the Python HELICS client, in order to allow users to implement their domain-specific cosimulations without needing to interact with core HELICS primitives. This will make adoption of HELICS and cosimulation in general easier, by exposing a simpler API. In the second part of the paper, we validate our library on a collection of different simulation examples, including the canonical IEEE 13 Bus Feeder. Lastly, we demonstrate using the SEAS Engine to directly call domain-specific code written in the Julia programming language. Our hope is that this will serve as a template for easily calling software in different programming languages via the SEAS Engine, thereby avoiding code duplication and complexity.

co-simulation↗
DOE OSTI2026

Robust Rapid Cellular Metabolite Sensing Using Benchtop NMR and SABRE-Hyperpolarized [1- 13 C]Pyruvate

Hyperpolarized NMR has emerged as a powerful analytical technique to significantly enhance targeted NMR signals, improving the sensitivity for investigations of unique chemical and biological dynamics. Here, we demonstrate the use of a hyperpolarization strategy based on Signal Amplification By Reversible Exchange (SABRE) to generate highly reproducible doses of a hyperpolarized [1- 13 C]pyruvate probe for benchtop characterization of yeast metabolism. This method allows rapid, scalable, and benchtop preparation of biocompatible hyperpolarized solutions suitable for live-cell experiments. We show that this production can be dove-tailed into a modular, compact workflow to characterize real-time metabolism in cell cultures, using Saccharomyces cerevisiae (Baker’s yeast) as a model organism. With high temporal resolution, we show that this method can resolve the conversion of hyperpolarized [1- 13 C]pyruvate into oxidative decarboxylation products CO 2 and bicarbonate. This conversion exhibits sustained and detectable metabolic activity for over 300 s after introduction of the agent to the cells. We model the metabolite kinetics to show decarboxylation activity and derive estimates of the pH over time from the CO 2 and bicarbonate (carbonic acid buffer system) equilibrium to probe changes in the cellular environment during active metabolism. These results highlight the utility of benchtop SABRE-hyperpolarized [1- 13 C]pyruvate as a scalable, specific probe for metabolic phenotyping of living cells using compact, low-cost instrumentation well-suited for future high-throughput applications across microbial engineering, drug response profiling, and dynamic metabolic screening.

fungi↗
DOE OSTI2024

Combatting melioidosis with chemical synthetic lethality

Burkholderia thailandensishas emerged as a nonpathogenic surrogate forBurkholderia pseudomallei, the causative agent of melioidosis, and an important Gram-negative model bacterium for studying the biosynthesis and regulation of secondary metabolism. We recently reported that subinhibitory concentrations of trimethoprim induce vast changes in both the primary and secondary metabolome ofB. thailandensis. In the current work, we show that the folate biosynthetic enzyme FolE2 is permissive under standard growth conditions but essential forB. thailandensisin the presence of subinhibitory doses of trimethoprim. Reasoning that FolE2 may serve as an attractive drug target, we screened for and identified ten inhibitors, including dehydrocostus lactone (DHL), parthenolide, and β-lapachone, all of which are innocuous individually but form a chemical-synthetic lethal combination with subinhibitory doses of trimethoprim. We show that DHL is a mechanism-based inhibitor of FolE2 and capture the structure of the covalently inhibited enzyme using X-ray crystallography. In vitro, the combination of subinhibitory trimethoprim and DHL is more potent than Bactrim, the current standard of care against melioidosis. Moreover, unlike Bactrim, this combination does not affect the growth of most commensal and beneficial gut bacteria tested, thereby providing a degree of specificity againstB. pseudomallei. Our work provides a path for identifying antimicrobial drug targets and for utilizing binary combinations of molecules that form a toxic cocktail based on metabolic idiosyncrasies of specific pathogens.

Science & Technology - Other Topics↗
DOE OSTI2024

Linking Threat Agents to Targeted Organizations: A Pipeline for Enhanced Cybersecurity Risk Metrics

In this study, we present a methodology leveraging Large Language Models (LLMs) to transform Cybersecurity Threat Intelligence (CTI) narratives into actionable insights for individual organizations. Our approach automates the extraction of machine-readable adversary SKRAM (Skills, Knowledge, Resources, Authorities, and Motivation) attributes from open-source reports, extending LLM utility beyond typical interactions. This innovation enables precise, automated assessments of cybersecurity risks posed by various adversaries. Using a chain-of-thought and multi-shot prompting strategy, our methodology advances the automation of cybersecurity feature extraction for new machine-learning models that predict the risk of adversary targeting. This approach is refined using a substantial dataset of over 150 analyst-validated threat reports and synthetic organizational data from 900 companies. Here, by bootstrapping the training data with a rule-based heuristic over synthetic data, we have developed a high-accuracy machine-learning model that allows entities to dynamically prioritize threats and defensive actions.

Cyber Threat Intelligence↗
DOE OSTI2024

Extreme Risk Mitigation in Reinforcement Learning using Extreme Value Theory

Risk-sensitive reinforcement learning (RL) has garnered significant attention in recent years due to the growing interest in deploying RL agents in real-world scenarios. A critical aspect of risk awareness involves modelling highly rare risk events (rewards) that could potentially lead to catastrophic outcomes. These infrequent occurrences present a formidable challenge for data-driven methods aiming to capture such risky events accurately. While risk-aware RL techniques do exist, they suffer from high variance estimation due to the inherent data scarcity. Our work proposes to enhance the resilience of RL agents when faced with very rare and risky events by focusing on refining the predictions of the extreme values predicted by the state-action value distribution. To achieve this, we formulate the extreme values of the state-action value function distribution as parameterized distributions, drawing inspiration from the principles of extreme value theory (EVT). We propose an extreme value theory based actor-critic approach, namely, Extreme Valued Actor-Critic (EVAC) which effectively addresses the issue of infrequent occurrence by leveraging EVT-based parameterization. Importantly, we theoretically demonstrate the advantages of employing these parameterized distributions in contrast to other risk-averse algorithms. Our evaluations show that the proposed method outperforms other risk averse RL algorithms on a diverse range of benchmark tasks, each encompassing distinct risk scenarios.

Wang, Yu↗
NASA NTRS2012

Aerosol Direct, Indirect, Semidirect, and Surface Albedo Effects from Sector Contributions Based on the IPCC AR5 Emissions for Preindustrial and Present-day Conditions

The anthropogenic increase in aerosol concentrations since preindustrial times and its net cooling effect on the atmosphere is thought to mask some of the greenhouse gas-induced warming. Although the overall effect of aerosols on solar radiation and clouds is most certainly negative, some individual forcing agents and feedbacks have positive forcing effects. Recent studies have tried to identify some of those positive forcing agents and their individual emission sectors, with the hope that mitigation policies could be developed to target those emitters. Understanding the net effect of multisource emitting sectors and the involved cloud feedbacks is very challenging, and this paper will clarify forcing and feedback effects by separating direct, indirect, semidirect and surface albedo effects due to aerosols. To this end, we apply the Goddard Institute for Space Studies climate model including detailed aerosol microphysics to examine aerosol impacts on climate by isolating single emission sector contributions as given by the Coupled Model Intercomparison Project Phase 5 (CMIP5) emission data sets developed for Intergovernmental Panel on Climate Change (IPCC) AR5. For the modeled past 150 years, using the climate model and emissions from preindustrial times to present-day, the total global annual mean aerosol radiative forcing is -0.6 W/m(exp 2), with the largest contribution from the direct effect (-0.5 W/m(exp 2)). Aerosol-induced changes on cloud cover often depends on cloud type and geographical region. The indirect (includes only the cloud albedo effect with -0.17 W/m(exp 2)) and semidirect effects (-0.10 W/m(exp 2)) can be isolated on a regional scale, and they often have opposing forcing effects, leading to overall small forcing effects on a global scale. Although the surface albedo effects from aerosols are small (0.016 W/m(exp 2)), triggered feedbacks on top of the atmosphere (TOA) radiative forcing can be 10 times larger. Our results point out that each emission sector has varying impacts by geographical region. For example, the single sector most responsible for a net positive radiative forcing is the transportation sector in the United States, agricultural burning and transportation in Europe, and the domestic emission sector in Asia. These sectors are attractive mitigation targets.

Bauer, Susanne E.↗
DOE OSTI2022

Quantum model learning agent: characterisation of quantum systems through machine learning

Accurate models of real quantum systems are important for investigating their behaviour, yet are difficult to distil empirically. Here, we report an algorithm—the quantum model learning agent (QMLA)—to reverse engineer Hamiltonian descriptions of a target system. We test the performance of QMLA on a number of simulated experiments, demonstrating several mechanisms for the design of candidate Hamiltonian models and simultaneously entertaining numerous hypotheses about the nature of the physical interactions governing the system under study. QMLA is shown to identify the true model in the majority of instances, when provided with limited a priori information, and control of the experimental setup. Our protocol can explore Ising, Heisenberg and Hubbard families of models in parallel, reliably identifying the family which best describes the system dynamics. We demonstrate QMLA operating on large model spaces by incorporating a genetic algorithm to formulate new hypothetical models. The selection of models whose features propagate to the next generation is based upon an objective function inspired by the Elo rating scheme, typically used to rate competitors in games such as chess and football. In all instances, our protocol finds models that exhibit F 1 score ≥ 0.88 when compared with the true model, and it precisely identifies the true model in 72% of cases, whilst exploring a space of over 250 000 potential models. By testing which interactions actually occur in the target system, QMLA is a viable tool for both the exploration of fundamental physics and the characterisation and calibration of quantum devices.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2001

Sensitivity Studies for Space-Based Global Measurements of Atmospheric Carbon Dioxide

Carbon dioxide (CO2) is well known as the primary forcing agent of global warming. Although the climate forcing due to CO2 is well known, the sources and sinks of CO2 are not well understood. Currently the lack of global atmospheric CO2 observations limits our ability to diagnose the global carbon budget (e.g., finding the so-called "missing sink") and thus limits our ability to understand past climate change and predict future climate response. Space-based techniques are being developed to make high-resolution and high-precision global column CO2 measurements. One of the proposed techniques utilizes the passive remote sensing of Earth's reflected solar radiation at the weaker vibration-rotation band of CO2 in the near infrared (approx. 1.57 micron). We use a line-by-line radiative transfer model to explore the potential of this method. Results of sensitivity studies for CO2 concentration variation and geophysical conditions (i.e., atmospheric temperature, surface reflectivity, solar zenith angle, aerosol, and cirrus cloud) will be presented. We will also present sensitivity results for an O2 A-band (approx. 0.76 micron) sensor that will be needed along with CO2 to make surface pressure and cloud height measurements.

Mao, Jian-Ping↗
DOE OSTI2022

Curriculum-based Reinforcement Learning for Distribution System Critical Load Restoration

This paper focuses on the critical load restoration problem in distribution systems following major outages. To provide fast online response and optimal sequential decision-making support, a reinforcement learning (RL) based approach is proposed to optimize the restoration. Due to the complexities stemming from the large policy search space, renewable uncertainty, and nonlinearity in a complex grid control problem, directly applying RL algorithms to train a satisfactory policy requires extensive tuning to be successful. To address this challenge, this paper leverages the curriculum learning (CL) technique to design a training curriculum involving a simpler steppingstone problem that guides the RL agent to learn to solve the original hard problem in a progressive and more effective manner. We demonstrate that compared with direct learning, CL facilitates controller training to achieve better performance. To study realistic scenarios where renewable forecasts used for decision-making are in general imperfect, the experiments compare the trained RL controllers against two model predictive controllers (MPCs) using renewable forecasts with different error levels and observe how these controllers can hedge against the uncertainty. Results show that RL controllers are less susceptible to forecast errors than the baseline MPCs and can provide a more reliable restoration process.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Robustness of cement-based materials: From dosage variations to yield stress fluctuations

The idea behind the present work is to assess robustness of fresh cement-based materials (i.e. their ability to display the same properties when submitted to variations in the way they are produced) from a generic and analytical point of view. First, analytical models from literature relating components proportions and yield stress are combined into a multi-scale and physical approach and compared to our experimental yield stress measurements. In the second part, the derivation of these analytical relationships is used to assess robustness as a function of components proportions, which allows for the most sensitive variations to be mapped onto component dosage from a yield stress point of view. In the case of fluid concretes, this approach showcases the interest of working at slightly higher water content and slightly lower aggregates fraction along with benefits of viscosity modifying agents / plasticizers combination.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Genome Sequence of the Chestnut Blight Fungus Cryphonectria parasitica EP155: A Fundamental Resource for an Archetypical Invasive Plant Pathogen

Cryphonectria parasitica is the causal agent of chestnut blight, a fungal disease that almost entirely eliminated mature American chestnut from North America over a 50-year period. Here, we formally report the genome of C. parasitica EP155 using a Sanger shotgun sequencing approach. After finishing and integration with simple-sequence repeat markers, the assembly was 43.8 Mb in 26 scaffolds (L 50 = 5; N 50 = 4.0Mb). Eight chromosomes are predicted: five scaffolds have two telomeres and six scaffolds have one telomere sequence. In total, 11,609 gene models were predicted, of which 85% show similarities to other proteins. Finally, this genome resource has already increased the utility of a fundamental plant pathogen experimental system through new understanding of the fungal vegetative incompatibility system, with significant implications for enhancing mycovirus-based biological control.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Blockchain for Fault-Tolerant Grid Operations Version 2.0

This report explores the potential of distributed ledger technology (DLT) as a transformative tool to enhance fault-tolerant operations in electrical distribution systems. Leveraging DLT's core attributes, including an immutable decentralized ledger, distributed consensus mechanisms, and state replication capabilities, this study focuses on three critical use cases. A central aspect of this research centers on the utilization of a consensus-driven ledger, providing actors within the system, such as distributed resources, with access to a reliable data repository. This empowers these actors to collaborate effectively and make informed decisions, all securely recorded on the blockchain. The first use case concentrates on data configuration, utilizing mathematical criteria---particularly, the chi-squared test for gross error detection---to identify trustworthy sensors for advanced decision-making. Building upon this foundation of trust, the second use case, topology identification, accurately determines circuit breaker states, unveiling the distribution network's topology. Ultimately, the third use case leverages this trust to execute switching actions, reconfiguring feeders and restoring power to disconnected customers after fault events. The concept of trust serves as a cornerstone in this approach, marking a departure from traditional fault location, isolation, and service restoration (FLISR) methods. Additionally, the blockchain-based architecture introduces decentralization, empowering disconnected areas to make autonomous decisions, even when communication with a central control center is disrupted. The primary contributions of this report are twofold: (1) a novel approach for evaluating distribution system voltage areas while preserving data ownership and (2) the implementation of interactions between distribution network areas using the actor model. Unlike the previous sequential approach for evaluating the area connection voltages, which required a radial network topology, this study's area model reduction enables a more versatile approach. The area model reduction addresses issues of prolonged data waiting times and multiple points of failure within the previous approach. Notably, the presented evaluation for the reduced network model area connection reveals a significant increase in the differences in voltage magnitudes. Simulation and evaluation of area agents across four distinct cases elucidate the area-level interaction behavior during a fault event. Simulations demonstrate that the proposed distributed FLISR (DFLISR) approach can successfully restore service to an affected area. Varying message delays and message loss probabilities in each simulation case underscore their impacts on restoration times, ranging from 3 min and 32 s to 6 min and 19 s. In contrast, power is not restored in an area in one of our simulation cases.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS2004

Towards On-Line Services Based on a Holistic Analysis of Human Activities

Very often computer scientists view computerization of services in terms of the logistics of human-machine interaction, including establishing a contract, accessing records, and of course designing an interface. But this analysis often moves too quickly to tactical details, failing to frame the entire service in human terms, and not recognizing the mutual learning required to define and relate goals, constraints, and the personalized value of available services. In particular, on-line services that "computerize communication" can be improved by constructing an activity model of what the person is trying to do, not just filtering, comparing, and selling piece-meal services. For example, from the customer s perspective the task of an on-line travel service is not merely to establish confirmed reservations, but to have a complete travel plan, usually integrating many days of transportation, lodging, and recreation into a happy experience. The task of the travel agent is not merely "ticketing", but helping the customer understand what they want and providing services that will connect everything together in an enjoyable way.

Clancey, William J.↗
NASA NTRS2004

Distributed network scheduling

Distributed Network Scheduling is the scheduling of future communications of a network by nodes in the network. This report details software for doing this onboard spacecraft in a remote network. While prior work on distributed scheduling has been applied to remote spacecraft networks, the software reported here focuses on modeling communication activities in greater detail and including quality of service constraints. Our main results are based on a Mars network of spacecraft and include identifying a maximum opportunity of improving traverse exploration rate a factor of three; a simulation showing reduction in one-way delivery times from a rover to Earth from as much as 5 to 1.5 hours; simulated response to unexpected events averaging under an hour onboard; and ground schedule generation ranging from seconds to 50 minutes for 15 to 100 communication goals.

planning↗
NASA NTRS2008

Control Architecture for Robotic Agent Command and Sensing

Control Architecture for Robotic Agent Command and Sensing (CARACaS) is a recent product of a continuing effort to develop architectures for controlling either a single autonomous robotic vehicle or multiple cooperating but otherwise autonomous robotic vehicles. CARACaS is potentially applicable to diverse robotic systems that could include aircraft, spacecraft, ground vehicles, surface water vessels, and/or underwater vessels. CARACaS incudes an integral combination of three coupled agents: a dynamic planning engine, a behavior engine, and a perception engine. The perception and dynamic planning en - gines are also coupled with a memory in the form of a world model. CARACaS is intended to satisfy the need for two major capabilities essential for proper functioning of an autonomous robotic system: a capability for deterministic reaction to unanticipated occurrences and a capability for re-planning in the face of changing goals, conditions, or resources. The behavior engine incorporates the multi-agent control architecture, called CAMPOUT, described in An Architecture for Controlling Multiple Robots (NPO-30345), NASA Tech Briefs, Vol. 28, No. 11 (November 2004), page 65. CAMPOUT is used to develop behavior-composition and -coordination mechanisms. Real-time process algebra operators are used to compose a behavior network for any given mission scenario. These operators afford a capability for producing a formally correct kernel of behaviors that guarantee predictable performance. By use of a method based on multi-objective decision theory (MODT), recommendations from multiple behaviors are combined to form a set of control actions that represents their consensus. In this approach, all behaviors contribute simultaneously to the control of the robotic system in a cooperative rather than a competitive manner. This approach guarantees a solution that is good enough with respect to resolution of complex, possibly conflicting goals within the constraints of the mission to be accomplished by the vehicle(s).

Huntsberger, Terrance↗
NASA NTRS2010

Envisioning Cognitive Robots for Future Space Exploration

Cognitive robots in the context of space exploration are envisioned with advanced capabilities of model building, continuous planning/re-planning, self-diagnosis, as well as the ability to exhibit a level of 'understanding' of new situations. An overview of some JPL components (e.g. CASPER, CAMPOUT) and a description of the architecture CARACaS (Control Architecture for Robotic Agent Command and Sensing) that combines these in the context of a cognitive robotic system operating in a various scenarios are presented. Finally, two examples of typical scenarios of a multi-robot construction mission and a human-robot mission, involving direct collaboration with humans is given.

robot consciousness↗
DOE OSTI2024

Resilient Control of Networked Microgrids Using Vertical Federated Reinforcement Learning: Designs and Real-Time Test-Bed Validations

Improving system-level resiliency of networked microgrids against adversarial cyber-attacks is an important aspect in the current regime of increased inverter-based resources (IBRs). To achieve that, this paper contributes in designing a hierarchical control layer, in conjunction with the existing control layers, resilient to adversarial attack signals. Considering model complexities, unknown dynamical behaviors of IBRs, and privacy issues regarding data sharing in multi-party-owned microgrids, designing such a control layer is non-trivial. Here, to tackle these issues, a novel federated reinforcement learning (Fed-RL) method is proposed. To grasp the interconnected dynamics of networked microgrids, the paper develops Federated Soft Actor-Critic (FedSAC) algorithm following the vertical structure of implementing Fed-RL. Next, utilizing the OpenAI Gym interface, we built a custom set-up in GridLAB-D/HELICS co-simulation platform, named Resilient RL Co-simulation (ResRLCoSIM), to train the RL agents with IEEE 123-bus benchmark comprising 3 interconnected microgrids. Finally, the learned policies in the simulation are transferred to the real-time hardware-in-the-loop (HIL) test-bed developed using the high-fidelity Hypersim platform. Finally, experiments show that the simulator-trained RL controllers achieve desirable performance with the test-bed platform, validating the minimization of the sim-to-real gap.

24 POWER TRANSMISSION AND DISTRIBUTION↗