Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “agent based model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 505 records · Page 28
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Data-driven Mori–Zwanzig modeling of Lagrangian particle dynamics in turbulent flows

The dynamics of Lagrangian particles in turbulence play a crucial role in mixing, transport, and dispersion in complex flows. Their trajectories exhibit highly nontrivial statistical behavior, motivating the development of surrogate models that can reproduce these trajectories without incurring the high computational cost of direct numerical simulations of the full Eulerian field. This task is particularly challenging because reduced-order models typically lack access to the full set of interactions with the underlying turbulent field. Novel data-driven machine learning techniques can be powerful in capturing and reproducing complex statistics of the reduced-order/surrogate dynamics. In this work, we show how one can learn a surrogate dynamical system that is able to evolve a turbulent Lagrangian trajectory in a way that is point-wise accurate for short-time predictions (with respect to Kolmogorov time) and stable and statistically accurate at long times. This approach is based on the Mori–Zwanzig formalism, which prescribes a mathematical decomposition of the full dynamical system into resolved dynamics that depend on the current state and the past history of a reduced set of observables, and the unresolved orthogonal dynamics due to unresolved degrees of freedom of the initial state. We show how by training this reduced order model on a point-wise error metric on short time-prediction, we are able to correctly learn the dynamics of Lagrangian turbulence, such that also the long-time statistical behavior is stably recovered at test time. This opens up a range of applications, for example, for the control of active Lagrangian agents in turbulence.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Distributed and communication-efficient solutions to linear equations with special sparse structure

In this paper we report two distributed and communication-efficient algorithms based on the multi-agent system are proposed to solve a system of linear equations with the Laplacian sparse system matrix. One algorithm is based on the gradient descent method in optimization. In this algorithm, the agents only share partial information instead of all of their collective state vectors to save significant communication. The other algorithm is obtained by approximating Newton’s method for a faster convergence rate. Although it requires twice as much communication as the first one, it is still communication-efficient given the low dimension of the information shared among agents. The convergence at a linear rate is proved for both algorithms, and a comprehensive comparison of their convergence rate, communication burden, and computation costs is also performed. The proposed algorithms can be applied to various systems to solve those problems that can be modeled as a system of linear equations with a Laplacian sparse system matrix. Simulation results with the electric power system illustrate their effectiveness.

42 ENGINEERING↗
DOE OSTI2020

Deep reinforcement learning for complex evaluation of one-loop diagrams in quantum field theory

In this paper we present a technique based on deep reinforcement learning that allows for numerical analytic continuation of integrals that are often encountered in one-loop diagrams in quantum field theory. To extract certain quantities of two-point functions, such as spectral densities, mass poles or multiparticle thresholds, it is necessary to perform an analytic continuation of the correlator in question. At one-loop level in Euclidean space, this results in the necessity to deform the integration contour of the loop integral in the complex plane of the square of the loop momentum, to avoid nonanalyticities in the integration plane. Using a toy model for which an exact solution is known, we train a reinforcement learning agent to perform the required contour deformations. We report our study shows great promise for an agent to be deployed in iterative numerical approaches used to compute nonperturbative two-point functions, such as the quark propagator Dyson-Schwinger equation, or more generally, Fredholm equations of the second kind, in the complex domain.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Towards Immobilized Proton-Coupled Electron Transfer Agents for Electrochemical Carbon Capture from Air and Seawater

Electrochemical CO 2 separation has drawn attention as a promising strategy for using renewable energy to mitigate climate change. Redox-active compounds that undergo proton-coupled electron transfer (PCET) are an impetus for pH-swing-driven CO 2 capture at low energetic costs. However, multiple barriers hinder this technology from maturing, including sensitivity to oxygen and the slow kinetics of CO 2 capture. Here, we use vapor phase chemistry to construct a textile electrode comprising an immobilized PCET agent, poly(1-aminoanthraquinone) (PAAQ), and incorporate it into redox flow cells. This design contrasts with others that use dissolved PCET agents by confining proton-storage to the surface of an electrode kept separate from an aqueous, CO 2 -capturing phase. This system facilitates carbon capture from gaseous sources (a 1% CO 2 feed and air), as well as seawater, with the latter at an energetic cost of 202 kJ/mol CO2 , and we find that quinone moieties embedded within the electrode are more stable to oxygen than dissolved counterparts. Simulations using a 1D reaction-transport model show that moderate energetic costs should be possible for air capture of CO 2 with higher loadings of polymer-bound PCET moieties. The remarkable stability of this system sets the stage for producing textile-based electrodes that facilitate pH-swing-driven carbon capture in practical situations.

Ali, Fawaz↗
DOE OSTI2020

Radiological Recovery Logistics Tool - 20161

Argonne is building and testing a tool, the Radiological Recovery Logistics Tool (RRLT), that can be used during the response and recovery from a radiological or nuclear incident to effectively allocate appropriate commercial and public works equipment to mitigate, remove, and contain radiological contamination. The requirements for this tool - as well as development of the resulting software - is overseen by a steering committee of stakeholders from DHS's National Urban Security Technology Laboratory (NUSTL), the Federal Emergency Management Agency (FEMA), and the Environmental Protection Agency (EPA). One essential requirement is for RRLT to support the efficient and appropriate allocation of resources for a radiological response. Subsequent discussions between ANL and stakeholders have solidified the nature of this support to include identification of the types of resources to be allocated. The study reported in this paper has both factored fundamental concepts and connections out of this identification process and created a Knowledge Base detailing support goals, response scenarios, and efficacy information on dozens of equipment types. In short, RRLT will dynamically apply these findings to situational conditions surrounding contamination incidents. RRLT's Domain, the model of elements, ideas and relationships with which the tool will work, draws concepts from technical reports and stakeholder vocabularies to connect response goals and scenarios to types of equipment that offer utility towards those goals in those scenarios. RRLT's Knowledge Base will contain details on dozens of equipment types and facilitate the operator's discovery and consumption of these details most pertinent to a dynamically selected subset of goals. The core of its Domain Model is based on a report authored by this team. This report [1] contains a comprehensive list of proposed equipment to accomplish various missions or scenarios that might arise after a large-scale radiological contamination incident in an urban environment or critical infrastructure. The report divides potential response and recovery efforts into five support goals: Survey and monitoring of the contaminated area; Mitigation of received dose to first responders: Decontamination (gross and final) of buildings, vehicles, roadways, parks, and other surfaces: Waste management of solid waste generated during recovery operations: and Containment of wastewater and other waste generated during the response and recovery phases. RRLT's development is driven by use cases. A use case is an intention with which a user approaches the software. Use cases are grouped into delivery increments to schedule development, testing, and presentation to stakeholders. This model partitions the system into seven increments: User Arrival and Authentication, Search and Navigation, Equipment Recommendation, Plan Management, Content Management, and Expanded Access. Once a user 15 authenticated, RRLT will present the user with a dashboard that allows them to explore or search RRLT's content. The dashboard will also include a 'Plan' panel for collecting decisions and relevant observations about an incident at hand to facilitate development of an equipment list. RRLT will offer three general modes of access to items in the knowledge base: - Keyword search for direct discovery of items, - Navigation along predetermined paths from recovery goal towards equipment types, and - Interactive guidance towards equipment types by an autonomous software agent: the Equipment Recommendation Wizard. This presentation will detail progress in the development of the RRLT and also discuss opportunities for those interested in providing feedback on its content and functionality. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2021

Dynamic Ride-Matching for Large-Scale Transportation Systems

Efficient dynamic ride-matching (DRM) in large-scale transportation systems is a key driver in transport simulations to yield answers to challenging problems. Although the DRM problem is simple to solve, it quickly becomes a computationally challenging problem in large-scale transportation system simulations. Therefore, this study thoroughly examines the DRM problem dynamics and proposes an optimization-based solution framework to solve the problem efficiently. To benefit from parallel computing and reduce computational times, the problem’s network is divided into clusters utilizing a commonly used unsupervised machine learning algorithm along with a linear programming model. Then, these sub-problems are solved using another linear program to finalize the ride-matching. At the clustering level, the framework allows users adjusting cluster sizes to balance the trade-off between the computational time savings and the solution quality deviation. A case study in the Chicago Metropolitan Area, U.S., illustrates that the framework can reduce the average computational time by 58% at the cost of increasing the average pick up time by 26% compared with a system optimum, that is, non-clustered, approach. Another case study in a relatively small city, Bloomington, Illinois, U.S., shows that the framework provides quite similar results to the system-optimum approach in approximately 62% less computational time.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2022

Discovering mechanisms for materials microstructure optimization via reinforcement learning of a generative model

Abstract The design of materials structure for optimizing functional properties and potentially, the discovery of novel behaviors is a keystone problem in materials science. In many cases microstructural models underpinning materials functionality are available and well understood. However, optimization of average properties via microstructural engineering often leads to combinatorically intractable problems. Here, we explore the use of the reinforcement learning (RL) for microstructure optimization targeting the discovery of the physical mechanisms behind enhanced functionalities. We illustrate that RL can provide insights into the mechanisms driving properties of interest in a 2D discrete Landau ferroelectrics simulator. Intriguingly, we find that non-trivial phenomena emerge if the rewards are assigned to favor physically impossible tasks, which we illustrate through rewarding RL agents to rotate polarization vectors to energetically unfavorable positions. We further find that strategies to induce polarization curl can be non-intuitive, based on analysis of learned agent policies. This study suggests that RL is a promising machine learning method for material design optimization tasks, and for better understanding the dynamics of microstructural simulations.

36 MATERIALS SCIENCE↗
DOE OSTI2023

AdenPredictor: accurate prediction of the adenylation domain specificity of nonribosomal peptide biosynthetic gene clusters in microbial genomes

Microbial natural products represent a major source of bioactive compounds for drug discovery. Among these molecules, nonribosomal peptides (NRPs) represent a diverse class that include antibiotics, immunosuppressants, anticancer agents, toxins, siderophores, pigments, and cytostatics. The discovery of novel NRPs remains a laborious process because many NRPs consist of nonstandard amino acids that are assembled by nonribosomal peptide synthetases (NRPSs). Adenylation domains (A-domains) in NRPSs are responsible for selection and activation of monomers appearing in NRPs. During the past decade, several support vector machine-based algorithms have been developed for predicting the specificity of the monomers present in NRPs. These algorithms utilize physiochemical features of the amino acids present in the A-domains of NRPSs. In this article, we benchmarked the performance of various machine learning algorithms and features for predicting specificities of NRPSs and we showed that the extra trees model paired with one-hot encoding features outperforms the existing approaches. Moreover, we show that unsupervised clustering of 453560 A-domains reveals many clusters that correspond to potentially novel amino acids. While it is challenging to predict the chemical structure of these amino acids, we developed novel techniques to predict their various properties, including polarity, hydrophobicity, charge, and presence of aromatic rings, carboxyl, and hydroxyl groups.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

GOOML - Finding Optimization Opportunities for Geothermal Operations: Preprint

Geothermal Operational Optimization with Machine Learning (GOOML) is a transferable and extensible component-based geothermal asset modeling framework that considers complex steamfield relationships and identifies optimization prospects using a data-driven approach. We have used this framework to develop digital twins that provide steamfield operators with an operational environment to analyze and understand historical and forecasted power production, explore new steamfield configuration possibilities, and seek optimal asset management for real world applications. The GOOML modeling software is built on a generic component-based systems framework that allows for both historical and forecast analysis. A GOOML model can perform historical data-assimilation using first-principal thermodynamics to create a meaningful data model. Historical production data can then be coupled with a forecast framework to train machine-learning models of steamfield components to predict future outputs. This modeling environment enables digital exploration of steamfield design configurations and operational scenarios. GOOML digital twins have been developed for steamfields in New Zealand and the United States representing differing power generation and field conditions. These digital twins have been validated by comparing hindcast predictions against historical production data. Reinforcement learning experiments were conducted to demonstrate the ability to programmatically explore the operations space using machine learning agents. Our initial results are compelling; two to five percent increases in annual energy production were demonstrated by the GOOML models with no additional infrastructure build required. GOOML offers a new approach to geothermal operations by applying state-of-the-art machine learning algorithms, comprehensive data analytics, and interaction with digital twins. Through application of these tools, operators will realize greater availability and higher net generation which will increase the cost effectiveness of geothermal energy projects.

access↗
DOE OSTI2025

Microstructural evolution and hardness changes in ion irradiated nickel-based Haynes 282 superalloy

This study investigated the crucial aspects of thermal and irradiation stability in precipitation hardened Haynes 282 Ni-based superalloy. The Haynes 282 Ni-based alloy was irradiated by 8 MeV Ni 3+ ions to assess its resistance to phase instabilities and mechanical property alterations. The mid-range doses (at a depth of ∼1 µm) were 1 and 10 displacements per atom (dpa) at temperatures of 600°C and 750°C. Nanoindentation tests provided insights into bulk equivalent hardness of the irradiated and pristine regions, while scanning transmission electron microscopy (STEM) and energy dispersive X-ray spectroscopy (EDS) were used to examine the microstructural evolution of irradiation-induced defects and γ′ -Ni 3 (Al, Ti) precipitates and defect structure under irradiation. These precipitates, with an average diameter of 29 nm and a number density of 5 x 10 21 /m 3 , acted as robust dispersion strengthening agents with high radiation point defect sink strength. Remarkably, irradiation did not significantly alter the size or number density of the γ′ precipitates, indicating exceptional thermal stability and radiation resistance of these precipitates at the examined conditions. Radiation-induced dislocation loops were observed at 600°C, albeit without a substantial impact on mechanical properties due to the dominance of γ′ precipitates on the overall alloy strength. The superior stability of γ′ precipitates observed in this study contrasts with several previous research findings on Ni-based alloys that reported poor precipitate stability. Plausible reasons for this difference are discussed. Moreover, this work explicitly outlines a physically grounded approach to ensure accurate microstructure-hardness correlations and clarifies the hardening model by addressing common misapplications of superposition in prior studies.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Efficient Reinforcement Learning for Real-Time Hardware-Based Energy System Experiments: Preprint

In the context of urgent climate challenges and the pressing need for rapid technology development, Reinforcement Learning (RL) stands as a compelling data-driven method for controlling real-world physical systems. However, RL implementation often entails time-consuming and computationally intensive data collection and training processes, rendering them inefficient for real-time applications that lack non-real-time models. To address these limitations, real-time emulation techniques have emerged as valuable tools for the lab-scale rapid prototyping of intricate energy systems. While emulated systems offer a bridge between simulation and reality, they too face constraints, hindering comprehensive characterization, testing, and development. In this research, we construct a surrogate model using limited data from simulated systems, enabling an efficient and effective training process for a Double Deep Q-Network (DDQN) agent for future deployment. Our approach is illustrated through a hydropower application, demonstrating the practical impact of our approach on climate-related technology development.

deep Q-learning↗
DOE OSTI2024

SEAS Communication Engine: An Extensible, Flexible Wrapper for Co-Simulation Agents

When modeling and analyzing the power grid and other large scale systems, researchers often express scenarios as optimization problems and feed them into advanced software solvers. In order to allow multiple solvers to communicate with each other and share data from different domains, the National Renewable Energy Laboratory (NREL) and associated Department of Energy (DOE) labs have developed a software framework called the Hierarchical Engine for Large-scale Infrastructure Co-Simulation (HELICS). HELICS allows cosimulation via a collection of client libraries for different languages that can be called from the appropriate optimization software. However, these client libraries do not provide a higher level of abstraction beyond reading and writing data off of the shared HELICS bus. In this paper, we describe a new software library called the SEAS Communication Engine that exposes a higher-level API for running cosimulation problems. The SEAS Engine provides a class-based abstraction on top of the Python HELICS client, in order to allow users to implement their domain-specific cosimulations without needing to interact with core HELICS primitives. This will make adoption of HELICS and cosimulation in general easier, by exposing a simpler API. In the second part of the paper, we validate our library on a collection of different simulation examples, including the canonical IEEE 13 Bus Feeder. Lastly, we demonstrate using the SEAS Engine to directly call domain-specific code written in the Julia programming language. Our hope is that this will serve as a template for easily calling software in different programming languages via the SEAS Engine, thereby avoiding code duplication and complexity.

co-simulation↗
DOE OSTI2026

Robust Rapid Cellular Metabolite Sensing Using Benchtop NMR and SABRE-Hyperpolarized [1- 13 C]Pyruvate

Hyperpolarized NMR has emerged as a powerful analytical technique to significantly enhance targeted NMR signals, improving the sensitivity for investigations of unique chemical and biological dynamics. Here, we demonstrate the use of a hyperpolarization strategy based on Signal Amplification By Reversible Exchange (SABRE) to generate highly reproducible doses of a hyperpolarized [1- 13 C]pyruvate probe for benchtop characterization of yeast metabolism. This method allows rapid, scalable, and benchtop preparation of biocompatible hyperpolarized solutions suitable for live-cell experiments. We show that this production can be dove-tailed into a modular, compact workflow to characterize real-time metabolism in cell cultures, using Saccharomyces cerevisiae (Baker’s yeast) as a model organism. With high temporal resolution, we show that this method can resolve the conversion of hyperpolarized [1- 13 C]pyruvate into oxidative decarboxylation products CO 2 and bicarbonate. This conversion exhibits sustained and detectable metabolic activity for over 300 s after introduction of the agent to the cells. We model the metabolite kinetics to show decarboxylation activity and derive estimates of the pH over time from the CO 2 and bicarbonate (carbonic acid buffer system) equilibrium to probe changes in the cellular environment during active metabolism. These results highlight the utility of benchtop SABRE-hyperpolarized [1- 13 C]pyruvate as a scalable, specific probe for metabolic phenotyping of living cells using compact, low-cost instrumentation well-suited for future high-throughput applications across microbial engineering, drug response profiling, and dynamic metabolic screening.

fungi↗
DOE OSTI2024

Combatting melioidosis with chemical synthetic lethality

Burkholderia thailandensishas emerged as a nonpathogenic surrogate forBurkholderia pseudomallei, the causative agent of melioidosis, and an important Gram-negative model bacterium for studying the biosynthesis and regulation of secondary metabolism. We recently reported that subinhibitory concentrations of trimethoprim induce vast changes in both the primary and secondary metabolome ofB. thailandensis. In the current work, we show that the folate biosynthetic enzyme FolE2 is permissive under standard growth conditions but essential forB. thailandensisin the presence of subinhibitory doses of trimethoprim. Reasoning that FolE2 may serve as an attractive drug target, we screened for and identified ten inhibitors, including dehydrocostus lactone (DHL), parthenolide, and β-lapachone, all of which are innocuous individually but form a chemical-synthetic lethal combination with subinhibitory doses of trimethoprim. We show that DHL is a mechanism-based inhibitor of FolE2 and capture the structure of the covalently inhibited enzyme using X-ray crystallography. In vitro, the combination of subinhibitory trimethoprim and DHL is more potent than Bactrim, the current standard of care against melioidosis. Moreover, unlike Bactrim, this combination does not affect the growth of most commensal and beneficial gut bacteria tested, thereby providing a degree of specificity againstB. pseudomallei. Our work provides a path for identifying antimicrobial drug targets and for utilizing binary combinations of molecules that form a toxic cocktail based on metabolic idiosyncrasies of specific pathogens.

Science & Technology - Other Topics↗
DOE OSTI2024

Linking Threat Agents to Targeted Organizations: A Pipeline for Enhanced Cybersecurity Risk Metrics

In this study, we present a methodology leveraging Large Language Models (LLMs) to transform Cybersecurity Threat Intelligence (CTI) narratives into actionable insights for individual organizations. Our approach automates the extraction of machine-readable adversary SKRAM (Skills, Knowledge, Resources, Authorities, and Motivation) attributes from open-source reports, extending LLM utility beyond typical interactions. This innovation enables precise, automated assessments of cybersecurity risks posed by various adversaries. Using a chain-of-thought and multi-shot prompting strategy, our methodology advances the automation of cybersecurity feature extraction for new machine-learning models that predict the risk of adversary targeting. This approach is refined using a substantial dataset of over 150 analyst-validated threat reports and synthetic organizational data from 900 companies. Here, by bootstrapping the training data with a rule-based heuristic over synthetic data, we have developed a high-accuracy machine-learning model that allows entities to dynamically prioritize threats and defensive actions.

Cyber Threat Intelligence↗
DOE OSTI2024

Extreme Risk Mitigation in Reinforcement Learning using Extreme Value Theory

Risk-sensitive reinforcement learning (RL) has garnered significant attention in recent years due to the growing interest in deploying RL agents in real-world scenarios. A critical aspect of risk awareness involves modelling highly rare risk events (rewards) that could potentially lead to catastrophic outcomes. These infrequent occurrences present a formidable challenge for data-driven methods aiming to capture such risky events accurately. While risk-aware RL techniques do exist, they suffer from high variance estimation due to the inherent data scarcity. Our work proposes to enhance the resilience of RL agents when faced with very rare and risky events by focusing on refining the predictions of the extreme values predicted by the state-action value distribution. To achieve this, we formulate the extreme values of the state-action value function distribution as parameterized distributions, drawing inspiration from the principles of extreme value theory (EVT). We propose an extreme value theory based actor-critic approach, namely, Extreme Valued Actor-Critic (EVAC) which effectively addresses the issue of infrequent occurrence by leveraging EVT-based parameterization. Importantly, we theoretically demonstrate the advantages of employing these parameterized distributions in contrast to other risk-averse algorithms. Our evaluations show that the proposed method outperforms other risk averse RL algorithms on a diverse range of benchmark tasks, each encompassing distinct risk scenarios.

Wang, Yu↗
DOE OSTI2022

Quantum model learning agent: characterisation of quantum systems through machine learning

Accurate models of real quantum systems are important for investigating their behaviour, yet are difficult to distil empirically. Here, we report an algorithm—the quantum model learning agent (QMLA)—to reverse engineer Hamiltonian descriptions of a target system. We test the performance of QMLA on a number of simulated experiments, demonstrating several mechanisms for the design of candidate Hamiltonian models and simultaneously entertaining numerous hypotheses about the nature of the physical interactions governing the system under study. QMLA is shown to identify the true model in the majority of instances, when provided with limited a priori information, and control of the experimental setup. Our protocol can explore Ising, Heisenberg and Hubbard families of models in parallel, reliably identifying the family which best describes the system dynamics. We demonstrate QMLA operating on large model spaces by incorporating a genetic algorithm to formulate new hypothetical models. The selection of models whose features propagate to the next generation is based upon an objective function inspired by the Elo rating scheme, typically used to rate competitors in games such as chess and football. In all instances, our protocol finds models that exhibit F 1 score ≥ 0.88 when compared with the true model, and it precisely identifies the true model in 72% of cases, whilst exploring a space of over 250 000 potential models. By testing which interactions actually occur in the target system, QMLA is a viable tool for both the exploration of fundamental physics and the characterisation and calibration of quantum devices.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Curriculum-based Reinforcement Learning for Distribution System Critical Load Restoration

This paper focuses on the critical load restoration problem in distribution systems following major outages. To provide fast online response and optimal sequential decision-making support, a reinforcement learning (RL) based approach is proposed to optimize the restoration. Due to the complexities stemming from the large policy search space, renewable uncertainty, and nonlinearity in a complex grid control problem, directly applying RL algorithms to train a satisfactory policy requires extensive tuning to be successful. To address this challenge, this paper leverages the curriculum learning (CL) technique to design a training curriculum involving a simpler steppingstone problem that guides the RL agent to learn to solve the original hard problem in a progressive and more effective manner. We demonstrate that compared with direct learning, CL facilitates controller training to achieve better performance. To study realistic scenarios where renewable forecasts used for decision-making are in general imperfect, the experiments compare the trained RL controllers against two model predictive controllers (MPCs) using renewable forecasts with different error levels and observe how these controllers can hedge against the uncertainty. Results show that RL controllers are less susceptible to forecast errors than the baseline MPCs and can provide a more reliable restoration process.

24 POWER TRANSMISSION AND DISTRIBUTION↗