Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “agent based”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Mic-hackathon 2024: hackathon on machine learning for electron and scanning probe microscopy

Microscopy is one of the primary sources of information on materials structure and functionality at the nanometer and atomic scales. The data generated through microscopy is often contained in well-structured datasets, enriched with extensive metadata and sample histories, although not always with the same level of detail or storage format. The broad incorporation of data management plans by major funding agencies ensures the preservation and accessibility of this data. However, deriving insights from these rich datasets remains challenging due to the lack of established code ecosystems, standardized benchmarks, and integration strategies. Correspondingly, the efficiency of data usage is very low, and time expenditures at the analysis stage are enormous. In addition to post-acquisition data analysis, the emergence of application programming interfaces by major microscope manufacturers now creates opportunities for real-time ML-based data analytics to enable automated decision making, and particularly ML-agent controlled real-time microscope operation. Despite these opportunities, there is a significant gap in integrating the ML community with the broader microscopy community, limiting the value that these methods bring to physics and materials discovery and materials optimization. Hackathons address these challenges by fostering collaboration between ML experts and microscopy professionals, encouraging the development of innovative solutions that leverage ML for microscopy and preparing the workforce of the future both for microscopy-intensive domains areas, instrument manufacturers, and ML scientists interested in real world applications for fundamental research, materials optimization, and manufacturing. The hackathon generated benchmark datasets and digital twins of microscopes that further contribute to the development of the field and establish data analysis ecosystems. All the codes can be found at GitHub(https://github.com/KalininGroup/Mic-hackathon-2024-codes-publication/tree/1.0.0.1) and Zenodo (https://zenodo.org/records/15579940).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Deep reinforcement learning with online data augmentation to improve sample efficiency for intelligent HVAC control

Deep Reinforcement Learning (DRL) has started showing success in real-world applications such as building energy optimization. Much of the research in this space utilized simulated environments to train RL-agent in an offline mode. Very few research have used DRL-based control in real-world systems due to two main reasons: 1) sample efficiency challenge---DRL approaches need to perform a lot of interactions with the environment to collect sufficient experiences to learn from, which is difficult in real systems, and 2) comfort or safety related constraints---user's comfort must never or at least rarely be violated. In this work, we propose a novel deep Reinforcement Learning framework with online Data Augmentation (RLDA) to address the sample efficiency challenge of real-world RL. We used a time series Generative Adversarial Network (TimeGAN) architecture as a data generator. We further evaluated the proposed RLDA framework using a case study of an intelligent HVAC control. With a ≈28% improvement in the sample efficiency, RLDA framework lays the way towards increased adoption of DRL-based intelligent control in real-world building energy management systems.

Kurte, Kuldeep↗
DOE OSTI2020

Optimal Renewable Resource Allocation and Load Scheduling of Resilient Communities

This paper presents a methodology for enhancing community resilience through optimal renewable resource allocation and load scheduling in order to minimize unserved load and thermal discomfort. The proposed control architecture distributes the computational effort and is easier to be scaled up than traditional centralized control. The decentralized control architecture consists of two layers: The community operator layer (COL) allocates the limited amount of renewable energy resource according to the power flexibility of each building. The building agent layer (BAL) addresses the optimal load scheduling problem for each building with the allowable load determined by the COL. Both layers are formulated as a model predictive control (MPC) based optimization. Simulation scenarios are designed to compare different combinations of building weighting methods and objective functions to provide guidance for real-world deployment by community and microgrid operators. The results indicate that the impact of power flexibility is more prominent than the weighting factor to the resource allocation process. Allocation based purely on occupancy status could lead to an increase of PV curtailment. Further, it is necessary for the building agent to have multi-objective optimization to minimize unserved load ratio and maximize comfort simultaneously.

model predictive control↗
DOE OSTI2022

Epoxy as Filler or Matrix for Polymer Composites

Epoxy is a widely used polymer because of its ease of processing, high adhesiveness, and high chemical resistance. Epoxy-based composites are commonly used in aerospace, automotive, and marine applications. The epoxy type, function, curing agent, and curing process are discussed in this chapter. Epoxy is used as either a filler or polymer matrix in composite applications. As a filler, the epoxy modification on the fiber is discussed. As a polymer matrix, the epoxy is reinforced by natural and synthetic fibers. The manufacturing process and the fabricated epoxy-based composites’ performance (e.g., mechanical and thermal properties) are investigated. The advantages and disadvantages of epoxy’s function are discussed and summarized. Epoxy modification is an effective approach to improve the composites’ performance.

Zhao, Xianhui↗
DOE OSTI2026

Learning to Trigger: Reinforcement Learning at the Large Hadron Collider

High-throughput scientific facilities such as the Large Hadron Collider depend on real-time event filtering (\textit{triggering}) under tight constraints on bandwidth, latency, and storage. In practice, trigger menus are largely static and hand-tuned and can become suboptimal as detector conditions, pileup, and background composition drift over time. We cast online threshold tuning as a sequential decision-making problem: a reinforcement learning agent ingests streaming summaries of recent rates and signal-sensitive features and updates trigger thresholds to maximize signal efficiency while tracking a target background rate within a tolerance band. We adapt Group-Filtered Policy Optimization (GFPO) to streaming control and introduce two variants (GFPO-F, GFPO-FR) that enforce background rate feasibility during training. On a benchmark that emulates realistic collider operation, we study two representative triggers: a total transverse energy ($H_{T}$) trigger sensitive to pileup variation, and an anomaly-detection (AD) trigger based on reconstruction loss for rare or non-standard signatures. On Monte Carlo streams, our agent increases the fraction of in-tolerance time intervals by 48% ($H_T$) and 28% (AD), with a cumulative gain of up to 2% in signal efficiency on those in-tolerance intervals. Transferring from simulation to \emph{real} collision data (CMS Run 283408), the same agent, without fine-tuning, achieves a 56% ($H_T$) and 28% (AD) in-tolerance improvement over baselines, with further signal-efficiency gain on both triggers. To our knowledge, this is the \emph{first} demonstration of RL-based trigger control on real Large Hadron Collider collision data. Code is available at https://github.com/Zixind/GFPO_LHC (see repo for details).

Ding, Zixin [Chicago U.]↗
DOE OSTI2025

Explainable and Differentiable Reinforcement Learning for Multi-objective Optimization in Particle Accelerators

Operating particle accelerators involves optimizing multiple goals simultaneously, which can be challenging due to trade-offs among objectives. While evolutionary algorithms like the genetic algorithm (GA) have been used for various Multi-Objective Optimization (MOO) tasks, they are not inherently suited for complex control problems. This talk highlights two variations of Reinforcement Learning (RL) for concurrently optimizing heat load and trip rates at the Continuous Electron Beam Accelerator Facility (CEBAF). The problem involves strict constraints on individual states, actions, and overall energy requirements of the beam. First, this talk highlights how differentiability can be harnessed through a Deep Differentiable Reinforcement Learning (DDRL) approach to address MOO issues within particle accelerators. We examine the DDRL method alongside Model Free Reinforcement Learning (MFRL), GA, and Bayesian Optimization (BO). The performance of these methods is assessed by generating a Pareto-front for two objectives. Our findings indicate that DDRL excels in handling high-dimensional problems more effectively than MFRL, BO, and GA. Next, we will show integration of explainable physics-based constraints into RL algorithms to enhance trans- parency and trust in decision-making processes by enabling users to verify that agents adhere to established physical principles. This surrogate function can be modeled using neural networks or sparse dictionary mod- els. By examining the mathematical form of the learned constraint function, we are able to confirm the agent has learned to use the established physics of each environment provided but the surrogate model. In addi- tion, we find that the introduction of a mathematical functional dictionary based surrogate model enables our reinforcement learning algorithms to reliably converge for difficult high-dimensional accelerator controls environments.

Rajput, Kishansingh [Thomas Jefferson National Acc↗
DOE OSTI2023

Direct synthesis of ordered mesoporous materials from thermoplastic elastomers

Abstract The ability to manufacture ordered mesoporous materials using low-cost precursors and scalable processes is essential for unlocking their enormous potential to enable advancement in nanotechnology. While templating-based methods play a central role in the development of mesoporous materials, several limitations exist in conventional system design, including cost, volatile solvent consumption, and attainable pore sizes from commercial templating agents. This work pioneers a new manufacturing platform for producing ordered mesoporous materials through direct pyrolysis of crosslinked thermoplastic elastomer-based block copolymers. Specifically, olefinic majority phases are selectively crosslinked through sulfonation reactions and subsequently converted to carbon, while the minority block can be decomposed to form ordered mesopores. We demonstrate that this process can be extended to different polymer precursors for synthesizing mesoporous polymer, carbon, and silica. Furthermore, the obtained carbons possess large mesopores, sulfur-doped carbon framework, with tailorable pore textures upon varying the precursor identities.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Multi-agent AI collaboration for digital twin development and assessment

Developing a digital twin (DT) model involves different steps that encompass formulating requirements, model development, implementation, and assessment with respect to real applications. Human expertise is required to coordinate and implement different steps in the DT development and assessment process. However, certain parts of this process can be automated using artificial intelligence (AI) agents for efficient workflow development. In this work, we test and analyze a multiagent AI collaboration with humans in the loop to automate different elements of the DT development and assessment process. To implement the workflow for multiagent AI DT development and assessment, we use Autogen, a multiagent framework developed by Microsoft. Autogen offers a modular and flexible framework for configuring and designing task-specific multiagent workflows. In this framework, large language models (LLMs) form the core intelligence of the AI agents where the quality and performance of the automated element is governed by the inherent capabilities and knowledge base of the LLM. We use retrieval augmented generation to supplement the LLM with relevant domain-specific information for DT requirement formulation. We illustrate this multiagent workflow using a case study on a thermal energy storage system, focusing on how AI agents can collaborate with humans to expedite and optimize different elements of DT development and assessment process.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2023

Inverse reinforcement learning control for building energy management

Reinforcement learning (RL) based control is widely considered a promising approach in building automation and control as it has demonstrated the potential to deal with complex objectives in adjacent domains like robotics, autonomous vehicles, gaming applications, and advertisement recommendations. When applied to any environment, model-free RL learns to improve its control performance over time without requiring a control model, by receiving and then analyzing feedback from the building environment after each control action. Operational objectives are becoming increasingly complex through the simultaneous consideration of thermal comfort, carbon emissions, grid services, and indoor air quality. In this context, conventional rule-based control approaches are proving sub-optimal, mostly heuristic, and inadequate. The model-free and self learning nature of RL appears promising and attractive as it may address the scalability issues associated with advanced control approaches. However, it suffers from long training times and unstable control behavior during the early stages of its learning process, which makes it unsuitable to be applied directly to buildings. This paper addresses these issues using an inverse reinforcement learning approach (IRL), a technique utilized to learn the objective of a controller agent which is considered an expert in its respective domain. Here, we consider a rule-based control as the expert demonstrator. IRL is different from a direct imitation (i.e., direct mapping of states to actions) of control actions as it tries to find the underlying intent of an expert's policy, providing the controller with a better-generalized policy for unseen states or environments with slightly different dynamics. This approach propels the RL controller's policy to levels similar to or better than that of a rule-based policy before it starts learning by interacting with the building. This makes RL for building energy management applications more practical as it prevents the erratic and exploratory behavior in the initial training period, simultaneously speeding up the learning process when compared to applying an untrained RL agent directly to a building environment.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2025

Towards philosophical reasoning with agentic LLMs: Socratic method for scientific assistance

As large language models (LLMs) become central tools in science, improving their reasoning capabilities is critical for meaningful and trustworthy applications. We introduce a Socratic agent for scientific reasoning, implemented through a structured system prompt that guides LLMs via classical principles of inquiry. Unlike typical prompt engineering or retrieval-based methods, our approach leverages definition, analogy, hypothesis elimination, and other Socratic techniques to generate more coherent, critical, and domain-aware responses. We evaluate the agent across diverse scientific domains and benchmark it on the abstraction and reasoning corpus challenge dataset, achieving 97.15% under a fixed prompting protocol and without fine-tuning or external tools. Expert evaluation shows improved reasoning depth, clarity, and adaptability over conventional LLM outputs, suggesting that structured prompting rooted in philosophical reasoning can improve the scientific utility of language models.

LLM reasoning↗
DOE OSTI2022

Physics Informed Reinforcement Learning for Power Grid Control using Augmented Random Search

Wide adoption of deep reinforcement learning need to overcome several challenges in energy system domain, including scalability, learning from limited samples, and high-dimensional continuous state and action spaces. In this paper, we integrated physics-based information from the normal generator operation state formula in the reinforcement learning agent's neural network loss function, and applied an augmented random search agent to optimize the generator control under dynamic contingency. Simulation results demonstrated the reliability performance improvements in training speed, reward convergence, sampling efficiency, scalability, and transferability.

physics informed ML, Physics Informed Neural Netwo↗
DOE OSTI2022

Likeness: a toolkit for connecting the social fabric of place to human dynamics

The ability to produce richly-attributed synthetic populations is key for understanding human dynamics, responding to emergencies, and preparing for future events, all while protecting individual privacy. The Likeness toolkit accomplishes these goals with a suite of Python packages: pymedm/pymedm\_legacy, livelike, and actlike. This production process is initialized in pymedm (or pymedm\_legacy) that utilizes census microdata records as the foundation on which disaggregated spatial allocation matrices are built. The next step, performed by livelike, is the generation of a fully autonomous agent population attributed with hundreds of demographic census variables. The agent population synthesized in livelike is then attributed with residential coordinates in actlike based on block assignment and, finally, allocated to an optimal daytime activity location via the street network. We present a case study in Knox County, Tennessee, synthesizing 30 populations of public K–12 school students \& teachers and allocating them to schools. Validation of our results shows they are highly promising by replicating reported school enrollment and teacher capacity with a high degree of fidelity.

Tuccillo, Joe↗
DOE OSTI2021

Carbide Nanoparticle Dispersion Techniques for Metal Powder Metallurgy

Nanoparticles (NP) embedded into a matrix material have been shown to improve mechanical properties such as strength, hardness, and wear-resistance. However, the tendency of NPs to agglomerate in the powder mixing process is a major concern. This study investigates five different mechanochemical processing (MCP) routes to mitigate agglomeration to achieve a uniform dispersion of ZrC NPs in an Fe-based metal matrix composite. Our results suggest that MCP with only process controlling agents is ineffective in avoiding aggregation of these NPs. Instead, the uniformity of the carbide NP dispersion is achieved by pre-dispersing the NPs under ultrasonication using suitable surfactants followed by mechanically mixing of the NPs with iron powders in an alcohol solvent which is then dried. High-energy MCP is then used to embed the NPs within the powders. These collective steps resulted in a uniform dispersion of ZrC in the sintered (consolidated) Fe sample.

08 HYDROGEN↗
DOE OSTI2021

Distributed Inference with Sparse and Quantized Communication

Here, we consider the problem of distributed inference where agents in a network observe a stream of private signals generated by an unknown state, and aim to uniquely identify this state from a finite set of hypotheses. We focus on scenarios where communication between agents is costly, and takes place over channels with finite bandwidth. To reduce the frequency of communication, we develop a novel event-triggered distributed learning rule that is based on the principle of diffusing low beliefs on each false hypothesis. Building on this principle, we design a trigger condition under which an agent broadcasts only those components of its belief vector that have adequate innovation, to only those neighbors that require such information. We prove that our rule guarantees convergence to the true state exponentially fast almost surely despite sparse communication, and that it has the potential to significantly reduce information flow from uninformative agents to informative agents. Next, to deal with finite-precision communication channels, we propose a distributed learning rule that leverages the idea of adaptive quantization. We show that by sequentially refining the range of the quantizers, every agent can learn the truth exponentially fast almost surely, while using just 1 bit to encode its belief on each hypothesis. For both our proposed algorithms, we rigorously characterize the trade-offs between communication-efficiency and the learning rate.

42 ENGINEERING↗
DOE OSTI2025

A Rapid, Sustainable, One‐step Mechanochemical Strategy for Synthesizing Gold Nanoparticle‐Doped Covalent Organic Frameworks

Doping gold nanoparticles within covalent organic frameworks (AuNPs@COFs) has garnered enormous momentum due to their unique properties and broad applications. Nevertheless, prevailing multi-step synthesis is plagued with low time efficiency, eco-unfriendliness, and tedious protocols. Herein, we introduce a rapid, sustainable, scalable, one-step mechanochemical strategy for synthesizing up to four AuNPs-doped COFs via steel ball milling within an hour under ambient conditions. This approach overcomes the synthetic barriers of conventional multi-step solution-based methods, such as extended reaction times (5 days), milligram scale, the use of toxic solvents, elevated temperatures, and reliance on external reducing agents. One exemplary AuNPs@COF (AuNPs@DMTP-TPB) exhibits high crystallinity, porosity, small AuNP size, and uniform dispersion (5.4±0.6 nm), surpassing its counterpart synthesized via multi-step solution-based methods (6.4±1.1 nm). Notably, the gram-scale synthesis of AuNPs@DMTP-TPB can be successfully achieved. Control experiments suggest that the in situ formation of AuNPs is attributed to the galvanic reduction of gold precursor by stainless steel apparatus. As a proof-of-concept catalytic application, AuNPs@DMTP-TPB demonstrates remarkable catalytic activity and recyclability for the aqueous reduction of 4-nitrophenol under ambient conditions. This study provides an environmentally benign and fast pathway to synthesize AuNPs@COFs via mechanochemistry for the first time, opening tremendous possibilities for heterogeneous catalysis and beyond.

Nailwal, Yogendra↗
DOE OSTI2023

Coded aperture and Compton imaging for the development of 225 Ac‐based radiopharmaceuticals

Abstract Background Targeted alpha‐particle therapy (TAT) has great promise as a cancer treatment. Arguably the most promising TAT radionuclide that has been proposed is 225 Ac. The development of 225 Ac‐based radiopharmaceuticals has been hampered due to the lack of effective means to study the daughter redistribution of these agents in small animals at the preclinical stage. Purpose The ability to directly image the daughters, namely 221 Fr and 213 Bi, via their gamma‐ray emissions would be a boon for preclinical studies. That said, conventional medical imaging modalities, including single photon emission computed tomography (SPECT) based on nonmultiplexed collimation, cannot be employed due to sensitivity limitations. Methods As an alternative, we propose the use of both coded aperture and Compton imaging with the former modality suited to the 218‐keV gamma‐ray emission of 221 Fr and the latter suited to the 440‐keV gamma‐ray emission of 213 Bi. Results This work includes coded aperture images of 221 Fr and Compton images of 213 Bi in tumor‐bearing mice injected with 225 Ac‐based radiopharmaceuticals. Conclusions These results are the first demonstration of visualizing and quantifying the 225 Ac daughters in small animals through the application of coded aperture and Compton imaging.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2024

Graph-Based Modeling for the Detection and Tracking of Sarin-Surrogate-Induced Neurotoxicity Using a Human-Relevant, In-Vitro Brain Model

Organophosphorus (OP) nerve agents are a chemical threat to the United States, to the civilian population (e.g., pesticides) and historically weaponized (e.g., sarin) as chemical warfare agents. The unprecedented, accelerated process from “bench-to-bedside” during the SARSCov2 pandemic has made it clear that technology and tools need to be readily available for immediate response. Advances in human organ tissue mimetic systems are a promising technology to evaluate the human-relevant response in vitro for basic and applied research and drug screening. In particular, current brain microphysiological systems (MPS) have the capability to monitor and detect changes in engineered human neural circuit activity. However, current data analytics approaches for these systems lack the granularity to functionally detect and distinguish the different mechanisms that occur in the brain following neurotoxicity, injury, and disease. The goal of this project was to advance the computational analytical capabilities of the brain MPS to detect functional changes in neural circuit structure at different stages of Sarin surrogate-induced neurotoxicity. We developed graph-based models to (1) identify the composition of the neural circuit structure; (2) detect and monitor how this structure changes following sarin-induced neurotoxicity; and (3) evaluate the analytical pipeline using known/promising oxime reactivators. Through experiments on the bMPS where in vitro neuronal cultures were exposed to a sarin surrogate, we demonstrated the capabilities of our computational pipeline to identify different responses in the functional networks of brain cells exposed to low and high concentrations of the nerve agent. We identified a biphasic response of human neural network activity following exposure to a sarin-surrogate that had not been reported in the literature before. The graph-based models and software developed in this project can be used for future studies that leverage the brain MPS technology, such as treatment efficacy assessment.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Internal Standards for Quantitative Analysis of Chemical Warfare Agents by the GC/MS Method: Nerve Agents

General conditions and requirements for an internal standard useful in the determination of chemical warfare agents (CWAs) by the method of gas chromatography coupled with mass detection (GC/MS) were defined. The determination is based on a GC/MS analysis of a mixture of a CWA with an internal standard, conversion of the TIC chromatogram to a chromatogram extracted at a particular m / z ratio, and calculation of the CWA concentration from the internal standard concentration, response factor, and chromatographic peak areas. Available internal standards were identified, and they were verified for seven organophosphorus nerve-paralysing agents. Corresponding response factors were determined as a ratio of gradients of the linear functions of the peak area and compound concentration. Linearity, repeatability, and accuracy of the measurements were evaluated. The determination can be performed on all GC/MS systems of the Fire Rescue Service of the Czech Republic (FRS), where no CWA standards are available.

Capoun, Tomas↗