Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “agent based model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

The DOE E3SM Model Version 2: Overview of the Physical Model and Initial Model Evaluation

Abstract This work documents version two of the Department of Energy's Energy Exascale Earth System Model (E3SM). E3SMv2 is a significant evolution from its predecessor E3SMv1, resulting in a model that is nearly twice as fast and with a simulated climate that is improved in many metrics. We describe the physical climate model in its lower horizontal resolution configuration consisting of 110 km atmosphere, 165 km land, 0.5° river routing model, and an ocean and sea ice with mesh spacing varying between 60 km in the mid‐latitudes and 30 km at the equator and poles. The model performance is evaluated with Coupled Model Intercomparison Project Phase 6 Diagnosis, Evaluation, and Characterization of Klima simulations augmented with historical simulations as well as simulations to evaluate impacts of different forcing agents. The simulated climate has many realistic features of the climate system, with notable improvements in clouds and precipitation compared to E3SMv1. E3SMv1 suffered from an excessively high equilibrium climate sensitivity (ECS) of 5.3 K. In E3SMv2, ECS is reduced to 4.0 K which is now within the plausible range based on a recent World Climate Research Program assessment. However, a number of important biases remain including a weak Atlantic Meridional Overturning Circulation, deficiencies in the characteristics and spectral distribution of tropical atmospheric variability, and a significant underestimation of the observed warming in the second half of the historical period. An analysis of single‐forcing simulations indicates that correcting the historical temperature bias would require a substantial reduction in the magnitude of the aerosol‐related forcing.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Impacts of anthropogenic gadolinium on the activity of the ammonia oxidizing bacterium Nitrosomonas europaea

Widespread use of gadolinium-based contrast agents in medical imaging has resulted in increased Gd inputs to municipal wastewater treatment plants. Others have reported that typical wastewater treatment does not attenuate Gd, resulting in discharges to natural waters. However, whether elevated Gd impacts the performance of biological treatment has not been investigated. We examined whether gadolinium chloride or Gd chelated with diethylenetriaminepentaacetic acid (DTPA) affected the activity of the model nitrifying bacterium Nitrosomonas europaea. At nominal GdCl 3 additions ranging from 1 to 500 µM, no impact was observed compared to the control. Most (>98%) of the added Gd precipitated, and extracellular GdPO 4 nanoparticles were observed. When chelated with DTPA, Gd remained soluble, but no statistically significant impact on ammonia oxidation was observed until the highest concentrations tested. At 300 and 500 µM Gd-DTPA, a temporary reduction of nitrite production relative to the control (effect size 1.3 mg l -1 and 1.5 mg l -1 , respectively, at 24 h) was seen. By itself, DTPA was highly inhibitory. Modeling suggested that DTPA likely chelated other metals, but adjusting the concentrations of the most abundant metals in the medium, calcium and magnesium, indicated that lowering their free ion activities was probably not the cause of inhibition. Complexation of other essential metals was more likely. Furthermore, our studies indicate that while the low bioavailability of Gd may limit its ecosystem impacts, the role of synthetic ligands used with Gd and other rare earth elements should e considered as the production, use and disposal of these elements increases.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Optimization-based approaches to control of connected and automated vehicles: Principles, complexities, applications, challenges, and outlook

Safe and optimal motion control for connected and automated vehicles (CAVs) poses a fundamental optimization challenge at the intersection of system complexity, environmental uncertainty, and stringent real-time constraints. Existing surveys address this challenge in isolation – focusing either on specific control techniques or individual uncertainty sources – without providing a unified framework that characterizes the trade-offs among computational tractability, performance verifiability, and adaptive generalization across paradigms. This review addresses that gap by presenting a cohesive analytical framework concentrated on the decision-making and trajectory optimization layers of the CAV autonomy stack. We systematically analyze three major optimization paradigms – first-principles model-based optimization, data-driven methods, and hybrid synergistic architectures – evaluating each against four core complexity axes: problem formulation, constraint handling, optimality guarantees, and robustness. Key applications including platooning, trajectory planning, collision avoidance, and cooperative control are examined to reveal recurring methodological patterns and critical operational constraints that limit real-world performance. Our synthesis identifies verifiable hybrid architectures, incentive-aligned multi-agent cooperation, and hardware-algorithm co-design as the defining research frontiers, and distills a targeted agenda for developing CAV control systems that are simultaneously safe, computationally efficient, and deployable in the full complexity of real-world traffic environments.

Muzahid, Abu Jafar Md [University of Tennessee, Kn↗
DOE OSTI2023

Understanding of three different polyvinylpyrrolidone (PVP) based battery binders blends on graphene surfaces from first principles via DFT simulations

Binders play a crucial role in binding electrodes, current collectors, and conductive agents during battery manufacturing. The selection of the binders and their blends considerably affect various properties such as mechanical durability, adhesion performance, ionic/electronic conductivities, and solid electrolyte interface stability for lithium-ion batteries. In this work, polyvinylidene di-fluoride (PVDF), polyacrylic acid (PAA), polyvinylpyrrolidone (PVP), and lithiated polyacrylic acid (Li-PAA) were selected as the model binders, and their blends were studied on graphene surfaces. Further, among them, PVDF is hardly soluble in water unlike the other binders, so in practice, the blend of PVP:PVDF is not thought to be a proper combination. However, the PVP:PVDF binder has also been included because the purpose of this work is to calculate several electronic properties such as binding energies, intermolecular interaction energies, bond critical points, electron density, and the Laplacian of electron density at critical bonding regions of all these blends at the molecular level via density functional theory (DFT) simulations in order to evaluate and compare how is the interaction strength and bonding type of three different lithium-ion batteries (LIBs) binder blends on graphene surfaces. Most stable binder pairs and their binding mechanisms on graphene surfaces were studied. The delocalization of lithium in Li-PAA was studied with the presence of water.

25 ENERGY STORAGE↗
DOE OSTI2021

A Real-time Agent Based Optimization and Control Approach for Residential Building Heating Ventilation and Air Conditioning Systems

The prevalence of the loT (Internet of Things) is fostering the development of new control options and decision making that was not previously available. This is creating a wealth of opportunities for real-time control approaches and systems that can optimize for a common goal. This paper presents a smart residential neighborhood with optimization at the residential level utilizing a system of agents. The optimization utilizes information and modeling to optimize variable speed HVAC real-time operation in actual residential buildings. Data is presented showing the performance of the optimization and conclusions are drawn on next steps for development.

Hall, Joni↗
DOE OSTI2020

Constrained Deep Reinforcement Learning for Energy Sustainable Multi-UAV Based Random Access IoT Networks With NOMA

In this paper, we apply the Non-Orthogonal Multiple Access (NOMA) technique to improve the massive channel access of a wireless IoT network where solar-powered Unmanned Aerial Vehicles (UAVs) relay data from IoT devices to remote servers. Specifically, IoT devices contend for accessing the shared wireless channel using an adaptive p-persistent slotted Aloha protocol; and the solar-powered UAVs adopt Successive Interference Cancellation (SIC) to decode multiple received data from IoT devices to improve access efficiency. To enable an energy-sustainable capacity-optimal network, we study the joint problem of dynamic multi-UAV altitude control and multi-cell wireless channel access management of IoT devices as a stochastic control problem with multiple energy constraints. We first formulate this problem as a Constrained Markov Decision Process (CMDP), and propose an online model-free Constrained Deep Reinforcement Learning (CDRL) algorithm based on Lagrangian primal-dual policy optimization to solve the CMDP. Extensive simulations demonstrate that our proposed algorithm learns a cooperative policy in which the altitude of UAVs and channel access probability of IoT devices are dynamically controlled to attain the maximal long-term network capacity while ensuring energy sustainability of UAVs, outperforming baseline schemes. The proposed CDRL agent can be trained on a small network, yet the learned policy can efficiently manage networks with a massive number of IoT devices and varying initial states, which can amortize the cost of training the CDRL agent.

42 ENGINEERING↗
DOE OSTI2025

Reinforcement Learning for In-Spill Optimization of the Mu2e Resonant Extraction: Compensating Non-Stationarity

We present design considerations and challenges for the fast machine learning component of a third-order resonant beam extraction regulation system being commissioned to deliver steady beam rates to the mu2e experiment at Fermilab. Dedicated quadrupoles drive the tune toward the 29/3 resonance each spill, extracting beam at kV multiwire septa. The overall Spill Regulation System consists of (1) a “slow” process using ~100-spill averages to adjust the base quad ramp infrequently, (2) a feedforward harmonic content compensator, and (3) the “fast” ML agent reacting during each ongoing spill with on-the-fly additive corrections to the sum of (1) and (2). We have demonstrated improved beam-rate steadying for a fast ML agent compared to a PID controller using a quasi-physical spill simulation, and demonstrated distillation of that simulation into a predictive surrogate model. Current work includes a data-and-training pipeline to generate data-aware surrogates with real-world dynamics, even as the dynamics shift unpredictably. The surrogates are to act as RL environments against which to train our fast ML control agents before deploying them on FPGA in the live system. Further current efforts focus on modeling and controlling beam loss around the storage ring, understanding additional available hardware inputs to the model, and the interplay of these with beam-steadying performance.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2020

Structure, Hydration, and Interactions of Native and Hydrophobically Modified Phytoglycogen Nanoparticles

Phytoglycogen is a highly branched polymer of glucose produced as soft, compact nanoparticles by sweet corn. Properties such as softness, porosity, and mechanical integrity, combined with nontoxicity and biodegradability, make phytoglycogen nanoparticles ideal for applications involving the human body, ranging from skin moisturizing and rejuvenation agents in personal care formulations to functional therapeutics in biomedicine. To further broaden the range of applications, phytoglycogen nanoparticles can be chemically modified with hydrophobic species such as octenyl succinic anhydride (OSA). In this study, we present a self-consistent model of the particle structure, water content, and degree of chemical modification of the particles, as well as the emergence of well-defined interparticle spacings in concentrated dispersions, based on small-angle neutron scattering (SANS) measurements of aqueous dispersions of native phytoglycogen nanoparticles and particles that were hydrophobically modified using octenyl succinic anhydride (OSA) in both its protiated (pOSA) and deuterated (dOSA) forms. Measurements on native particles with reduced polydispersity have allowed us to refine the particle morphology, which is well described by a hairy particle (core-chain) geometry with short chains decorating the surface of the particles. The isotopic variants of OSA-modified particles enhanced the scattering contrast for neutrons, revealing lightly modified hairy chains for small degrees of substitution (DS) of OSA, and a raspberry particle geometry for the largest DS value, where the OSA-modified hairy chains collapse to form small seeds on the surface of the particles. This refined model of native and OSA-modified phytoglycogen nanoparticles establishes a quantitative basis for the development of new applications of this promising sustainable nanotechnology.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Adsorptive behavior of poly (vinylidene fluoride) membranes for the recovery of lignin-derived hydrophobic deep eutectic solvents

Abstract Recently, membrane technology has gained significant traction as an energy-efficient alternative to traditional thermal processes for solvent recovery. Deep eutectic solvents (DESs) have emerged as sustainable alternatives to conventional organic solvents, yet a systematic methodology for selecting compatible membrane materials for their recovery remains underdeveloped. This study established a predictive framework for membrane material selection in hydrophobic DES applications using Hansen Solubility Parameters (HSP) with inverted criteria targeting materials with relative energy difference (RED) values greater than 1.0. Flat sheet membranes were fabricated via the non-solvent induced phase separation (NIPS) technique. Four NIPS fabricated polymer membranes were evaluated: polysulfone, cellulose acetate, polyvinylidene fluoride (PVDF) fabricated with polyethylene glycol (PEG) as a pore-forming agent, and polybenzimidazole (PBI). The HSP approach successfully predicted membrane-solvent compatibility, with polysulfone (RED = 0.6) and cellulose acetate (RED = 0.9) dissolving completely within 24 h, while PVDF (RED = 1.9) and PBI (RED = 1.1) maintained structural integrity throughout a 7-day exposure period. Furthermore, PVDF demonstrated superior performance with minimal weight gain (3.0%), hydrophobic surface characteristics (122° water contact angle), and enhanced mechanical properties following DES exposure. Comprehensive chemical and morphological characterization confirmed PVDF’s chemical stability and revealed a surface-selective interaction mechanism involving simultaneous PEG (pore-forming agent) extraction and DES component adsorption. Adsorption kinetics followed pseudo-first-order behavior with reversible characteristics, best described by the Temkin isotherm model (R² = 0.9987). PVDF membranes-maintained separation functionality with average lignin rejection (75.2 ± 7.69%) and demonstrated filtration permeability of 2.0 ± 0.34 LMH/bar. This methodology provides a rational approach for membrane selection in emerging solvent systems, contributing to the advancement of sustainable separation technologies for DES-based biomass processing applications.

Science & Technology - Other Topics↗
DOE OSTI2025

Reinforcement Learning‐Based Adaptation of Grid Following Inverter's Internal Controller to Networked Microgrids' Strengths

The varying topological configurations, generator commitments and dispatches, and dynamic load demand lead to changing system's strengths during the operations of networked microgrids. When the system's strengths significantly change, the fixed control gains at large devices may result in unsatisfactory system performance; this necessitates the tuning of the control gains at large devices to adapt to the changing system's strengths. In this paper, observer-based reinforcement learning (RL) is utilised to automatically tune the proportional-integral (PI) gains of phase lock loop (PLL) controller of grid-following (GFL) inverters to adapt to the changing strengths of microgrids and networked microgrids. The RL agent in this framework augments an observer predicting system's strengths, from which the RL control policy will adjust accordingly to tune the PLL controller's gains towards the system's strengths. Also, to enhance the control performance, the recently introduced Barrier function-based RL framework is leveraged for the design of reward function to prevent the high frequency nadir. An operational 26 kV electric distribution system, which is modelled as networked microgrids, is used to illustrate the need and effectiveness of the proposed RL-tuned control.

frequency response↗
DOE OSTI2024

Computational and Systems Biology Advances to Enable Bioagent Agnostic Signatures

Enumerated threat agent lists have long driven biodefense priorities. The global SARS-CoV-2 pandemic demonstrated the limitations of searching for known threat agents as compared to a more agnostic approach. Recent technological advances are enabling agent-agnostic biodefense, especially through the integration of multi-modal observations of host-pathogen interactions directed by a human immunological model. Although well-developed technical assays exist for many aspects of human-pathogen interaction, the analytic methods and pipelines to combine and holistically interpret the results of such assays are immature and require further investments to exploit new technologies. In this manuscript, we discuss potential immunologically based bioagent-agnostic approaches and the computational tool gaps the community should prioritize filling.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Multi-Agent Hierarchical Deep Reinforcement Learning for HVAC Control With Flexible DERs

As electricity consumption in commercial and residential buildings continues to rise, reducing energy costs presents an increasing challenge. Heating, ventilating, and air-conditioning (HVAC) systems, which typically account for 40%-50% of a building's energy use, are prime targets for energy savings. Intelligent control of HVAC temperature through the exploitation of HVAC load flexibility brings significant potential to reduce energy consumption and electricity expenses. The nonlinear models of HVAC systems challenge traditional control methods, while the uncertainty introduced by HVAC load flexibility complicates distributed energy resource (DER) management using conventional optimal dispatch techniques. In response to these challenges, we propose a hierarchical multi-agent deep reinforcement learning (DRL) approach. The lower-level agents focus on balancing comfort and energy conservation, while the upper-level DRL agents optimize the use of DERs to reduce peak demand based on the control outcomes of the HVAC by the lower-level agents. Here, in the upper-level agents, we incorporate a multi-agent structure based on ensemble learning, which acts based on historical and current data without relying on precise load forecasting to address the delayed rewarding issue in DRL. This allows for the effective reduction of energy costs. The proposed method is tested using a real-world microgrid comprising 413 buildings in Southern California, and the results demonstrate that our approach can significantly reduce overall electricity bills while ensuring the comfort of consumers and residents.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Evaluate the System-Level Impact of Connected and Automated Vehicles Coupled with Shared Mobility: An Agent-based Simulation Approach

With the rapid growth of information and communication technologies, Connected and Automated Vehicles (CAVs) are deemed to be disruptive with the potential to significantly improve overall transportation system efficiency, however, may bring Vehicle Miles Traveled (VMT) increase or other issues. Further, shared mobility systems are another disruptive force that is reshaping our travel patterns. To quantify the combined impact of CAV and shared mobility on travel behavior, traffic performance and energy efficiency, we develop a mesoscopic simulation-based framework for mobility and energy efficiency evaluation considering the disruptive transportation technologies. Under this framework, we develop novel models for energy intensity and modal activity, and evaluated a variety of energy scenarios for different combinations of CAV applications, various levels of automation, roadway characteristics, and traffic conditions, while also varying different vehicle types and fuel/powertrain technologies. Based applying this modeling suite to a calibrated BEAM simulation network in Riverside California, it was found that cooperative automated driving in general will improve mobility, but automated vehicles, even when deployed in a shared autonomous fleet, will likely bring an increase of VMT (up to 36%) due to mode shifts and deadheading. Ride-hailing vehicles typically have better energy efficiency and a higher share of electric vehicles, which helps offset the negative impact from VMT increases when estimating the system-level energy consumption. In general, simulation results show a 6% increase in energy consumption for the scenarios with an increasing shift to ride-hailing modes.

42 ENGINEERING↗
DOE OSTI2023

Deep Reinforcement Learning Based Smart Water Heater Control for Reducing Electricity Consumption and Carbon Emission

Water heating is the third largest electricity consumer in U.S. households, after space heating and cooling. Thus, water heaters represent a significant potential for reducing electricity consumption and associated CO2 emissions of residential buildings. To this end, this study proposes a model-free deep reinforcement learning (RL) approach that aims to minimize the electricity consumption and the CO2 emissions of a heat pump water heater without affecting user comfort. In this approach, a set of RL agents focusing on either electricity saving or emission reduction, with different look ahead periods, were trained using the deep Q-networks (DQN) algorithm and their performance was tested on different hot water usage and Marginal Operating Emissions Rate (MOER) profiles. The testing results showed that the RL agents that focus on electricity saving can save electricity in the range of 12–22% by operating the water heater with maximum heat pump efficiency and minimum electric element utilization. On the other hand, the RL agents that focus on emission reduction reduced emissions in the range of 18–37% by making use of the variable MOER values. These RL agents used the heat pump and/or an element when the MOER values are low due to the availability of renewable energy sources (e.g., solar and wind) and mostly avoided the periods of carbon-intensive periods. Overall, these results showed that the proposed RL approach can help minimize the electricity consumption and the CO2 emissions of a heat pump water heater without having any prior knowledge about the device.

Amasyali, Kadir↗
DOE OSTI2021

Decentralized Voltage Control of Large-Scale Distribution System with PVs Based on MADRL

This paper proposes a model-free decentralized control framework for the voltage regulation of large-scale distribution systems through the coordinated control of PV inverters. This is achieved by developing a novel interaction mechanism between the surrogate model and the centralized training and decentralized execution multiagent deep reinforcement learning framework. Specifically, the sparse Gaussian processes regression method is first utilized to develop the surrogate model of the original distribution system for reward calculation during the training stage, where each agent represents a sub-region in the centralized fashion for coordination strategy learning. After that, the learned control rules are used to inform controllers within each sub-region for real-time decisions with only local measurements. Comparative tests among various methods on the EPRI Ckt5 test system demonstrate the effectiveness of the proposed method.

distribution system↗
DOE OSTI2022

Removal of Non-Specifically Bound Proteins Using Rayleigh Waves Generated on ST-Quartz Substrates

Label-free biosensors are plagued by the issue of non-specific protein binding which negatively affects sensing parameters such as sensitivity, selectivity, and limit-of-detection. In the current work, we explore the possibility of using the Rayleigh waves in ST-Quartz devices to efficiently remove non-specifically bound proteins via acoustic streaming. A coupled-field finite element (FE) fluid structure interaction (FSI) model of a surface acoustic wave (SAW) device based on ST-Quartz substrate in contact with a liquid loading was first used to predict trends in forces related to SAW-induced acoustic streaming. Based on model predictions, it is found that the computed SAW body force is sufficient to overcome adhesive forces between particles and a surface while lift and drag forces prevent reattachment for a range of SAW frequencies. We further performed experiments to validate the model predictions and observe that the excitation of Rayleigh SAWs removed non-specifically bound (NSB) antigens and antibodies from sensing and non-sensing regions, while rinsing and blocking agents were ineffective. An amplified RF signal applied to the device input disrupted the specific interactions between antigens and their capture antibody as well. ST-quartz allows propagation of Rayleigh and leaky SH-SAW waves in orthogonal directions. Thus, the results reported here could allow integration of three important biosensor functions on a single chip, i.e., removal of non-specific binding, mixing, and sensing in the liquid phase.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Regeneration of critical-sized defects, in a goat model, using a dextrin-based hydrogel associated with granular synthetic bone substitute

Abstract The development of injectable bone substitutes (IBS) have obtained great importance in the bone regeneration field, as a strategy to reach hardly accessible defects using minimally invasive techniques and able to fit to irregular topographies. In this scenario, the association of injectable hydrogels and bone graft granules is emerging as a well-established trend. Particularly, in situ forming hydrogels have arisen as a new IBS generation. An in situ forming and injectable dextrin-based hydrogel (HG) was developed, aiming to act as a carrier of granular bone substitutes and bioactive agents. In this work, the HG was associated to a granular bone substitute (Bonelike®) and implanted in goat critical-sized calvarial defects (14 mm) for 3, 6 and 12 weeks. The results showed that HG improved the handling properties of the Bonelike® granules and did not affect its osteoconductive features, neither impairing the bone regeneration process. Human multipotent mesenchymal stromal cells from the umbilical cord, extracellular matrix hydrolysates and the pro-angiogenic peptide LLKKK18 were also combined with the IBS. These bioactive agents did not enhance the new bone formation significantly under the conditions tested, according to micro-computed tomography and histological analysis.

Pereira, Isabel↗
DOE OSTI2024

Modulating Ce 3+ Sites in Ce-Zr Oxide Nanocatalysts through Protamine Biomineralization for Organophosphate Dephosphorylation

Catalytic is a crucial reaction for environmental detoxication of pesticides and neutralization of various molecules classified as chemical warfare agents. Herein, we report on a series of tunable Ce-Zr-based metal oxides, (Zr a Ce 1–a O x ) prepared using a facile biomineralization technique, as catalysts for organophosphates dephosphorylation. Synchrotron scattering and spectroscopy methods showcase that Zr a Ce 1–a O x catalysts are highly defective and exhibit an abundance of Ce 3+ sites that promote oxygen vacancies needed for enhanced dephosphorylation reactions. The catalytic performance was assessed using a model para-nitrophenyl phosphate reaction and showcases a strong dependence on Zr dopant concentration and subsequent tuning of the Ce 3+ /Ce 4+ ratio. Analysis of synchrotron datasets allowed structure-performance correlations between the Ce 3+ concentration and associated oxygen vacancies, the dephosphorylation rate constant, and Zr concentration to be established, confirming that Ce 3+ as active sites is positively correlated with the rate constant. We envision that similar biomineralization approaches can be used to fabricate Ce 3+ -rich Ce-Zr oxide for environmental application in dephosphorylation and other hydrolysis reactions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗