Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “agent based simulation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Modeling and simulation to investigate the electrification potential of medium- and heavy-duty vehicle fleets

This project involves developing and integrating new modeling tools to simulate the dynamics of electric medium- and heavy-duty fleet vehicle adoption. A technical and economic modeling tool, combining a data-driven hardware cost model with a cost-optimal charging strategy microsimulation, enables tailored analysis of the costs and benefits of electrifying individual fleets. Next, a novel text synthesis process, applied to a curated corpus of literature, quantifies trade-offs between technical, economic, and other factors in the fleet vehicle procurement decision. The outcomes of these tasks combine with knowledge from recent literature on fleet decision processes to specify the vehicle procurement model used by fleets in an agent-based model of the medium- and heavy-duty electric vehicle market. This model embodies an especially disaggregated approach to adoption modeling, internalizing factors and dynamics that conventional adoption models externalize. In particular, explicitly modeling the formation and diffusion of opinions among agents enables experiments that conventional models cannot support. Demonstrations show, for example, that increasing the extent of interactions between populations with different proclivities to electric vehicles has an asymmetrical outcome. High-proclivity electric vehicle adoption is generally unaffected as interactions increase, but low-proclivity adoption is accelerated. By representing individual fleets' requirements and costs at a high level of detail, incorporating an adoption decision model informed by a wide body of empirical research, and broadening the array of variables and dynamics available for experimentation, this integrated model offers a new way to understand the urgent challenge of eliminating emissions from the most emissions-intensive transportation sectors.

Trinko, David A.↗
DOE OSTI2021

Polyphenols Weaken Pea Protein Gel by Formation of Large Aggregates with Diminished Noncovalent Interactions

Polyphenols are well-known native cross-linkers and gel strengthening agents for many animal proteins. However, their role in modifying plant protein gels remains unclear. In this study, multiple techniques were applied to unravel the influence of green tea polyphenols (GTP) on pea protein gels and the underlying mechanisms. We found that the elasticity and viscosity of pea protein gels decreased with increased GTP. The protein backbone became less rigid when GTP was present based on shortened T 1ρ H in relaxation solid-state NMR measurements. Electron microscopy and small-angle X-ray scattering showed that gels weakened by GTP possessed disrupted networks with the presence of large protein aggregates. Solvent extraction and molecular dynamic simulation revealed a reduction in hydrophobic interactions and hydrogen bonds among proteins in gels containing GTP. The current findings may be applicable to other plant proteins for greater control of gel structures in the presence of polyphenols, expanding their utilization in food and biomedical applications.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Distributional Deep Reinforcement Learning-Based Emergency Frequency Control

Emergency frequency control is one of the most critical approaches to maintain power system stability after major disturbances. With the increasing number of grid-connected renewable energy sources, existing model-based methods of frequency control are facing up with challenges of computational speed and scalability for large-scale systems. In this paper, the emergency frequency control problem is formulated as a Markov Decision Process (MDP) and solved through a novel Distributional Deep Reinforcement Learning (DDRL) method, namely the distributional soft actor critic (DSAC) method. Compared with other RL methods that only estimate the mean value, the proposed DSAC model estimates the distribution of value function over returns. This advancement can lead to more insights and knowledge for the agent, with the benefit of a much faster and more stable learning process, and the improved frequency control performance. Here, the simulation results on IEEE 39-bus and IEEE 118-bus systems demonstrate the effectiveness and robustness of proposed models, as well as the advantage compared to other state-of-the-art DRL algorithms.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Agent-based Disease Surveillance and Transmission Model

The SARS-CoV-2 pandemic has driven a need for models that can predict transmission among populations and test the effectiveness of proposed interventions to control it such as testing, quarantining, vaccinations, and nonpharmaceutical interventions. Some questions of interest include:1. How rapidly will the virus spread if no action is taken? 2. How frequently should people be tested and quarantined if necessary? 3. Should unvaccinated persons be tested more frequently than the vaccinated? 4. If test results are delayed, how will that affect the spread? 5. How do test false negative rates affect the overall efficacy of testing programs? 6. How do constraints on testing resources impact the overall efficacy of testing programs? 7. How does the patient’s failure to recognize COVID symptoms and quarantine affect the spread? 8. What level of vaccination coverage is needed to suppress the epidemic? 9. What is the risk of a large outbreak (probability exceeding x infections)? 10. What is the cost effectiveness of interventions (dollars per avoided infection)? This report describes the structure of a discrete event simulation model that provides actionable information to address these questions. Parameter inputs and results of analyses have been reported in briefings provided to LLNL management and are not discussed here.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Resilient Design of Continuous-time Distributed Optimization Algorithm in the Presence of Cyber-attacks

This paper presents a continuous-time resilient distributed optimization algorithm based on competitive interaction design method on connected graphs in the presence of adversaries. Here, the competitive interaction method allows us to design a network that protects the multi-agent systems from adversaries without requiring high network connectivity. In addition, the proposed algorithm does not require the global information about the number of adversaries. First, we show that the proposed distributed algorithm solves the resilient distributed optimization problem with no attack on the communication links. Second, we show that the proposed continuous-time distributed optimization algorithm on connected graphs converges to the small neighborhood of the optimal solution in the presence of cyber-attacks onto the communication channel. Simulations are presented to illustrate our theoretical results.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Understanding and Manipulating Helical Nanofilaments in Binary Systems with Achiral Dopants

Here, in this paper, we report the relationship between helical pitch of the helical nanofilament (HNF) phase formed by bent-core molecule NOBOW and the concentration of achiral dopants 5CB and octane, using linearly polarized resonant soft X-ray scattering (RSoXS). Utilizing theory-based simulation, which fits well with the experiments, the molecular helices in the filament were probed and the superstructure of helical 5CB directed by groove of HNFs was observed. Quantitative pitch determination with RSoXS reveals that helical pitch variation is related to 5CB concentration with no temperature dependence. Doping rodlike immiscible 5CB led to a pitch shortening of up to 30%, which was attributed to a change in interfacial tension. By shedding light not only on phase behavior of binary systems but also enabling control over pitch length, our work may benefit various applications of HNF-containing binary systems, including optical rotation devices, circularly polarized light emitters, and chirality transfer agents.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

An Edge-Cloud Integrated Solution for Buildings Demand Response Using Reinforcement Learning

Buildings, as major energy consumers, can provide great untapped demand response (DR) resources for grid services. However, their participation remains low in real-life. One major impediment for popularizing DR in buildings is the lack of cost-effective automation systems that can be widely adopted. Existing optimization-based smart building control algorithms suffer from high costs on both building-specific modeling and on-demand computing resources. To tackle these issues, this paper proposes a cost-effective edge-cloud integrated solution using reinforcement learning (RL). Beside RL’s ability to solve sequential optimal decision-making problems, its adaptability to easy-to-obtain building models and the off-line learning feature are likely to reduce the controller’s implementation cost. Using a surrogate building model learned automatically from building operation data, an RL agent learns an optimal control policy on cloud infrastructure, and the policy is then distributed to edge devices for execution. Simulation results demonstrate the control efficacy and the learning efficiency in buildings of different sizes. A preliminary cost analysis on a 4-zone commercial building shows the annual cost for optimal policy training is only 2.25% of the DR incentive received. Results of this study show a possible approach with higher return on investment for buildings to participate in DR programs.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Towards Immobilized Proton-Coupled Electron Transfer Agents for Electrochemical Carbon Capture from Air and Seawater

Electrochemical CO 2 separation has drawn attention as a promising strategy for using renewable energy to mitigate climate change. Redox-active compounds that undergo proton-coupled electron transfer (PCET) are an impetus for pH-swing-driven CO 2 capture at low energetic costs. However, multiple barriers hinder this technology from maturing, including sensitivity to oxygen and the slow kinetics of CO 2 capture. Here, we use vapor phase chemistry to construct a textile electrode comprising an immobilized PCET agent, poly(1-aminoanthraquinone) (PAAQ), and incorporate it into redox flow cells. This design contrasts with others that use dissolved PCET agents by confining proton-storage to the surface of an electrode kept separate from an aqueous, CO 2 -capturing phase. This system facilitates carbon capture from gaseous sources (a 1% CO 2 feed and air), as well as seawater, with the latter at an energetic cost of 202 kJ/mol CO2 , and we find that quinone moieties embedded within the electrode are more stable to oxygen than dissolved counterparts. Simulations using a 1D reaction-transport model show that moderate energetic costs should be possible for air capture of CO 2 with higher loadings of polymer-bound PCET moieties. The remarkable stability of this system sets the stage for producing textile-based electrodes that facilitate pH-swing-driven carbon capture in practical situations.

Ali, Fawaz↗
DOE OSTI2026

Optimization and stabilization of Fermilab Booster using hybrid Bayesian/RL framework

PIPII project will raise Fermilab Booster intensity and ramp rate. Beam losses will limit average power and are hard to simulate. Presently, Booster uses operator-guided empirical tuning. This task is challenging due to high dimensionality, multiple objectives, critical safety constraints, and drifts. We developed a synergistic suite of Bayesian optimization (BO) and reinforcement learning (RL) tools to optimize and stabilize beam losses. First, active learning was used to build a rough model. Data was collected parasitically using two novel safety constraint types – nonlinear input space restrictions (based on optics model), and uncertainty constraints (to stop bad steps/beam aborts). We then applied online multi-objective BO with scalarized objectives and fitting to improve/rebalance losses, increasing safety margins by 25%. Using BO model as a safety veto, we tried several on/off-policy RL agents for long term stabilization; SAC had best performance. We found that adding contextual (state) information further improved performance, eventually integrating key knobs like linac phase and temperature into the parameter space. Long term testing is ongoing to enable operational use.

Kuklev, Nikita [Fermilab]↗
DOE OSTI2022

Stochastic simulation of occupant-driven energy use in a bottom-up residential building stock model

The residential buildings sector is one of the largest electricity consumers worldwide and contributes disproportionally to peak electricity demand in many regions. Strongly driven by occupant activities, household energy consumption is stochastic and heterogeneous in nature. However, most residential energy models applied by industry use homogeneous, deterministic activity schedules, which work well for predictions of annual energy consumption, but can result in unrealistic hourly or sub-hourly electric load profiles, with exaggerated or muted peaks. The increasing proportion of variable renewable energy generators means that representing the heterogeneity and stochasticity of occupant behavior is now crucial for reliable planning at both bulk-power and distribution-system scales. This work presents a novel and open-source occupancy simulation approach that can simulate a diverse set of individual occupant and household event schedules for all major electricity, fuel, and hot water end uses. To accomplish this, we evaluated three alternative occupant activity simulation approaches before selecting a hybrid combining time-inhomogeneous Markov chains and probability-sampling of event durations and magnitudes. Further, we integrated the stochastic occupancy simulation with an open-source bottom-up physics-simulation building stock model and published a set of 550,000 diverse household end-use activity schedules representing a national housing stock. The simulator was verified against time-use survey data, and simulation results were validated against measured end-use electricity data for accuracy and reliability. While we use data for the United States, our application demonstrates how similar approaches could be applied using the time-use survey data collected in many countries around the world.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

A Self-Organizing Nano Grid (SONG) for Energy Access Clusters

In this paper, a bottom-up dc Self-Organizing Nano Grid (SONG) for rural electrification at ultra-low-cost through a flexible and scalable topology is presented. The proposed nano grid requires no communication and uses dc-bus signaling to convey excess energy in the system through the bus voltage, allowing independent agents to conduct energy transactions based on their needs in a decentralized manner. Moreover, this article proposes a concept that uses a novel multi-source, multi-segment droop control strategy with demand coordination, allowing for dynamic self-balancing of the system under fluctuating operating conditions. The unique use of a surplus photovoltaic (PV) power droop mode precisely estimates energy availability to minimize load chattering that helps in stabilizing the dc-bus. In addition, this architecture uses standard dc/dc converters for a low-cost, modular, Plug and Play ( PnP ) solution with highly automated operation, enabling local assembly, operation, and maintenance by non-technical personnel. The proposed SONG concept was validated in simulation and with a 700W, laboratory-scale, custom made nano grid system. The simulation and experimental test cases show the feasibility of the SONG system by demonstrating the proposed multi-source, multi-segment droop control strategy without load chattering under varying operating conditions.

dc-dc converter↗
DOE OSTI2023

Distributed Optimal Power Management for Battery Energy Storage Systems: A Novel Accelerated Tracking ADMM Approach

Optimal power management (OPM) is critical for large-scale battery energy storage systems. Today’s methods often require formidable computational effort due to the design based on centralized numerical optimization. Thus, this paper investigates computationally distributed OPM where the agents based on the cells communicate over a network to cooperatively solve the OPM problem. We propose an accelerated tracking alternating direction method of multipliers (ADMM) algorithm to solve the distributed OPM. The proposed algorithm embeds dynamic average consensus and Nesterov’s acceleration technique in the ADMM algorithm. Not only is the proposed algorithm fully distributed without a need for fusion or aggregating nodes, but it also accelerates convergence. The paper formulates the OPM in a model predictive control framework where it seeks to regulate the charging/discharging power of each battery cell to minimize the total power losses and promote balanced use of the constituent cells while complying with the safety constraints. The paper provides ample simulation results to demonstrate the effectiveness and advantages of the proposed distributed OPM in terms of computation and convergence.

Farakhor, Amir↗
DOE OSTI2026

Surrogates for Valve-Controlled Pipe Flow: Accelerating Nuclear Reactor Design

Neural surrogate models are developed to replace expensive steady-state RANS CFD simulations for valve-controlled pipe flow in nuclear reactor design. Using parametric CFD data generated with MOOSE Pronghorn across a range of valve geometry and flow conditions, three approaches are compared: a POD-based reduced-order model, a structured UNet on a cylindrical grid, and unstructured models (DeepONet and BiStride MeshGraphNet) on nondimensionalized point clouds. POD achieves the highest accuracy (99%) with fast inference but requires storing all solution snapshots, while the DeepONet and BSMS-GNN both achieve ~89% accuracy at sub-second inference, with the BSMS-GNN offering superior geometric generalizability. These surrogates enable rapid ranking of candidate valve designs and can warm-start CFD solvers to accelerate convergence, supporting agentic design iteration on the Prometheus platform.

42 - ENGINEERING↗
DOE OSTI2023

Transactive Energy System Deployment Over Insecure Communication Links

Here, in this paper, the privacy and security issues associated with the transactive energy system (TES) deployment over insecure communication links are addressed. In particular, it is ensured that 1) individual agents’ bidding information is kept private throughout hierarchical market-based interactions; and 2) any extraneous data injection attack can be quickly and easily detected. An implementation framework is proposed to enable the cryptography-based enhancement of privacy and security for the deployment of any general hierarchical systems including TESs. Under the proposed framework, a unified cryptography-based approach is developed to achieve both privacy and security simultaneously. Specifically, privacy preservation is realized by an enhanced Paillier encryption scheme, where a block design is proposed to significantly improve computational efficiency. Attack detection is further achieved by an enhanced Paillier digital signature scheme, where a stamp-concatenation mechanism is proposed to enable detection of data replace and reorder attacks. Simulation results verify the effectiveness of the proposed cyber-resilient design for transactive energy systems. Note to Practitioners—This paper is motivated by addressing the issues of cyber resiliency for practically deploying transactive energy system (TES) but it is also applicable to the problem of enhancing the privacy and security for any general hierarchical control systems. TES is an emerging control approach that engages energy suppliers and customers through market operations and uses the price to optimally allocate energy resources. While it has been shown to be promising for power system applications, the underlying market-based interactions raise significant concerns of privacy (data leakage) and security (data tampering). However, existing TES works only focus on the coordination mechanism instead of privacy and security issues. This paper proposes a new cryptography-based TES design for practical deployment. Specifically, to protect privacy, individual supply and demand amounts to be exchanged are all encrypted in a particular way such that the original amounts cannot be inferred from the encrypted amounts, while the desired computation for setting the market clearing price can be carried out over the encrypted amounts, thus generating an encrypted result which, when decrypted, matches that of the same computation over the original amounts. To achieve security, for each exchanged data, its sender generates a particular digital signature which is exchanged together with the data. This enables the receiver to automatically detect the integrity by checking whether a mathematical relationship holds for the pair of data and signature. In our future research, we will investigate more challenging scenarios where some suppliers and customers themselves could be corrupted and purposely submit distorted amounts.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Efficient Reinforcement Learning for Real-Time Hardware-Based Energy System Experiments: Preprint

In the context of urgent climate challenges and the pressing need for rapid technology development, Reinforcement Learning (RL) stands as a compelling data-driven method for controlling real-world physical systems. However, RL implementation often entails time-consuming and computationally intensive data collection and training processes, rendering them inefficient for real-time applications that lack non-real-time models. To address these limitations, real-time emulation techniques have emerged as valuable tools for the lab-scale rapid prototyping of intricate energy systems. While emulated systems offer a bridge between simulation and reality, they too face constraints, hindering comprehensive characterization, testing, and development. In this research, we construct a surrogate model using limited data from simulated systems, enabling an efficient and effective training process for a Double Deep Q-Network (DDQN) agent for future deployment. Our approach is illustrated through a hydropower application, demonstrating the practical impact of our approach on climate-related technology development.

deep Q-learning↗
DOE OSTI2024

SEAS Communication Engine: An Extensible, Flexible Wrapper for Co-Simulation Agents

When modeling and analyzing the power grid and other large scale systems, researchers often express scenarios as optimization problems and feed them into advanced software solvers. In order to allow multiple solvers to communicate with each other and share data from different domains, the National Renewable Energy Laboratory (NREL) and associated Department of Energy (DOE) labs have developed a software framework called the Hierarchical Engine for Large-scale Infrastructure Co-Simulation (HELICS). HELICS allows cosimulation via a collection of client libraries for different languages that can be called from the appropriate optimization software. However, these client libraries do not provide a higher level of abstraction beyond reading and writing data off of the shared HELICS bus. In this paper, we describe a new software library called the SEAS Communication Engine that exposes a higher-level API for running cosimulation problems. The SEAS Engine provides a class-based abstraction on top of the Python HELICS client, in order to allow users to implement their domain-specific cosimulations without needing to interact with core HELICS primitives. This will make adoption of HELICS and cosimulation in general easier, by exposing a simpler API. In the second part of the paper, we validate our library on a collection of different simulation examples, including the canonical IEEE 13 Bus Feeder. Lastly, we demonstrate using the SEAS Engine to directly call domain-specific code written in the Julia programming language. Our hope is that this will serve as a template for easily calling software in different programming languages via the SEAS Engine, thereby avoiding code duplication and complexity.

co-simulation↗
DOE OSTI2021

Distributed and communication-efficient solutions to linear equations with special sparse structure

In this paper we report two distributed and communication-efficient algorithms based on the multi-agent system are proposed to solve a system of linear equations with the Laplacian sparse system matrix. One algorithm is based on the gradient descent method in optimization. In this algorithm, the agents only share partial information instead of all of their collective state vectors to save significant communication. The other algorithm is obtained by approximating Newton’s method for a faster convergence rate. Although it requires twice as much communication as the first one, it is still communication-efficient given the low dimension of the information shared among agents. The convergence at a linear rate is proved for both algorithms, and a comprehensive comparison of their convergence rate, communication burden, and computation costs is also performed. The proposed algorithms can be applied to various systems to solve those problems that can be modeled as a system of linear equations with a Laplacian sparse system matrix. Simulation results with the electric power system illustrate their effectiveness.

42 ENGINEERING↗
DOE OSTI2020

Evaluating the Adaptability of Reinforcement Learning Based HVAC Control for Residential Houses

Intelligent Heating, Ventilation, and Air Conditioning (HVAC) control using deep reinforcement learning (DRL) has recently gained a lot of attention due to its ability to optimally control the complex behavior of the HVAC system. However, more exploration is needed on understanding the adaptability challenges that the DRL agent could face during the deployment phase. Using online learning for such applications is not realistic due to the long learning period and likely poor comfort control during the learning process. Alternatively, DRL can be pre-trained using a building model prior to deployment. However, developing an accurate building model for every house and deploying a pre-trained DRL model for HVAC control would not be cost-effective. In this study, we focus on evaluating the ability of DRL-based HVAC control to provide cost savings when pre-trained on one building model and deployed on different house models with varying user comforts. We observed around 30% of cost reduction by pre-trained model over baseline when validated in a simulation environment and achieved up to 21% cost reduction when deployed in the real house. This finding provides experimental evidence that the pre-trained DRL has the potential to adapt to different house environments and comfort settings.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗