SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
PPO and Friends (PPOAF) is a pytorch implementation of proximal policy optimization for single- and multi-agent reinforcement learning (the PPO), along with several optimizations and add-ons (the Friends) to enable efficient MPI-parallelized model training on HPC clusters.
The Behavior, Energy, Autonomy, and Mobility (BEAM) model is an integrated, agent-based travel demand simulation framework. Individual agents express preferences through a utility- maximizing evolutionary algorithm that minimizes each individual’s cost and time spent traveling via diverse modal options, including the competition for scarce supply resources such as parking spaces and charging infrastructure. BEAM simulates the essential elements that compose a dynamic transportation system. From the road network, parking and charging infrastructure, to the transit system and a synthetic population with plans and preferences, the virtual system is an amalgamation of multiple spatially resolved layers that together represent an integrated transportation system. BEAM is an extension to the MATSim (Multi-Agent Transportation Simulation) model, where agents employ reinforcement learning across successive simulated days to maximize their personal utility through plan mutation (exploration) and selecting between previously executed plans (exploitation). The BEAM model shifts some of the behavioral emphasis in MATSim from across-day planning to within- day planning, where agents dynamically respond to the state of the system during the mobility simulation. In BEAM, agents can plan across all major modes of travel including driving, walking, biking, transit, and demand-responsive ride hailing. It is designed to integrate with other open source transportation models, such as ActivitySim.
The approach used is the Multi-Agent System (MAS) paradigms, where systems components are represented as agents, interacting both with each other, and with the environment in which they evolved. Agents behaviors correspond to components in the real Integrated Water-Power System. The model simulate actions and interactions of these (autonomous) agents to analyze their effects on the overall system. Agents in the water system capture components of water collection, treatment, transportation, distribution and use (e.g. pipe, canal, pump, water demands for agriculture, etc.). Agents in the power system capture components of power generation, transportation, distribution and use (electricity demands, sources, etc.).
This codebase contains a multi-agent, actor-critic reinforcement learning implementation for cyber-resilient grid voltage control. It uses a 123-bus OpenDSS system as the environment, with three-phase power flow translating nodal power injections into solved nodal voltages. The reward function penalizes deviations from nominal voltage as well as reactive power dispatch, while encouraging agents to take actions that result in fast convergence to nominal conditions. The codebase models false data injection attacks and includes functionality for training, testing, hyper-parameter tuning, and visualization.
MADA-tools (Multi-Agent Design Assistant tools) is a library for defining MCP (Model Context Protocol) servers that can be used by AI agents in the MADA project. Each MCP server provides a focused set of tools that enhances an LLM's knowledge and capabilities for a specific domain, for example, how to launch jobs with Flux versus Slurm. The library makes it easy to configure and start multiple MCP servers using configuration files or command line options. Once running, these servers are intended to be consumed by one or more agents in the MADA ecosystem. The system is designed to be extensible so that future projects can contribute their own MCP servers, skills, and toolsets.
The Workflow Orchestration Language Framework (WOLF) is an agentic framework grounded in natural language with an architecture inspired by reinforcement learning (RL)—designed to orchestrate, scale, and accelerate complex workflows. The concept of WOLF was born out of the very successful ASC Tri-lab Multi-Agent Design Assistant (MADA) project, but extends beyond its domain-specific design agents to provide a more general and extensible architecture. WOLF capitalizes on the lessons learned from MADA and is fully aligned with Sutton’s The Bitter Lesson—that the most enduring progress in AI comes from general-purpose methods that scale with computation, rather than narrow techniques built on domain-specific human knowledge. In this spirit, WOLF enables agents to autonomously learn workflows, capture strategies as reusable playbooks, and build a growing corpus of interpretable, auditable “wisdom artifacts.” These artifacts, expressed in natural language, bridge human and machine understanding while preserving adaptability and scalability as computational power continues to expand.
Generative artificial intelligence (AI) has captured into the mainstream, demonstrating capabilities that once belonged solely to the realm of human cognition. From defeating world champions in complex games to generating human-quality text and images, Generative AI has proven its potential to revolutionize countless industries. The electric power grid is no exception. Generative AI's ability to process vast amounts of data rapidly, assist decision support and identify patterns could significantly enhance power grid operations. For example, Generative AI could improve state estimation where measurements are not available or integrate renewable energy sources more efficiently with probabilistic forecasting. The key contributions of this whitepaper are outlined below: (1) Comprehensive overview of Generative AI's applications in power grid operations: It highlights the opportunities in areas such as forecasting, state estimation, and demonstrating the potential for enhancing efficiency, reliability, and resilience. (2) Expanding Generative AI's impact through synergies with emerging technologies: The paper introduce NREL developed eGridGPT and explores how AI orchestration, multi-agent systems, and Digital Twins can collaborate to optimize grid operations, addressing the complexities of a decarbonized and electrified future. (3) In-depth analysis of challenges in implementing Generative AI: This includes considerations like data availability and quality, model validation, certification, and ethical concerns, ensuring responsible AI deployment. (4) Emphasizing human-AI collaboration: The whitepaper underscores the importance of trustworthy, transparency, and explainability in AI systems to promote seamless interaction between human operators and AI, ultimately improving decision-making. (5) Exploring future research and development: It identifies critical areas for further advancement to fully realize Generative AI's potential in power grid operations. This whitepaper serves as a valuable resource for researchers, practitioners, and policymakers looking to harness Generative AI for a more reliable, stable, and cost-effective power grid.
The objective of this work is to develop and demonstrate novel, adaptive, lightweight algorithms that enable the decision-making agents in a large cyber-physical network to act both autonomously and in collaborative harmony to enforce assured resilience across spatiotemporal layers, even under unforeseen adversarial scenarios (e.g., high- impact-low-probability events). Towards this end, the proposed solution will serve as minimally invasive add-on layers that bridge the existing (faster, reactive) local myopic controls and (slower, predictive) centralized optimization. Importantly, the proposed algorithms will enable the multi-agent network to autonomously and collaboratively enforce resilient operation under no or limited communication environment typical of severe cyber- physical adversarial events. The expected outcome of this effort is a suite of prototype, open-source, software algorithms for safety-aware local autonomous and semi-cooperative control (SLAC3R), demonstrated on networked microgrids (via RD2C/Thrust-1 OPAL-RT testbed).
Prior efforts have shown that deep reinforcement learning (DRL) may provide a new method for controlling networked power systems. Though successful, prior approaches have not yet demonstrated their behavior on systems of realistic scale. This effort examined multiple theoretical and technical approaches to allow a DRL model to operate over a system of 2,000 buses or more. We find that allowing the DRL models to run training episodes in parallel provides near limitless efficiency gains, allowing us to train successful agents to behave on our Kuramoto transmission model of up to 4,000 buses. We further show that we can expand our PowerWorld DRL implementation to systems of up to 25 buses but struggle to go beyond this limit due to PowerWorld’s inability to run multiple instances at once. Finally, we examine a multi-agent approach and find that it performs as well if not better than our existing centralized approach.
One year ago, the AISLE roadmap argued that autonomous laboratories operated as isolated islands and proposed a grassroots network organized around five critical dimensions. The field has since moved faster than that roadmap anticipated: multi-agent systems have produced experimentally validated hypotheses, self-driving laboratories have grown more interoperable and orchestrated, reasoning-trained and domain foundation models have raised the capability ceiling, and the Genesis Mission has placed autonomous experimentation at the center of U.S. federal science strategy, with industry emerging as a primary actor. Progress has met a sobering counter-current, including a corrected flagship discovery result, benchmarks showing that agents which rival experts on closed-ended questions still complete only a fraction of open-ended research, and fabricated citations surfacing at leading venues. We read this as the defining tension of the field: producing a candidate discovery is no longer the hard part, but verifying it is, and this asymmetry now limits autonomous science more than raw model capability. Accordingly, we update the roadmap around seven dimensions, revisiting the original five and elevating two former cross-cutting concerns, trust, verification, and reproducibility, and safety, security, and governance, to first-class status. We assess the original milestones (M1 through M14) as achieved, partially achieved, reframed, or open, add four new milestones (M15 through M18) for the elevated dimensions, and scope the path forward to a two-year horizon, with the first year concentrating on interfaces, protocol adoption, and the scaffolding of verification, and the second targeting federation, zero-trust coordination, and governance. Throughout, we position the grassroots network as the interoperability fabric that lets national programs, international initiatives, and commercial platforms connect rather than re-silo.
Photonic quantum communication enables secure and high-fidelity information transfer beyond classical limits, with direct relevance to emerging quantum networks operating in free-space environments. While physical-layer models of depolarizing noise, Gamma–Gamma turbulence statistics, entanglement swapping, and decoy-state QKD security bounds are individually well established, prior work typically treats these components in isolation or under fixed network assumptions. In this work, we develop a unified topology-aware analytical framework that simultaneously integrates free-space optical link budgets, turbulence-induced visibility degradation, depolarizing qubit noise, multi-hop entanglement cascade dynamics, teleportation fidelity thresholds, CHSH nonlocality certification, and asymptotic decoy-state secret key rate bounds across star, mesh, and ring graph structures. Rather than introducing new physical channel models, we demonstrate that identical physical links exhibit fundamentally different end-to-end performance once embedded within different network topologies. Mesh architectures minimize visibility cascade through hop-count reduction but incur quadratic hardware scaling. Star topologies minimize link count but concentrate noise and synchronization overhead at the hub. Ring configurations offer linear hardware scaling with multiplicative fidelity degradation. The results establish topology as a first-order design parameter in near-term free-space quantum networks operating without full quantum repeater infrastructures. While motivated by distributed multi-agent architectures, the framework applies broadly to terrestrial, airborne, and satellite-assisted photonic quantum communication systems.
This paper presents a novel model-free multi-agent Reinforcement Learning (RL) control method to enhance the resilience of community energy systems in island mode, which coordinates multiple objectives without the necessity of identifying system models that require expert knowledge. Specifically, a community-level coordinator agent is designed to allocate renewable energy resources among different buildings, and multiple building-level agents are developed to optimize load schedules based on limited energy resources and requirements of building loads and occupants’ comfort. In a two-day evaluation, our RL approach demonstrated a similar performance against MPC without requiring system models and formulation of optimization problems as required in MPC.
As particle accelerators grow in complexity, traditional control methods face increasing challenges in achieving optimal performance. This paper envisions a paradigm shift: a decentralized multi-agent framework for accelerator control, powered by Large Language Models (LLMs) and distributed among autonomous agents. We present a proposition of a self-improving decentralized system where intelligent agents handle high-level tasks and communication and each agent is specialized control individual accelerator components. This approach raises some questions: What are the future applications of AI in particle accelerators? How can we implement an autonomous complex system such as a particle accelerator where agents gradually improve through experience and human feedback? What are the implications of integrating a human-in-the-loop component for labeling operational data and providing expert guidance? We show two examples, where we demonstrate viability of such architecture.
Liquid cooling is critical for thermal management in high-density data centers with the rising AI workloads. However, machine learning-based controllers are essential to unlock greater energy efficiency and reliability, promoting sustainability. We present LC-Opt, a Sustainable Liquid Cooling (LC) benchmark environment, for reinforcement learning (RL) control strategies in energy-efficient liquid cooling of high-performance computing (HPC) systems. Built on the baseline of a high-fidelity digital twin of Oak Ridge National Lab's Frontier Supercomputer cooling system, LC-Opt provides detailed Modelica-based end-to-end models spanning site-level cooling towers to data center cabinets and server blade groups. RL agents optimize critical thermal controls like liquid supply temperature, flow rate, and granular valve actuation at the IT cabinet level, as well as cooling tower (CT) setpoints through a Gymnasium interface, with dynamic changes in workloads. This environment creates a multi-objective real-time optimization challenge balancing local thermal regulation and global energy efficiency, and also supports additional components like a heat recovery unit (HRU). We benchmark centralized and decentralized multi-agent RL approaches, demonstrate policy distillation into decision and regression trees for interpretable control, and explore LLM-based methods that explain control actions in natural language through an agentic mesh architecture designed to foster user trust and simplify system management. LC-Opt democratizes access to detailed, customizable liquid cooling models, enabling the ML community, operators, and vendors to develop sustainable data center liquid cooling control solutions.
This paper considers the problem of distributed beam scheduling and power allocation problem in millimeter- Wave (mmWave) cellular networks, in which multiple Base Stations (BSs) operate as individual operators over a shared spectrum. We propose a novel learning-aided approach that integrates the Lyapunov Drift-Plus-Penalty (LDPP) framework and Multi-agent Deep Deterministic Policy Gradient (MADDPG) reinforcement learning algorithms. This offers a powerful approach to learning stable and constraint-aware policies, reaping the joint benefit of both LDPP and MADDPG, in complex multiagent environments. The major challenge for this approach is to integrate these two approaches in a meaningful and effective manner. The key idea to solve this problem is to introduce a novel feature of local observation that incorporates potential negative value of the reward function due to the stochastic constraints introduced by the LDPP framework. Empirical results demonstrate that our proposed scheme outperforms the baseline methods under various conditions.
The rapid progression in artificial intelligence has facilitated the emergence of large language models like ChatGPT, offering potential applications extending into specialized engineering modeling, especially physics-based building energy modeling. This paper investigates the innovative integration of large language models with building energy modeling software, focusing specifically on the fusion of ChatGPT with EnergyPlus. A literature review is first conducted to reveal a growing trend of incorporating large language models in engineering modeling, albeit limited research on their application in building energy modeling. We underscore the potential of large language models in addressing building energy modeling challenges and outline potential applications including simulation input generation, simulation output analysis and visualization, conducting error analysis, co-simulation, simulation knowledge extraction and training, and simulation optimization. Three case studies reveal the transformative potential of large language models in automating and optimizing building energy modeling tasks, underscoring the pivotal role of artificial intelligence in advancing sustainable building practices and energy efficiency. The case studies demonstrate that selecting the right large language model techniques is essential to enhance performance and reduce engineering efforts. The findings advocate a multidisciplinary approach in future artificial intelligence research, with implications extending beyond building energy modeling to other specialized engineering modeling.
Not Available
Additive manufacturing (AM) of large objects has, over the last decade, required the scaling of existing material extrusion processes. The current generation of large-scale printers are primarily gantry robots with high-throughput extrusion systems. With workspaces approaching 50 m 3 , these printers have pushed the boundaries of achievable print volume while allowing the utilization of low-cost feedstocks, such as cementitious materials and polymer pellets, like those used in injection molding. Continued workspace expansion requires an examination of the inherent trade-offs, which impact capital and operational costs. Here, in this work, the authors examine these trade-offs to determine fundamental scaling laws for existing system architectures, survey the state of the art for alternative system configurations, and pose recommendations for future system designers to continue the evolution of large-scale AM systems.